Prior to Maven 3.2.1 how Maven built multi-module projects was fixed. There was a serial strategy and parallel strategy of stepping through projects to build, and the way projects could reference one another as dependencies was not pluggable. These two aspects of building multi-module projects and resolving inter-project dependencies is loosely referred to as the reactor and it’s behaviour the source of much annoyance. Normally we tell users to build everything from the top-level directory build everything. But if you want to build individual projects we suggest using something like:

mvn test --projects :project1,...,:projectN `  

If you want to make the dependencies of those projects use:

mvn test --projects :project1,...,:projectN -am

This is not too bad for building, but while trying to develop a product this can be annoying. I myself never build applications from the command line as I’m accustomed to M2Eclipse with workspace resolution where I develop, run, and debug applications or plugins from within the Eclipse workspace. Things I typically do in a day inside Eclipse with M2Eclipse:

• Develop Maven, Maven plugins with sample projects where I run the sample project against the version of Maven in the workspace using Maven plugins found in the workspace.
• Develop Nexus plugins where Nexus is booted from Eclipse and the plugin manager loads my Nexus plugin from the workspace and I can debug and test from within the workspace.
• Develop Dropwizard applications with dependencies resolved from the workspace. I develop, run, test, and debug all from within the Eclipse workspace.

We have developed fairly sophisticated tools in Eclipse over the years so my attention to the command line is probably lacking. But in all my use cases above, nothing needs to be installed to the local repository, all my dependencies resolve from the workspace and what I’m working on executes from within Eclipse. I just wouldn’t be able to develop Maven efficiently if I was messing around on the command line all day. In the case of Maven, I also run Maven’s integration tests from within Eclipse where a process forks, executes Maven from my workspace running with any Maven plugins found in my workspace and I debug it all from within Eclipse. No remote debugging, and we have source lookups that work by using an agent that traces a class back to its origin JAR, finds the pom.properties within the JAR to find the Maven coordinates of the JAR and we use that to retrieve the right source JAR and links it in automatically.

Trying to develop an application, like Maven, from the command line would honestly drive me bat shit crazy so I can understand why people are frustrated with application development from the command line with Maven. We have developed tooling in Eclipse to make this very efficient and over the years we have done this for Maven, Nexus, Tycho and general webapp development with Webby. I personally don’t see how you can really be efficient working on something relatively sophisticated on the command line. That said there is no reason that our workspace type resolution we have in Eclipse with M2Eclipse can’t be achieved from the command line for more productive command line development.

How Maven can build now?

With the changes that were recently made to Maven it would be possible to have a model akin to that of an Eclipse workspace, which I believe is very productive. Just to be clear I’m not saying we’ve fully implemented this yet for Maven but we’re getting there. I still don’t think it’s terribly efficient to develop an application on the command line but some may still prefer that, and it does have significant implications for CI if you could treat all projects in CI like a workspace.

So what has changed in Maven and how did those changes come about? For a customer I needed to experiment with different ways to build Maven projects while retaining all of Maven’s normal behaviour. I needed to build faster and I needed different resolution semantics from the reactor to be more like an Eclipse workspace.

To control the way a project builds I created a new Builder interface where implementations can encapsulate a strategy for stepping through projects to build. There existed a serial mode, and a simple parallel mode for building projects: I took this existing code and refactored them into separate builder implementations. I then worked with our customer to create a faster parallel mode of execution and that also became a builder implementation. Now from Maven’s command line you can specify --builder builder-id to use a different builder. Our implementation is packaged up in a separte JAR, dropped into Maven’s lib/ext folder and it’s dynamically picked up and made available.

To create workspace like resolution I made Maven’s WorkspaceReader pluggable. The existing implementation is what is used to see if a request for a dependency can be satisfied from the set of projects currently being built. I wanted to change it such that I might only be building one project in a multi-module project, but if the output (target/classes) exist in my workspace then resolve the dependency from the workspace.

I basically turned all the projects in a multi-module build into a workspace that I can resolve from like in Eclipse. So I end up with the projects I’m building, the buildspace, and all the projects I want to resolve against, the workspace. It was easy to play around with different implementations by creating new implementations of Builders and WorkspaceReaders. If you wanted to create a model where you stepped into a directory and automatically built everything the project depends on you can now.

With these two new extension points it is possible to completely change the way Maven builds and resolves inter-project dependencies in the context of multi-module projects. For the customer we’re working with we haven’t fully figured out the pattern as we are trying to solve for working simultaneously on multiple branches which represent multiple workspaces, and reconciling that with the local repository. Thus far we’ve started designing a workspace descriptor that sits at the top of your multi-module project that effectively pins all those projects into your workspace. You can control what projects you want to build from the CLI or by configuring your workspace descriptor. We have also started creating a layered local repository implementation so that multiple branches can be developed at the same time without interferring with one another.

This workspace mode of building is related to the incremental build support we have done for the same customer. Ultimately I see this workspace mode of building combined with our incremental plugins and possibly run from a shell to effectively give you the same workspace resolution and incremental behaviour that is available while developing from within Eclipse. If you move around directories in your build and perform different actions its all within the defined workspace so we can make the behaviour consistent. All new implementations can also be created without affecting existing behaviour. With our heavy use the dependency injection in Maven’s core and our extension mechanism, it’s very easy to define new implementations that can selectively override existing ones. That combined with the changes made in Maven 3.2.1 it’s really possible to build in new ways.

What becomes standard I don’t know yet, but it’s been very easy to experiment now which makes it fun again! I believe this is a necessary change and allows us to develop ingenious new ways of building projects that users can selectively try. It means that we can move forward with new techniques and if you rely on the existing behaviour you will not be forced off that behaviour, you can move to new behaviour when you choose. This is all good news for Maven.

Next up: Incremental builds? No problem!

Comments