In the last few months at Takari we’ve primarily been working on performance and process improvements for Maven as part of a comprehensive conversion from Ant. On the performance side we’ve create an incremental build API for Maven and a more efficient way to to schedule builds in parallel. On the process side we’ve been working on a form of continuous delivery for Maven that we call generations. On the fun side we’ve been poking at Polyglot for Maven and the Shell for Maven.

Incremental build

Maven takes a lot of heat for not being incremental but this will change in the near future. There have been two previous APIs for incremental build that we have leveraged in M2E but there have only been no-op implementations for the CLI. We used what we have learned in the past to create an implementation for the CLI that we’re getting comfortable with. We have revised the API several times while creating plugin implementations for a customer and we’re opening up the API to review by a small group of people this week.

Making all Maven builds fully incremental will likely take some time as the API is just the first step. In order make incremental builds work generally we have introduced a new custom Guice scope in Maven and we have made an implementation of the standard build lifecycle that is fully incremental, including an incremental compiler based on JDT. We are currently working on integrating this into M2E and will need to sort out some issues before incremental plugins work transparently in Eclipse. Our next blog entry will be dedicated to describing our incremental build strategy in more detail so stay tuned.

Smart scheduling

Our new smart scheduling for Maven consists of a new parallelization model that is more aggressive, and optimized prioritization of project builds based on recording their execution times and persisting them for subsequent analysis. The current model of parallelization builds everything at a given level in the graph until proceeding to the next while our new form recursively topo-sorts to find all available projects that can be built (projects where all upstream dependencies have been satisfied). For the optimized prioritization we are working from our clients crafty Ant-based implementation which is based on the work found in Static vs. Dynamic List-Scheduling Performance Comparison. In order to make our new implementation pluggable we have refactored the current strategies in Maven and created a new provisional Builder interface which all existing implementation will now use.

Generations

We hope that our generations model becomes the standard way to do continuous delivery in Maven. We’re really excited to see this start being used in production after several months of work. We do not have the complete model working yet, but we are deploying a cohesive set of projects that corresponds to a source control revision which requires no further transformation to be used, no rebuilds are required and no maven-release-plugin. For both deployment and consumption there is a delta protocol to reduce the payload to what has changed between the source and target revision. Generations will be the subject of many further blog entries so stay tuned. We will also be looking for feedback and review of the whole system in March.

The combination of incremental builds, the smart scheduling and generations is going to make Maven incredibly fast not only from a performance perspective but also from a process perspective.

In addition to client work we’ve been doing some fun work on Polyglot and the Shell. These two projects may become client work at some point in the near future which would be great validation for their value like the other features we’ve been working on.

Polyglot for Maven

Polyglot for Maven is still moving along and we’ve seen a lot of activity recently on the Ruby DSL by Cristian Meier from the JRuby team, and the Scala DSL by Chris Hunt from Typesafe. We hope that this initial work can serve as inspiration for helping the POM evolve. For those interested in what the XML alternatives can look like you can take a look at the existing implementations:

• Ruby DSL
• Groovy DSL
• Scala DSL
• YAML
• Atom

Shell for Maven

The long term hope for the Shell is that it serve as a housing for a long-lived Maven process. It will take advantage of POM caching, leverage the incremental build APIs and provide better resolution semantics and be akin to the workspace style resolution that is present in M2E. There is also some cool Git and Github integration in the shell.

Really, if you’re interested in any of these features then you might want to watch our last webinar which gives a quick over view of all the features and a demo of the Shell at the end.

Comments