mesos/spark — pull request #310 by tomdz (2012-11-21 01:00:52+00:00)
Title: Maven build setup

Some notes about this pull request:

It requires Maven 3 (we use 3.0.4) and Java 1.6 or newer (works fine with Java 1.7 which is what we use).

It includes two profiles for building for Hadoop 1 or Hadoop 2. One of these **has** to be chosen explicitly, there is no default:

```
$ mvn -Phadoop1 clean install
```
for Hadoop 1 (using 0.20.205.0), or
```
$ mvn -Phadoop2 clean install
```
for Hadoop 2 (using 2.0.0-mr1-cdh4.1.1).

It uses the [scala-maven-plugin](http://davidb.github.com/scala-maven-plugin/) which supports incremental and continuous compilation. E.g.
```
mvn -Phadoop2 scala:cc
```
should run continuous compilation (i.e. wait for changes). We have not tested this extensively however.

This setup works fine in IntelliJ IDEA 11.1.4. After opening the project via the `pom.xml` file in the project root folder, you only need to activate either the `hadoop1` or `hadoop2` profile in the "Maven Properties" popout. We have not tried Eclipse/Scala IDE with this.

It includes support for building a Debian package containing a 'fat-jar' which includes the repl, the examples and bagel. This can be created by specifying the `deb` profile:
```
$ mvn -Phadoop2,deb clean install
```
The debian package can then be found under `repl/target`. We added the short commit hash to the file name so that we can distinguish individual packages build for SNAPSHOT versions.

Tests are run via the [scalatest-maven-plugin](http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin). With this you can do things like:
```
$ mvn -DskipTests -Phadoop2 clean install
```
for skipping test execution (but not compilation), and
```
repl$ mvn -Phadoop2 -Dsuites=spark.repl.ReplSuite test
```
for running a specific test suite.

Unfortunately the Twirl bits only work as a SBT plugin, so I had to extract the API source code into a separate artifact (https://github.com/tomdz/twirl-api) and create a Maven plugin (https://github.com/tomdz/twirl-maven-plugin). Both of these are released to the Sonatype's OSS repository and should show up in Maven central within the next day or so. I also have an open issue with the Spray project about a shared twirl-api (and maybe twirl-compiler) project (https://github.com/spray/twirl/issues/8), but I haven't heard back from them yet.

Last but not least I included support for deploying to OSS Sonatype (which is the official way to get into the Maven central repository) as per https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide. More specifically, the `pom.xml` files should be setup for deployment, and only the JIRA & GPG steps in that document should be what's left to do in order to be able deploy Spark to the Sonatype's OSS maven repository (which is automatically sync'd to Maven central once a day or so) .