Misc style changes in the 'yarn' package.
Merge branch 'master' into yarn-cleanup

Conflicts:
	yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
	yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
	yarn/src/main/scala/org/apache/spark/deploy/yarn/WorkerRunnable.scala
	yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
A few more style fixes in `yarn` package.
Add a "new-yarn" directory in SPARK_HOME, intended to contain Hadoop-2.2 API changes.
Hadoop 2.2 YARN API migration for `SPARK_HOME/new-yarn`
Add optional Hadoop 2.2 settings in sbt build.

If the Hadoop used is version 2.2 or derived from it, then Spark
will be compiled against protobuf-2.5 and a protobuf-2.5 version of
Akka 2.0.5.
Merge remote-tracking branch 'origin/master' into yarn-2.2

Conflicts:
	yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
Port yarn-client mode for new-yarn
Merge pull request #1 from colorant/yarn-client-2.2

Port yarn-client mode for new-yarn
Use published "org.spark-project.akka-*" in sbt build for Hadoop-2.2 dependencies.

This also includes:
-Change `isNewYarn` to `isNewHadoop`, since the protobuf-2.5 dependency is from Hadoop-2.2 itself.
-Regexp bugix

Credits to @alig for this patch.
Fix pom.xml for maven build
Merge pull request #2 from colorant/yarn-client-2.2

Fix pom.xml for maven build
Document missing configs and set shuffle consolidation to false.
Small changes from Matei review
Merge pull request #199 from harveyfeng/yarn-2.2

Hadoop 2.2 migration

Includes support for the YARN API stabilized in the Hadoop 2.2 release, and a few style patches.

Short description for each set of commits:

a98f5a0 - "Misc style changes in the 'yarn' package"
a67ebf4 - "A few more style fixes in the 'yarn' package"
Both of these are some minor style changes, such as fixing lines over 100 chars, to the existing YARN code.

ab8652f - "Add a 'new-yarn' directory ... "
Copies everything from `SPARK_HOME/yarn` to `SPARK_HOME/new-yarn`. No actual code changes here.

4f1c3fa - "Hadoop 2.2 YARN API migration ..."
API patches to code in the `SPARK_HOME/new-yarn` directory. There are a few more small style changes mixed in, too.
Based on @colorant's Hadoop 2.2 support for the scala-2.10 branch in #141.

a1a1c62 - "Add optional Hadoop 2.2 settings in sbt build ... "
If Spark should be built against Hadoop 2.2, then:
a) the `org.apache.spark.deploy.yarn` package will be compiled from the `new-yarn` directory.
b) Protobuf v2.5 will be used as a Spark dependency, since Hadoop 2.2 depends on it. Also, Spark will be built against a version of Akka v2.0.5 that's built against Protobuf 2.5, named `akka-2.0.5-protobuf-2.5`. The patched Akka is here: https://github.com/harveyfeng/akka/tree/2.0.5-protobuf-2.5, and was published to local Ivy during testing.

There's also a new boolean environment variable, `SPARK_IS_NEW_HADOOP`, that users can manually set if their `SPARK_HADOOP_VERSION` specification does not start with `2.2`, which is how the build file tries to detect a 2.2 version. Not sure if this is necessary or done in the best way, though...
Forcing shuffle consolidation in DiskBlockManagerSuite
Merge pull request #228 from pwendell/master

Document missing configs and set shuffle consolidation to false.