Move disk warmup code into Cluster class.
Introduce SparkBuildManager to maintain build cache:

SparkBuildManager maintains a cache of binary Spark distributions,
indexed by their SHAs. This makes it much faster to test against
multiple Spark versions because we don’t have to completely re-build
Spark for each test run.

I removed the SPARK_SKIP_PREP configuration option, since it’s not very
useful anymore: if we’re running multiple tests against the same Spark
version, we’ll just use the cached build.

This commit part of the groundwork for automated testing against
multiple Spark versions.
Several build cleanups / improvements:

- Upgrade all subprojects to use newer SBT launch script from Spark; remove
  sbt JARs, since they will now be automatically downloaded.
- Use newer sbt, sbt-assembly, Scala, and ScalaTest versions.
- Remove unused dependency from streaming tests.
- Switch builds to use .scala build definitions instead of .sbt files; this
  will make it easier to introduce a test support library that can be shared
  across test projects.
- Add MLlib perf tests to Travis tests.
- Run `sbt assembly` in Travis tests.
- Add Travis badge to README.