amplab/spark-perf — pull request #14 by tdas (2013-12-22 01:35:42+00:00)
Title: Added key-value tests, and updated spark-tests and streaming-tests to work Spark + scala 2.10.3

- Added KVDataTest along with tests for updateStateByKey, reduceByKeyAndWindow and groupByKeyAndWindow.
- Updated spark-tests and streaming-tests to use scala 2.10.3. Converting back to 2.9.3 should just require changing the scala version in build.sbt to 2.9.3, though I havent tested it.
- Added new configuration in run and config.py to skip all tests of a project - SKIP_SPARK_TESTS, SKIP_STREAMING_TESTS, SKIP_SHARK_TESTS.
- Refactored some of the variables in the run script to make sure the dependency of compiling tests and cross- dependency between projects (shark-tests need spark-test's jar) are handled cleanly. For example, earlier the existence of spark-tests' jar would be verified even if there was no spark-tests.