databricks/spark-perf — pull request #29 by jkbradley (2014-11-24 21:29:06+00:00)
Title: [mllib] All updates for MLlib for QA for Spark 1.2

Major additions:
* added Python tests
* updated tests to do JSON logging
* updated test settings to be more reasonable for QA
* split Scala tests into directories for Spark 1.0, 1.1, 1.2 (since APIs are incompatible)
* Note: These could share some code, but it is unclear how much we need to support spark-perf for more than the latest 1 or 2 versions of MLlib.

Items remaining (before next Spark release QA period):
* Change Scala testName to match Python test names: “glm-regression” —> GLMRegressionTest
* Make parameter names match. (num-examples, num-rows, etc.)
* Refactor correlation tests so pearson/spearman is a parameter.
* Better data generation in Python