mesos/spark — pull request #346 by JoshRosen (2012-12-30 02:11:46+00:00)
Title: Python API (PySpark)

This is a first version of the Spark Python API (PySpark).

This pull request contains the PySpark code, plus a Python Programming Guide, Epydoc API docs, and additions to the Quick Start guide.

I would appreciate any feedback, but I'm especially interested in any comments on these topics:

* __Testing__: I've tested this code locally and on the Spark EC2 image, but it would be useful to see if the out-of-the-box configuration works on other systems.
* __Python Programming Guide__: I'd appreciate additional examples or suggestions for this guide.
* __Code clarity__: PySpark's code has a few complicated parts, particularly at the interfaces between Java and the Python worker processes. I'd love feedback on which parts are confusing and require comments.
* __Fixing the LR example__: I'd like to fix [SPARK-646](https://spark-project.atlassian.net/browse/SPARK-646) so that I can run the LR example on a large dataset.

If you want to try out PySpark, follow the instructions in the documentation included in this branch.