apache/incubator-spark — pull request #283 by tmyklebu (2013-12-21 06:48:44+00:00)
Title: Python bindings for most of mllib

This pull request contains Python bindings for the regression, clustering, and classification tools in mllib. This pull request notably does not contain bindings for the recommendation engine based on alternating least squares; that will likely happen in a future pull request.

For each 'train' frontend exposed, there is a Scala stub in PythonMLLibAPI.scala and a Python stub in mllib.py. The Python stub serialises the input RDD and any vector/matrix arguments into a mutually-understood format and calls the Scala stub. The Scala stub deserialises the RDD and the vector/matrix arguments, calls the appropriate 'train' function, serialises the resulting model, and returns the serialised model.

I have tested these bindings on an x86_64 machine running Linux. There is a risk that these bindings may fail on some choose-your-own-endian platform if Python's endian differs from java.nio.ByteBuffer's idea of the native byte order.