mesos/spark — pull request #837 by AndreSchumacher (2013-08-15 23:40:37+00:00)
Title: SPARK-878: Adding of zip and egg files to PYTHONPATH

Previously it seems that only the local working directory was added to sys,path. See the original discussion:

https://groups.google.com/forum/#!searchin/spark-users/pyfiles/spark-users/jG8VC17vTe4/z_DhBoRWuAMJ

This pull request lets PySpark's SparkContext record the *.zip and *.egg files. These are then recorded in Shark's PythonRDD, which serializes their names into the worker's input stream, similarly to the broadcast variables. Then the worker adds them to its sys.path prior to the function execution. Also a test case for the local mode is included.