apache/spark — pull request #2144 by davies (2014-08-26 19:27:33+00:00)
Title: [SPARK-3094] [PySpark] compatitable with PyPy

After this patch, we can run PySpark in PyPy (testing with PyPy 2.3.1 in Mac 10.9), for example:

```
PYSPARK_PYTHON=pypy ./bin/spark-submit wordcount.py
```

The performance speed up will depend on work load (from 20% to 3000%). Here are some benchmarks:

Job | CPython 2.7 | PyPy 2.3.1 | Speed up
-------|------------ | ------------- |
Word Count | 41s | 15s | 2.7x
Stats | 21s | 0.7s | 30x
Sort | 55s | 44s | 1.25x