apache/incubator-spark — pull request #481 by hornairs (2014-01-21 04:25:32+00:00)
Title: Allow controlling the number of cores pyspark uses using the `-c` option  like spark-shell

I couldn't find a way right now to control the number of cores an interactive `pyspark` session uses, which is annoying if more than one person is trying to work on a cluster interactively at once.

So,

- Use the python 2.7 stdlib argparse library to pull out the `-c` option
- Change the bin/pyspark shell script to pass all arguments to the python script instead of allowing the python interpreter program to parse any of them.