apache/spark — pull request #3424 by lvsoft (2014-11-24 07:56:51+00:00)
Title: [SPARK-2313] PySpark pass port rather than stdin

This patch will fix [SPARK-2313].

It peek available free port number, and pass the port number to Py4j.Gateway for binding via command line argument.
The initial value of the port number is scanned beginning at the mod of PID, which could avoid potential concurrency issues such as supporting multiple PySpark instances in future. And the port number printed from Py4j in STDIN is also parsed for double check.