apache/spark — pull request #1197 by mattf (2014-06-24 19:01:09+00:00)
Title: [SPARK-2244] Fix hang introduced by SPARK-1466

The fix to SPARK-1466 (sha 38702487) opens a buffer for stderr, but
does not drain it under normal operation. The result is an eventual
hang during IPC.

The fix here is to close stderr after it is no longer used.

Related, but not addressed here, SPARK-1466 also removes stderr from
the console in the pyspark shell. It should be reintroduced with a
-verbose option.