The broken pipe error manifests itself in the test_short_lived_command() test in the script above.  What happens is that for each split (in our example, there are two) we call the command (in our example, it's "echo hello").  The echo command doesn't take input and returns almost immediately.  As we continue to pass data into proc.stdin, the pipe is closed because the command completed and we get the broken pipe exception. Ignoring the broken pipe handles the case where the command finishes before we're done sending in data.  If you take a look at the [pipe with env variable](https://github.com/mesos/spark/blob/master/core/src/test/scala/spark/PipedRDDSuite.scala?source=c#L22) test in Spark, this is the expected output.  (Except that they handle setting envVars which you do not.)