Shopify/spark — pull request #6 by hornairs (2014-01-24 01:45:31+00:00)
Title: Fix broken quoting in spark-env.sh and echo to stderr to not break stdout communication with pyspark

This fixes a couple things:

- my handy "Connecting to shopify cluster..." message breaks the stdout communication pyspark uses during boot with Py4J (the JVM bridge), so echo stuff in the spark conf to stderr instead of stdout
- I completely incorrectly wrote the bash which infers the local IP such that it was just erroring every time. Whoops.

This works for a standalone pyspark app for me :)

@tyro89 @dklassen look ok?