Shopify/spark — pull request #1 by hornairs (2014-01-21 15:36:28+00:00)
Title: Autoconnect VPN

`conf/spark-env.sh` is our local configuration which manages a bunch of stuff. This runs an applescript to tell viscosity to connect to the VPN if it can't infer the local IP to bind to, which is necessary since the spark master and the console need to communicate. Example execution:

```
oration ~/C/spark (master*) ➜ bin/pyspark
Sparkify: Connecting to chicago spark cluster ...
WARNING: could not find an VPN interface to connect to the Shopify Spark Cluster! Trying to autoconnect...
251:257: syntax error: Expected end of line, etc. but found identifier. (-2741)
Python 2.7.6 (default, Dec 10 2013, 15:39:48)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
^CTraceback (most recent call last):
File "/Users/hornairs/Code/spark/python/pyspark/shell.py", line 32, in
sc = SparkContext(os.environ.get("MASTER", "local"), "PySparkShell", pyFiles=add_files)
File "/Users/hornairs/Code/spark/python/pyspark/context.py", line 123, in __init__
self._jsc = self._jvm.JavaSparkContext(self._conf._jconf)
File "/Users/hornairs/Code/spark/python/lib/py4j-0.8.1-src.zip/py4j/java_gateway.py", line 667, in __call__
File "/Users/hornairs/Code/spark/python/lib/py4j-0.8.1-src.zip/py4j/java_gateway.py", line 363, in send_command
File "/Users/hornairs/Code/spark/python/lib/py4j-0.8.1-src.zip/py4j/java_gateway.py", line 472, in send_command
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 430, in readline
data = recv(1)
KeyboardInterrupt
>>>
oration ~/C/spark (master*) ➜ bin/pyspark
Sparkify: Connecting to chicago spark cluster ...
WARNING: could not find an VPN interface to connect to the Shopify Spark Cluster! Trying to autoconnect...
Connected!
Python 2.7.6 (default, Dec 10 2013, 15:39:48)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
14/01/21 10:33:31 INFO Utils: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
*snip*
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/__ / .__/\_,_/_/ /_/\_\ version 0.9.0-SNAPSHOT
/_/

Using Python version 2.7.6 (default, Dec 10 2013 15:39:48)
Spark context available as sc.
>>>
```

@bouke @honkfestival see any problems with this?