A comment on this file from the version that was merged in by Reynold: The MetadadataCleaner seems like it may be creating non-daemon threads by using TimerTask. Will this prevent the JVM from exiting until this is stopped? Or does Java use daemon threads for TimerTask? Basically it would be nice if programs will still exit even if they don't call SparkContext.stop(). Try that out, and if it doesn't work, switch this to using a DaemonThread with a sleep loop instead of TimerTask.