Fix SPARK-698. From ExecutorRunner, launch java directly instead via the run scripts.
Incorporate feedback from mateiz:
- we do not need getEnvOrEmpty
- Instead of saving SPARK_NONDAEMON_JAVA_OPTS, it would be better to modify the scripts to use a different variable name for the JAVA_OPTS they do eventually use
adding typesafe repo to streaming resolvers so that akka-zeromq is found
adding checkpoint dir to .gitignore
Made changes to the SparkContext to have a DynamicVariable for setting local properties that can be passed down the stack. Added an implementation of the fair scheduler
Changed the name of the system property to set the allocation xml
Updated/Refactored the Fair Task Scheduler. It does not inherit ClusterScheduler anymore. Rather, ClusterScheduler internally uses TaskSetQueuesManager that handles the scheduling of taskset queues. This is the class that should be extended to support other scheduling policies
Cleaned up the code
Removed some commented code
changing streaming resolver for akka
Merge pull request #517 from Reinvigorate/sm-build-fixes

Build fixes for streaming /w SBT
KafkaInputDStream improvements. Allows more Kafka configurability
fixing memory leak in kafka MessageHandler
removing dependency on ZookeeperConsumerConnector + purging last relic of kafka reliability that never solidified (ie- setOffsets)
refactor fair scheduler implementation
1.Chage "pool" properties to be the memeber of ActiveJob
2.Abstract the Schedulable of Pool and TaskSetManager
3.Abstract the FIFO and FS comparator algorithm
4.Miscellaneous changing of class define and construction
keeping JavaStreamingContext in sync with StreamingContext + adding comments for better clarity
Bugfix: WorkerWebUI must respect workDirPath from Worker
Add a shuffle parameter to coalesce.

This is useful for when you want just 1 output file (part-00000) but
still up the upstream RDD to be computed in parallel.
Fix are/our typo.
Just update typo and comments