- Allow for finer control of cleaner
- Address review comments, move to incubator spark
- Also includes a change to speculation - including preventing exceptions in rare cases.
Add an optional closure parameter to HadoopRDD instantiation to used when creating any local JobConfs.
Add a zookeeper compile dependency to fix build in maven
Remove unnecessary mutable imports
Merge pull request #32 from mridulm/master

Address review comments, move to incubator spark

Also includes a small fix to speculative execution.

&lt;edit> Continued from https://github.com/mesos/spark/pull/914 &lt;/edit>
Merge pull request #53 from witgo/master

Add a zookeeper compile dependency to fix build in maven

 Add a zookeeper compile dependency to fix build in maven
Merge pull request #54 from aoiwelle/remove_unused_imports

Remove unnecessary mutable imports

It appears that the imports aren't necessary here.
Remove the new HadoopRDD constructor from SparkContext API, plus some minor style changes.
Merge pull request #52 from harveyfeng/hadoop-closure

Add an optional closure parameter to HadoopRDD instantiation to use when creating local JobConfs.

Having HadoopRDD accept this optional closure eliminates the need for the HadoopFileRDD added earlier. It makes the HadoopRDD more general, in that the caller can specify any JobConf initialization flow.
Refactor BlockId into an actual type

This is an unfortunately invasive change which converts all of our BlockId
strings into actual BlockId types. Here are some advantages of doing this now:

+ Type safety

+ Code clarity - it's now obvious what the key of a shuffle or rdd block is,
  for instance. Additionally, appearing in tuple/map type signatures is a big
  readability bonus. A Seq[(String, BlockStatus)] is not very clear.
  Further, we can now use more Scala features, like matching on BlockId types.

+ Explicit usage - we can now formally tell where various BlockIds are being used
  (without doing string searches); this makes updating current BlockIds a much
  clearer process, and compiler-supported.
  (I'm looking at you, shuffle file consolidation.)

+ It will only get harder to make this change as time goes on.

Since this touches a lot of files, it'd be best to either get this patch
in quickly or throw it on the ground to avoid too many secondary merge conflicts.
Add unit test and address rest of Reynold's comments
Change BlockId filename to name + rest of Patrick's comments
Address Matei's comments
Merge pull request #57 from aarondav/bid

Refactor BlockId into an actual type

Converts all of our BlockId strings into actual BlockId types. Here are some advantages of doing this now:

+ Type safety
+  Code clarity - it's now obvious what the key of a shuffle or rdd block is, for instance. Additionally, appearing in tuple/map type signatures is a big readability bonus. A Seq[(String, BlockStatus)] is not very clear. Further, we can now use more Scala features, like matching on BlockId types.
+ Explicit usage - we can now formally tell where various BlockIds are being used (without doing string searches); this makes updating current BlockIds a much clearer process, and compiler-supported.
  (I'm looking at you, shuffle file consolidation.)
+ It will only get harder to make this change as time goes on.

Downside is, of course, that this is a very invasive change touching a lot of different files, which will inevitably lead to merge conflicts for many.
Merge branch 'master' of github.com:apache/incubator-spark into kill

Conflicts:
	core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala