Address review comments, move to incubator spark

Also includes a small fix to speculative execution.

&lt;edit> Continued from https://github.com/mesos/spark/pull/914 &lt;/edit>