Merge pull request #155 from rxin/jobgroup

Don't reset job group when a new job description is set.

(cherry picked from commit 83bf1920c85c33d8d725831b28bc5d11b53c27bd)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #149 from tgravescs/fixSecureHdfsAccess

Fix secure hdfs access for spark on yarn

https://github.com/apache/incubator-spark/pull/23 broke secure hdfs access. Not sure if it works with secure hdfs on standalone. Fixing it at least for spark on yarn.

The broadcasting of jobconf change also broke secure hdfs access as it didn't take into account things calling the getPartitions before sparkContext is initialized. The DAGScheduler does this as it tries to getShuffleMapStage.

(cherry picked from commit 8af99f2356ed19fe43d722ada02f8802cbd46d40)
Signed-off-by: Reynold Xin <rxin@apache.org>