[SPARK-2585] Remove special handling of Hadoop JobConf.

This is based on #1498. Diff here: https://github.com/rxin/spark/commit/5904cb6649b03d48e3465ccab664b506cc69327b