[SPARK-2098] All Spark processes should support spark-defaults.conf, config file

This is another implementation about #1256
cc @andrewor14 @vanzin
Then if any of those don't fit in one line, I would do
```
val sparkConfDirDefaults =
  sys.env.get("SPARK_CONF_DIR").map(...)
val sparkHomeDefaults =
  sys.env.get("SPARK_HOME").map(...)
sparkConfDirDefaults.orElse(sparkHomeDefaults).map(_.getAbsolutePath).orNull
```