[SPARK-2098] All Spark processes should support spark-defaults.conf, config file

This is another implementation about #1256
cc @andrewor14 @vanzin
This looks kinda funky. How about the following:
```
sys.env.get("SPARK_CONF_DIR").map(...)
  .orElse(sys.env.get("SPARK_HOME").map(...))
  .map(_.getAbsolutePath)
  .orNull
```