SPARK-2058: Overriding config from SPARK_HOME with SPARK_CONF_DIR

Fixes https://issues.apache.org/jira/browse/SPARK-2058

The general idea is to use SPARK_CONF_DIR content instead of SPARK_HOME/conf when defined.

 - When SPARK_CONF_DIR is defined, spark-defaults.conf will be loaded from there. 
 - SPARK_CONF_DIR is added to the classpath, overriding the default conf from SPARK_HOME/conf
This whole paragraph is a little verbose. I think it's sufficient to say something like

To specify a different configuration directory other than the default "SPARK_HOME/conf", you can set SPARK_CONF_DIR. Spark will look for the following configuration files in this directory:
...