[WIP]SPARK-2098: All Spark processes should support spark-defaults.conf, config file

Since you're touching this, could you remove the "required" here, and update docs/monitoring.md so that the config option (spark.history.fs.logDirectory) is listed in the table of options?

The option is described there in the file but in a different place, so it's easy to miss.