Spark Core - [SPARK-3620] - Refactor of SparkSubmit Argument parsing code

Argument processing seems to have gotten a lot of attention lately, so I thought I might throw my contribution into the ring.  Attached for consideration and to prompt discussion is a revamp of argument handling in SparkSubmit aimed at making things a lot more consistent. The only things that have been modified are the way that configuration properties are read/ processed and prioritised 

Things to note include:
* All configuration parameters can now be consistently set via config file

* Configuration parameters defaults have been removed from the code, and placed into a property file which is read from the class path on startup.  There should be no need to trace through 5 files to see what a config parameter defaults to if it is not specified, or have different default values applied in multiple places throughout the code.

* Configuration parameter validation is now done once all configuration parameters have been read in and resolved from various locations, not just when reading the command line.

* All property files (including spark_default_conf) are parsed by Java property handling code. All custom parsing code has been removed. Escaping of characters should now be consistent everywhere.

* All configuration parameters are overridden in the same consistent way - configuration parameters for sparkSubmit are pulled form the following sources in order of priority
 1. Entries specified on the command line (except from --conf entries)
 2. Entries specified on the command line with --conf
 3. Environment variables (including legacy variable mappings)
 4. System config variables (eg by using -Dspark.var.name)
 5. $(SPARK_DEFAULT_CONF)/spark-defaults.conf or $(SPARK_HOME)/conf/spark-defaults.conf if either exist
 6. Hard coded defaults in class path at spark-submit-defaults.prop

* A property file specified by one of the sources listed above gets read in and the properties are considered to be at the priority of the configuration source that specified the file. A property specified in a property file will not override an existing config value already specifiedby that configuration source

The existing argument handling is pretty finicky - chances are high that I’ve missed some behaviour - if this PR is going to be accepted/approved let me know any bugs and I’ll fix them up and document the behaviour for future reference
Hi @vanzin, I believe the existing behaviour is driven by environment variables and files in the file system as well as command line options (though command line options seem to have priority (the existing code base treats priority on a variable-by-variable basis, so It can be easy to miss exceptions to the rule).

if $SPARK_CONF_DIR/spark-defaults.conf iexists then it is loaded into a hashmap. if SPARK_HOME/conf.spark-defaults.conf exists then it will replace the above mentioned hashmap.
  What ever is left is then overridden by what ever --conf properties have been specified no the command line.

These values are then overridden by various command line specified options.

Finally legacy environment variables are checked and config values assigned if they have yet to be assigned.