[SPARK-1395] Fix "local:" URI support in Yarn mode (again).

Recent changes ignored the fact that path may be defined with "local:"
URIs, which means they need to be explicitly added to the classpath
everywhere a remote process is started. This change fixes that by:

- Using the correct methods to add paths to the classpath
- Creating SparkConf settings for the Spark jar itself and for the
  user's jar
- Propagating those two settings to the remote processes where needed

This ensures that both in client and in cluster mode, the driver has
the necessary info to build the executor's classpath and have things
still work when they contain "local:" references.

On the cleanup front, I removed the hacky way that log4j configuration
was being propagated to handle the "local:" case. It's much more cleanly
(and generically) handled by using spark-submit arguments (--files to
upload a config file, or setting spark.executor.extraJavaOptions to pass
JVM arguments and use a local file).

Can you elaborate? SparkConf will include all the system properties that start with "spark", which is exactly what the previous code was doing when reading system properties directly.

I'll re-test without this fix (my tests were failing without it but after reading the rest of the code I'm not so sure it was because of this), but in general I think this is a cleaner way of doing it (since it treats both cases as the same).