apache/spark — pull request #853 by andrewor14 (2014-05-22 08:36:53+00:00)
Title: [SPARK-1900] Fix running PySpark files on YARN

If I run the following on a YARN cluster
```
bin/spark-submit sheep.py --master yarn-client
```
it fails because of a mismatch in paths. Spark submit thinks that `sheep.py` resides on HDFS, and balks when it can't find the file there. A natural workaround is to add the `file:` prefix to the file:
```
bin/spark-submit file:/path/to/sheep.py --master yarn-client
```
However, this also fails, this time because python does not understand URI schemes.

This PR fixes this by automatically resolving all paths passed as command line argument to spark-submit properly. This has the added benefit of keeping file and jar paths consistent across different cluster modes.

Much of the code is written by @mengxr.