[SPARK-2894][Core] Fixes spark-shell and pyspark CLI options

This PR tries to fix SPARK-2894 in a way different from #1825, namely filtering out all `spark-submit` options from user application options, since we have control over option list of `spark-submit`. By extracting the filtering into a utility shell function, fixing other shell scripts can be much easier, and adding new `spark-submit` options wouldn't need too much duplicated work. In the mid term, we still need a cleaner solution such as what has been discussed in #1715.

All mode `pyspark` supported (Python, IPython and deprecated `pyspark app.py` style) should work. @JoshRosen @davies It would be great if you can help review PySpark related code, thanks!

An open issue is Windows scripts need also to be updated.
Any reason to spread this out over 4 cases? Why not just group them in 1? (You could use backslash to escape new line)