[SPARK-2894][Core] Fixes spark-shell and pyspark CLI options

This PR tries to fix SPARK-2894 in a way different from #1825, namely filtering out all `spark-submit` options from user application options, since we have control over option list of `spark-submit`. By extracting the filtering into a utility shell function, fixing other shell scripts can be much easier, and adding new `spark-submit` options wouldn't need too much duplicated work. In the mid term, we still need a cleaner solution such as what has been discussed in #1715.

All mode `pyspark` supported (Python, IPython and deprecated `pyspark app.py` style) should work. @JoshRosen @davies It would be great if you can help review PySpark related code, thanks!

An open issue is Windows scripts need also to be updated.
This code didn't work in GNU Bash 4.3.0 and 4.1.2. I guess you use BSD right? I think, this is BSD specific issue.
I re-PRed #1825 and modified to use bask slash based multiline and that worked in GNU Bash 4.3.0 and 4.1.2.

@liancheng can you check whether that works or not in 4.3.8?