bugfix: overflow of graphx Edge compare function

Author: Zhen Peng <zhenpeng01@baidu.com>

Closes #769 from zhpengg/bugfix-graphx-edge-compare and squashes the following commits:

8a978ff [Zhen Peng] add ut for graphx Edge.lexicographicOrdering.compare
413c258 [Zhen Peng] there maybe a overflow for two Long's substraction

(cherry picked from commit fa6de408a131a3e84350a60af74a92c323dfc5eb)
Signed-off-by: Reynold Xin <rxin@apache.org>
[SQL] Implement between in hql

Author: Michael Armbrust <michael@databricks.com>

Closes #804 from marmbrus/between and squashes the following commits:

ae24672 [Michael Armbrust] add golden answer.
d9997ef [Michael Armbrust] Implement between in hql.
9bd4433 [Michael Armbrust] Better error on parse failures.

(cherry picked from commit 032d6632ad4ab88c97c9e568b63169a114220a02)
Signed-off-by: Reynold Xin <rxin@apache.org>
Tweaks to Mesos docs

- Mention Apache downloads first
- Shorten some wording

Author: Matei Zaharia <matei@databricks.com>

Closes #806 from mateiz/doc-update and squashes the following commits:

d9345cd [Matei Zaharia] typo
a179f8d [Matei Zaharia] Tweaks to Mesos docs

(cherry picked from commit fed6303f29250bd5e656dbdd731b38938c933a61)
Signed-off-by: Matei Zaharia <matei@databricks.com>
SPARK-1864 Look in spark conf instead of system properties when propagating configuration to executors.

Author: Michael Armbrust <michael@databricks.com>

Closes #808 from marmbrus/confClasspath and squashes the following commits:

4c31d57 [Michael Armbrust] Look in spark conf instead of system properties when propagating configuration to executors.
(cherry picked from commit a80a6a139e729ee3f81ec4f0028e084d2d9f7e82)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Version bump of spark-ec2 scripts

This will allow us to change things in spark-ec2 related to the 1.0 release.

Author: Patrick Wendell <pwendell@gmail.com>

Closes #809 from pwendell/spark-ec2 and squashes the following commits:

59117fb [Patrick Wendell] Version bump of spark-ec2 scripts
(cherry picked from commit c0ab85d7320cea90e6331fb03a70349bc804c1b1)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
[SPARK-1808] Route bin/pyspark through Spark submit

**Problem.** For `bin/pyspark`, there is currently no other way to specify Spark configuration properties other than through `SPARK_JAVA_OPTS` in `conf/spark-env.sh`. However, this mechanism is supposedly deprecated. Instead, it needs to pick up configurations explicitly specified in `conf/spark-defaults.conf`.

**Solution.** Have `bin/pyspark` invoke `bin/spark-submit`, like all of its counterparts in Scala land (i.e. `bin/spark-shell`, `bin/run-example`). This has the additional benefit of making the invocation of all the user facing Spark scripts consistent.

**Details.** `bin/pyspark` inherently handles two cases: (1) running python applications and (2) running the python shell. For (1), Spark submit already handles running python applications. For cases in which `bin/pyspark` is given a python file, we can simply call pass the file directly to Spark submit and let it handle the rest.

For case (2), `bin/pyspark` starts a python process as before, which launches the JVM as a sub-process. The existing code already provides a code path to do this. All we needed to change is to use `bin/spark-submit` instead of `spark-class` to launch the JVM. This requires modifications to Spark submit to handle the pyspark shell as a special case.

This has been tested locally (OSX and Windows 7), on a standalone cluster, and on a YARN cluster. Running IPython also works as before, except now it takes in Spark submit arguments too.

Author: Andrew Or <andrewor14@gmail.com>

Closes #799 from andrewor14/pyspark-submit and squashes the following commits:

bf37e36 [Andrew Or] Minor changes
01066fa [Andrew Or] bin/pyspark for Windows
c8cb3bf [Andrew Or] Handle perverse app names (with escaped quotes)
1866f85 [Andrew Or] Windows is not cooperating
456d844 [Andrew Or] Guard against shlex hanging if PYSPARK_SUBMIT_ARGS is not set
7eebda8 [Andrew Or] Merge branch 'master' of github.com:apache/spark into pyspark-submit
b7ba0d8 [Andrew Or] Address a few comments (minor)
06eb138 [Andrew Or] Use shlex instead of writing our own parser
05879fa [Andrew Or] Merge branch 'master' of github.com:apache/spark into pyspark-submit
a823661 [Andrew Or] Fix --die-on-broken-pipe not propagated properly
6fba412 [Andrew Or] Deal with quotes + address various comments
fe4c8a7 [Andrew Or] Update --help for bin/pyspark
afe47bf [Andrew Or] Fix spark shell
f04aaa4 [Andrew Or] Merge branch 'master' of github.com:apache/spark into pyspark-submit
a371d26 [Andrew Or] Route bin/pyspark through Spark submit
(cherry picked from commit 4b8ec6fcfd7a7ef0857d5b21917183c181301c95)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
[SPARK-1824] Remove <master> from Python examples

A recent PR (#552) fixed this for all Scala / Java examples. We need to do it for python too.

Note that this blocks on #799, which makes `bin/pyspark` go through Spark submit. With only the changes in this PR, the only way to run these examples is through Spark submit. Once #799 goes in, you can use `bin/pyspark` to run them too. For example,

```
bin/pyspark examples/src/main/python/pi.py 100 --master local-cluster[4,1,512]
```

Author: Andrew Or <andrewor14@gmail.com>

Closes #802 from andrewor14/python-examples and squashes the following commits:

cf50b9f [Andrew Or] De-indent python comments (minor)
50f80b1 [Andrew Or] Remove pyFiles from SparkContext construction
c362f69 [Andrew Or] Update docs to use spark-submit for python applications
7072c6a [Andrew Or] Merge branch 'master' of github.com:apache/spark into python-examples
427a5f0 [Andrew Or] Update docs
d32072c [Andrew Or] Remove <master> from examples + update usages
(cherry picked from commit cf6cbe9f76c3b322a968c836d039fc5b70d4ce43)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Make deprecation warning less severe

Just a small change. I think it's good not to scare people who are using the old options.

Author: Patrick Wendell <pwendell@gmail.com>

Closes #810 from pwendell/warnings and squashes the following commits:

cb8a311 [Patrick Wendell] Make deprecation warning less severe
(cherry picked from commit 442808a7482b81c8de887c901b424683da62022e)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit e5436b8c1a79ce108f3af402455ac5f6dc5d1eb3.
Revert "[maven-release-plugin] prepare release v1.0.0-rc8"

This reverts commit 80eea0f111c06260ffaa780d2f3f7facd09c17bc.
[maven-release-plugin] prepare release v1.0.0-rc9
[maven-release-plugin] prepare for next development iteration