[SPARK-3032][Shuffle] Fix key comparison integer overflow introduced sorting exception

Previous key comparison in `ExternalSorter` will get wrong sorting result or exception when key comparison overflows, details can be seen in [SPARK-3032](https://issues.apache.org/jira/browse/SPARK-3032). Here fix this and add a unit test to prove it.

Author: jerryshao <saisai.shao@intel.com>

Closes #2514 from jerryshao/SPARK-3032 and squashes the following commits:

6f3c302 [jerryshao] Improve the unit test according to comments
01911e6 [jerryshao] Change the test to show the contract violate exception
83acb38 [jerryshao] Minor changes according to comments
fa2a08f [jerryshao] Fix key comparison integer overflow introduced sorting exception

(cherry picked from commit dab1b0ae29a6d3017bdca23464f22a51d51eaae1)
Signed-off-by: Matei Zaharia <matei@databricks.com>
[graphX] GraphOps: random pick vertex bug

When `numVertices > 50`, probability is set to 0. This would cause infinite loop.

Author: yingjieMiao <yingjie@42go.com>

Closes #2553 from yingjieMiao/graphx and squashes the following commits:

6adf3c8 [yingjieMiao] [graphX] GraphOps: random pick vertex bug

(cherry picked from commit 51229ff7f4d3517706a1cdc1a2943ede1c605089)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
Fixed the condition in StronglyConnectedComponents Issue: SPARK-3635

Author: oded <oded@HP-DV6.c4internal.c4-security.com>

Closes #2486 from odedz/master and squashes the following commits:

dd7890a [oded] Fixed the condition in StronglyConnectedComponents Issue: SPARK-3635

(cherry picked from commit dc30e4504abcda1774f5f09a08bba73d29a2898b)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
[SPARK-3734] DriverRunner should not read SPARK_HOME from submitter's environment

When using spark-submit in `cluster` mode to submit a job to a Spark Standalone
cluster, if the JAVA_HOME environment variable was set on the submitting
machine then DriverRunner would attempt to use the submitter's JAVA_HOME to
launch the driver process (instead of the worker's JAVA_HOME), causing the
driver to fail unless the submitter and worker had the same Java location.

This commit fixes this by reading JAVA_HOME from sys.env instead of
command.environment.

Author: Josh Rosen <joshrosen@apache.org>

Closes #2586 from JoshRosen/SPARK-3734 and squashes the following commits:

e9513d9 [Josh Rosen] [SPARK-3734] DriverRunner should not read SPARK_HOME from submitter's environment.

(cherry picked from commit b167a8c7e75d9e816784bd655bce1feb6c447210)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
[SPARK-3709] Executors don't always report broadcast block removal properly back to the driver (for branch-1.1)

Author: Reynold Xin <rxin@apache.org>

Closes #2591 from rxin/SPARK-3709-1.1 and squashes the following commits:

ab99cc0 [Reynold Xin] [SPARK-3709] Executors don't always report broadcast block removal properly back to the driver
SPARK-3745 - fix check-license to properly download and check jar

for details, see: https://issues.apache.org/jira/browse/SPARK-3745

Author: shane knapp <incomplete@gmail.com>

Closes #2596 from shaneknapp/SPARK-3745 and squashes the following commits:

c95eea9 [shane knapp] SPARK-3745 - fix check-license to properly download and check jar

(cherry picked from commit a01a30927d107a8d9496f749eb9d89eda6dda9d7)
Signed-off-by: Josh Rosen <joshrosen@apache.org>

Conflicts:
	dev/check-license
[SPARK-3747] TaskResultGetter could incorrectly abort a stage if it cannot get result for a specific task

Author: Reynold Xin <rxin@apache.org>

Closes #2599 from rxin/SPARK-3747 and squashes the following commits:

a74c04d [Reynold Xin] Added a line of comment explaining NonFatal
0e8d44c [Reynold Xin] [SPARK-3747] TaskResultGetter could incorrectly abort a stage if it cannot get result for a specific task

(cherry picked from commit eb43043f411b87b7b412ee31e858246bd93fdd04)
Signed-off-by: Reynold Xin <rxin@apache.org>
[SPARK-3755][Core] Do not bind port 1 - 1024 to server in spark

Non-root user use port 1- 1024 to start jetty server will get the exception " java.net.SocketException: Permission denied", so not use these ports

Author: scwf <wangfei1@huawei.com>

Closes #2610 from scwf/1-1024 and squashes the following commits:

cb8cc76 [scwf] do not use port 1 - 1024

(cherry picked from commit 6390aae4eacbabfb1c53fb828b824f6a6518beff)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
SPARK-2626 [DOCS] Stop SparkContext in all examples

Call SparkContext.stop() in all examples (and touch up minor nearby code style issues while at it)

Author: Sean Owen <sowen@cloudera.com>

Closes #2575 from srowen/SPARK-2626 and squashes the following commits:

5b2baae [Sean Owen] Call SparkContext.stop() in all examples (and touch up minor nearby code style issues while at it)

Conflicts:
	examples/src/main/python/parquet_inputformat.py
[SPARK-3756] [Core]check exception is caused by an address-port collision properly

Jetty server use MultiException to handle exceptions when start server
refer https://github.com/eclipse/jetty.project/blob/jetty-8.1.14.v20131031/jetty-server/src/main/java/org/eclipse/jetty/server/Server.java

So in ```isBindCollision``` add the logical to cover MultiException

Author: scwf <wangfei1@huawei.com>

Closes #2611 from scwf/fix-isBindCollision and squashes the following commits:

984cb12 [scwf] optimize the fix
3a6c849 [scwf] fix bug in isBindCollision

(cherry picked from commit 2fedb5dddcc10d3186f49fc4996a7bb5b68bbc85)
Signed-off-by: Patrick Wendell <pwendell@gmail.com>

Conflicts:
	core/src/main/scala/org/apache/spark/util/Utils.scala
Typo error in KafkaWordCount example

topicpMap to topicMap

Author: Gaspar Munoz <munozs.88@gmail.com>

Closes #2614 from gasparms/patch-1 and squashes the following commits:

00aab2c [Gaspar Munoz] Typo error in KafkaWordCount example

(cherry picked from commit b81ee0b46d63c2122b88941696654100fd736942)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile

This patch forces use of commons http client 4.2 in Kinesis-asl profile so that the AWS SDK does not run into dependency conflicts

Author: aniketbhatnagar <aniket.bhatnagar@gmail.com>

Closes #2535 from aniketbhatnagar/Kinesis-HttpClient-Dep-Fix and squashes the following commits:

aa2079f [aniketbhatnagar] Merge branch 'Kinesis-HttpClient-Dep-Fix' of https://github.com/aniketbhatnagar/spark into Kinesis-HttpClient-Dep-Fix
73f55f6 [aniketbhatnagar] SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile
70cc75b [aniketbhatnagar] deleted merge files
725dbc9 [aniketbhatnagar] Merge remote-tracking branch 'origin/Kinesis-HttpClient-Dep-Fix' into Kinesis-HttpClient-Dep-Fix
4ed61d8 [aniketbhatnagar] SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile
9cd6103 [aniketbhatnagar] SPARK-3638 | Forced a compatible version of http client in kinesis-asl profile

(cherry picked from commit 93861a5e876fa57f509cce82768656ddf8d4ef00)
Signed-off-by: Josh Rosen <joshrosen@apache.org>
Merge branch 'branch-1.1' of github.com:apache/spark into csd-1.1