[SPARK-3647] Add more exceptions to Guava relocation.

Guava's Optional refers to some package private classes / methods, and
when those are relocated the code stops working, throwing exceptions.
So add the affected classes to the exception list too, and add a unit
test.

(Note that this unit test only really makes sense in maven, since we
don't relocate in the sbt build. Also, JavaAPISuite doesn't seem to
be run by "mvn test" - I had to manually add command line options to
enable it.)

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #2496 from vanzin/SPARK-3647 and squashes the following commits:

84f58d7 [Marcelo Vanzin] [SPARK-3647] Add more exceptions to Guava relocation.

Conflicts:
	assembly/pom.xml
	core/pom.xml
	core/src/test/java/org/apache/spark/JavaAPISuite.java
Merge branch 'branch-1.1' of github.com:apache/spark into csd-1.1
[SQL][Docs] Update the output of printSchema and fix a typo in SQL programming guide.

We have changed the output format of `printSchema`. This PR will update our SQL programming guide to show the updated format. Also, it fixes a typo (the value type of `StructType` in Java API).

Author: Yin Huai <huai@cse.ohio-state.edu>

Closes #2630 from yhuai/sqlDoc and squashes the following commits:

267d63e [Yin Huai] Update the output of printSchema and fix a typo.

(cherry picked from commit 82a6a083a485140858bcd93d73adec59bb5cca64)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-3755][Core] avoid trying privileged port when request a non-privileged port

pwendell, ```tryPort``` is not compatible with old code in last PR, this is to fix it.
And after discuss with srowen renamed the title to "avoid trying privileged port when request a non-privileged port". Plz refer to the discuss for detail.

Author: scwf <wangfei1@huawei.com>

Closes #2623 from scwf/1-1024 and squashes the following commits:

10a4437 [scwf] add comment
de3fd17 [scwf] do not try privileged port when request a non-privileged port
42cb0fa [scwf] make tryPort compatible with old code
cb8cc76 [scwf] do not use port 1 - 1024

(cherry picked from commit 8081ce8bd111923db143abc55bb6ef9793eece35)
Signed-off-by: Andrew Or <andrewor14@gmail.com>

Conflicts:
	core/src/main/scala/org/apache/spark/util/Utils.scala
[DEPLOY] SPARK-3759: Return the exit code of the driver process

SparkSubmitDriverBootstrapper.scala now returns the exit code of the driver process, instead of always returning 0.

Author: Eric Eijkelenboom <ee@userreport.com>

Closes #2628 from ericeijkelenboom/master and squashes the following commits:

cc4a571 [Eric Eijkelenboom] Return the exit code of the driver process

(cherry picked from commit 42d5077fd3f2c37d1cd23f4c81aa89286a74cb40)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
SPARK-2058: Overriding SPARK_HOME/conf with SPARK_CONF_DIR

Update of PR #997.

With this PR, setting SPARK_CONF_DIR overrides SPARK_HOME/conf (not only spark-defaults.conf and spark-env).

Author: EugenCepoi <cepoi.eugen@gmail.com>

Closes #2481 from EugenCepoi/SPARK-2058 and squashes the following commits:

0bb32c2 [EugenCepoi] use orElse orNull and fixing trailing percent in compute-classpath.cmd
77f35d7 [EugenCepoi] SPARK-2058: Overriding SPARK_HOME/conf with SPARK_CONF_DIR

(cherry picked from commit f0811f928e5b608e1a2cba3b6828ba0ed03b701d)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
[SPARK-3696]Do not override the user-difined conf_dir

https://issues.apache.org/jira/browse/SPARK-3696

We see if SPARK_CONF_DIR is already defined before assignment.

Author: WangTaoTheTonic <barneystinson@aliyun.com>

Closes #2541 from WangTaoTheTonic/confdir and squashes the following commits:

c3f31e0 [WangTaoTheTonic] Do not override the user-difined conf_dir

(cherry picked from commit 9d320e222c221e5bb827cddf01a83e64a16d74ff)
Signed-off-by: Andrew Or <andrewor14@gmail.com>

Conflicts:
	sbin/spark-config.sh
[SPARK-3535][Mesos] Fix resource handling.

Author: Brenden Matthews <brenden@diddyinc.com>

Closes #2401 from brndnmtthws/master and squashes the following commits:

4abaa5d [Brenden Matthews] [SPARK-3535][Mesos] Fix resource handling.

(cherry picked from commit a8c52d5343e19731909e73db5de151a324d31cd5)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
[SPARK-3775] Not suitable error message in spark-shell.cmd

Modified some sentence of error message in bin\*.cmd.

Author: Masayoshi TSUZUKI <tsudukim@oss.nttdata.co.jp>

Closes #2640 from tsudukim/feature/SPARK-3775 and squashes the following commits:

3458afb [Masayoshi TSUZUKI] [SPARK-3775] Not suitable error message in spark-shell.cmd

(cherry picked from commit 358d7ffd01b4a3fbae313890522cf662c71af6e5)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
[SPARK-3774] typo comment in bin/utils.sh

Modified the comment of bin/utils.sh.

Author: Masayoshi TSUZUKI <tsudukim@oss.nttdata.co.jp>

Closes #2639 from tsudukim/feature/SPARK-3774 and squashes the following commits:

707b779 [Masayoshi TSUZUKI] [SPARK-3774] typo comment in bin/utils.sh

(cherry picked from commit e5566e05b1ac99aa6caf1701e47ebcdb68a002c6)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
Merge branch 'csd-1.1' of github.com:markhamstra/spark into csd-1.1
[SPARK-3597][Mesos] Implement `killTask`.

The MesosSchedulerBackend did not previously implement `killTask`,
resulting in an exception.

Author: Brenden Matthews <brenden@diddyinc.com>

Closes #2453 from brndnmtthws/implement-killtask and squashes the following commits:

23ddcdc [Brenden Matthews] [SPARK-3597][Mesos] Implement `killTask`.

(cherry picked from commit 32fad4233f353814496c84e15ba64326730b7ae7)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
SPARK-1656: Fix potential resource leaks

JIRA: https://issues.apache.org/jira/browse/SPARK-1656

Author: zsxwing <zsxwing@gmail.com>

Closes #577 from zsxwing/SPARK-1656 and squashes the following commits:

c431095 [zsxwing] Add a comment and fix the code style
2de96e5 [zsxwing] Make sure file will be deleted if exception happens
28b90dc [zsxwing] Update to follow the code style
4521d6e [zsxwing] Merge branch 'master' into SPARK-1656
afc3383 [zsxwing] Update to follow the code style
071fdd1 [zsxwing] SPARK-1656: Fix potential resource leaks

(cherry picked from commit a7c73130f1b6b0b8b19a7b0a0de5c713b673cd7b)
Signed-off-by: Andrew Or <andrewor14@gmail.com>
[SPARK-3792][SQL] Enable JavaHiveQLSuite

Do not use TestSQLContext in JavaHiveQLSuite, that may lead to two SparkContexts in one jvm and enable JavaHiveQLSuite

Author: scwf <wangfei1@huawei.com>

Closes #2652 from scwf/fix-JavaHiveQLSuite and squashes the following commits:

be35c91 [scwf] enable JavaHiveQLSuite

(cherry picked from commit 58f5361caaa2f898e38ae4b3794167881e20a818)
Signed-off-by: Michael Armbrust <michael@databricks.com>
Merge branch 'branch-1.1' of github.com:apache/spark into csd-1.1