[SPARK-3310][SQL] Directly use currentTable without unnecessary implicit conversion

We can directly use currentTable there without unnecessary implicit conversion.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #2203 from viirya/direct_use_inmemoryrelation and squashes the following commits:

4741d02 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into direct_use_inmemoryrelation
b671f67 [Liang-Chi Hsieh] Can directly use currentTable there without unnecessary implicit conversion.
[SPARK-2219][SQL] Added support for the "add jar" command

Adds logical and physical command classes for the "add jar" command.

Note that this PR conflicts with and should be merged after #2215.

Author: Cheng Lian <lian.cs.zju@gmail.com>

Closes #2242 from liancheng/add-jar and squashes the following commits:

e43a2f1 [Cheng Lian] Updates AddJar according to conventions introduced in #2215
b99107f [Cheng Lian] Added test case for ADD JAR command
095b2c7 [Cheng Lian] Also forward ADD JAR command to Hive
9be031b [Cheng Lian] Trims Jar path string
8195056 [Cheng Lian] Added support for the "add jar" command
[SPARK-3392] [SQL] Show value spark.sql.shuffle.partitions for mapred.reduce.tasks

This is a tiny fix for getting the value of "mapred.reduce.tasks", which make more sense for the hive user.
As well as the command "set -v", which should output verbose information for all of the key/values.

Author: Cheng Hao <hao.cheng@intel.com>

Closes #2261 from chenghao-intel/set_mapreduce_tasks and squashes the following commits:

653858a [Cheng Hao] show value spark.sql.shuffle.partitions for mapred.reduce.tasks
[SPARK-3391][EC2] Support attaching up to 8 EBS volumes.

Please merge this at the same time as https://github.com/mesos/spark-ec2/pull/66

Author: Reynold Xin <rxin@apache.org>

Closes #2260 from rxin/ec2-ebs-vol and squashes the following commits:

b9527d9 [Reynold Xin] Removed io1 ebs type.
bf9c403 [Reynold Xin] Made EBS volume type configurable.
c8e25ea [Reynold Xin] Support up to 8 EBS volumes.
adf4f2e [Reynold Xin] Revert git repo change.
020c542 [Reynold Xin] [SPARK-3391] Support attaching more than 1 EBS volumes.
[Docs] fix minor MLlib case typo

Also make the list of features consistent in style.

Author: Nicholas Chammas <nicholas.chammas@gmail.com>

Closes #2278 from nchammas/patch-1 and squashes the following commits:

56df319 [Nicholas Chammas] [Docs] fix minor MLlib case typo
[SPARK-3260] yarn - pass acls along with executor launch

Pass along the acl settings when we launch a container so that they can be applied to viewing the logs on a running NodeManager.

Author: Thomas Graves <tgraves@apache.org>

Closes #2185 from tgravescs/SPARK-3260 and squashes the following commits:

6f94b5a [Thomas Graves] make unit test more robust
28b9dd3 [Thomas Graves] yarn - pass acls along with executor launch
[SPARK-3375] spark on yarn container allocation issues

If yarn doesn't get the containers immediately it stops asking for them and the yarn application hangs with never getting any executors.

The issue here is that we are sending the number of containers as 0 after we send the original one of X. on the yarn side this clears out the original request.

For a ping we should just send empty asks.

Author: Thomas Graves <tgraves@apache.org>

Closes #2275 from tgravescs/SPARK-3375 and squashes the following commits:

74b6820 [Thomas Graves] send empty resource requests when we aren't asking for containers
[SPARK-3399][PySpark] Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>

Closes #2270 from sarutak/SPARK-3399 and squashes the following commits:

7613be6 [Kousuke Saruta] Modified pyspark script to ignore environment variables YARN_CONF_DIR and HADOOP_CONF_DIR while testing
SPARK-3211 .take() is OOM-prone with empty partitions

Instead of jumping straight from 1 partition to all partitions, do exponential
growth and double the number of partitions to attempt each time instead.

Fix proposed by Paul Nepywoda

Author: Andrew Ash <andrew@andrewash.com>

Closes #2117 from ash211/SPARK-3211 and squashes the following commits:

8b2299a [Andrew Ash] Quadruple instead of double for a minor speedup
e5f7e4d [Andrew Ash] Update comment to better reflect what we're doing
09a27f7 [Andrew Ash] Update PySpark to be less OOM-prone as well
3a156b8 [Andrew Ash] SPARK-3211 .take() is OOM-prone with empty partitions