[SPARK-4765] Make GC time always shown in UI.

This commit removes the GC time for each task from the set of
optional, additional metrics, and instead always shows it for
each task.

cc pwendell

Author: Kay Ousterhout <kayousterhout@gmail.com>

Closes #3622 from kayousterhout/gc_time and squashes the following commits:

15ac242 [Kay Ousterhout] Make TaskDetailsClassNames private[spark]
e71d893 [Kay Ousterhout] [SPARK-4765] Make GC time always shown in UI.

(cherry picked from commit 1f5110630c1abb13a357b463c805a39772923b82)
Signed-off-by: Kay Ousterhout <kayousterhout@gmail.com>
SPARK-4567. Make SparkJobInfo and SparkStageInfo serializable

Author: Sandy Ryza <sandy@cloudera.com>

Closes #3426 from sryza/sandy-spark-4567 and squashes the following commits:

cb4b8d2 [Sandy Ryza] SPARK-4567. Make SparkJobInfo and SparkStageInfo serializable

(cherry picked from commit 5e4c06f8e54265a4024857f5978ec54c936aeea2)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
SPARK-4805 [CORE] BlockTransferMessage.toByteArray() trips assertion

Allocate enough room for type byte as well as message, to avoid tripping assertion about capacity of the buffer

Author: Sean Owen <sowen@cloudera.com>

Closes #3650 from srowen/SPARK-4805 and squashes the following commits:

9e1d502 [Sean Owen] Allocate enough room for type byte as well as message, to avoid tripping assertion about capacity of the buffer

(cherry picked from commit d8f84f26e388055ca7459810e001d05ab60af15b)
Signed-off-by: Aaron Davidson <aaron@databricks.com>
[SPARK-4740] Create multiple concurrent connections between two peer nodes in Netty.

It's been reported that when the number of disks is large and the number of nodes is small, Netty network throughput is low compared with NIO. We suspect the problem is that only a small number of disks are utilized to serve shuffle files at any given point, due to connection reuse. This patch adds a new config parameter to specify the number of concurrent connections between two peer nodes, default to 2.

Author: Reynold Xin <rxin@databricks.com>

Closes #3625 from rxin/SPARK-4740 and squashes the following commits:

ad4241a [Reynold Xin] Updated javadoc.
f33c72b [Reynold Xin] Code review feedback.
0fefabb [Reynold Xin] Use double check in synchronization.
41dfcb2 [Reynold Xin] Added test case.
9076b4a [Reynold Xin] Fixed two NPEs.
3e1306c [Reynold Xin] Minor style fix.
4f21673 [Reynold Xin] [SPARK-4740] Create multiple concurrent connections between two peer nodes in Netty.

(cherry picked from commit 2b9b72682e587909a84d3ace214c22cec830eeaf)
Signed-off-by: Reynold Xin <rxin@databricks.com>
Config updates for the new shuffle transport.

Author: Reynold Xin <rxin@databricks.com>

Closes #3657 from rxin/conf-update and squashes the following commits:

7370eab [Reynold Xin] Config updates for the new shuffle transport.

(cherry picked from commit 9bd9334f588dbb44d01554f9f4ca68a153a48993)
Signed-off-by: Aaron Davidson <aaron@databricks.com>