SPARK-1695: java8-tests compiler error: package com.google.common.co...

...llections does not exist

Author: witgo <witgo@qq.com>

Closes #611 from witgo/SPARK-1695 and squashes the following commits:

d77a887 [witgo] Fix SPARK-1695: java8-tests compiler error: package com.google.common.collections does not exist
(cherry picked from commit f25ebed9f4552bc2c88a96aef06729d9fc2ee5b3)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
delete no use var

Author: wangfei <wangfei_hello@126.com>

Closes #613 from scwf/masterIndex and squashes the following commits:

1463056 [wangfei] delete no use var: masterIndex
(cherry picked from commit 4bf24f7897e1c67ca5f96dec05480e571f05ee1d)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
SPARK-1492. Update Spark YARN docs to use spark-submit

Author: Sandy Ryza <sandy@cloudera.com>

Closes #601 from sryza/sandy-spark-1492 and squashes the following commits:

5df1634 [Sandy Ryza] Address additional comments from Patrick.
be46d1f [Sandy Ryza] Address feedback from Marcelo and Patrick
867a3ea [Sandy Ryza] SPARK-1492. Update Spark YARN docs to use spark-submit
(cherry picked from commit 2b961d88079d7a3f9da63d5175d7b61f6dec762b)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
SPARK-1700: Close socket file descriptors on task completion

This will ensure that sockets do not build up over the course of a job, and that cancellation successfully cleans up sockets.

Tested in standalone mode. More file descriptors spawn than expected (around 1000ish rather than the expected 8ish) but they do not pile up between runs, or as high as before (where they went up to around 5k).

Author: Aaron Davidson <aaron@databricks.com>

Closes #623 from aarondav/pyspark2 and squashes the following commits:

0ca13bb [Aaron Davidson] SPARK-1700: Close socket file descriptors on task completion

(cherry picked from commit 0a14421765b672305e8f32ded4a9a1f6f7241d8d)
Signed-off-by: Aaron Davidson <aaron@databricks.com>
Update SchemaRDD.scala

Modify spelling errors

Author: ArcherShao <ArcherShao@users.noreply.github.com>

Closes #619 from ArcherShao/patch-1 and squashes the following commits:

2957195 [ArcherShao] Update SchemaRDD.scala
(cherry picked from commit 9347565f4188cf1574c6dc49fcde91eb286be955)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>