[SPARK-4584] [yarn] Remove security manager from Yarn AM.

The security manager adds a lot of overhead to the runtime of the
app, and causes a severe performance regression. Even stubbing out
all unneeded methods (all except checkExit()) does not help.

So, instead, penalize users who do an explicit System.exit() by leaving
them in "undefined behavior" territory: if they do that, the Yarn
backend won't be able to report the final app status to the RM.
The result is that the final status of the application might not match
the user's expectations.

One side-effect of the change is that users who do an explicit
System.exit() will lose the AM retry functionality. Since there is
no way to know if the exit was because of success or failure, the
AM right now errs on the side of it being a successful exit.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #3484 from vanzin/SPARK-4584 and squashes the following commits:

21f2502 [Marcelo Vanzin] Do not retry apps that use System.exit().
4198b3b [Marcelo Vanzin] [SPARK-4584] [yarn] Remove security manager from Yarn AM.

(cherry picked from commit 915f8eeb3a493a0bb4b8d05d795ddd21f373d2ff)
Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Preparing Spark release v1.2.0-rc1
Preparing development version 1.2.1-SNAPSHOT
Revert "Preparing development version 1.2.1-SNAPSHOT"

This reverts commit fc7bff00ac731d2632213a98cd92dc5e84ce7dcd.
Revert "Preparing Spark release v1.2.0-rc1"

This reverts commit 39c7d1c1f9a7785285cf4c20dfbffd96f72d5634.
Updating version in package.scala
Preparing Spark release v1.2.0-rc1
Preparing development version 1.2.1-SNAPSHOT
HOTFIX: Rolling back incorrect version change
[SPARK-4597] Use proper exception and reset variable in Utils.createTempDir()

`File.exists()` and `File.mkdirs()` only throw `SecurityException` instead of `IOException`. Then, when an exception is thrown, `dir` should be reset too.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #3449 from viirya/fix_createtempdir and squashes the following commits:

36cacbd [Liang-Chi Hsieh] Use proper exception and reset variable.

(cherry picked from commit 49fe8797e64f10c574e0790b32a8c3fdc7e594a0)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
[DOCS][BUILD] Add instruction to use change-version-to-2.11.sh in 'Building for Scala 2.11'.

To build with Scala 2.11, we have to execute `change-version-to-2.11.sh` before Maven execute, otherwise inter-module dependencies are broken.

Author: Takuya UESHIN <ueshin@happy-camper.st>

Closes #3361 from ueshin/docs/building-spark_2.11 and squashes the following commits:

1d29126 [Takuya UESHIN] Add instruction to use change-version-to-2.11.sh in 'Building for Scala 2.11'.

(cherry picked from commit 0fcd24cc542040ff3555290eec7b021062e7e6ac)
Signed-off-by: Patrick Wendell <pwendell@gmail.com>
SPARK-2143 [WEB UI] Add Spark version to UI footer

This PR adds the Spark version number to the UI footer; this is how it looks:

![screen shot 2014-11-21 at 22 58 40](https://cloud.githubusercontent.com/assets/822522/5157738/f4822094-7316-11e4-98f1-333a535fdcfa.png)

Author: Sean Owen <sowen@cloudera.com>

Closes #3410 from srowen/SPARK-2143 and squashes the following commits:

e9b3a7a [Sean Owen] Add Spark version to footer
[SPARK-4656][Doc] Typo in Programming Guide markdown

Grammatical error in Programming Guide document

Author: lewuathe <lewuathe@me.com>

Closes #3412 from Lewuathe/typo-programming-guide and squashes the following commits:

a3e2f00 [lewuathe] Typo in Programming Guide markdown

(cherry picked from commit a217ec5fd5cd7addc69e538d6ec6dd64956cc8ed)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
[DOC] Fixes formatting typo in SQL programming guide

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/3498)
<!-- Reviewable:end -->

Author: Cheng Lian <lian@databricks.com>

Closes #3498 from liancheng/fix-sql-doc-typo and squashes the following commits:

865ecd7 [Cheng Lian] Fixes formatting typo in SQL programming guide

(cherry picked from commit 2a4d389f70b2066b1ac32b081bef44e61fefb03c)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
SPARK-2192 [BUILD] Examples Data Not in Binary Distribution

Simply, add data/ to distributions. This adds about 291KB (compressed) to the tarball, FYI.

Author: Sean Owen <sowen@cloudera.com>

Closes #3480 from srowen/SPARK-2192 and squashes the following commits:

47688f1 [Sean Owen] Add data/ to distributions

(cherry picked from commit 6384f42ab2e5c2b3e767ab4a428cda20a8ddcbe1)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
[SPARK-4661][Core] Minor code and docs cleanup

Author: zsxwing <zsxwing@gmail.com>

Closes #3521 from zsxwing/SPARK-4661 and squashes the following commits:

03cbe3f [zsxwing] Minor code and docs cleanup

(cherry picked from commit 30a86acdefd5428af6d6264f59a037e0eefd74b4)
Signed-off-by: Reynold Xin <rxin@databricks.com>
Documentation: add description for repartitionAndSortWithinPartitions

Author: Madhu Siddalingaiah <madhu@madhu.com>

Closes #3390 from msiddalingaiah/master and squashes the following commits:

cbccbfe [Madhu Siddalingaiah] Documentation: replace <b> with <code> (again)
332f7a2 [Madhu Siddalingaiah] Documentation: replace <b> with <code>
cd2b05a [Madhu Siddalingaiah] Merge remote-tracking branch 'upstream/master'
0fc12d7 [Madhu Siddalingaiah] Documentation: add description for repartitionAndSortWithinPartitions

(cherry picked from commit 2b233f5fc4beb2c6ed4bc142e923e96f8bad3ec4)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
[SPARK-4258][SQL][DOC] Documents spark.sql.parquet.filterPushdown

Documents `spark.sql.parquet.filterPushdown`, explains why it's turned off by default and when it's safe to be turned on.

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/3440)
<!-- Reviewable:end -->

Author: Cheng Lian <lian@databricks.com>

Closes #3440 from liancheng/parquet-filter-pushdown-doc and squashes the following commits:

2104311 [Cheng Lian] Documents spark.sql.parquet.filterPushdown

(cherry picked from commit 5db8dcaf494e0dffed4fc22f19b0334d95ab6bfb)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SQL] add @group tab in limit() and count()

group tab is missing for scaladoc

Author: Jacky Li <jacky.likun@gmail.com>

Closes #3458 from jackylk/patch-7 and squashes the following commits:

0121a70 [Jacky Li] add @group tab in limit() and count()

(cherry picked from commit bafee67ebad01f7aea2cd393a70b57eb8345eeb0)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-4358][SQL] Let BigDecimal do checking type compatibility

Remove hardcoding max and min values for types. Let BigDecimal do checking type compatibility.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #3208 from viirya/more_numericLit and squashes the following commits:

e9834b4 [Liang-Chi Hsieh] Remove byte and short types for number literal.
1bd1825 [Liang-Chi Hsieh] Fix Indentation and make the modification clearer.
cf1a997 [Liang-Chi Hsieh] Modified for comment to add a rule of analysis that adds a cast.
91fe489 [Liang-Chi Hsieh] add Byte and Short.
1bdc69d [Liang-Chi Hsieh] Let BigDecimal do checking type compatibility.

(cherry picked from commit b57365a1ec89e31470f424ff37d5ebc7c90a39d8)
Signed-off-by: Michael Armbrust <michael@databricks.com>