[SPARK-2658][SQL] Add rule for true = 1.

Author: Michael Armbrust <michael@databricks.com>

Closes #1556 from marmbrus/fixBooleanEqualsOne and squashes the following commits:

ad8edd4 [Michael Armbrust] Add rule for true = 1 and false = 0.

(cherry picked from commit 78d18fdbaa62d8ed235c29b2e37fd6607263c639)
Signed-off-by: Reynold Xin <rxin@apache.org>
[SPARK-2603][SQL] Remove unnecessary toMap and toList in converting Java collections to Scala collections JsonRDD.scala

In JsonRDD.scalafy, we are using toMap/toList to convert a Java Map/List to a Scala one. These two operations are pretty expensive because they read elements from a Java Map/List and then load to a Scala Map/List. We can use Scala wrappers to wrap those Java collections instead of using toMap/toList.

I did a quick test to see the performance. I had a 2.9GB cached RDD[String] storing one JSON object per record (twitter dataset). My simple test program is attached below.
```scala
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext._

val jsonData = sc.textFile("...")
jsonData.cache.count

val jsonSchemaRDD = sqlContext.jsonRDD(jsonData)
jsonSchemaRDD.registerAsTable("jt")

sqlContext.sql("select count(*) from jt").collect
```
Stages for the schema inference and the table scan both had 48 tasks. These tasks were executed sequentially. For the current implementation, scanning the JSON dataset will materialize values of all fields of a record. The inferred schema of the dataset can be accessed at https://gist.github.com/yhuai/05fe8a57c638c6666f8d.

From the result, there was no significant difference on running `jsonRDD`. For the simple aggregation query, results are attached below.
```
Original:
Run 1: 26.1s
Run 2: 27.03s
Run 3: 27.035s

With this change:
Run 1: 21.086s
Run 2: 21.035s
Run 3: 21.029s
```

JIRA: https://issues.apache.org/jira/browse/SPARK-2603

Author: Yin Huai <huai@cse.ohio-state.edu>

Closes #1504 from yhuai/removeToMapToList and squashes the following commits:

6831b77 [Yin Huai] Fix failed tests.
09b9bca [Yin Huai] Merge remote-tracking branch 'upstream/master' into removeToMapToList
d1abdb8 [Yin Huai] Remove unnecessary toMap and toList.

(cherry picked from commit b352ef175c234a2ea86b72c2f40da2ac69658b2e)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-2464][Streaming] Fixed Twitter stream stopping bug

Stopping the Twitter Receiver would call twitter4j's TwitterStream.shutdown, which in turn causes an Exception to be thrown to the listener. This exception caused the Receiver to be restarted. This patch check whether the receiver was stopped or not, and accordingly restarts on exception.

Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes #1577 from tdas/twitter-stop and squashes the following commits:

011b525 [Tathagata Das] Fixed Twitter stream stopping bug.

(cherry picked from commit a45d5480f65d2e969fc7fbd8f358b1717fb99bef)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit baf92a0f2119867b1be540085ebe9f1a1c411ae8.
Revert "[maven-release-plugin] prepare release v1.0.1-rc3"

This reverts commit 70ee14f76d6c3d3f162db6bbe12797c252a0295a.
Updating versions for 1.0.2 release.
[SPARK-2529] Clean closures in foreach and foreachPartition.

Author: Reynold Xin <rxin@apache.org>

Closes #1583 from rxin/closureClean and squashes the following commits:

8982fe6 [Reynold Xin] [SPARK-2529] Clean closures in foreach and foreachPartition.

(cherry picked from commit eb82abd8e3d25c912fa75201cf4f429aab8d73c7)
Signed-off-by: Reynold Xin <rxin@apache.org>
[maven-release-plugin] prepare release v1.0.2-rc1
[maven-release-plugin] prepare for next development iteration
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit edbd02fc6873676e080101d407916efb64bdf71a.
Revert "[maven-release-plugin] prepare release v1.0.2-rc1"

This reverts commit 919c87f26a2655bfd5ae03958915b6804367c1d6.
[maven-release-plugin] prepare release v1.0.2-rc1
[maven-release-plugin] prepare for next development iteration
Updated CHANGES.txt
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 54df1b8c31fa2de5b04ee4a5563706b2664f34f3.
Revert "[maven-release-plugin] prepare release v1.0.2-rc1"

This reverts commit 08f601328ad9e7334ef7deb3a9fff1343a3c4f30.
[maven-release-plugin] prepare release v1.0.2-rc1
[maven-release-plugin] prepare for next development iteration
[SPARK-2696] Reduce default value of spark.serializer.objectStreamReset

The current default value of spark.serializer.objectStreamReset is 10,000.
When trying to re-partition (e.g., to 64 partitions) a large file (e.g., 500MB), containing 1MB records, the serializer will cache 10000 x 1MB x 64 ~= 640 GB which will cause out of memory errors.

This patch sets the default value to a more reasonable default value (100).

Author: Hossein <hossein@databricks.com>

Closes #1595 from falaki/objectStreamReset and squashes the following commits:

650a935 [Hossein] Updated documentation
1aa0df8 [Hossein] Reduce default value of spark.serializer.objectStreamReset

(cherry picked from commit 66f26a4610aede57322cb7e193a50aecb6c57d22)
Signed-off-by: Matei Zaharia <matei@databricks.com>
[SPARK-2580] [PySpark] keep silent in worker if JVM close the socket

During rdd.take(n), JVM will close the socket if it had got enough data, the Python worker should keep silent in this case.

In the same time, the worker should not print the trackback into stderr if it send the traceback to JVM successfully.

Author: Davies Liu <davies.liu@gmail.com>

Closes #1625 from davies/error and squashes the following commits:

4fbcc6d [Davies Liu] disable log4j during testing when exception is expected.
cc14202 [Davies Liu] keep silent in worker if JVM close the socket

(cherry picked from commit ccd5ab5f82812abc2eb518448832cc20fb903345)
Signed-off-by: Josh Rosen <joshrosen@apache.org>