[maven-release-plugin] prepare for next development iteration
[SPARK-3342] Add SSDs to block device mapping

On `m3.2xlarge` instances the 2x80GB SSDs are inaccessible if not added to the block device mapping when the instance is created. They work when added with this patch. I have not tested this with other instance types, and I do not know much about this script and EC2 deployment in general. Maybe this code needs to depend on the instance type.

The requirement for this mapping is described in the AWS docs at:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html#InstanceStore_UsageScenarios

"For M3 instances, you must specify instance store volumes in the block
device mapping for the instance. When you launch an M3 instance, we
ignore any instance store volumes specified in the block device mapping
for the AMI."

Author: Daniel Darabos <darabos.daniel@gmail.com>

Closes #2081 from darabos/patch-1 and squashes the following commits:

1ceb2c8 [Daniel Darabos] Use %d string interpolation instead of {}.
a1854d7 [Daniel Darabos] Only specify ephemeral device mapping for M3.
e0d9e37 [Daniel Darabos] Create ephemeral device mapping based on get_num_disks().
6b116a6 [Daniel Darabos] Add SSDs to block device mapping
[MLlib] Squash bug in IndexedRowMatrix

Kill this bug fast before it does damage.

Author: Reza Zadeh <rizlar@gmail.com>

Closes #2224 from rezazadeh/indexrmbug and squashes the following commits:

53386d6 [Reza Zadeh] Squash bug in IndexedRowMatrix

(cherry picked from commit 0f16b23cd17002fac05f3ecc58899be1b1121b82)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
[SPARK-3332] Revert spark-ec2 patch that identifies clusters using tags

This reverts #1899 and #2163, two patches that modified `spark-ec2` so that clusters are identified using tags instead of security groups.  The original motivation for this patch was to allow multiple clusters to run in the same security group.

Unfortunately, tagging is not atomic with launching instances on EC2, so with this approach we have the possibility of `spark-ec2` launching instances and crashing before they can be tagged, effectively orphaning those instances.  The orphaned instances won't belong to any cluster, so the `spark-ec2` script will be unable to clean them up.

Since this feature may still be worth supporting, there are several alternative approaches that we might consider, including detecting orphaned instances and logging warnings, or maybe using another mechanism to group instances into clusters.  For the 1.1.0 release, though, I propose that we just revert this patch.

Author: Josh Rosen <joshrosen@apache.org>

Closes #2225 from JoshRosen/revert-ec2-cluster-naming and squashes the following commits:

0c18e86 [Josh Rosen] Revert "SPARK-2333 - spark_ec2 script should allow option for existing security group"
c2ca2d4 [Josh Rosen] Revert "Spark-3213 Fixes issue with spark-ec2 not detecting slaves created with "Launch More like this""
[Build] merge changes to run-tests-jenkins from master branch

Author: Nicholas Chammas <nicholas.chammas@gmail.com>
Author: nchammas <nicholas.chammas@gmail.com>

Closes #2237 from nchammas/branch-1.1 and squashes the following commits:

39bdd5e [Nicholas Chammas] merge updates from master
f5aa841 [nchammas] Merge pull request #3 from apache/branch-1.1
SPARK-3328 fixed make-distribution script --with-tachyon option.

Directory path for dependencies jar and resources in Tachyon 0.5.0 has been changed.

Author: Prudhvi Krishna <prudhvi953@gmail.com>

Closes #2228 from prudhvije/SPARK-3328/make-dist-fix and squashes the following commits:

d1d2c22 [Prudhvi Krishna] SPARK-3328 fixed make-distribution script --with-tachyon option.
(cherry picked from commit 644e31524a6a9a22c671a368aeb3b4eaeb61cf29)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
[SPARK-2981][GraphX] EdgePartition1D Int overflow

minor fix
detail is here: https://issues.apache.org/jira/browse/SPARK-2981

Author: Larry Xiao <xiaodi@sjtu.edu.cn>

Closes #1902 from larryxiao/2981 and squashes the following commits:

88059a2 [Larry Xiao] [SPARK-2981][GraphX] EdgePartition1D Int overflow

(cherry picked from commit aa7de128c5987fd2e134736f07ae913ad1f5eb26)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
[SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis

- Include kinesis in the unidocs
- Hide non-public classes from docs

Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes #2239 from tdas/kinesis-doc-fix and squashes the following commits:

156e20c [Tathagata Das] More fixes, based on PR comments.
e9a6c01 [Tathagata Das] Fixed docs related to kinesis

(cherry picked from commit e9bb12bea9fbef94332fbec88e3cd9197a27b7ad)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
[SPARK-2823][GraphX]fix GraphX EdgeRDD zipPartitions

If the users set “spark.default.parallelism” and the value is different with the EdgeRDD partition number, GraphX jobs will throw:
java.lang.IllegalArgumentException: Can't zip RDDs with unequal numbers of partitions

Author: luluorta <luluorta@gmail.com>

Closes #1763 from luluorta/fix-graph-zip and squashes the following commits:

8338961 [luluorta] fix GraphX EdgeRDD zipPartitions

(cherry picked from commit 9b225ac3072de522b40b46aba6df1f1c231f13ef)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
SPARK-3358: [EC2] Switch back to HVM instances for m3.X.

During regression tests of Spark 1.1 we discovered perf issues with
PVM instances when running PySpark. This reverts a change added in #1156
which changed the default type for m3 instances to PVM.

Author: Patrick Wendell <pwendell@gmail.com>

Closes #2244 from pwendell/ec2-hvm and squashes the following commits:

1342d7e [Patrick Wendell] SPARK-3358: [EC2] Switch back to HVM instances for m3.X.
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 865e6f63f63f5e881a02d1a4e3b4c5d0e86fcd8e.
Revert "[maven-release-plugin] prepare release v1.1.0-rc3"

This reverts commit b2d0493b223c5f98a593bb6d7372706cc02bebad.
[maven-release-plugin] prepare release v1.1.0-rc4
Merge tag 'v1.1.0-rc4' of github.com:apache/spark into csd-1.1

[maven-release-plugin]  copy for tag v1.1.0-rc4

Conflicts:
	pom.xml