[maven-release-plugin] prepare for next development iteration
[SPARK-2419][Streaming][Docs] Updates to the streaming programming guide

Updated the main streaming programming guide, and also added source-specific guides for Kafka, Flume, Kinesis.

Author: Tathagata Das <tathagata.das1565@gmail.com>
Author: Jacek Laskowski <jacek@japila.pl>

Closes #2254 from tdas/streaming-doc-fix and squashes the following commits:

e45c6d7 [Jacek Laskowski] More fixes from an old PR
5125316 [Tathagata Das] Fixed links
dc02f26 [Tathagata Das] Refactored streaming kinesis guide and made many other changes.
acbc3e3 [Tathagata Das] Fixed links between streaming guides.
cb7007f [Tathagata Das] Added Streaming + Flume integration guide.
9bd9407 [Tathagata Das] Updated streaming programming guide with additional information from SPARK-2419.

(cherry picked from commit a5224079286d1777864cf9fa77330aadae10cd7b)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
[SPARK-3372] [MLlib] MLlib doesn't pass maven build / checkstyle due to multi-byte character contained in Gradient.scala

Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>

Closes #2248 from sarutak/SPARK-3372 and squashes the following commits:

73a28b8 [Kousuke Saruta] Replaced UTF-8 hyphen with ascii hyphen

(cherry picked from commit 1bed0a3869a526241381d2a74ba064e5b3721336)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
[HOTFIX] [SPARK-3400] Revert 9b225ac "fix GraphX EdgeRDD zipPartitions"

9b225ac3072de522b40b46aba6df1f1c231f13ef has been causing GraphX tests
to fail nondeterministically, which is blocking development for others.

Author: Ankur Dave <ankurdave@gmail.com>

Closes #2271 from ankurdave/SPARK-3400 and squashes the following commits:

10c2a97 [Ankur Dave] [HOTFIX] [SPARK-3400] Revert 9b225ac "fix GraphX EdgeRDD zipPartitions"

(cherry picked from commit 00362dac976cd05b06638deb11d990d612429e0b)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
[SPARK-3401][PySpark] Wrong usage of tee command in python/run-tests

Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>

Closes #2272 from sarutak/SPARK-3401 and squashes the following commits:

2b35a59 [Kousuke Saruta] Modified wrong usage of tee command in python/run-tests

(cherry picked from commit 4feb46c5feca8d48ec340dc9c8d0eccbcd41f505)
Signed-off-by: Reynold Xin <rxin@apache.org>
[Docs] fix minor MLlib case typo

Also make the list of features consistent in style.

Author: Nicholas Chammas <nicholas.chammas@gmail.com>

Closes #2278 from nchammas/patch-1 and squashes the following commits:

56df319 [Nicholas Chammas] [Docs] fix minor MLlib case typo

(cherry picked from commit 6a37ed838b3cbf96f7a904f3d3dabf99141729f5)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
SPARK-3211 .take() is OOM-prone with empty partitions

Instead of jumping straight from 1 partition to all partitions, do exponential
growth and double the number of partitions to attempt each time instead.

Fix proposed by Paul Nepywoda

Author: Andrew Ash <andrew@andrewash.com>

Closes #2117 from ash211/SPARK-3211 and squashes the following commits:

8b2299a [Andrew Ash] Quadruple instead of double for a minor speedup
e5f7e4d [Andrew Ash] Update comment to better reflect what we're doing
09a27f7 [Andrew Ash] Update PySpark to be less OOM-prone as well
3a156b8 [Andrew Ash] SPARK-3211 .take() is OOM-prone with empty partitions

(cherry picked from commit ba5bcaddecd54811d45c5fc79a013b3857d4c633)
Signed-off-by: Matei Zaharia <matei@databricks.com>
[SPARK-2419][Streaming][Docs] More updates to the streaming programming guide

- Improvements to the kinesis integration guide from @cfregly
- More information about unified input dstreams in main guide

Author: Tathagata Das <tathagata.das1565@gmail.com>
Author: Chris Fregly <chris@fregly.com>

Closes #2307 from tdas/streaming-doc-fix1 and squashes the following commits:

ec40b5d [Tathagata Das] Updated figure with kinesis
fdb9c5e [Tathagata Das] Fixed style issues with kinesis guide
036d219 [Chris Fregly] updated kinesis docs and added an arch diagram
24f622a [Tathagata Das] More modifications.

(cherry picked from commit baff7e936101635d9bd4245e45335878bafb75e0)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
[SPARK-3394] [SQL] Fix crash in TakeOrdered when limit is 0

This resolves https://issues.apache.org/jira/browse/SPARK-3394

Author: Eric Liang <ekl@google.com>

Closes #2264 from ericl/spark-3394 and squashes the following commits:

c87355b [Eric Liang] refactor
bfb6140 [Eric Liang] change RDD takeOrdered instead
7a51528 [Eric Liang] fix takeordered when limit = 0

(cherry picked from commit 6754570d83044c4fbaf0d2ac2378a0e081a93629)
Signed-off-by: Matei Zaharia <matei@databricks.com>
[SQL] Update SQL Programming Guide

Author: Michael Armbrust <michael@databricks.com>
Author: Yin Huai <huai@cse.ohio-state.edu>

Closes #2258 from marmbrus/sqlDocUpdate and squashes the following commits:

f3d450b [Michael Armbrust] fix brackets
bea3bfa [Michael Armbrust] Davies suggestions
3a29fe2 [Michael Armbrust] tighten visibility
a71aa36 [Michael Armbrust] Draft of doc updates
52932c0 [Michael Armbrust] Merge remote-tracking branch 'origin/master' into sqlDocUpdate
1e8c849 [Yin Huai] Update the example used for applySchema.
9457c39 [Yin Huai] Update doc.
31ba240 [Yin Huai] Merge remote-tracking branch 'upstream/master' into dataTypeDoc
29bc668 [Yin Huai] Draft doc for data type and schema APIs.

(cherry picked from commit 39db1bfdab434c867044ad4c70fe93a96fb287ad)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-3408] Fixed Limit operator so it works with sort-based shuffle.

Author: Reynold Xin <rxin@apache.org>

Closes #2281 from rxin/sql-limit-sort and squashes the following commits:

1ef7780 [Reynold Xin] [SPARK-3408] Fixed Limit operator so it works with sort-based shuffle.

(cherry picked from commit e2614038e78f4693fafedeee15b6fdf0ea1be473)
Signed-off-by: Reynold Xin <rxin@apache.org>
Fixed typos in make-distribution.sh

`hadoop.version` and `yarn.version` are properties rather then profiles, should use `-D` instead of `-P`.

/cc pwendell

Author: Cheng Lian <lian.cs.zju@gmail.com>

Closes #2121 from liancheng/fix-make-dist and squashes the following commits:

4c49158 [Cheng Lian] Also mentions Hadoop version related Maven profiles
ed5b42a [Cheng Lian] Fixed typos in make-distribution.sh
(cherry picked from commit 9d69a782bd2fc45193f269d8d8434795ea1580a4)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
[SPARK-938][doc] Add OpenStack Swift support

See compiled doc at
http://people.apache.org/~rxin/tmp/openstack-swift/_site/storage-openstack-swift.html

This is based on #1010. Closes #1010.

Author: Reynold Xin <rxin@apache.org>
Author: Gil Vernik <gilv@il.ibm.com>

Closes #2298 from rxin/openstack-swift and squashes the following commits:

ff4e394 [Reynold Xin] Two minor comments from Patrick.
279f6de [Reynold Xin] core-sites -> core-site
dfb8fea [Reynold Xin] Updated based on Gil's suggestion.
846f5cb [Reynold Xin] Added a link from overview page.
0447c9f [Reynold Xin] Removed sample code.
e9c3761 [Reynold Xin] Merge pull request #1010 from gilv/master
9233fef [Gil Vernik] Fixed typos
6994827 [Gil Vernik] Merge pull request #1 from rxin/openstack
ac0679e [Reynold Xin] Fixed an unclosed tr.
47ce99d [Reynold Xin] Merge branch 'master' into openstack
cca7192 [Gil Vernik] Removed white spases from pom.xml
99f095d [Reynold Xin] Pending openstack changes.
eb22295 [Reynold Xin] Merge pull request #1010 from gilv/master
39a9737 [Gil Vernik] Spark integration with Openstack Swift
c977658 [Gil Vernik] Merge branch 'master' of https://github.com/gilv/spark
2aba763 [Gil Vernik] Fix to docs/openstack-integration.md
9b625b5 [Gil Vernik] Merge branch 'master' of https://github.com/gilv/spark
eff538d [Gil Vernik] SPARK-938 - Openstack Swift object storage support
ce483d7 [Gil Vernik] SPARK-938 - Openstack Swift object storage support
b6c37ef [Gil Vernik] Openstack Swift support
(cherry picked from commit eddfeddac19870fc265ef406d87e1c3db9b54249)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Merge branch 'branch-1.1' of github.com:apache/spark into csd-1.1

Conflicts:
	assembly/pom.xml
	bagel/pom.xml
	core/pom.xml
	examples/pom.xml
	external/flume-sink/pom.xml
	external/flume/pom.xml
	external/kafka/pom.xml
	external/mqtt/pom.xml
	external/twitter/pom.xml
	external/zeromq/pom.xml
	extras/kinesis-asl/pom.xml
	extras/spark-ganglia-lgpl/pom.xml
	graphx/pom.xml
	mllib/pom.xml
	pom.xml
	repl/pom.xml
	sql/catalyst/pom.xml
	sql/core/pom.xml
	sql/hive-thriftserver/pom.xml
	sql/hive/pom.xml
	streaming/pom.xml
	tools/pom.xml
	yarn/pom.xml
	yarn/stable/pom.xml