Version number in scala and python shell
Removed 'incubator'
bump for release.
Merge branch 'csd-2.10' of github.com:clearstorydata/spark into branch-0.9-csd
[maven-release-plugin] prepare release spark-parent-0.9.2-candidate-csd-2
[maven-release-plugin] prepare for next development iteration
[SPARK-979] Randomize order of offers.

This commit randomizes the order of resource offers to avoid scheduling
all tasks on the same small set of machines.

This is a much simpler solution to SPARK-979 than #7.

Author: Kay Ousterhout <kayousterhout@gmail.com>

Closes #27 from kayousterhout/randomize and squashes the following commits:

435d817 [Kay Ousterhout] [SPARK-979] Randomize order of offers.

Conflicts:
	core/src/test/scala/org/apache/spark/scheduler/cluster/ClusterSchedulerSuite.scala
	core/src/test/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManagerSuite.scala

Conflicts:
	core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
	core/src/test/scala/org/apache/spark/scheduler/ClusterSchedulerSuite.scala
	core/src/test/scala/org/apache/spark/scheduler/FakeTask.scala
	core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
adding full outer join support

Conflicts:
	core/src/main/scala/org/apache/spark/api/java/JavaPairRDD.scala
	streaming/src/main/scala/org/apache/spark/streaming/PairDStreamFunctions.scala
update comment in pyspark code
SPARK-1601 & SPARK-1602: two bug fixes related to cancellation

This should go into 1.0 since it would return wrong data when the bug happens (which is pretty likely if cancellation is used). Test case attached.

1. Do not put partially executed partitions into cache (in task killing).

2. Iterator returned by CacheManager#getOrCompute was not an InterruptibleIterator, and was thus leading to uninterruptible jobs.

Thanks @aarondav and @ahirreddy for reporting and helping debug.

Author: Reynold Xin <rxin@apache.org>

Closes #521 from rxin/kill and squashes the following commits:

401033f [Reynold Xin] Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/spark into kill
7a7bdd2 [Reynold Xin] Add a new line in the end of JobCancellationSuite.scala.
35cd9f7 [Reynold Xin] Fixed a bug that partially executed partitions can be put into cache (in task killing).

Conflicts:
	core/src/main/scala/org/apache/spark/CacheManager.scala
	core/src/main/scala/org/apache/spark/executor/Executor.scala
	core/src/test/scala/org/apache/spark/JobCancellationSuite.scala

Conflicts:
	core/src/main/scala/org/apache/spark/CacheManager.scala
SPARK-1582 Invoke Thread.interrupt() when cancelling jobs

Sometimes executor threads are blocked waiting for IO or monitors, and the current implementation of job cancellation may never recover these threads. By simply invoking Thread.interrupt() during cancellation, we can often safely unblock the threads and use them for subsequent work.

Note that this feature must remain optional for now because of a bug in HDFS where Thread.interrupt() may cause nodes to be marked as permanently dead (as the InterruptedException is reinterpreted as an IOException during communication with some node).

Author: Aaron Davidson <aaron@databricks.com>

Closes #498 from aarondav/cancel and squashes the following commits:

e52b829 [Aaron Davidson] Don't use job.properties when null
82f78bb [Aaron Davidson] Update DAGSchedulerSuite
b67f472 [Aaron Davidson] Add comment on why interruptOnCancel is in setJobGroup
4cb9fd6 [Aaron Davidson] SPARK-1582 Invoke Thread.interrupt() when cancelling jobs

Conflicts:
	core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala
	core/src/main/scala/org/apache/spark/executor/Executor.scala
	core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
	core/src/main/scala/org/apache/spark/scheduler/SchedulerBackend.scala
	core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
	core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala
	core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
Changes to accommodate backporting of SPARK-1582, SPARK-1601 and SPARK-1602

Conflicts:
	core/src/main/scala/org/apache/spark/scheduler/SchedulerBackend.scala
	core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala
	core/src/main/scala/org/apache/spark/scheduler/local/LocalScheduler.scala
	core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala
streaming iterable

Conflicts:
	core/src/main/scala/org/apache/spark/rdd/RDD.scala

Conflicts:
	core/src/main/scala/org/apache/spark/rdd/RDD.scala
updated streaming iterable

Conflicts:
	core/src/main/scala/org/apache/spark/rdd/RDD.scala
	core/src/main/scala/org/apache/spark/util/RDDiterable.scala

Conflicts:
	core/src/main/scala/org/apache/spark/rdd/RDD.scala
	core/src/main/scala/org/apache/spark/util/RDDiterable.scala
SPARK-1689: Spark application should die when removed by Master

scheduler.error() will mask the error if there are active tasks. Being removed is a cataclysmic event for Spark applications, and should probably be treated as such.

Author: Aaron Davidson <aaron@databricks.com>

Closes #832 from aarondav/i-love-u and squashes the following commits:

9f1200f [Aaron Davidson] SPARK-1689: Spark application should die when removed by Master

Conflicts:
	core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
Merge pull request #11 from markhamstra/SPARK-979

SKIPME SPARK-979 Randomize order of offers.
Merge pull request #12 from markhamstra/FullOuterJoin

SKIPME Full outer join
Merge pull request #13 from markhamstra/SPY-317

SPY-317
Merge pull request #14 from markhamstra/streaming-iterable

SKIPME Streaming iterable
Merge pull request #15 from markhamstra/SPARK-1689

SKIPME backport SPARK-1689