Merge pull request #928 from jerryshao/fairscheduler-refactor

Refactor FairSchedulableBuilder
(cherry picked from commit 834686b108ce31cbee531d89de6c6e80913448f4)

Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #941 from ilikerps/master

Add "org.apache." prefix to packages in spark-class
(cherry picked from commit f06f2da2cb2fd6bd25b57e5f5fd6f8e5d37ab1a3)

Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #937 from jerryshao/localProperties-fix

Fix PR926 local properties issues in Spark Streaming like scenarios
(cherry picked from commit a2ea069a5f2ed83268109deade456dc0fc9b79ee)

Signed-off-by: Reynold Xin <rxin@apache.org>
Merge remote-tracking branch 'pr/12'

Fix spacing so java.io.tmpdir doesn't run on with SPARK_JAVA_OPTS

(cherry picked from commit 7220e8f90b7d0bf5d5e3e4ad906a6b257ca75c53)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #9 from rxin/limit

Smarter take/limit implementation.

(cherry picked from commit 6566a19b38204d754c5e8f821b4276616e90abc6)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #7 from wannabeast/memorystore-fixes

some minor fixes to MemoryStore

This is a repeat of #5, moved to its own branch in my repo.

This makes all updates to   on ; it skips on synchronizing the reads where it can get away with it.

(cherry picked from commit 560ee5c9bba3f9fde380c831d0c6701343b2fecf)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge branch 'master' of github.com:markhamstra/incubator-spark

(cherry picked from commit ff540a015b970f722ae6aeba7dece481f9ad863e)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #14 from kayousterhout/untangle_scheduler

Improved organization of scheduling packages.

This commit does not change any code -- only file organization.
Please let me know if there was some masterminded strategy behind
the existing organization that I failed to understand!

There are two components of this change:
(1) Moving files out of the cluster package, and down
a level to the scheduling package. These files are all used by
the local scheduler in addition to the cluster scheduler(s), so
should not be in the cluster package. As a result of this change,
none of the files in the local package reference files in the
cluster package.

(2) Moving the mesos package to within the cluster package.
The mesos scheduling code is for a cluster, and represents a
specific case of cluster scheduling (the Mesos-related classes
often subclass cluster scheduling classes). Thus, the most logical
place for it seems to be within the cluster package.

The one thing about the scheduling code that seems a little funny to me
is the naming of the SchedulerBackends.  The StandaloneSchedulerBackend
is not just for Standalone mode, but instead is used by Mesos coarse grained
mode and Yarn, and the backend that *is* just for Standalone mode is instead called SparkDeploySchedulerBackend. I didn't change this because I wasn't sure if there
was a reason for this naming that I'm just not aware of.

(cherry picked from commit 70a0b993d4f1a72a1b38f9a17fcf269acc1b394f)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #930 from holdenk/master

Add mapPartitionsWithIndex
(cherry picked from commit c514cd15879d510406712b1cebdaf0c0db9151c9)

Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #17 from rxin/optimize

Remove -optimize flag

(cherry picked from commit 714fdabd99bbff3a0cdec5dcf06b021a3a3f2da8)
Signed-off-by: Reynold Xin <rxin@apache.org>