Fixed issue with canonical edge partitioner.
Merge branch 'cleanup'
Merge branch 'master' of github.com:amplab/graphx
Added logging to Graph, GraphLab, and Pregel.
Merge branch 'master' of github.com:amplab/graphx
Merge branch 'master' of github.com:amplab/graphx
Allow spark on yarn to be run from HDFS. Allows the spark.jar, app.jar, and log4j.properties to be put into hdfs.
Merge pull request #128 from shimingfei/joblogger-doc

add javadoc to JobLogger, and some small fix

against Spark-941

add javadoc to JobLogger, output more info for RDD, modify recordStageDepGraph to avoid output duplicate stage dependency information

(cherry picked from commit 518cf22eb2436d019e4f7087a38080ad4a20df58)
Signed-off-by: Reynold Xin <rxin@apache.org>
Never store shuffle blocks in BlockManager

After the BlockId refactor (PR #114), it became very clear that ShuffleBlocks are of no use
within BlockManager (they had a no-arg constructor!). This patch completely eliminates
them, saving us around 100-150 bytes per shuffle block.
The total, system-wide overhead per shuffle block is now a flat 8 bytes, excluding
state saved by the MapOutputTracker.
Merge pull request #139 from aarondav/shuffle-next

Never store shuffle blocks in BlockManager

After the BlockId refactor (PR #114), it became very clear that ShuffleBlocks are of no use
within BlockManager (they had a no-arg constructor!). This patch completely eliminates
them, saving us around 100-150 bytes per shuffle block.
The total, system-wide overhead per shuffle block is now a flat 8 bytes, excluding
state saved by the MapOutputTracker.

Note: This should *not* be merged directly into 0.8.0 -- see #138
Using compact case class pattern matching syntax to simplify code in DAGScheduler.processEvent
Merge pull request #142 from liancheng/dagscheduler-pattern-matching

Using case class deep match to simplify code in DAGScheduler.processEvent

Since all `XxxEvent` pushed in `DAGScheduler.eventQueue` are case classes, deep pattern matching is more convenient to extract event object components.
Ignore a task update status if the executor doesn't exist anymore.
Merge pull request #143 from rxin/scheduler-hang

Ignore a task update status if the executor doesn't exist anymore.

Otherwise if the scheduler receives a task update message when the executor's been removed, the scheduler would hang.

It is pretty hard to add unit tests for these right now because it is hard to mock the cluster scheduler. We should do that once @kayousterhout finishes merging the local scheduler and the cluster scheduler.
Removed unused return value in SparkContext.runJob
Attempt to fix SparkListenerSuite breakage

Could not reproduce locally, but this test could've been flaky if the
build machine was too fast.
Merge pull request #145 from aarondav/sls-fix

Attempt to fix SparkListenerSuite breakage

Could not reproduce locally, but this test could've been flaky if the build machine was too fast, due to typo. (index 0 is intentionally slowed down to ensure total time is >= 1 ms)

This should be merged into branch-0.8 as well.
Merge pull request #144 from liancheng/runjob-clean

Removed unused return value in SparkContext.runJob

Return type of this `runJob` version is `Unit`:

    def runJob[T, U: ClassManifest](
        rdd: RDD[T],
        func: (TaskContext, Iterator[T]) => U,
        partitions: Seq[Int],
        allowLocal: Boolean,
        resultHandler: (Int, U) => Unit) {
        ...
    }

It's obviously unnecessary to "return" `result`.
Add Spark multi-user support for standalone mode and Mesos
Merge branch 'master' of github.com:amplab/graphx