Merge pull request #75 from JoshRosen/block-manager-cleanup

Code de-duplication in BlockManager

The BlockManager has a few methods that duplicate most of their code.  This pull request extracts the duplicated code into private doPut(), doGetLocal(), and doGetRemote() methods that unify the storing/reading of bytes or objects.

I believe that I preserved the logic of the original code, but I'd appreciate some help in reviewing this.
(cherry picked from commit edc5e3f8f44a658e9829f2ee65d5fb32b464121b)

Signed-off-by: Aaron Davidson <aaron@databricks.com>

Conflicts:
	core/src/main/scala/org/apache/spark/storage/BlockManager.scala
Never store shuffle blocks in BlockManager

After the BlockId refactor (PR #114), it became very clear that ShuffleBlocks are of no use
within BlockManager (they had a no-arg constructor!). This patch completely eliminates
them, saving us around 100-150 bytes per shuffle block.
The total, system-wide overhead per shuffle block is now a flat 8 bytes, excluding
state saved by the MapOutputTracker.
Merge pull request #142 from liancheng/dagscheduler-pattern-matching

Using case class deep match to simplify code in DAGScheduler.processEvent

Since all `XxxEvent` pushed in `DAGScheduler.eventQueue` are case classes, deep pattern matching is more convenient to extract event object components.

(cherry picked from commit 9f7b9bb1cd157e8278d43a3908d5f778b54aed55)
Signed-off-by: Reynold Xin <rxin@apache.org>
Merge pull request #140 from aarondav/merge-75

Manually merge BlockManager refactor #75

PR #75 was unfortunately not cherry-picked into 0.8.0, which caused a build breakage earlier and now again due to later changes being cherry-picked. This merges #75 and replays #139 on top of it. I have confirmed that #75 has made exactly the changes it did on master. So long as all BlockManager patches from master should have gone into branch-0.8, this PR should do the right thing.