mesos/spark — pull request #587 by rxin (2013-04-30 20:32:40+00:00)
Title: A set of shuffle map output related changes

1. Added a BlockObjectWriter interface for writing out a series of jvm objects to disk.

2. Added a getBlockWriter interface to DiskStore.

3. Added a new method to BlockManager for writing out shuffle files. This method provides a short-circuited way to write shuffle data out directly to disk.

4. Added ShuffleBlockManager and updated ShuffleMapTask to use the manager to write map outputs out to disk (ultimately via the BlockObjectWriter interface).

5. Allow specifying a shuffle serilaizer on a per-shuffle basis.