mesos/spark — pull request #557 by shane-huang (2013-04-07 07:06:34+00:00)
Title: Shuffle Performance Optimization

The patch includes below optimizations for shuffle performance.
1. Use netty embeded OIO file server instead of Connection Manager for block shuffling.
2. Skip 0-byte block requests to reduce the network transport in shuffle

A system property "spark.shuffle.use.netty" is provided for user to choose whether enable or disable this optimization. If spark.shuffle.use.netty=true, the optimization is enabled, otherwise it is disabled. By default it is disabled.

This patch is moved from against branch-0.6 to master. The original pull request is closed( https://github.com/mesos/spark/pull/522).