mesos/spark — pull request #522 by shane-huang (2013-03-13 08:14:56+00:00)
Title: Shuffle Performance fix: Use netty embeded OIO file server instead of Co...

...nnectionManager

This patch provides an alternative way to do shuffle. Basically it uses a embeded netty OIO file server to serve blocks to BlockManager instead of the current message exchanging ConnectionManager. We provide an option for user to choose between the two alternatives. Set system property "spark.shuffle.use.netty=true" to enable this feature. Default is disabled.
In our tests using netty can speedup jobs from 2x~5x.