mesos/spark — pull request #610 by JoshRosen (2013-05-14 20:13:18+00:00)
Title: Throw exception if TaskResult exceeds Akka frame size

This addresses [SPARK-747](https://spark-project.atlassian.net/browse/SPARK-747): "Throw an exception on slaves if a message is sent that is larger than Akka's max frame size".

If serialized task results are within 1024 bytes of the maximum frame size, the Executors die with an appropriate exception. I didn't know how much overhead Akka adds to the message payload, but I assume that it won't be more than 1024 bytes.

I decided to pull the frame size directly from the ActorSystem's actual configuration since that's the actual limit that matters.

A better long-term approach might be to collect large task results through the BlockManager (as proposed in [SPARK-669](https://spark-project.atlassian.net/browse/SPARK-669)), but that involves a number of subtleties that I'll need more time to think through.