mesos/spark — pull request #847 by rxin (2013-08-19 03:24:43+00:00)
Title: Allow subclasses of Product2 in all key-value related classes

(ShuffleDependency, PairRDDFunctions, etc)

This change would allow more performance optimizations for Spark applications, e.g.:

1. Across elements, reuse a mutable implementation of Tuple2 to reduce temporary object creation (unfortunately the current iterator model makes it harder for escape analysis to do on stack allocation for these temporary tuples).

2. Making the key element of the tuple transient. This can result in huge shuffle savings when the application is using Spark in BSP fashion (the key is often used to route the message to a specific partition; once the partition code is calculated, the key is no longer needed).