apache/incubator-spark — pull request #18 by nkronenfeld (2013-09-26 21:27:36+00:00)
Title: RDD.sliding method and other associated methods

Adding a few partition-based methods on RDDs - getPartition, prepend, append, sliding, and index.

Sliding is the most interesting one, and the one that I think warrants pushing into the main code, though I don't see a reason not to push the rest.

Sliding is intended to mimic Seq.sliding, though I've left off the step size parameter for now. I think I see how to add it if necessary, but I think it would require a two-pass approach, where the second pass depends on the results of the first - i.e., the results would be pass2(pass1(data), data) - which seemed less than ideal to me. The current approach technically has two passes, but they are independent - the results are essentially (pass1) union (pass2).