mesos/spark — pull request #649 by ryanlecompte (2013-06-13 23:15:23+00:00)
Title: Add top K method to RDD using a bounded priority queue

This pull request contains an implementation for a new top K operation for an RDD[T] based on a specific Ordering[T]. This operation should be a more performant alternative to the usual first `sortByKey` and then `take(k)` for large data sets to avoid shuffling. The implementation currently relies on `mapPartitions`. I am not sure if there is a better approach, so it would be great to have other people look at it. Note that I am using Java's PriorityQueue because Scala's PriorityQueue currently has serialization issues (a bug has been filed already on Scala's version).