Well, I'll point out the obvious that this is exactly what's been done in ParallelCollectionRDD, and thus could feasibly be replicated here, as it already partially is.