Ok in that case -- adding a while loop here will have the effect of packing multiple tasks from the same task set on a small number of executors.  I know there have been other efforts to do the opposite of this by trying to spread tasks across the cluster, if possible (see the comment here: https://github.com/apache/incubator-spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L203).  I think the reason for trying to spread tasks out is that, when the cluster is mostly empty, the job will run faster if the tasks are spread across executors, because there's less resource contention.  Was there a reason for moving away from that model?