This actually leads to potentially suboptimal scheduling.
Consider the case of 4 workers ready for scheduling, with the level set to rack local (anything worse than instance local).
The best scheduling will be in the order instance local, node local, rack local, others.

But that need not necessarily be gauranteed here - for example, if task1 has offers(0) rack local, offers(1) node local and offers(2) instance local.
Ideally, we will want task1 to be scheduled on offers(2), not even on offers(1) ... which is what the earlier messy blocks were attempting to do - to minimize intra node (added as an after thought thanks to @rxin's suggestion !), followed by inter node, and then intra-rack data movement.