Matei - what are the semantics of these different waits? Say that you want a task to bump from PROCESS_LOCAL to NODE_LOCAL after 10 seconds. Would you set the wait to 10 seconds for PROCESS_LOCAL or NODE_LOCAL? It would be good to make that clear in the docs.

It might make sense to have the default wait be different for different levels so that the "turn key" performance is pretty good. For instance, it's probably fine to make the PROCESS_LOCAL -> NODE_LOCAL switch after only a few seconds. But going from PROCESS_LOCAL to RACK_LOCAL should be a lot longer.