Update synchronization comment
Correct semantics of adding executors + update comments

Previously, we just kept asking for more executors whether or not
we are granted the ones we have asked for in the past. This is
extremely not cautious and could lead to an overflow of add executor
requests, if the cluster manager cannot accommodate all the requests,
for instance. This commit adds the correct guards against attempting
to request more executors before the pending ones have all registered.

An important TODO at this point is to retry these requests if
they're not immediately granted. We can do this after a timeout.
The tricky thing there is that if the retry is too eager then we
may end up exceeding the upper bound on the number of executors.