continuuity/loom — pull request #439 by albertshau (2014-08-28 04:13:14+00:00)
Title: changing the retry logic so that rollback tasks must be done

before the retries begin instead of letting them happen at the
same time. To make this concrete, if a node confirm task fails,
the node delete task must happen before another node create task
can begin.

This was done so that the number of nodes provisioned for a
cluster never goes over the cluster size. Before, when the node
delete happened concurrently with the node create retry, you could
actually go over the cluster size if the node create happened
before the node delete. Enforcing ordering also has the benefit
of ensuring that hostnames are always unique since the node is
guaranteed to be deleted before another one with the same config
is created.