NixOS/nixops — pull request #177 by ocharles (2014-04-06 12:56:30+00:00)
Title: Retry setting Route53 DNS records on PriorRequestNotComplete

For NixOps-managed networks with a lot of machines, the Route53
processing can often be too fast. As mentioned in the documentation [1],
Route53 can only process one request at a time per hosted zone, and
changes.commit() returning does not necessarily imply the request has
been processed.

Instead, we use a retry loop with 3 retries with a 500ms sleep between
each request. This should give AWS enough time to process the
requests. If not, an exception is thrown, which matches the current
behaviour.

[1]: http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/DNSLimitations.html