thecity/rds-s3-backup — pull request #5 by sandinmyjoints (2014-03-26 14:33:39+00:00)
Title: Bug 68268570 increase fog timeout

@CarsonBanov

This is the script that the cron job runs to create the S3 snapshot and do the backup. The error message we have been seeing is:

```
/var/lib/gems/1.8/gems/fog-1.20.0/lib/fog/core/wait_for.rb:10:in `wait_for': The specified wait_for timeout (600 seconds) was exceeded (Fog::Errors::TimeoutError)
from /var/lib/gems/1.8/gems/fog-1.20.0/lib/fog/core/model.rb:64:in `wait_for'
from /usr/local/bin/rds-s3-backup.rb:59:in `s3_dump'
from /var/lib/gems/1.8/gems/thor-0.18.1/lib/thor/command.rb:27:in `__send__'
from /var/lib/gems/1.8/gems/thor-0.18.1/lib/thor/command.rb:27:in `run'
from /var/lib/gems/1.8/gems/thor-0.18.1/lib/thor/invocation.rb:120:in `invoke_command'
from /var/lib/gems/1.8/gems/thor-0.18.1/lib/thor.rb:363:in `dispatch'
from /var/lib/gems/1.8/gems/thor-0.18.1/lib/thor/base.rb:439:in `start'
from /usr/local/bin/rds-s3-backup.rb:160
```

There are several calls to `wait_for` in our script, and I think the one that is failing here is waiting for the new DB snapshot to finish. So here’s what I think happens:
- cron runs this script, which kicks off the db snapshot to our dump instance.
- 10 minutes pass, Fog errors out and this script dies.
- the snapshot finishes sometime later, but this script is no longer running so the backup to S3 doesn’t happen and the dump instance is not deleted.
- cron runs this script a day later, sees the old backup dump instance, and bails.

According to what [this thread says](https://groups.google.com/forum/#!topic/ruby-fog/hlLvYMLIxMg), I think we can simply increase `Fog.timeout` to fix this problem. Fwiw, I verified that `Fog.timeout` is currently set to 600 on our db-backup rw instance:

```
william@aws-prod-site-rw-precise-c1x-10:~$ irb
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'fog'
=> true
irb(main):004:0> Fog.timeout
=> 600
```

/cc @nanek

[#68268570]