allourideas/allourideas.org — Issue #7: ECONNRESET in Photocracy MungeAndNotifyJob
Author: demery
Created: 2012-02-24 20:01:15+00:00
State: open

Hi,

A client is not able to request and receive results and image list emails from a Photocracy installation I set up for them.

The application is up and working as expected otherwise. When requests are made for results emails, the job is enqueued, however, it always fails with the following error and stacktrace:

```text
Errno::ECONNRESET: Connection reset by peer - Connection lost
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:483:in `read_reply'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:455:in `process_command'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:453:in `map'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:453:in `process_command'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:442:in `raw_call_command'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:463:in `call'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:463:in `maybe_lock'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:439:in `raw_call_command'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:346:in `call_command'
/usr/lib/ruby/gems/1.8/gems/redis-1.0.7/lib/redis/client.rb:392:in `method_missing'
/home/dylan/rails/allourideas.org/lib/munge_and_notify_job.rb:19:in `perform'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/backend/base.rb:74:in `invoke_job'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:122:in `run'
/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:122:in `run'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:121:in `run'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:173:in `reserve_and_run_one_job'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:106:in `work_off'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:105:in `times'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:105:in `work_off'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:80:in `start'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:79:in `start'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:76:in `loop'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/worker.rb:76:in `start'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/command.rb:100:in `run'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/command.rb:79:in `run_process'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/application.rb:249:in `call'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/application.rb:249:in `start_proc'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/daemonize.rb:200:in `call'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/daemonize.rb:200:in `call_as_daemon'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/application.rb:253:in `start_proc'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/application.rb:293:in `start'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/controller.rb:70:in `run'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons.rb:197:in `run_proc'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/cmdline.rb:109:in `call'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons/cmdline.rb:109:in `catch_exceptions'
/usr/lib/ruby/gems/1.8/gems/daemons-1.1.6/lib/daemons.rb:196:in `run_proc'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/command.rb:78:in `run_process'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/command.rb:72:in `daemonize'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/command.rb:70:in `times'
/usr/lib/ruby/gems/1.8/gems/delayed_job-2.0.6/lib/delayed/command.rb:70:in `daemonize'
```
I've confirmed that mail is working properly as configured. The problem comes in the `MungeAndNotifyJob` call:

```ruby
thekey, zlibcsv = r.blpop(redis_key, (60*10).to_s) # Timeout - 10 minutes
```

Playing around in the console, I've found that a call to `Redis.blpop` works as expected. The expected Redis key is *not* in the database.

I'm stumped. This may be a configuration problem, but I've not been able to solve it. I thought that perhaps the problem lay on the pairwise side, but I've rechecked the applications' configurations and everything *appears* to be in order.

Thanks.

Doug