PlasticLizard/ruote-mongodb — Issue #1: "fatal: deadlock detected" error in production
Author: reedlaw
Created: 2011-12-15 13:47:10+00:00
State: open

I have a low-usage app running on a production EC2 instance and get this error intermittently (so far 15 times in just over a week).

Stacktrace:

/usr/lib/ruby/1.9.2/thread.rb:71:in `sleep'
/usr/lib/ruby/1.9.2/thread.rb:71:in `wait'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/util/pool.rb:171:in `block (2 levels) in checkout'
:
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/util/pool.rb:152:in `block in checkout'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/util/pool.rb:145:in `loop'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/util/pool.rb:145:in `checkout'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/connection.rb:579:in `checkout_reader'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/connection.rb:477:in `receive_message'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:407:in `block in send_initial_query'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/connection.rb:609:in `instrument'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:406:in `send_initial_query'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:371:in `refresh'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:87:in `next_document'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:248:in `each'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:267:in `to_a'
[GEM_ROOT]/gems/mongo-1.3.1/lib/mongo/cursor.rb:267:in `to_a'
[GEM_ROOT]/bundler/gems/ruote-mongodb-781a57caac4a/lib/ruote-mongodb/mongodb_storage.rb:123:in `block in get_many'
/usr/lib/ruby/1.9.2/monitor.rb:201:in `mon_synchronize'
[GEM_ROOT]/bundler/gems/ruote-mongodb-781a57caac4a/lib/ruote-mongodb/mongodb_storage.rb:106:in `get_many'
[GEM_ROOT]/bundler/gems/ruote-48ee8e5dd284/lib/ruote/part/storage_participant.rb:422:in `fetch_all'
[GEM_ROOT]/bundler/gems/ruote-48ee8e5dd284/lib/ruote/part/storage_participant.rb:437:in `do_select'
[GEM_ROOT]/bundler/gems/ruote-48ee8e5dd284/lib/ruote/part/storage_participant.rb:217:in `by_participant'
app/contexts/review.rb:7:in `report_list'
app/controllers/reports_controller.rb:6:in `index'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/abstract_controller/base.rb:167:in `process_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/rendering.rb:10:in `process_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/callbacks.rb:434:in `_run__791114626__process_action__775585695__callbacks'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/callbacks.rb:386:in `_run_process_action_callbacks'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/abstract_controller/callbacks.rb:17:in `process_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/rescue.rb:17:in `process_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/notifications.rb:53:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/notifications.rb:53:in `instrument'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/params_wrapper.rb:201:in `process_action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/abstract_controller/base.rb:121:in `process'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/abstract_controller/rendering.rb:45:in `process'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal.rb:193:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_controller/metal.rb:236:in `block in action'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/routing/route_set.rb:65:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/routing/route_set.rb:65:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/routing/route_set.rb:29:in `call'
[GEM_ROOT]/gems/rack-mount-0.8.3/lib/rack/mount/route_set.rb:152:in `block in call'
[GEM_ROOT]/gems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:96:in `block in recognize'
[GEM_ROOT]/gems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:75:in `optimized_each'
[GEM_ROOT]/gems/rack-mount-0.8.3/lib/rack/mount/code_generation.rb:95:in `recognize'
[GEM_ROOT]/gems/rack-mount-0.8.3/lib/rack/mount/route_set.rb:141:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/routing/route_set.rb:532:in `call'
[GEM_ROOT]/bundler/gems/stagehand-40aee1bf3628/lib/stagehand/rack/middleware.rb:54:in `block in call'
[GEM_ROOT]/bundler/gems/stagehand-40aee1bf3628/lib/stagehand/rack/middleware.rb:12:in `catch'
[GEM_ROOT]/bundler/gems/stagehand-40aee1bf3628/lib/stagehand/rack/middleware.rb:12:in `call'
[GEM_ROOT]/gems/mongoid-2.3.4/lib/rack/mongoid/middleware/identity_map.rb:33:in `block in call'
[GEM_ROOT]/gems/mongoid-2.3.4/lib/mongoid.rb:132:in `unit_of_work'
[GEM_ROOT]/gems/mongoid-2.3.4/lib/rack/mongoid/middleware/identity_map.rb:33:in `call'
[GEM_ROOT]/gems/sass-3.1.11/lib/sass/plugin/rack.rb:54:in `call'
[GEM_ROOT]/gems/client_side_validations-3.1.3/lib/client_side_validations/middleware.rb:18:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/etag.rb:23:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/conditionalget.rb:25:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/head.rb:14:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/flash.rb:247:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/session/abstract/id.rb:195:in `context'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/session/abstract/id.rb:190:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/cookies.rb:331:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/callbacks.rb:392:in `_run_call_callbacks'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/callbacks.rb:28:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/sendfile.rb:101:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'
[GEM_ROOT]/gems/actionpack-3.1.3/lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'
[GEM_ROOT]/gems/railties-3.1.3/lib/rails/rack/logger.rb:13:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/methodoverride.rb:24:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/runtime.rb:17:in `call'
[GEM_ROOT]/gems/activesupport-3.1.3/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
[GEM_ROOT]/gems/rack-1.3.5/lib/rack/lock.rb:15:in `call'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:132:in `forward'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:241:in `fetch'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:181:in `lookup'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:65:in `call!'
[GEM_ROOT]/gems/rack-cache-1.1/lib/rack/cache/context.rb:50:in `call'
[GEM_ROOT]/gems/railties-3.1.3/lib/rails/engine.rb:456:in `call'
[GEM_ROOT]/gems/railties-3.1.3/lib/rails/railtie/configurable.rb:30:in `method_missing'
[GEM_ROOT]/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:528:in `process_client'
[GEM_ROOT]/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:600:in `worker_loop'
[GEM_ROOT]/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:485:in `spawn_missing_workers'
[GEM_ROOT]/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:135:in `start'
[GEM_ROOT]/gems/unicorn-4.1.1/bin/unicorn_rails:209:in `'
[GEM_ROOT]/bin/unicorn_rails:19:in `load'
[GEM_ROOT]/bin/unicorn_rails:19:in `

'

I using RuoteKit without a dedicated worker at the moment. Could that be the issue or is it something I can fix through configuration?