mongodb/mongo-ruby-driver — pull request #142 by cheald (2012-12-21 01:41:25+00:00)
Title: Thread leak fixes

The current 1.8 release prevents threads from being garbage collected because it uses threads as hash keys in a couple of places, which are then either occasionally or never cleaned up.

For brevity's sake, I'll be referring to thread local variables at TLVs.

I have a number of fixes in this PR:

1) Small housekeeping things; fixed Mocha stubs clobbering later tests in one test, and fixed a variable scope issue in the mongo_config replica set tests.
2) Fixed the existing TLV `:manager` to use a less ambiguous name. TLVs are effectively globals, so distinct names will prevent potential conflicts with other libraries.
3) Replaced the @threads_to_sockets lookup hash with TLVs. This eliminates the need for the periodic pruner.
4) Replaced the PID change tracking lookup tables by storing the current PID on the socket object when it's created, and then checking it in the same fashion as before.
5) Replaced the pinned pool tracking with TLVs, eliminating yet another lookup table. This is the one that was actually holding onto Thread references and causing long-term leaks.

Additionally, the thread socket affinity has been preserved, but the socket acquisition has been changed so that existing checked-in sockets will be preferred over creating a new socket when a thread does not yet have a socket affiliated. This fixes issues where every new thread would open a new socket in cases like Sidekiq, which uses short-lived worker threads to do work (and was burning up all our file descriptors over time as a result). Instead, sockets will be reused if available even if you rotate thousands of threads through the socket pool.

I used a variant of the unit test in [RUBY-500](https://jira.mongodb.org/browse/RUBY-500?focusedCommentId=220861#comment-220861) to test the leaks.

require 'mongo'

DB = Mongo::Connection.new('localhost', 27017)['test_db']
@collection = DB['test_collection']
code = -> { @collection.find({"Name" => "aaron.symplicity.com"}).each {|d| d} }

loop do
th = Thread.new do
code.call
end
th.join
GC.start
p "Threads: #{ObjectSpace.each_object(Thread) {}}"
end

Using the 1.8 driver, this code causes the thread count to increase by 1 for every iteration. Using my patched driver, this remains consistent at 2 threads (the primary thread and the scheduler thread).

This patch passes the standard test suite, the replica_test suite, and the sharded_cluster suite on:

* ruby-1.9.3-p327
* jruby-1.7.0

Additionally, the basic test suite passes on:

* ruby-1.8.7-p371

However, it segfaults when running the replica set and sharded cluster tests suites in bson_ext/cbson.so, so I was unable to do testing there. The same segfault occurred for me on master, so I am confident that my changes did not introduce it.

I haven't yet had a chance to test on jruby 1.6, and have to run out, but once I get back I'll see if I can get 1.8.7 fixed and run jruby 1.6 tests.