mwmitchell/rsolr — Issue #39: memory leaks?
Author: jtoy
Created: 2012-02-01 16:44:10+00:00
State: open

I've tracked down a memory leak to rsolr, where I normally create some threads to push data into solr. The ruby process uses up all the memory, but If I comment out rsolr so the while loop just pops items off the queue, the app stays at a constant 22 mb of use. This loop is used to push in millions of docs into solr.

25.times do |t|
threads << Thread.new do
rsolr = RSolr.connect :url => (CONFIG[$env]["solr"] ||'http://localhost:8983/solr')
while true
a = $q.pop
if a
begin
rsolr.add a
$inserts += 1
rescue Exception => e
puts e
$ecount += 1
end
end
end
end
end