rubygems/rubygems — pull request #341 by amatsuda (2012-06-11 02:40:34+00:00)
Title: symbolize operators in order to reduce String instances

Here's a patch that reduces number of un-GCed String objects in memory.

Inspired by @elisehuard's GC talk at EuRuKo (she pointed out that even a simple Rails app creates hundred-thousands of String objects which cause severe overhead on GC), I investigated the `ObjectSpace` to check what kind of String objects are actually created.

```ruby
require 'pp'
p ObjectSpace.each_object(String).count
pp ObjectSpace.each_object(String).group_by {|o| o}.map {|o, a| [o, a.length]}.sort_by {|_o, c| -c}.first(20)
```
The result on a vanilla Rails app was like this.

* Ruby 1.9.3 p194, Rails 3.2.5, RubyGems 1.8.24

```
% rails runner script/count_obj.rb
67439
[[">=", 1443],
["~>", 948],
["0", 916],
["ruby", 509],
["", 494],
["lib", 493],
["bin", 486],
["1.8.24", 467],
["initialize", 415],
["README.rdoc", 385],
["=", 343],
[" ", 189],
["\n", 188],
["rake", 169],
["http://www.rubyonrails.org", 160],
["David Heinemeier Hansson", 160],
["rspec", 151],
[")", 147],
["README.md", 137],
["LICENSE", 129]]
```

I didn't expect to see this lot ">=" or "~>". Where did they come from?
Next, I created a vacant dummy gem and run the same script against the gem.

* Ruby 1.9.3 p194, RubyGems 1.8.24

```
% ruby -rdummy_gem script/count_obj.rb
21772
[[">=", 716],
["0", 589],
["~>", 461],
["1.8.24", 352],
[">= 0", 334],
["lib", 264],
["ruby", 260],
["bin", 244],
["README.rdoc", 203],
["/", 180],
["1.2.0", 167],
["/Users/a_matsuda/.rvm/gems/ruby-1.9.3-p194/specifications", 166],
["\n", 149],
["2", 145],
["1", 130],
["-", 124],
["3", 120],
["", 120],
["=", 116],
["rake", 99]]
```

Still there are hundreds of ">=", "~>", and "=".

I tried RubyGems GH master, and it returned almost same result.

* Ruby 1.9.3 p194, RubyGems master (before patch)

```
% ruby -rdummy_gem script/count_obj.rb
17246
[["~>", 461],
[">=", 273],
["ruby", 260],
["lib", 245],
["bin", 243],
["1.8.24", 233],
["README.rdoc", 193],
["", 170],
["=", 128],
["/Users/a_matsuda/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/fileutils.rb",
90],
["/Users/a_matsuda/.rvm/gems/ruby-1.9.3-p194", 85],
["/Users/a_matsuda/.rvm/gems/ruby-1.9.3-p194/gems", 82],
["David Heinemeier Hansson", 80],
["http://www.rubyonrails.org", 80],
["rake", 78],
["rspec", 76],
["README.md", 69],
["LICENSE", 65],
["--main", 63],
["activesupport", 53]]
```

Apparently, these comparison operators originally came from Gemspecs, and are living perpetually in the memory without being GCed because the loaded specs are memoized in class variables.

So, I wrote a patch to use Symbol for those variables in the loaded specs.

* Ruby 1.9.3 p194, RubyGems master (after patch)

```
% ruby -rdummy_gem script/count_obj.rb
16839
[["0", 453],
["ruby", 260],
["lib", 245],
["bin", 243],
["1.8.24", 233],
["README.rdoc", 193],
["", 170],
["/Users/a_matsuda/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/fileutils.rb",
90],
["/Users/a_matsuda/.rvm/gems/ruby-1.9.3-p194", 85],
["/Users/a_matsuda/.rvm/gems/ruby-1.9.3-p194/gems", 83],
["http://www.rubyonrails.org", 80],
["David Heinemeier Hansson", 80],
["rake", 78],
["rspec", 76],
["README.md", 69],
["LICENSE", 65],
["--main", 63],
["activesupport", 53],
["1.8.7", 52],
["\n", 49]]
```

You see the GC cleaned up all the operator Strings now.