kevinweil/elephant-bird — pull request #245 by dvryaboy (2012-09-15 18:57:26+00:00)
Title: Stop using DefaultDataBags, use NonSpillable ones instead.

DefaultDataBags create WeakReferences in SpillableMemoryManager, which costs 32 bytes (!) per instance, and are poorly GC'd. It doesn't make sense to pay that overhead for bags we know to be small.

I ran a simple benchmark looking at memory utilization of ToBag(), which had the same problem, and saw that 60K strings being inserted into bags created 500KB overhead in DefaultDataBag objects (not counting the strings themselves), and another 2MB in WeakReferences. Switching to NonSpillableDataBag dropped overhead to 175KB.