amccreight/heapgraph — pull request #10 by froydnj (2013-11-11 20:38:28+00:00)
Title: Memory usage reduction

These changes enable parsing and displaying information about a 540MB heap dump on my work machine (12GB RAM, several gigabytes occupied by emacs). Max memory usage with the patches is ~7.7GB. Max memory usage within the patches is well over 8GB; the process gets killed at about 8.5GB while reversing the CC graph.

I'm not entirely sure if both parts are necessary. I know the string internment one is not sufficient on its own, though I think it did seem to help a little. (The process still got killed and I wasn't sure if it made it any farther.) I can try just the integer node/edge patch if you like.

Any speed decrease seems to be negligible. With these patches applied, I can get to producing output from find_roots.py at about the same time the process without these patches applied goes OOM. I'd be interested to hear if you see differences on any smaller CC graphs that you have lying about.