jokkedk/webgrind — pull request #36 by bionoren (2012-08-13 20:50:26+00:00)
Title: Performance optimizations preprocessing callgrind data

The 36Mb file processes in ~12 seconds vs ~14 seconds previously, and the 152Mb file processes in ~109 seconds vs ~137.

Memory usage should be slightly lower, but I haven't been able to reproduce the issues with runaway allocations that I was experiencing last week. Maybe we were running into memory leaks in the interpreter? Wouldn't be the first time :(

For reference, SQLite didn't work out because the database calls take too long (even to an in-memory DB with stored procedures), and batching them up caused significant memory overhead. The best I could do was ~16.5 seconds vs the original 14, which I understand isn't acceptable.