honestbleeps/Reddit-Enhancement-Suite — pull request #1844 by TheRook (2014-12-08 03:35:36+00:00)
Title: Lower Memory usage with RES!

I got a new Microsoft Slate, and the memory usage on RES is too much for the poor tablet. So, I decided to fork RES and see if I can reduce memory overhead for loading new pages. If this patch goes well, I'd like to further reduce RES's memory usage.

This patch could use more testing. Loading 10 pages with the current RES takes roughly 804M or ram, after this patch the page consumed 764M with the same dataset. Still needs more work, but there is a noticeable reduction in memory usage with normal browsing habits.

Improvements:
- A bloomfilter is the most efficient way of keeping track of dupes. A bloomfilter doesn't need to store the full string in order to keep track of uniqueness. Additionally, insertion into a bloom filter is always O(1), where as dictionaries need to be expanded when they become full.

- "Hiding" elements with CSS doesn't make them go away. These elements need to be deleted properly to free up memory. Hiding elements lead to a **nasty** memory leak. Reddit has enough re-posts, there is no need for RES to introduce duplicates to the feed. One thing all redditors can agree upon is that duplicates == garbage.