markfasheh/duperemove — pull request #6 by orezpraw (2014-07-11 22:16:05+00:00)
Title: Fix poor performance on huge quantities of files, don't dedup already deduped files.

Pros: Fast enough to be used on whole systems (tested on 2M+ inodes) if you have enough RAM.

Cons: Uses more memory.
Doesn't always find the best possible dedup (but it always finds a good one, in practice this seems to be insignificant)