github.com/F483/dejavu (go): Package dejavu offers quick detection of already witnessed data.  Limited memory of witnessed data, oldest are forgotten. Library is thread safe. Offers deterministic and probabilistic (over an order of magnatude less memory consuming) implementation.
Keywords: command-line|command-line-tool|deduplication|duplicate-values|duplicates|go|golang|history|memory|probabilistic