Probabilistic near-duplicate detection using simhash
Approximate nearest neighbors: towards removing the curse of dimensionality