版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。
海量数据挖掘Mining Massive Datasets(MMDs) -Jure Leskoveccourses学习笔记之 Locality-Sensitive Hashing(LSH) 局部敏感哈希
局部敏感哈希LSH简介
LSH聚焦在很可能相关的事物上,而避免对不可能是我们想要查找的事物进行检测,这个和原始的hash有一定相似,也就是直接查找到我们想要的记录而不用遍历整个数据集。A fundamental problem of scale: If we have even a million sets, the number of pairs of sets is half a trillion.We don't have the resources to compare them all, so we need some magic defor,focus us on the pairs that are likely to be highly similar,never looking at the vast majority of pairs.
shingling, a way to convert the informal notion of similar documents into a formal test for similarity of sets.
min hashing, allows us to replace a large set by a much smaller list of values.the similarity of the small lists,called signatures,predicts the similarity of the whole sets.
locality sensitive hashing,is another bit of magic.we are pointed right at the similar pairs without having to wade through the morass of all pairs.find simil