Repository: ekzhu/datasketch

Issue:
Implementing MinHash retrieval from keys for MinHashLSHForest
Hi, I've been using the `MinHashLSHForest` class to do some deduplication, and part of that pipeline is to retrieve the top-k items and then estimate the Jaccard similarities with each of those items, obviously this requires reconstructing the `MinHash` object related to the given key returned by `MinHashLSHForest.query()`.

This seems like a decently common use-case since we often screen the results of LSH Forest using the Jaccard Similarity, my question is do you feel that this is common enough to implement such a function as a part of the MinHashLSHForest class?

I've implemented a simple way to recompute the original `hashvalues` array from the `keys` dictionary in `MinHashLSHForest` as follows, I'd be happy to submit a PR but just wanted to know how this aligned with the vision for this package

```python
"""
Takes the list of bytes-like generated by the LSH Forest 
that corresponds to some given key and recovers the hashvalues
which can be converted back into a MinHash to compute Jaccard Similarity

Given a number of prefix trees, L, when we insert a (key, MinHash) pair 
the LSH Forest creates L byteslike items each corresponding to a range of 
hashvalues from the original MinHash object for a given key. Each range is of
size num_perm / L. Therefore here we convert these items from byteslikes back into
arrays of unsigned integers and then concatenate them so that they are in a representation
that we can build a MinHash object with. Namely, we return an array of unsigned integers
of length num_perm that represent hashvalues from each of num_perm hash functions
chosen during the MinHash creation.
"""
def byteslist_to_hashvalues(byteslist):
    hashvalues = np.array([], dtype=np.uint64)
    for item in byteslist:
        # unswap the bytes, as their representation is flipped during storage
        hv_segment = np.frombuffer(item, dtype=np.uint64).byteswap()
        hashvalues = np.append(hashvalues, hv_segment)
    
    return hashvalues
```

where we might call this by using 

```python
lsh = MinHashLSHForest(...)
hv = byteslist_to_hashvalues(lsh.keys[mykey])
mh = MinHash(hashvalues=hv)

# now use mh.jaccard() ...
...
```

A unit test would involve inserting a `MinHash` into `MinHashLSHForest` and then reconstructing it and checking for `jaccard_sim == 1.0`.

Discussion:
123epsilon: @ekzhu Just pushed changes to preallocate the buffer

Patch:
diff --git a/datasketch/lshforest.py b/datasketch/lshforest.py
index 9f3455b..a02569d 100644
--- a/datasketch/lshforest.py
+++ b/datasketch/lshforest.py
@@ -1,5 +1,6 @@
 from collections import defaultdict
 from typing import Hashable, List
+import numpy as np
 
 from datasketch.minhash import MinHash
 
@@ -128,6 +129,30 @@ class MinHashLSHForest(object):
             r -= 1
         return list(results)
 
+    def get_minhash_hashvalues(self, key: Hashable) -> np.ndarray:
+        """
+        Returns the hashvalues from the MinHash object that corresponds to the given key in the LSHForest,
+        if it exists. This is useful for when we want to reconstruct the original MinHash
+        object to manually check the Jaccard Similarity for the top-k results from a query.
+
+        Args:
+            key (Hashable): The key whose MinHash hashvalues we want to retrieve.
+
+        Returns:
+            hashvalues: The hashvalues for the MinHash object corresponding to the given key.
+        """
+        byteslist = self.keys.get(key, None)
+        if byteslist is None:
+            raise KeyError(f"The provided key does not exist in the LSHForest: {key}")
+        hashvalue_byte_size = len(byteslist[0])//8
+        hashvalues = np.empty(len(byteslist)*hashvalue_byte_size, dtype=np.uint64)
+        for index, item in enumerate(byteslist):
+            # unswap the bytes, as their representation is flipped during storage
+            hv_segment = np.frombuffer(item, dtype=np.uint64).byteswap()
+            curr_index = index*hashvalue_byte_size
+            hashvalues[curr_index:curr_index+hashvalue_byte_size] = hv_segment
+        return hashvalues
+
     def _binary_search(self, n, func):
         """
         https://golang.org/src/sort/search.go?s=2247:2287#L49


Test patch:
diff --git a/test/test_lshforest.py b/test/test_lshforest.py
index 77e7bf4..400a9af 100644
--- a/test/test_lshforest.py
+++ b/test/test_lshforest.py
@@ -62,6 +62,18 @@ class TestMinHashLSHForest(unittest.TestCase):
             results = forest.query(data[key], 10)
             self.assertIn(key, results)
 
+    def test_get_minhash_hashvalues(self):
+        forest, data = self._setup()
+        for key in data:
+            minhash_ori = data[key]
+            hashvalues = forest.get_minhash_hashvalues(key)
+            minhash_retrieved = MinHash(hashvalues=hashvalues)
+            retrieved_hashvalues = minhash_retrieved.hashvalues
+            self.assertEqual(len(hashvalues), len(retrieved_hashvalues))
+            self.assertEqual(minhash_retrieved.jaccard(minhash_ori), 1.0)
+            for i in range(len(retrieved_hashvalues)):
+                self.assertEqual(hashvalues[i], retrieved_hashvalues[i])
+
     def test_pickle(self):
         forest, _ = self._setup()
         forest2 = pickle.loads(pickle.dumps(forest))