Repository: piskvorky/gensim

Issue:
gensim.similarities.Similarity merges results from shards incorrectly (LSI model)
If "num_best" is used, `gensim.similarities.Similarity` runs the query against each of the shards (MatrixSimilarity objects) and then merges the results.

MatrixSimilarity uses `matutils.full2sparse_clipped()` to pick "num_best" results which sorts by the absolute value.

gensim.similarities.Similarity on the other hand, just uses `heapq.nlargest` (in `__getitem__`) to merge the results from each of the shards. So negative sims are either pushed down the list or cut off completely.


Discussion:
You're right. This seems an inconsistency / bug introduced by PR #811.

@avoskresensky in your use case – is `abs` desirable, or is the raw similarity desirable?
If my assumption that it's the absolute values that indicate similarity is correct, then I don't care that much about the raw values. `abs` values are okay. That's how we ended up "fixing" this inconsistency in out current implementation: we're just abs'ing the results in our `Similarity.query_shards` override.

But if I'm wrong and sign of the sim values does mean some form of dissimilarity, then the whole idea of sorting while merging becomes questionable.
Negative similarity values (close to -1) mean "opposite vectors", which is can also be interpreted as "very similar", because it indicates a strong semantic connection between the two inputs. Cossim values around 0.0 mean perpendicular = unrelated.

Whether "opposite" really means "similar" or not will depend on how you generated your vectors, and what you're using them for. That's why I ask: this is an "upstream" question, not a technical one.

Either way: can you open a PR that fixes Similarity to sort on `abs` values as well? We definitely want this logic consistent across the similarity classes. Cheers.
@avoskresensky Ping on this. Are you able to make a PR?
Not in the next couple of weeks.

Resolving patch:
diff --git a/gensim/similarities/docsim.py b/gensim/similarities/docsim.py
--- a/gensim/similarities/docsim.py
+++ b/gensim/similarities/docsim.py
@@ -233,6 +233,30 @@ def query_shard(args):
     return result
 
 
+def _nlargest(n, iterable):
+    """Helper for extracting n documents with maximum similarity.
+
+    Parameters
+    ----------
+    n : int
+        Number of elements to be extracted
+    iterable : iterable of list of (int, float)
+        Iterable containing documents with computed similarities
+
+    Returns
+    -------
+    :class:`list`
+        List with the n largest elements from the dataset defined by iterable.
+
+    Notes
+    -----
+    Elements are compared by the absolute value of similarity, because negative value of similarity
+    does not mean some form of dissimilarity.
+
+    """
+    return heapq.nlargest(n, itertools.chain(*iterable), key=lambda item: abs(item[1]))
+
+
 class Similarity(interfaces.SimilarityABC):
     """Compute cosine similarity of a dynamic query against a corpus of documents ('the index').
 
@@ -539,7 +563,7 @@ def convert(shard_no, doc):
             if not is_corpus:
                 # user asked for num_best most similar and query is a single doc
                 results = (convert(shard_no, result) for shard_no, result in enumerate(shard_results))
-                result = heapq.nlargest(self.num_best, itertools.chain(*results), key=lambda item: item[1])
+                result = _nlargest(self.num_best, results)
             else:
                 # the trickiest combination: returning num_best results when query was a corpus
                 results = []
@@ -548,7 +572,7 @@ def convert(shard_no, doc):
                     results.append(shard_result)
                 result = []
                 for parts in zip(*results):
-                    merged = heapq.nlargest(self.num_best, itertools.chain(*parts), key=lambda item: item[1])
+                    merged = _nlargest(self.num_best, parts)
                     result.append(merged)
         if pool:
             # gc doesn't seem to collect the Pools, eventually leading to


Test patch:
diff --git a/gensim/test/test_similarities.py b/gensim/test/test_similarities.py
--- a/gensim/test/test_similarities.py
+++ b/gensim/test/test_similarities.py
@@ -30,6 +30,7 @@
 from gensim.similarities import UniformTermSimilarityIndex
 from gensim.similarities import SparseTermSimilarityMatrix
 from gensim.similarities import LevenshteinSimilarityIndex
+from gensim.similarities.docsim import _nlargest
 from gensim.similarities.levenshtein import levdist, levsim
 
 try:
@@ -532,6 +533,11 @@ def testChunksize(self):
         self.assertTrue(numpy.allclose(expected, sims))
         index.destroy()
 
+    def testNlargest(self):
+        sims = ([(0, 0.8), (1, 0.2), (2, 0.0), (3, 0.0), (4, -0.1), (5, -0.15)],)
+        expected = [(0, 0.8), (1, 0.2), (5, -0.15)]
+        self.assertTrue(_nlargest(3, sims), expected)
+
 
 class TestWord2VecAnnoyIndexer(unittest.TestCase):