Repository: piskvorky/gensim

Issue:
Feature suggestion: relative cosine similarity for word2vec 
Hi all,
Based on this paper, do you think it worths the effort to implement relative cosine similarity measure? 
https://ufal.mff.cuni.cz/pbml/105/art-leeuwenberg-et-al.pdf

Note that I'm not suggesting this as a potential contributor but as a grateful user.
Thank you,
Viktor



Discussion:
Thanks for request @viplexke, I quickly looked at the article: IMO doesn't look very useful for including it to gensim. Also, the formula for `relative cosine similarity` looks pretty simple (i.e. any person who need this can implement it self).

CC: @gojomo @piskvorky 
It's interesting that it seems to help highlight synonyms, as opposed to other kinds of related words. If it could be done as a single short method in KeyedVectors, I think it'd be a good contribution – even though it's not hard for other to implement, sometimes people only discover new techniques by browing APIs. (Any implementation should cite this origin paper.) 


@gojomo @menshikh-iv How exactly would I go about implementing this "as a single short method in KeyedVectors" (as you say @gojomo ). Excuse the question - I'm very new to Gensim. Thanks!
Section 3.5 of the referenced paper introduced the "relative cosine similarity" measure, which is essentially a measure of how-much-more-similar to word-A that word-B is, compared to the top-N-other-most-similar-to-word-A words. Essentially, it seems they observed that when one word was much "more similar" to a target word than the next N, it was especially likely to be a true synonym. (This "better than the others" was more reliable than any absolute cutoff of cosine-similarity; see the paper for the details and full reasoning.)

So given two words and a top-n value, and a set of word-vectors, this new measure can be calculated.   That naturally suggests a single new method on `KeyedVectors` with a signature like:

    def relative_cosine_similarity(word_a, word_b, topn=10):
        ...

A pull-request that implements this method, matching the definition in the paper, with an explanatory doc-comment (with link to the paper) and some tests (which manage to somewhat confirm expected behavior along the lines of that described in the paper) would be a useful contribution. 
@gojomo Thanks so much! I got it working now. 

> Hi dear @ailsamm I need to implement the same measure, can you provide me the code please since it works with you
> I really need it :(


@rawannasser Sure! How should I send it?
@ailsamm
Thanks so much! 
 I don't know if I can write my email here but maybe you can upload it in your GitHub?
@ailsamm Can you submit your implementation as a pull-request for potential integration to the project?
Hi,
Is anyone working on this?
If not, I would like to take this up.
@jenishah go ahead please :)
If anyone is not working on this can I contribute?
feel free to contribute @rsdel2007
@rsdel2007 yes, please 

Resolving patch:
diff --git a/gensim/models/keyedvectors.py b/gensim/models/keyedvectors.py
--- a/gensim/models/keyedvectors.py
+++ b/gensim/models/keyedvectors.py
@@ -1385,6 +1385,36 @@ def init_sims(self, replace=False):
             logger.info("precomputing L2-norms of word weight vectors")
             self.vectors_norm = _l2_norm(self.vectors, replace=replace)
 
+    def relative_cosine_similarity(self, wa, wb, topn=10):
+        """Compute the relative cosine similarity between two words given top-n similar words,
+        by `Artuur Leeuwenberga, Mihaela Velab , Jon Dehdaribc, Josef van Genabithbc "A Minimally Supervised Approach
+        for Synonym Extraction with Word Embeddings" <https://ufal.mff.cuni.cz/pbml/105/art-leeuwenberg-et-al.pdf>`_.
+
+        To calculate relative cosine similarity between two words, equation (1) of the paper is used.
+        For WordNet synonyms, if rcs(topn=10) is greater than 0.10 then wa and wb are more similar than
+        any arbitrary word pairs.
+
+        Parameters
+        ----------
+        wa: str
+            Word for which we have to look top-n similar word.
+        wb: str
+            Word for which we evaluating relative cosine similarity with wa.
+        topn: int, optional
+            Number of top-n similar words to look with respect to wa.
+
+        Returns
+        -------
+        numpy.float64
+            Relative cosine similarity between wa and wb.
+
+        """
+        sims = self.similar_by_word(wa, topn)
+        assert sims, "Failed code invariant: list of similar words must never be empty."
+        rcs = float(self.similarity(wa, wb)) / (sum(sim for _, sim in sims))
+
+        return rcs
+
 
 class Word2VecKeyedVectors(WordEmbeddingsKeyedVectors):
     """Mapping between words and vectors for the :class:`~gensim.models.Word2Vec` model.


Test patch:
diff --git a/gensim/test/test_keyedvectors.py b/gensim/test/test_keyedvectors.py
--- a/gensim/test/test_keyedvectors.py
+++ b/gensim/test/test_keyedvectors.py
@@ -106,6 +106,29 @@ def test_most_similar_topn(self):
         predicted = self.vectors.most_similar('war', topn=None)
         self.assertEqual(len(predicted), len(self.vectors.vocab))
 
+    def test_relative_cosine_similarity(self):
+        """Test relative_cosine_similarity returns expected results with an input of a word pair and topn"""
+        wordnet_syn = [
+            'good', 'goodness', 'commodity', 'trade_good', 'full', 'estimable', 'honorable',
+            'respectable', 'beneficial', 'just', 'upright', 'adept', 'expert', 'practiced', 'proficient',
+            'skillful', 'skilful', 'dear', 'near', 'dependable', 'safe', 'secure', 'right', 'ripe', 'well',
+            'effective', 'in_effect', 'in_force', 'serious', 'sound', 'salutary', 'honest', 'undecomposed',
+            'unspoiled', 'unspoilt', 'thoroughly', 'soundly'
+        ]   # synonyms for "good" as per wordnet
+        cos_sim = []
+        for i in range(len(wordnet_syn)):
+            if wordnet_syn[i] in self.vectors.vocab:
+                cos_sim.append(self.vectors.similarity("good", wordnet_syn[i]))
+        cos_sim = sorted(cos_sim, reverse=True)  # cosine_similarity of "good" with wordnet_syn in decreasing order
+        # computing relative_cosine_similarity of two similar words
+        rcs_wordnet = self.vectors.similarity("good", "nice") / sum(cos_sim[i] for i in range(10))
+        rcs = self.vectors.relative_cosine_similarity("good", "nice", 10)
+        self.assertTrue(rcs_wordnet >= rcs)
+        self.assertTrue(np.allclose(rcs_wordnet, rcs, 0, 0.125))
+        # computing relative_cosine_similarity for two non-similar words
+        rcs = self.vectors.relative_cosine_similarity("good", "worst", 10)
+        self.assertTrue(rcs < 0.10)
+
     def test_most_similar_raises_keyerror(self):
         """Test most_similar raises KeyError when input is out of vocab."""
         with self.assertRaises(KeyError):