Repository: piskvorky/gensim

Issue:
Wrong return types in docstrings for wv.evaluate_word_analogies and wv.evaluate_word_pairs
This is the documented return type for `KeyedVectors.evaluate_word_analogies`:
https://github.com/RaRe-Technologies/gensim/blob/6a4424d3e465dea0afcddc4f4e2410cf0f52dba1/gensim/models/keyedvectors.py#L1070

The dicts in the second element of the return tuple have as values either a string or a list of 4-tuples. For instance:
```
{
    'section': 'family',
    'correct': [('he', 'she', 'dad', 'mom'), ...],
    'incorrect': [('man', 'woman', 'groom', 'bride'), ...]
}
```
So I believe the return value of `evaluate_word_analogies` is actually `Tuple[float, List[Dict[str, Union[str, List[Tuple[str, str, str, str]]]]]` (in PEP-484 style annotation).

Then, in `KeyedVectors.evaluate_word_pairs`, the documented return type is:
https://github.com/RaRe-Technologies/gensim/blob/6a4424d3e465dea0afcddc4f4e2410cf0f52dba1/gensim/models/keyedvectors.py#L1284

Here, the second and third elements of the return tuple are actually pairs of floats, since they are the values returned from `scipy.stats.pearsonr` and `spearmanr`, which return both the correlation and a p-value. I don't know whether the p-value should be discarded or the documented return type be updated to `(float, (float, float), (float, float))`.


Discussion:
Hello @Stigjb, thanks for the report

> So I believe the return value of evaluate_word_analogies is actually Tuple[float, List[Dict[str, Union[str, List[Tuple[str, str, str, str]]]]] (in PEP-484 style annotation).

I would be happy to use PEP484 annotation style, but I'm not sure how to use it in docstring (we use sphinx & napoleon plugin for numpy-style docstrings), also, we can't annotate arguments directly in the definition using `typing` backport lib (because of no new syntax support). Have you any thought about it?

Of course, this is a bug in current return type and should be fixed, but maybe you have some good ideas how to apply PEP484 in our case?



Resolving patch:
diff --git a/gensim/models/keyedvectors.py b/gensim/models/keyedvectors.py
--- a/gensim/models/keyedvectors.py
+++ b/gensim/models/keyedvectors.py
@@ -1067,8 +1067,12 @@ def evaluate_word_analogies(self, analogies, restrict_vocab=300000, case_insensi
 
         Returns
         -------
-        (float, list of dict of (str, (str, str, str))
-            Overall evaluation score and full lists of correct and incorrect predictions divided by sections.
+        score : float
+            The overall evaluation score on the entire evaluation set
+        sections : list of dict of {str : str or tuple of (str, str, str, str)}
+            Results broken down by each section of the evaluation set. Each dict contains the name of the section
+            under the key 'section', and lists of correctly and incorrectly predicted 4-tuples of words under the
+            keys 'correct' and 'incorrect'.
 
         """
         ok_vocab = [(w, self.vocab[w]) for w in self.index2word[:restrict_vocab]]
@@ -1281,9 +1285,13 @@ def evaluate_word_pairs(self, pairs, delimiter='\t', restrict_vocab=300000,
 
         Returns
         -------
-        (float, float, float)
-            Pearson correlation coefficient, Spearman rank-order correlation coefficient between the similarities
-            from the dataset and the similarities produced by the model itself, ratio of pairs with unknown words.
+        pearson : tuple of (float, float)
+            Pearson correlation coefficient with 2-tailed p-value.
+        spearman : tuple of (float, float)
+            Spearman rank-order correlation coefficient between the similarities from the dataset and the
+            similarities produced by the model itself, with 2-tailed p-value.
+        oov_ratio : float
+            The ratio of pairs with unknown words.
 
         """
         ok_vocab = [(w, self.vocab[w]) for w in self.index2word[:restrict_vocab]]


Test patch: