Repository: piskvorky/gensim

Issue:
Negative exponent with value -1 (minus one) raises error when loading Doc2Vec model
<!--
**IMPORTANT**:

- Use the [Gensim mailing list](https://groups.google.com/forum/#!forum/gensim) to ask general or usage questions. Github issues are only for bug reports.
- Check [Recipes&FAQ](https://github.com/RaRe-Technologies/gensim/wiki/Recipes-&-FAQ) first for common answers.

Github bug reports that do not include relevant information and context will be closed without an answer. Thanks!
-->

#### Problem description

I try to vary the value of the negative exponent parameter. When I use a value of -1, training works fine, saving the model too, but when I try to load the model afterwards with Doc2Vec.load() it raises the error "ValueError: Integers to negative integer powers are not allowed."

This is due to the following line: https://github.com/RaRe-Technologies/gensim/blob/266a01455ade51a93a08dba5950e87b4d98e0724/gensim/models/word2vec.py#L836

Here, numpy does not raise an integer by the power of another, but negative integer.

I guess this could be solved by converting the exponent to a float in this case?





Discussion:
Yes - I'd expect if `count` (which may have only recently became a `numpy`-typed int rather than a generic Python int) were converted to a float (either Python `float()` or `np.float32()`), this error would be eliminated. 

Can you share the full error stack traceback? I'm curious why this only shows on `.load()`, not earlier – and while the fix is likely just a few characters, ideally we'd want a new minimal unit test that triggers the failure, pre-fix, as well.

(Also, given how rarely `ns_exponent` is changed, especially to negative values, there's at least a little risk other downstream assumptions about types/ranges may be fragile - so it'd be great to know, post-fix, if everything seems to continue working as desired.)
@edg-stg I would love to work on this issue. Is it available to assign?
@pranav1610 - this is quite likely a 1-line fix, but would ideally be accompanied by a test case that fails before the fix is applied, & succeeds after. You could just prepare those in a PR & ask it to be considered for inclusion - no pre-assignment of the issue is required. 

Resolving patch:
diff --git a/gensim/models/word2vec.py b/gensim/models/word2vec.py
--- a/gensim/models/word2vec.py
+++ b/gensim/models/word2vec.py
@@ -833,11 +833,11 @@ def make_cum_table(self, domain=2**31 - 1):
         train_words_pow = 0.0
         for word_index in range(vocab_size):
             count = self.wv.get_vecattr(word_index, 'count')
-            train_words_pow += count**self.ns_exponent
+            train_words_pow += count**float(self.ns_exponent)
         cumulative = 0.0
         for word_index in range(vocab_size):
             count = self.wv.get_vecattr(word_index, 'count')
-            cumulative += count**self.ns_exponent
+            cumulative += count**float(self.ns_exponent)
             self.cum_table[word_index] = round(cumulative / train_words_pow * domain)
         if len(self.cum_table) > 0:
             assert self.cum_table[-1] == domain


Test patch:
diff --git a/gensim/test/test_doc2vec.py b/gensim/test/test_doc2vec.py
--- a/gensim/test/test_doc2vec.py
+++ b/gensim/test/test_doc2vec.py
@@ -720,6 +720,15 @@ def test_train_warning(self, loglines):
     def test_load_on_class_error(self):
         """Test if exception is raised when loading doc2vec model on instance"""
         self.assertRaises(AttributeError, load_on_instance)
+
+    def test_negative_ns_exp(self):
+        """The model should accept a negative ns_exponent as a valid value."""
+        model = doc2vec.Doc2Vec(sentences, ns_exponent=-1, min_count=1, workers=1)
+        tmpf = get_tmpfile('d2v_negative_exp.tst')
+        model.save(tmpf)
+        loaded_model = doc2vec.Doc2Vec.load(tmpf)
+        loaded_model.train(sentences, total_examples=model.corpus_count, epochs=1)
+        assert loaded_model.ns_exponent == -1, loaded_model.ns_exponent
 # endclass TestDoc2VecModel
 
 
diff --git a/gensim/test/test_fasttext.py b/gensim/test/test_fasttext.py
--- a/gensim/test/test_fasttext.py
+++ b/gensim/test/test_fasttext.py
@@ -762,6 +762,15 @@ def test_vectors_for_all_without_inference(self):
         predicted = vectors_for_all['responding']
         assert np.allclose(expected, predicted)
 
+    def test_negative_ns_exp(self):
+        """The model should accept a negative ns_exponent as a valid value."""
+        model = FT_gensim(sentences, ns_exponent=-1, min_count=1, workers=1)
+        tmpf = get_tmpfile('fasttext_negative_exp.tst')
+        model.save(tmpf)
+        loaded_model = FT_gensim.load(tmpf)
+        loaded_model.train(sentences, total_examples=model.corpus_count, epochs=1)
+        assert loaded_model.ns_exponent == -1, loaded_model.ns_exponent
+
 
 @pytest.mark.parametrize('shrink_windows', [True, False])
 def test_cbow_hs_training(shrink_windows):
diff --git a/gensim/test/test_word2vec.py b/gensim/test/test_word2vec.py
--- a/gensim/test/test_word2vec.py
+++ b/gensim/test/test_word2vec.py
@@ -1054,6 +1054,15 @@ def test_compute_training_loss(self):
         training_loss_val = model.get_latest_training_loss()
         self.assertTrue(training_loss_val > 0.0)
 
+    def test_negative_ns_exp(self):
+        """The model should accept a negative ns_exponent as a valid value."""
+        model = word2vec.Word2Vec(sentences, ns_exponent=-1, min_count=1, workers=1)
+        tmpf = get_tmpfile('w2v_negative_exp.tst')
+        model.save(tmpf)
+        loaded_model = word2vec.Word2Vec.load(tmpf)
+        loaded_model.train(sentences, total_examples=model.corpus_count, epochs=1)
+        assert loaded_model.ns_exponent == -1, loaded_model.ns_exponent
+
 
 # endclass TestWord2VecModel