Repository: piskvorky/gensim

Issue:
ZeroDivisionError in Doc2Vec.build_vocab
When I call `model.build_vocab(docs_iterator)` sometimes I got `ZeroDivisionError`:

```
Traceback (most recent call last):
  File "patient_notes_doc2vec.py", line 45, in <module>
    model.build_vocab(docs_iterator)
  File "/data2/aromanov/virt_env/theano/lib/python3.4/site-packages/gensim/models/word2vec.py", line 495, in build_vocab
    self.scan_vocab(sentences, trim_rule=trim_rule)  # initial survey
  File "/data2/aromanov/virt_env/theano/lib/python3.4/site-packages/gensim/models/doc2vec.py", line 634, in scan_vocab
    interval_rate = (total_words - interval_count) / (default_timer() - interval_start)
ZeroDivisionError: float division by zero
```

A few lines above in doc2vec.py `interval_start` is defined as `interval_start = default_timer()` and in the for loop we have

```
interval_rate = (total_words - interval_count) / (default_timer() - interval_start)
```

Perhaps my system is too fast :smiley: 

ZeroDivisionError in Doc2Vec.build_vocab
When I call `model.build_vocab(docs_iterator)` sometimes I got `ZeroDivisionError`:

```
Traceback (most recent call last):
  File "patient_notes_doc2vec.py", line 45, in <module>
    model.build_vocab(docs_iterator)
  File "/data2/aromanov/virt_env/theano/lib/python3.4/site-packages/gensim/models/word2vec.py", line 495, in build_vocab
    self.scan_vocab(sentences, trim_rule=trim_rule)  # initial survey
  File "/data2/aromanov/virt_env/theano/lib/python3.4/site-packages/gensim/models/doc2vec.py", line 634, in scan_vocab
    interval_rate = (total_words - interval_count) / (default_timer() - interval_start)
ZeroDivisionError: float division by zero
```

A few lines above in doc2vec.py `interval_start` is defined as `interval_start = default_timer()` and in the for loop we have

```
interval_rate = (total_words - interval_count) / (default_timer() - interval_start)
```

Perhaps my system is too fast :smiley: 



Discussion:
Too fast... or the clock too coarse! 

This definitely shouldn't happen; I'll likely add a +0.001 (or similar). Then the rate-estimate may occasionally be off, but the code won't error. 

Thanks for the report!

Also reported by @kikohs in #529.

@gojomo Pinging this issue. Is it an easy fix that can be include in this month's release?

Too fast... or the clock too coarse! 

This definitely shouldn't happen; I'll likely add a +0.001 (or similar). Then the rate-estimate may occasionally be off, but the code won't error. 

Thanks for the report!

Also reported by @kikohs in #529.

@gojomo Pinging this issue. Is it an easy fix that can be include in this month's release?


Resolving patch:
diff --git a/gensim/models/doc2vec.py b/gensim/models/doc2vec.py
--- a/gensim/models/doc2vec.py
+++ b/gensim/models/doc2vec.py
@@ -626,7 +626,7 @@ def scan_vocab(self, documents, progress_per=10000, trim_rule=None):
         document_no = -1
         total_words = 0
         min_reduce = 1
-        interval_start = default_timer()
+        interval_start = default_timer() - 0.00001  # guard against next sample being identical
         interval_count = 0
         vocab = defaultdict(int)
         for document_no, document in enumerate(documents):
diff --git a/gensim/models/word2vec.py b/gensim/models/word2vec.py
--- a/gensim/models/word2vec.py
+++ b/gensim/models/word2vec.py
@@ -807,7 +807,7 @@ def job_producer():
             thread.start()
 
         example_count, trained_word_count, raw_word_count = 0, 0, word_count
-        start, next_report = default_timer(), 1.0
+        start, next_report = default_timer() - 0.00001, 1.0
 
         while unfinished_worker_count > 0:
             report = progress_queue.get()  # blocks if workers too slow
@@ -844,7 +844,7 @@ def job_producer():
         elapsed = default_timer() - start
         logger.info(
             "training on %i raw words (%i effective words) took %.1fs, %.0f effective words/s",
-            raw_word_count, trained_word_count, elapsed, trained_word_count / elapsed if elapsed else 0.0)
+            raw_word_count, trained_word_count, elapsed, trained_word_count / elapsed)
         if job_tally < 10 * self.workers:
             logger.warn("under 10 jobs per worker: consider setting a smaller `batch_words' for smoother alpha decay")
 
@@ -968,7 +968,7 @@ def worker_loop():
         self.clear_sims()
         logger.info(
             "scoring %i sentences took %.1fs, %.0f sentences/s",
-            sentence_count, elapsed, sentence_count / elapsed if elapsed else 0.0)
+            sentence_count, elapsed, sentence_count / elapsed)
         return sentence_scores[:sentence_count]
 
     def clear_sims(self):


Test patch: