apresta/tagger — Issue #6: ZeroDivisionError: float division by zero
Author: jonaslejon
Created: 2012-01-11 14:46:21+00:00
State: open

$ python build_dict.py -o data/dict4.pkl -s stopwords.txt /ebs/category/Internet.txt
Processing corpus...
ls -lah /ebs/category/Internet.txt
Processing stopwords...
Building dictionary...
Traceback (most recent call last):
File "build_dict.py", line 134, in
build_dict_from_files(output_file, corpus, stopwords_file, verbose=True)
File "build_dict.py", line 115, in build_dict_from_files
dictionary = build_dict(corpus, stopwords, measure)
File "build_dict.py", line 72, in build_dict
dictionary[w] = math.log(corpus_size / (cnt + 1)) / scale
ZeroDivisionError: float division by zero
$ ls -lah /ebs/category/Internet.txt
-rw-r--r-- 1 ubuntu ubuntu 2.3M 2012-01-10 14:23 /ebs/category/Internet.txt