piskvorky/gensim — Issue #68: distributed LDA:  'NoneType' object is not subscriptable 
Author: piskvorky
Created: 2011-11-29 12:41:52+00:00
State: open
Labels: bug

Distributed LDA sometimes throws

> 2011-11-23 21:19:58,881 : INFO : resetting worker #11
> Exception in thread Thread-19
> Traceback (most recent call last):
> File "/bb/blaw/PYTHON/lib/python2.7/threading.py", line 530, in
> __bootstrap_inner
> self.run()
> File "/bb/blaw/PYTHON/lib/python2.7/threading.py", line 483, in run
> self.__target(*self.__args, **self.__kwargs)
> File
> "/bb/blaw/tools/Python-2.7.1/lib/python2.7/site-packages/gensim-0.8.2-py2.7 .egg/gensim/models/lda_worker.py",
> line 58, in requestjob
> self.processjob(job)
> File
> "/bb/blaw/PYTHON/lib/python2.7/site-packages/gensim-0.8.2-py2.7.egg/gensim/ utils.py",
> line 49, in _synchronizer
> result = func(self, *args, **kwargs)
> File
> "/bb/blaw/tools/Python-2.7.1/lib/python2.7/site-packages/gensim-0.8.2-py2.7 .egg/gensim/models/lda_worker.py",
> line 65, in processjob
> self.model.do_estep(job)
> File
> "/bb/blaw/PYTHON/lib/python2.7/site-packages/gensim-0.8.2-py2.7.egg/gensim/ models/ldamodel.py",
> line 371, in do_estep
> gamma, sstats = self.inference(chunk, collect_sstats=True)
> File
> "/bb/blaw/PYTHON/lib/python2.7/site-packages/gensim-0.8.2-py2.7.egg/gensim/ models/ldamodel.py",
> line 324, in inference
> expElogbetad = self.expElogbeta[:, ids]
> TypeError: 'NoneType' object is not subscriptable

Reported by several people, e.g. by Dale here: http://groups.google.com/group/gensim/browse_thread/thread/a57c68342e58c7b2

Check the flow around the `clear()` method in distributed LDA, which is likely culprit.