nltk/nltk — Issue #165: nltk.MaxentClassifier.train can't optimize with scipy 0.9.0
Author: alexrudnick
Created: 2012-01-17 06:27:33+00:00
State: open

NLTK Version: 2.0.1rc1

What steps will reproduce the problem? (e.g. include Python source code)

1. Install scipy 0.9.0.

2. The demo reproduces the problem:

import nltk.classify.maxent
nltk.classify.maxent.demo()

The problem is not unique to the demo. It holds whenever nltk.MaxentClassifier.train is called using one of the scipy algorithms. Everything works fine with the other algorithms, or with scipy 0.8.0.

What is the expected output? What do you see instead?

The demo should train and assess a names classifier. Instead, it displays "Grad eval #0" and never moves off of that.

Please use labels and text to provide additional information.

I'm running Python 2.6.6, and I was trying with the corresponding "2.6" versions of numpy and scipy.

I'm on a Mac.

I eventually settled on NumPy 1.4.1 and SciPy 0.8.0, which seem to do well together and get along with NLTK. NumPy 1.5.1 works with SciPy 0.8.0, but it delivers a lot of deprecation warnings.

Migrated from http://code.google.com/p/nltk/issues/detail?id=688

earlier comments

leon@dcs.shef.ac.uk said, at 2011-07-14T14:55:26.000Z:

I'm sometimes seeing this with NumPy 1.3.0, SciPy 0.7.0 in both NLTK 2.0b9 and the current revision (2.0.1rc1 r8802):

Training classifier
training: 1326 test: 200
Grad eval #0
/usr/lib/python2.6/dist-packages/numpy/lib/utils.py:108: DeprecationWarning: rmatvec is deprecated
warnings.warn(str1, DeprecationWarning)
/usr/lib/python2.6/dist-packages/numpy/lib/utils.py:108: DeprecationWarning: matvec is deprecated
warnings.warn(str1, DeprecationWarning)
norm of gradient = 1.76242441379e-14
Function eval # 0
dual is 0.0
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 0
Function evaluations: 1
Gradient evaluations: 1