nltk/nltk — Issue #55: nltk_data python package
Author: alexrudnick
Created: 2012-01-17 06:24:59+00:00
State: open

Create a new Python package trunk/nltk/nltk_data, containing modules for
building the models that get distributed from trunk/nltk_data. This will
make it possible to rebuild models when the associated code is changed.
Users will also be able to see how the models were trained, and be able to
modify the parameters if necessary.

Migrated from http://code.google.com/p/nltk/issues/detail?id=301

earlier comments

StevenBird1 said, at 2009-02-20T07:25:23.000Z:

I've done this for a new tagsets package.

edloper said, at 2009-02-20T18:22:40.000Z:

This seems like a good approach; I'll migrate the code that trains some of the
existing models to this directory.

Some of that code depends on having access to corpora that aren't distributed with
NLTK. E.g., currently named_entity.py has the very ugly path
"/tmp/ace.old/data/ace.dev/text" hard-coded into it. I propose that we use
"/usr/share/corpora" as a standard location for corpora that are needed to train the
models, but which we don't distribute with them (such as the ACE corpus, or the
complete treebank).

StevenBird1 said, at 2009-02-20T21:48:34.000Z:

"/usr/share/corpora" as the standard place for non-distributed corpora sounds fine.

On the output side, we might want to package up the output files, and have a Makefile
that puts them in the right place in the working copy (cf the "publish" directives in
doc/Makefile).