nltk/nltk — Issue #140: Use data with annotations in ISO GrAF format in NLTK
Author: alexrudnick
Created: 2012-01-17 06:26:59+00:00
State: open

We have developed a corpus reader for annotations in ISO GrAF format that rely on a Python API for GrAF. We are very anxious to have our MASC corpus usable in NLTK, and hopefully distributed with it. MASC includes multiple types of annotations--parts have about a dozen different ones--so it should be a very interesting dataset for students and other NLTK users. The Open American National Corpus is also in GrAF format, so users will be able to use that data with its annotations as well.

We are not sure exactly how to best incorporate all of this into NLTK. You can look at our current code by going to
http://www.anc.org/~ide/NLTK
The main question we have is: is it ok that the Python API to GrAF has to be included in NLTK?

We'd appreciate it if you could look over what we have and let us know what else may need to be done, or if you need more information. We would be very grateful to see MASC included in the NLTK distribution as soon as possible!

Best,
Nancy Ide

Migrated from http://code.google.com/p/nltk/issues/detail?id=630

earlier comments

StevenBird1 said, at 2011-02-17T05:05:34.000Z:

Thanks for this contribution! It can be distributed with NLTK Contrib right away.

However, in order to be included in core NLTK (and then maintained by us, and included with NLTK's standard distribution), the code would need to follow our coding practices, which includes requirements on naming, documentation, package structure, new-style classes, and so forth, as set out here:

http://code.google.com/p/nltk/wiki/DevelopersGuide

Note that the corpus reader would need to implement NLTK's corpus reader API, in order to be incorporated with NLTK. The API is documented here:

http://nltk.googlecode.com/svn/trunk/doc/howto/corpus.html

I expect it would be 2-3 days of work for a programmer to do this. Questions could be posted on the nltk-dev mailing list.

I hope this doesn't present too big a challenge. It would be good to have this functionality in NLTK.

StevenBird1 said, at 2011-03-21T23:34:46.000Z:

Issue 658 has been merged into this issue.

smatysik said, at 2011-05-05T21:12:16.000Z:

Code for the corpus reader and the GrAF API package are attached, I tried to commit these to the nltk_contrib subversion, but think I need access first.
Thanks!