nltk/nltk — Issue #45: BracketParsedCorpusReader's error-recovery behavior could be improved
Author: alexrudnick
Created: 2012-01-17 06:24:45+00:00
State: open

Mike M said:

If you look at the _parse() function in BracketParsedCorpusReader
(http://nltk.org/doc/api/nltk.corpus.reader.bracket_parse-
pysrc.html#BracketParseCorpusReader._parse) it writes to the standard
output whenever it encounters an unexpected situation.

Does anyone know a quick and dirty way to prevent this??

Also, wouldn't it make more sense to throw an exception here instead
of writing to stderr??

Then Edward Loper said:

The motivation behind *not* raising an exception here is robustness -- if
your corpus file isn't quite valid (i.e., a single paren is off) then
throwing an exception here would make it impossible to read any portion of
the corpus file. This can be problematic for parsed corpora that consist
of a single big file.

The best solution here would probably be to default to throwing an
exception; but to allow an argument to the corpus reader that specifies
whether it should try to recover from bad trees. If a bad tree is
detected, it might be better to use the warnings module than to just print
to stderr.

Migrated from http://code.google.com/p/nltk/issues/detail?id=218