nltk/nltk — Issue #105: Tokenizers
Author: alexrudnick
Created: 2012-01-17 06:26:11+00:00
State: open

NLTK's word tokenizers aren't as good as they could be (see issue 128 and
issue 376). We need high performance tokenizer(s) for multiple languages.

Migrated from http://code.google.com/p/nltk/issues/detail?id=524

earlier comments

StevenBird1 said, at 2010-03-12T22:58:44.000Z:

Issue 376 has been merged into this issue.