use nltk_data or nltk_data_3 as the directory name for data files depending on the current python version
use http://nltk.github.com/nltk_data/ or http://nltk.github.com/nltk_data_3/ as the download url depending on the current python version
reuse code from nltk.data to get the default download path, if no existing nltk data directories are present
fixed chunk/regexp.py chunk parser usage examples
use _im_class instead of im_class to avoid conflict with nose.suite.ContextSuiteFactory.ancestory seeing the compat module as a class
fixed xml encoding detection bug, can be reproduced by the following code: "from nltk import book"
open gzip files in text mode in the tadm classifier trainer
encoding option for doctest files in runtests.py
PY3: fix nltk.classify.tadm
rename compat._im_class to compat.get_im_class because it is not private
improve word_tokenize speed

Runs roughly twice as fast on my test set of 40000 sentences. According
to kernprof (http://pythonhosted.org/line_profiler/), the culprits of the
previously bad performance were

    r"([^' ])('ll|'re|'ve|n't|) "

and

    r"([^' ])('LL|'RE|'VE|N'T|) "

which were matching all words due to the | at the end.

Also removed some useless whitespace handling that was being undone by the
str.split just before return.

The contractions handling is now the slowest part @~20% of the time.
removed stale test framework
Lowest Common Hypernyms Change & Tests

- Set lowest_common_hypernyms to use max_depth by default
- Updated wup_similarity() to use min_depth with LCH
- Added a tests file for wordnet (only tests lowest_common_hypernyms()
at the moment)
- Added an additional LCH example to demo() which illustrates a case
where a target synset is also the LCH for itself and another synset
doctest and docstring updates
Explicitly mention setuptools 0.6 in install.rst

Setuptools merged with Distribute in 0.7. I think setup.py needs to be migrated to 0.7?
nltk works for setuptools 0.8 already; resolved issue #435
fixed failing doctests, caused by floating point precision issue