make chi-square feature selection work again
change tokenizer for collocation search
make everything unicode
Merge pull request #20 from escherba/m_preprocess

make wordpuncttokenizer default instead of treebank