uwdata/termite — pull request #2 by ifalk (2013-11-26 10:36:11+00:00)
Title: Better unicode support

- tokenization option in config: kind of tokenization (whitespace, alpha, alphanumeric or unicode) can now be specified in the config file. Needed whitespace tokenization for unicode input files.

- fixed output when using unicode terms: `str` did not work on unicode strings.