hltfbk/Excitement-Open-Platform — pull request #387 by gilnoh (2014-04-15 06:59:09+00:00)
Title: Better tokenizer for TreeTagger pipeline (TreeTaggerLN) 

Exchanged the simplest (breakiteratorsegmenter) tokenizer to a model based (open-nlp) tokenizer.
Unit tests are now un- @ignore -d. (hope Jenkins is all okay)