Refactored tokenization into separate module
Refactored tokenization into separate module
simplyfied as_unicode and as_binary functions
Unit test for core as_unicode as_binary functions
First steps toward integrating estner system. Settings do not now have global state in ner system.
Adjusted the trainer and tagger. This is still work in progress.
Finished basic NER functionality
Started writing more user friendly trainer and tagger classes for NER
Ner integration finished. But not covered with unit tests yet