* Refactoring get_atoms, improving tokens API
* Remove the feature array stuff from Tokens class, and replace vector with array-based implementation, with padding.
* Upd for refactored Tokens class. Now gets 95.74, 185ms training on swbd_wsj_ewtb, eval on onto_web, Google POS tags.
* Fix padding on tokens
* Add tests for string intern
* Large refactor, particularly to Python API
* Seems to be working after refactor. Need to wire up more POS tag features, and wire up save/load of POS tags.
* Rewriting Lexeme serialization.
* Update lexeme serialization, using a binary file format
* Fix POS tagger, so that it loads correctly. Lexemes are being read in.
* Remove lexemes vector from Lexicon, and the id and hash attributes from Lexeme
* Fix Token after lexeme changes
* Fiddle with data types on Lexeme, to compress them to a much smaller size.
* Small efficiency tweak to lexeme init
* Tighten the interface to Language
* Tighten interfaces
* Upd tests for tighter interface
* Add count_tags functionto pos.pyx, which should probably live in another file. Feature set achieves 97.9 on wsj19-21, 95.85 on onto web.
* Add count_tags functionto pos.pyx, which should probably live in another file. Feature set achieves 97.9 on wsj19-21, 95.85 on onto web.
* Restore id attribute to lexeme, and rename pos field to postype, to store clustered tag dictionaries