Initial commit
* Add build/setup stuff
* Add initial docs stuff
* Add gitignore
* Add ext stuff, while I figure out how to get it working as a different project
* Initial commit. Tests passing for punctuation handling. Need contractions, file transport, tokenize function, etc.
* Possessive test passing
* Fixes to tokenization. Now segment sequences of the same punctuation.
* Rejigged tests. Working possessives, but no other contractions
* Reading in tokenization rules correctly. Passing tests.
* Working tokenization. en doesn't match PTB perfectly. Need to reorganize before adding more schemes.
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals
* Tests passing for reorganized version
* Tests passing for reorganized version
* Compile string_tools module
* Begin work on full PTB-compatible English tokenization
* Generalize tokenization rules to capitals
* Remove cpp files
* Upd ptb tokenization rules
* Fix error in tokenization rules