sentence indexer
unfinished indexing using lucene
Factor out the model object into a separate class. Needs more factorization still
Move outputStats to the PerceptronModel
Refactor another method
Add a flag for retrainAfterCutoff
Move the code to repeat the training deeper into the train() call
Treebank loading is now refactored
Factor out EmergencyTransition
Oops... need to include the PerceptronModel in the codebase
midway between indexing using lucene
Merge branch 'master' into sonal
Merge branch 'master' into sonal
make surfacepattern class with array of Token objects; still working on indexing generalizable objects; surfacepattern is not compiling
Factor out objects which are used at inference or training time but not anywhere else
SurfacePattern is now sequence of Token s, inverted index is working, lucene is yet to test. batch processing is not yet tested
lucene index is working! todo: save and load the index from disk
saving and loading index; bug fix for batch process sentences
regex file patterns; ensuredir in saving index
making it back compatible?