factorie/factorie — pull request #112 by strubell (2014-01-22 19:48:20+00:00)
Title: Parser fixes + updated OntoNotes loader

Summary
-------------

Our transition-based parser should now get 91.08/92.59 LAS/UAS on OntoNotes (using auto tags and not including punctuation tokens in evaluation) at ~10,000 tokens/sec (parallelized).

Our OntoNotes loader was not compatible with the newer version of OntoNotes that Jinho has been using, so I updated our OntoNotes loader to be compatible with this newer data, which we should use instead of the old data.

The parser changes are combined with the OntoNotes loader changes because one of the modifications I made to the OntoNotes loader was adding parameters to specify whether to load auto or old values, which is used by the updated parser to load auto pos tags and lemmas.

Parser changes
----------------------
* Added and removed features, including grand-head, sibling, 2nd-left- and right-most dependents
* Wrote a test-class for the parser
* Fixed bugs in Token (parse-related), ParseTree and ParserEval
* Fixed feature generation bugs in the parser itself
* Exclude features

OntoNotes loader changes
--------------------------------------
* Old loader is now LoadOntonotesOld; file formats are not compatible. May want to rename/remove this.
* New format has new features and different order of features, and NER data are in BILOU format by default
* Loader now has options for loading either auto or gold tags
* Removed duplicate code between fromFilename and fromLines methods