factorie/factorie — pull request #86 by strubell (2013-10-23 01:00:36+00:00)
Title: Speed increase in ForwardPOS, test methods in pos, ner, parser

This consists of changes I made while doing benchmarking.

Mostly minor, adding methods that allow easy testing (returning accuracy and speed given a list of sentences), and parameters to the trainers to take a directory from which to grab training/testing files.

Integrated Alex's optimization to ForwardPOS that considers token-tag pairs seen > 1000 times in training as sure.

I also added a FileUtils class in util that contains a method that gets a list of file names with the given suffix from a directory. This is useful for getting training/testing files from a directory.