Initial crack at Cython
supersenseFeatureExtractor.py: optimization - store len on freeze
timeit
refactor DataSet, SupersenseDataSet, and SupersenseFeaturizer into
dataFeaturizer.py; change printWeights() signature; remove maxInstances
arg to train(); {SupersenseDataSet,SupersenseFeaturizer}.reset();
various optimizations
use Cython type inference
slight optimization for printWeights()
IndexedFeatureMap.__repr__()
- refactor train() into train() and decode(), which supports steps
common to the training and test scenarios
- rename command line flags useBIO -> bio, useCostAug -> costAug, and
pass their values through to train() and decode()
- support serialization of the full DiscriminativeTagger instance:
saveModel(), loadModel()
- always pass the data as arguments, don't store it as a member of
DiscriminativeTagger
- when creating features from the training data, store tag frequencies
- remove duplicate legalTagBigram() implementation
memoize legalTagBigram() for a huge speedup in --bio mode
remove debugging print of weights
debugging print comment
avoid comparing sentence index to None
Ensure sizes are ints when growing DP tables
i_viterbi(): iterative Viterbi (pruning not implemented yet). Also,
--excludeFirstOrder option.
quash EVIL BUG
pruning for iterative Viterbi!
iterative Viterbi: actually enable pruning and apply some other
optimizations. but much slower than conventional Viterbi when
considering label bigrams (with --bio or without --excludeFirstOrder);
perhaps first-order scores for degenerate label pairs need to be cached
as well
i_viterbi(): properly cache bigram scores when there are degenerate
labels
morph.py: don't rely on current working directory in path to morphology files
supersenseFeatureExtractor.py: fix paths