a little bit of cleanup
tokenizer support for utf8 patterns
build ngram_query binary
KenLM 58da338b
russian abbrevs
proper topological ordering
Completely untested Raw read call
Clean up dead code
Scored grammars from online extraction.  Don't trust them yet.
Online grammars now diff with incremental suffix array (except lex, TODO)
Bilexical scores for online rules
For now, don't use online bilex counts
fcount -> paircount
Baum Welch training for HMMs
fix possible link problem
new metric
add thing
Merge remote-tracking branch 'upstream/master'