overhaul of ngram code: allow for backoff models and sub-sequence prob
fixed some remaining ngram compilation errors