Delayed adding new tokens until after prediction

This fixes the problem of new tokens being immeiately chosen, although
there still is the problem that unseen words are rated very highly. It
makes me wonder if the regret formulation is not the way to go and
explicit priors might be better.