fix bug in failure diags, kill old "ngram"s
add some assertions of unicodiness

And a couple of tidy-ups of dictionary usage ("k in d" instead of "k in
d.keys()" for example).
make token literals in tests be unicodes

Not really important, except that now assertions of unicodiness in
VariationProposer will not be triggered by the literals from the tests.
fix bug in error diags
shutdown "ngram" better, fix bug in err diags