Additional speedup of Tokenizer

I did some further tuning of Tokenizer. Made it about 2 x faster, but had to change the code around considerably more - mostly reordering things. See the commit message for details.

Also eliminated few drops of trailing whitespace.