tenderlove/rkelly — pull request #20 by nene (2013-06-05 17:02:02+00:00)
Title: Additional speedup of Tokenizer

I did some further tuning of Tokenizer. Made it about 2 x faster, but had to change the code around considerably more - mostly reordering things. See the commit message for details.

Also eliminated few drops of trailing whitespace.