Separate 'tokenize' unit tests from 'main'.