Split tokenizer up into helper functions

In an attempt to make it more readable, and seperate parsing
logic from tokenizing logic.