Separating tokening and lexing so we can be stricter about the syntax we accept.