Now it also stores the characters that created the tokens, because you'll need those to build NFAs.