Tokenizer should ignore non-token characters