making lexer immutable and updating tokenizer to use immutable lexer.
trying to keep things fucnctional! :)