fixed the lexer, now it returns the correct tokens, while being memory efficent.