Moved token definitions and lexer to a dedicated file.