final lexer and beginning of parsing table generation