a more detailed lexer plus FILE input and output