simplify the filter logic
add eof support for lexer
tweak
simplify token
simplify tokens further
simplify lexer token
finish the token simplification
tweak
renaming finished