improves docstring
simplifies GeneralLexer
removes duplicated tests
restores testEmptyInput
disables some lexer tests