Expanded documentation for LexerEngine class.