Units tests of LexerEngine cover up to TC46

Units testing covers the test case TC1 up to TC46.