Completed primary lexer features and unittests.