Added unit tests for lexer and fixed a bug.