Add Lexer tests for edge cases

Also fix some existing tests that after careful reading of the spec
turned out to be wrong.