Introduce unit testing.

### Lexer:
* Expand public interface of `lexer` module to allow testing.
* Use a `DummyLogger` if no logger is provided.
* Rewrite lexing of char and string literals:
  * Simplify and annotate regexes; build master regexes piece by piece.
  * Use python built-ins to return properly escaped values.
  * Terminate string literals with null byte.
  * Fix return value in literals ending with a single or double quote.

### Error:
* Add `DummyLogger` class.

### Tests:
* Rename `test` folder to `tests`.
* Add unit tests for `lexer` module.
* Drop functional tests replaced by unit tests.

### misc:
* Add unit tests to build.
* Beautify code.