Refactor and test lex API. Simplify standalone use of lex module. Fix couple of bugs. Beautify test code.

### Lex:
* [will fill]

### Lex test:
* [will fill]
I believe silencing warnings/errors should never be the default behaviour.