Factor code to use a simple wrapper lexer

Handling indentation, logical lines, duplicate line removal, etc. seems
to result in awful code if all of it is done in the lexer, so this
commit begins a refactor to run the lexer on each logical line, with a
separate custom lexer for handling the lines themselves.

Also add a test for duplicate line removal, currently failing.
Refactor the lexer code to use generators

Instead of using PLY's interface for getting tokens from a lexer, we
convert to and from generators, so that we can use yield.