Bring back the best of the old lexer

Figured out that yacc was actually reliant on the whitespace in between tokens.
Adding that in the right place seems to have made this a lot happier, and should
make handling semicolons and optional whitespace a LOT easier.