Support zero-length tokens in the lexer (fixes #688)
Support zero-length empty lexer rules with lexer commands (fixes #687)
Merge pull request #691 from sharwell/fix-688

Improved support for zero-length lexer tokens