Don't delete lines following lex'd `ignore:end`. Fixes gh-1879

Previously, lines would be cleared (set to the empty string) even after `ignore:end`
is found, if the comment was not processed first.

This change performs some minor comment processing in the lexer itself in order to
solve this problem.

Closes #1879