antlr/antlr4 — pull request #747 by sharwell (2014-10-26 20:53:36+00:00)
Title: Fix handling of lexer accept positions for zero-length tokens and at EOF

In the beginning (prior to supporting zero-length tokens), the `captureSimState` method was called prior to consuming the final matched character of a token. The input index was recorded directly, but a fixup was later applied in `accept` by calling `consume`. The introduction of support for zero-length tokens resulted in inconsistent state when `captureSimState` was called, so a fixup was applied to the input index for just that case when it occurred. Unfortunately the fixup did not properly handling the `line` and `charPositionInLine` fields.

This pull request makes the following changes.

1. `captureSimState` is called with the input positioned exactly where it should be in all cases.
2. The recorded `index`, `line`, and `charPositionInLine` values can therefore be applied directly with no fixup either when they are recorded or when they are later applied.