ariya/esprima — pull request #272 by benjamn (2014-07-29 01:10:55+00:00)
Title: When consumeSemicolon fails to find a semicolon, don't consume anything.

When an `ExpressionStatement` is not terminated by a semicolon, the `consumeSemicolon` function terminates the `range`/`loc` of the node at the beginning of the next node, including any whitespace or comments trailing after the `ExpressionStatement` node. Instead, it should revert back to the end of the previous node.

If you look carefully at the test changes included in this commit (all of which I made by hand), I think you'll agree they make a lot of sense. The only ones I'm not sure about are the end-of-`Program` positions: for instance, the `Program` location for `'\n42\n\n'` now only includes the `42`, and not any of the whitespace. This was already true for the starting position, but now it is also true for the ending position.

This inclusion of trailing comments/whitespace by `consumeSemicolon` has been a very long-standing issue that I dealt with in https://github.com/benjamn/recast using the [`LocationFixer`](https://github.com/benjamn/recast/blob/6feb25024c6883bcfa027b95d3b5fa476ac179e9/lib/parser.js#L72-L101) class (first introduced in September 2012), but I want to stop using that class for performance reasons, and I've come to realize I can't correctly fix all cases anyway (I can rewind over whitespace, but not so easily over comments, without implementing something like `unskipComment`), so I believe this logic is better implemented in esprima.js.