beautify-web/js-beautify — pull request #530 by bitwiseman (2014-09-16 23:37:00+00:00)
Title: Initial refactor to fully tokenize before formatting

We've been dealing with limitations from incomplete parsing (#200) for ages.

Instead attempting full parsing, this change only tokenizes fully before starting formatting. This should open the door to simplifying parsing and formatting rules.

For example, to find an object literal currently, we have to check repeatedly later on for a matching pattern. With this change we could determine as a block is opened whether to treat it as on object-literal or not (for features like #114, #314, #370).

Next steps could also include a second refactor move the tokenizing code outside of the beautifier for clarity, cleaning up the factoring for the recognizers for each token, and refactoring the beautification rules to make use of the information now available to them.

@evocateur, @einars - please take a look when you have a chance. Any feedback welcome.