Constellation/escodegen — pull request #133 by getify (2013-09-22 19:54:29+00:00)
Title: Support (and output) arbitrary whitespace/comments attached as "extras" to any node of the AST

I have need for the ability to attach extra information to the standard AST. The extra information is whitespace and comments that appear before (leading) or after (trailing) any node in the AST (such as Identifier, Expression, etc).

**NOTE:** I have implemented this (see the attached PR) in such a way that only AST's which have this extra data/nodes in them will activate the new behavior, so all existing usages of existing standard AST's should be unaffected by my changes (either functionally or performance).

------------
The reason for this new feature is I'm building [a tool](http://github.com/getify/esre) which parses a JS file, custom modifies its styling/formatting by adding such information to the normal AST, then needs to re-generate the code from that AST.

In other words, I plan to turn on escodegen's `compact` config, so that none of its automatic whitespace/identation/etc happens, but instead have my own "control" on exactly how whitespace is outputted by having that data encoded into the AST via `extras` annotations.

Moreover, the existing mechanism in *escodegen* for `leadingComments` and `trailingComments` is unfortunately insufficient, because there are many places where comments can validly appear which are not handled by that mechanism. It only tracks comments attached before or after a statement. For example, `foo(/* blah */);`, the comment there isn't leading or trailing to a statement, and is thus not trackable by the current comments mechanism.

By contrast, though, the "extras" mechanism I've made allows any node in the AST to have leading or trailing "extras", which can either be whitespace, single-line comments, or multiline comments. This allows comments to be output in any of the valid locations, rather than only before/after statements as currently. Perhaps my suggested mechanism here can obviate the need for the existing comments tracking, but note that my attached PR didn't make that bold assumption, so it leaves that stuff entirely alone (for now).

My attached PR here implements this feature as I suggest here in this discussion. It also adds a number of test cases to the "AST" test to verify that it's working correctly.

Here's an example (which is also in the test suite). This example AST shows several places where an `extras: { .. }` key is added to various different nodes, which can contain either/both `leading` or `trailing` extras, which respectively can have either `Whitespace`, `LineComment` and/or `MultilineComment` nodes:

```js
{
type: 'Program',
body: [{
type: 'ExpressionStatement',
expression: {
type: 'CallExpression',
callee: {
type: 'Identifier',
name: 'foo',
extras: {
trailing: [
{
type: 'MultilineComment',
value: '2'
},
],
}
},
arguments: [
{
type: 'Literal',
value: 4,
raw: '4',
extras: {
leading: [
{
type: 'LineComment',
value: '3'
}
],
trailing: [
{
type: 'MultilineComment',
value: '5'
},
// NOTE: testing custom line-comment markers,
// in support of HTML-style line-comment markers
// Ref: http://javascript.spec.whatwg.org/#comment-syntax
{
type: 'LineComment',
value: '6',
marker: '