Support future regular expression flags

https://code.google.com/p/esprima/issues/detail?id=557
Fix the tests for new ES6 regex flags.

Keep the tracking of value for regex literal, even when it is null.
Once this needs to run on an ES6-compliant environment, the check
needs to be revisited.

Also adjust SM AST compatibility test so that the special regex property
is ignored during the comparison.

https://code.google.com/p/esprima/issues/detail?id=557
Ensure that the tokens carry the new regex property.

https://code.google.com/p/esprima/issues/detail?id=557
Update the baseline syntax tree and tokens for test libraries.

This accounts for the new `regex` object in regular expression literals.

https://code.google.com/p/esprima/issues/detail?id=557
Stub for change documentation.

https://code.google.com/p/esprima/issues/detail?id=600
Document syntax tree variants in version 1.0.

https://code.google.com/p/esprima/issues/detail?id=600
Document the comments array in version 1.0.

https://code.google.com/p/esprima/issues/detail?id=600
Remove the upper limit of hex digits in Unicode code point escapes

E.g. `'\u{00000000000000000001D306}' == '\u{1D306}'`.

https://code.google.com/p/esprima/issues/detail?id=612