antlr/antlr4 — pull request #505 by ziuchkovski (2014-03-21 01:56:15+00:00)
Title: Expose TokenSources from ParserRuleContext and TokenStreamRewriter

TL; DR -- Add toSource() methods on ParserRuleContext and TokenStreamRewriter to stream portions of the parse tree and to stream rewrite operations. I'm using this for a c-preprocessor-like implementation.

This PR adds the following:

1. Ability to retrieve the Parser instance associated with ParserRuleContext objects.
2. A new TokenSource implementation, StreamSource, that wraps a TokenStream as a new TokenSource. It can wrap the full stream or an Interval portion of the stream. Returned Tokens are copied from the source stream so that changes don't affect the original Tokens.
3. A new toSource() method on ParserRuleContext that combines (1) and (2) above to return a context node as a TokenSource.
4. Methods on TokenStreamRewriter to accept rewrite operations as Tokens.
5. A new toSource() method on TokenStreamRewriter that combines all of the above to stream rewrites as a TokenSource.

I'm currently using these features in a C-preprocessor-like implementation that makes multiple replacement passes on the tokens underneath different context nodes. By adding the toSource() methods above, the macro replacement is simplified. It takes a TokenSource as the input and returns a TokenSource as output, recursing and iterating as many times as needed, each with TokenSource input and output. Since the individual tree nodes can be streamed as a TokenSource, the replacements can be made in isolated portions of the tree. The replacement results are collected at the end from the full tree and then passed to a different Lexer/Parser for compiling.

_This PR has a lot of changes, so I'm quite open to suggestions/feedback_