erikrose/parsimonious — pull request #40 by bloff (2014-05-09 10:00:21+00:00)
Title: Added the ability of parsimonious to parse lists of tokens.

Not sure you wanted this in parsimonious, but I needed it. I implemented a language with Python-like indentation; this indentation is usually implemented in the lexer. I am using PLY.

However, LALR(1) grammars suck, so I wanted something with more lookahead. But all PEG grammars insisted on doing the lexing themselves (bah!).

But parsimonious was so simply written that it was trivial to change to do what I needed.

Now if you call grammar.parse on a list, it will assume that the list is a list of "tokens": a token is an instance with a member called "type".

To match a token of a given type, you may use the syntax %TOKEN_TYPE% in the grammar.

Example usage:

grammar = Grammar("""foo = %TOKEN1% %TOKEN2%""")
assert(grammar.parse([Token("TOKEN1"), Token("TOKEN2")]) is not None)

I've already tested it on my project, it works just fine :-)