Package: petitparser
PetitParser for PythonGrammars for programming languages are traditionally specified statically. They are hard to compose and reuse due to ambiguities that inevitably arise. PetitParser combines ideas from scannnerless parsing, parser combinators, parsing expression grammars (PEG) and packrat parsers to model grammars and parsers as objects that can be reconfigured dynamically.PetitParser was originally implemented in Smalltalk. This Python implementation (and parts of the README as well) is mostly a direct translation of theJava version.TutorialWriting a simple grammarWriting grammars with PetitParser is as simple as writing python code. For example to write a grammar that can parse identifiers that start with a letter followed by zero or more letters or digits is defined as follows:frompetitparserimportcharacterascident=c.letter()&(c.letter()|c.digit()).star()Parsing some inputTo parse a string we can use theParser.parsemethod:id1=ident.parse('yeah')id2=ident.parse('f12')The method returns aResultwhich is either aSuccessor aFailure(can be checked withResult.is_success/Result.is_failureproperties). If it's a success (like above) we can get the result withResult.valueproperty:print(id1.value)# ['y', ['e', 'a', 'h']]print(id2.value)# ['f', ['1', '2']]If we try to parse somehing invalid, we get an instance ofFailureid3=ident.parse('123')print(id3.message)# letter expectedprint(id3.position)# 0If you are only interested if a string matches or not, you can useParser.accept()method:print(ident.accept('foo'))# Trueprint(ident.accept('123'))# FalseDiferent kinds of parsersPetitParser provide a large set of ready-made parser that you can compose to consume and transform arbitrarily complex languages. The terminal parsers are the most simple ones. We've already seen a few of those:petitparser.character.of('a')parses a single charactera.petitparser.string.of('abc')parses the stringabc.petitparser.character.any()parses any characterpetitparser.character.digit()parses any digit (usingstr.isdigit)petitparser.character.letter()parses any leter (usingstr.isalpha)petitparser.character.word()parses any letter or figit (usingstr.isalnum)Other parsers are available inpetitparser.characterandpetitparser.stringmodules.So instead of using the letter and digit predicate, we could have written our identifier parser like this:ident=c.letter()&c.word().star()Next set of parsers are used to combine other persers together:p1.seq(p2)orp1 & p2: parsesp1followed byp2p1.or_(p2)orp1 | p2: pasresp1, and if that fails parsesp2(ordered choice)p.star()parsespzero or more timesp.plus()parsespone or more timesp.optional()parsespif possiblep[m:n]parsespbetweenm(default 0) andn(default unlimited) timesp.and_()or+pparsesp, but does not consume inputp.not_()or-pparsespand succeeds if that fails, but does not consume input.p.end()parsespand succeeds at the end of input.Note:some methods are suffixe with an underscore to keep their original names, and to not conflict with the Python keywordsTo attach an action or transformation to a parser we can use the following methods:p.map(lambda x: ...)performs the transformation given by the function.p.pick(n)returns then-th element of the listpreturns.p.flatten()creates a string from the result ofp.p.token()creates aTokenfrom the result ofp.p.trim()trims whitespace before and afterp.To return the string of the parsed identifier, we can modify our parser like this:ident=(c.letter()&c.word().star()).flatten()To conveniently find all matches in a given input string you can useParser.matches_skipping():print(ident.matches_skipping('foo 123 bar4'))# ['foo', 'bar4']Writing more complicated grammarNow we are able to write a more complicated grammar for evaluating simple arithmetic expressions. Within a file we start with the grammar for a number (actually an integer):number=c.digit().plus().flatten().trim().map(int)Then we define the productions for addition and multiplication in order of precedence. Note that we instantiate the productions with undefined parsers upfront, because they recursively refer to each other. Later on we can resolve this recursion by setting their reference:frompetitparserimportSettableParserterm=SettableParser.undefined()prod=SettableParser.undefined()prim=SettableParser.undefined()term.set((prod&c.of('+').trim()&term).map(lambdax:x[0]+x[2])|prod)prod.set((prim&c.of('*').trim()&prod).map(lambdax:x[0]*x[2])|prim)prim.set((c.of('(').trim()&term&c.of(')').trim()).map(lambdax:x[1])|number)To make sure that our parser consumes all input we wrap it with theend()parser into the start production:start=term.end()That's it, now we can test our parser and evaluator:print(start.parse('1 + 2 * 3').value)# 7print(start.parse('(1 + 2) * 3').value)# 9As an exercise we could extend the parser to also accept negative numbers and floating point numbers, not only integers. Furthermore it would be useful to support subtraction and division as well. All these features can be added with a few lines of PetitParser code.Using the Expression BuilderWriting such expression parsers is pretty common and can be quite tricky to get right. To simplify things, PetitParser comes with a builder that can help you to define such grammars easily. It supports the definition of operator precedence; and prefix, postfix, left- and right-associative operators.frompetitparserimportExpressionBuilderbuilder=ExpressionBuilder()Then we define the operator-groups in descending precedence. The highest precedence are the literal numbers themselves. This time we accept floating point numbers, not just integers. In the same group we add support for parenthesis (note that the wrapper action receives 3 arguments, including both wrapper parts):builder.group()\.primitive((c.digit().plus()&(c.of('.')&c.digit().plus()).optional()).flatten().trim().map(float))\.wrapper(c.of('(').trim(),c.of(')').trim(),lambda_l,x,_r:x)Then come the normal arithmetic operators. Note, that the action blocks receive both of the terms and the parsed operator in the order they appear in the parsed input, and the operator:# negation is a prefix operatorbuilder.group()\.prefix(c.of('-'),lambda_op,x:-x)# power is right-associativebuilder.group()\.right(c.of('^'),lambdalhs,_op,rhs:lhs**rhs)# multiplication and addition are left-assicoativebuilder.group()\.left(c.of('*'),lambdalhs,_op,rhs:lhs*rhs)\.left(c.of('/'),lambdalhs,_op,rhs:lhs/rhs)builder.group()\.left(c.of('+'),lambdalhs,_op,rhs:lhs+rhs)\.left(c.of('-'),lambdalhs,_op,rhs:lhs-rhs)Finally we can build the parser:parser=builder.build().end()After executing the above code we get an efficient parser that correctly evaluates expressions like:print(parser.parse('-8').value)# -8.0print(parser.parse('1+2*3').value)# 7print(parser.parse('1*2+3').value)# 5print(parser.parse('8/4/2').value)# 1print(parser.parse('2^2^3').value)# 256Using Grammar DefinitionUsing theGrammarDefinitionclass is a bit different than in other PetitParser implementations, and is done so to make it simpler to write grammars. It uses some metaclass trickery to allow simple definitions of complex grammars.Here is a lambda expression grammar in PEG syntax:expression  ::= variable / abstraction / application
variable    ::= letter word*
abstraction ::= '\' variable '.' expression
application ::= '(' expression expression ')'

start       ::= expression EOFAnd here it is translated into PetitParser Grammar DefinitionfrompetitparserimportGrammarDefinition,reffrompetitparserimportcharacterascclassLambdaGrammar(GrammarDefinition):start=ref('expression').end()expression=(ref('variable')|ref('abstraction')|ref('application'))variable=(c.letter()&c.word().star()).flatten().trim()abstraction=(c.of('\\').trim()&ref('variable')&c.of('.').trim()&ref('expression'))application=(c.of('(').trim()&ref('expression')&ref('expression')&c.of(')').trim())LicenseThe MIT License, seeLICENSE