Package: lark-dynamic
Dynamic grammar generator forLark parsing toolkitWhy?While writing EBNF by hand is a preferred approach for most use cases (because of rich syntax), it's only viable for static grammars.If your grammar is variable (e.g. based on some kind of configuration), it should be built with code on the fly.As perlark#439, no API for grammar exists, so building a string is only solution (it's pretty fast though).This README implies you are familiar with Lark and Lark EBNF.Basic usagefromlark_dynamicimport*g=Grammar()g.DIGIT=g.NONZERO|"0"g.NONZERO=Range("1","9")g.INTEGER=makeBoolVariable("zero_leading_numbers",true=Many(g.DIGIT),false=(g.NONZERO,Some(g.DIGIT)))g.FLOAT[2]=Group(g.INTEGER,".",Some(g.DIGIT))|(".",Many(g.DIGIT))g.number=Alias.integer(g.INTEGER)|Alias.float(g.FLOAT)print(g.generate(zero_leading_numbers=False))print(g.generate(zero_leading_numbers=True))Which outputs:zero_leading_numbers=False:DIGIT: NONZERO | "0"
NONZERO: "1".."9"
INTEGER: ( NONZERO ( DIGIT )* )
FLOAT.2: ( INTEGER "." ( DIGIT )* ) | ( "." ( DIGIT )+ )

number: INTEGER -> integer | FLOAT -> floatzero_leading_numbers=TrueDIGIT: NONZERO | "0"
NONZERO: "1".."9"
INTEGER: ( DIGIT )+
FLOAT.2: ( INTEGER "." ( DIGIT )* ) | ( "." ( DIGIT )+ )

number: INTEGER -> integer | FLOAT -> floatGetting startedInstall with:python -m pip install lark-dynamicTo create a grammar, you need to createGrammarobject:fromlark_dynamicimport*# it is recommended to create grammar in a separate file, so * import is fineg=Grammar()# short name is convenientThen write rules and terminals:# this is a terminal (uppercase)g.NUMBER=RegExp(r'[0-9]+')g.SIGN=Literal("+")|"-"# first (or second) string needs to be wrapped into a Literal to support |g.test=Maybe(g.SIGN),Many(g.TEST)ReferenceAll code snippets use this setup:fromlark_dynamicimport*g=Grammar()GrammarMain object, stores rules, terminals, templates and directivesTo create a rule, you can use a shorthand:g.any_lowercase_identifier=...# rule contents hereSame for terminal:g.ANY_UPPERCASE_IDENTIFIER=...# terminal contents hereTo create a template (name{arg1, arg2} = ...) you need to use[]:g.template_name[g.arg1,g.arg2]=...# template contents hereDirectives have different syntaxes, so no special shorthand is present:g.make_directive("name","directive contents here")To build grammar, you need to call.generate(**context):g.generate(some_variable=True)# variables are discussed belowVariableSince the purpose of the module is to create grammars dynamically, this class exists.It takes a function, which accepts one argument (context) - a dictionary of everything passed to.generate().Function sould return anything renderable (e.g. a token or a tuple of tokens):definteger_term(context):ifcontext.get("zero_leading_numbers"):returnMany(g.DIGIT)returng.NONZERO,Some(g.DIGIT)g.DIGIT=g.NONZERO|"0"g.NONZERO=Range("1","9")g.INTEGER=Variable(integer_term)g.FLOAT[2]=Group(g.INTEGER,".",Some(g.DIGIT))|(".",Many(g.DIGIT))g.number=Alias.integer(g.INTEGER)|Alias.float(g.FLOAT)(this is an example from Basic usage section rewritten to use Variable)BoolVariable and makeBoolVariableCommon use case for Variable is to render different content based on a boolean key (such as in Basic usage example).BoolVariablemakes it more convenient:definteger_term(zln):ifzln:returnMany(g.DIGIT)returng.NONZERO,Some(g.DIGIT)g.DIGIT=g.NONZERO|"0"g.NONZERO=Range("1","9")g.INTEGER=BoolVariable(integer_term,key="zero_leading_numbers")g.FLOAT[2]=Group(g.INTEGER,".",Some(g.DIGIT))|(".",Many(g.DIGIT))g.number=Alias.integer(g.INTEGER)|Alias.float(g.FLOAT)makeBoolVariablegoes even further, providing arguments forTrueandFalsevalues:g.DIGIT=g.NONZERO|"0"g.NONZERO=Range("1","9")g.INTEGER=makeBoolVariable("zero_leading_numbers",true=Many(g.DIGIT),false=(g.NONZERO,Some(g.DIGIT)))g.FLOAT[2]=Group(g.INTEGER,".",Some(g.DIGIT))|(".",Many(g.DIGIT))g.number=Alias.integer(g.INTEGER)|Alias.float(g.FLOAT)LiteralUsed for literal strings:g.hello=Literal("Hello"),Literal("World")yields:hello: "Hello" "World"In most cases you can just use a string. ExplicitLiteral(...)is needed to support|(seeCombinators/Option)Using regexp flags on Literalg.hello=Literal("Hello").si,Literal("World").iyields:hello: "Hello"si "World"iCombinatorsMaybe (aka QuestionMark)Corresponds to?combinator in Lark:g.hello="Hello",Maybe(","),"World",QuestionMark("!")# QuestionMark is just an alias for Maybeyields:hello: "Hello" (",")? "World" ("!")?Some (aka Star)Corresponds to*combinator in Lark:g.hello="Hello",Maybe(","),"World",Some("!")yields:hello: "Hello" (",")? "World" ("!")*Many (aka Plus)Corresponds to+combinator in Lark:g.POSITION=Literal("two number 9s")|"a number 9 large"|"a number 6 with extra dip"|"a number 7"|"two number 45s"|"one with cheese"|"a large soda"g.order="I'll have ",g.position,Many(", ",g.position),Maybe(", and ",g.position)g.position=g.POSITIONyields:POSITION: "two number 9s" | "a number 9 large" | "a number 6 with extra dip" | "a number 7" | "two number 45s" | "one with cheese" | "a large soda"

order: "I'll have " position ( ", " position )+ ( ", and " position )?
position: POSITIONOptional (aka Brackets)Corresponds to[...]combinator in Lark:g.hello="Hello",Optional(","),"World",["!"]yields:hello: "Hello" [","] "World" ["!"]Can also be written as[...](as a list, basically), but be aware of|usage rules (below)OptionCorresponds to|combinator in Lark:g.hello="Hello",Maybe(","),(Option("World","dlroW"),),Some("!")# note Option is placed inside a tuple, (explained below)yields:hello: "Hello" (",")? ("World" | "dlroW") ("!")*Can also be written as|:g.hello="Hello",Maybe(","),(Literal("World")|"dlroW",),Some("!")# note Literal (explained below)Option usage rules|priority in Lark is higher than in Python:Maybe(","), Option("World", "dlroW"), Maybe("!")yields(",")? "World" | "dlroW" ("!")?which, in turn, is equal to(","? "World") | ("dlroW" "!"?)You should wrap option with any other combinator (e.g.Group()or tuple. Remember that(x)is not a tuple) or just useOptionG.The reason why defaultOptiondoesn't do this is that wrapping top-level option in parens would break alias creation.|operator would not work with python built-in types (str, list or tuple)You need at least oneToken(rule, combinator, literal etc.) on either side of|for it to work.Group (aka Parens)Corresponds to(...)combinator in Lark:g.hello="Hello",Group(Maybe(","),"World"),(Some("!"),)yields:hello: "Hello" ((",")? "World") (("!")*)Can also be written as(...)(as a tuple, basically), but be aware of|usage rulesSome,ManyandMaybealso will render in parenthesis by default to avoid combinator collision (e.g. to avoid rendering(("!")+)?as"!"+?which is invalid).SomeSeparated and ManySeparatedCorrespond tocontent (sep, content)*andcontent (sep, content)+patterns, respectively:g.POSITION=Literal("two number 9s")|"a number 9 large"|"a number 6 with extra dip"|"a number 7"|"two number 45s"|"one with cheese"|"a large soda"g._COMMA=",",Maybe(g.WS)g.WS=" "g.order="I'll have ",SomeSeparated(g._COMMA,g.position),Maybe(g._COMMA,"and ",g.position)g.position=g.POSITIONyields:POSITION: "two number 9s" | "a number 9 large" | "a number 6 with extra dip" | "a number 7" | "two number 45s" | "one with cheese" | "a large soda"
_COMMA: "," ( WS )?
WS: " "

order: "I'll have " (position ( _COMMA position )*) ( _COMMA "and " position )?
position: POSITIONRegExpCorresponds to Lark regexp syntax (/content/flags):g.WORD=RegExp(r"\w+")# flags are provided as a second argument (e.g. RegExp(r"\w+", "is"))g.some_phrase=g.WORD,Maybe(","),g.WORD,Some("!")yields:WORD: /\w+/

some_phrase: WORD ( "," )? WORD ( "!" )*RepeatCorresponds to Lark repeat syntax:item ~ nanditem ~ n..m:g.three_lemons=Repeat("Lemon",3)g.three_to_five_apples=Repeat("Lemon",[3,5])yields:three_lemons: ( "Lemon" ) ~ 3
three_to_five_apples: ( "Lemon" ) ~ 3..5PrerenderedIf you want to insert a prerendered content into a rule/terminal/template, you can usePrerendered:g.hello="Hello",Prerendered('((",")? "World")'),Some("!")yields:hello: "Hello" ((",")? "World") ("!")*EmptySometimes (when using Variable) you'll need to render nothing.It can be achieved withPrerendered(""), or its alias,EmptyModifiersRule and terminal modifiers (such as?and!) can be used withModifierclass (works only on top level of rule/terminal):g.parens=Modifier.INLINE_SINGLE("(",g.word,")")g.word=g.WORDg.WORD=RegExp(r"\w+")yields:WORD: /\w+/

?parens: "(" word ")"
word: WORDPriorityTo use terminal/rule priority, you can use[priority]notation (similar to templates, but with a number):g.DIGIT=g.NONZERO|"0"g.NONZERO=Range("1","9")g.INTEGER=g.NONZERO,Some(g.DIGIT)g.FLOAT[2]=Group(g.INTEGER,".",Some(g.DIGIT))|(".",Many(g.DIGIT))g.number=Alias.integer(g.INTEGER)|Alias.float(g.FLOAT)yields:DIGIT: NONZERO | "0"
NONZERO: "1".."9"
INTEGER: NONZERO ( DIGIT )*
FLOAT.2: ( INTEGER "." ( DIGIT )* ) | ( "." ( DIGIT )+ )

number: INTEGER -> integer | FLOAT -> floatAdvanced usageGrammarobject can be edited after its creation.First, you need to get a grammar wrapper:wrapper = grammar.use_wrapper()Get a definitiong=Grammar()g.HELLO="Hello"wrapper=g.use_wrapper()print(wrapper.get_def("HELLO"))# prints# RuleDef(#     "Hello"# )Replace definition contentsg=Grammar()g.HELLO="Hello"wrapper=g.use_wrapper()wrapper.replace("HELLO","World")print(wrapper.get_def("HELLO"))# prints# RuleDef(#     "World"# )Edit definitiong=Grammar()g.HELLO="Hello"wrapper=g.use_wrapper()wrapper.edit("HELLO",priority=10)print(wrapper.get_def("HELLO"))# prints# RuleDef(#     "World"# )Extend definitiong=Grammar()g.HELLO="Hello"wrapper=g.use_wrapper()wrapper.extend("HELLO","World")print(wrapper.get_def("HELLO"))# prints# RuleDef(#     Option(#         "Hello"#         "World"#     )# )Getting all definitions / directivesYou can get dictionaries with rules, terminals, templates, or a list of all directives with a wrapper:g=Grammar()wrapper=g.use_wrapper()wrapper.ruleswrapper.terminalswrapper.templateswrapper.directivesWhy a wrapper?Because grammar object itself is used to create definitions with arbitrary names. Creating methods with common names would easily create a problem:g=Grammar()g.edit="EDIT",Some("blah")g.save="SAVE",Some(Option("lorem","ipsum","dolor","sit","amet"))g.command=Option(g.edit,g.save)# g.edit used here as a rule, not method