Package: grammpy-transforms
grammpy-transformsTHIS PACKAGE IS DEPRECATED, USEgrammpyLIBRARY INSTEAD.Version: 1.2.5Package for transforming grammpy grammars.Currently only small subset of operations are implement.Subset of methods for each type of grammar will be stored in separate object (currently only ContextFree is supported).For each method, you can decide to modify or copy the grammar. Default behaviour is to copy grammar before each modification.
You can disable it for every method by passingtransform_grammar=Trueas parameter.Regular grammarsNot implemented.Context-Free grammarsOnly methods allowing to transform grammar into Chomsky normal form.Removing of useless symbolsIncluding removing of unreachable symbols and symbols, thats not generate.fromgrammpy_transformsimportContextFreenew_g=ContextFree.remove_unreachable_symbols(g)new_g=ContextFree.remove_nongenerating_nonterminals(g)new_g=ContextFree.remove_useless_symbols(g)ContextFree.is_grammar_generating(g)Epsilon rules eliminationMethod create new rules that will replace rules with nonterminal rewritable to epsilon rule.
You can also only search  for nonterminals, that are in rule with epsilon on the right side.fromgrammpyimport*fromgrammpy_transformsimportContextFreeclassOldRules(Rule):rules=[([A],[B,C]),([B],[EPS])]ContextFree.find_nonterminals_rewritable_to_epsilon(g)# list of nonterminalsnew_g=ContextFree.remove_rules_with_epsilon(g)classNewRules(Rule):rules=[([A],[B,C]),([A],[C])]assertnew_g.have_rule(NewRules)isTrueLibrary creating own type of rule, so you can backtrack the changes.
The type isContextFree.EpsilonRemovedRule.classCreatedRule(Rule):rule=([A],[C])created=new_g.get_rule(CreatedRule)assertcreated.from_rule.rule==([A],[B,C])assertcreated.replace_index==0assertissubclass(created,ContextFree.EpsilonRemovedRule)Removing of unit rulesAs with epsilon rules removing, method for removing unit rules create new ones as so as removing invalid ones.
You can transform the grammar or just find reachable symbols.fromgrammpyimport*fromgrammpy_transformsimportContextFreeclassOldRules(Rule):rules=[([A],[B]),([B],[C]),([B],[0]),([C],[1])]reach=ContextFree.find_nonterminals_reachable_by_unit_rules(g)# instance of ContextFree.UnitSymbolRechablingResultsassertreach.reach(A,B)isTrueassertreach.reachables(A)==[A,B,C]path=reach.path_rules(B,C)# list of unit rulesassertpath[0].rule==([B],[C])new_g=ContextFree.remove_unit_rules(g)classNewRules(Rule):rules=[([A],[0]),([A],[1]),([B],[0]),([C],[1])]assertnew_g.have_rule(NewRules)isTrueMethod creates own type of rule (ContextFree.ReducedUnitRule) and you can backtrack the tranformations.classAto1Rule(Rule):rule=([A],[1])created=new_g.get_rule(Ato1Rule)assertcreated.by_rules[0].rule==([A],[B])assertcreated.by_rules[1].rule==([B],[C])assertcreated.end_rule.rule==([C],[1])Transformation to Chomsky normal formMethod transfer grammar into Chomsky normal form.This operations create a lot of own types to allow easy backtracking of transformations.Base classes areContextFree.ChomskyNonterminalandContextFree.ChomskyRule, that are base classes for other.As nonterminals method useContextFree.ChomskyTermNonterminalthat represent nonterminal rewritable to terminal (A->a). Nonterminal have propertyfor_term, where it stores terminal (as Terminal class).
Second class isContextFree.ChomskyGroupNonterminal, that represent group of symbols (for example in rule A->BCD will this nonterminal represent CD). This nonterminal have propertygroup, where it stores list of symbols, that represent.For rules method create list of classes, where each class have different meaning:ContextFree.ChomskySplitRule: Represent rule, that was split to contain only two symbols. In propertyfrom_ruleis stored original rule.ContextFree.ChomskyRestRule: Represent right part after splitting of rule. As previous, infrom_ruleproperty is stored original rule.
When splitting, ChomskySplitRule and ChomskyRestRule represent original whole rule:A->BCDE ==> A->BX and X->CDE.ContextFree.ChomskyTerminalReplaceRule: This class is used in situations, where rule contains nonterminal with terminal. Rule is transformed into state, where terminal is replace with nonterminal rewritable to that terminal.
Class havefrom_ruleproperty that stores original rule andreplace_indexproperty, that indicate which terminal were replace.ContextFree.ChomskyTermRule: It is class for rule, that directly rewrite nonterminal to terminal.Inverse operationsEliminating of epsilon rules, removing of unit rules and transforming into Chomsky normal form have their inverse operations.
They are implemented onInverseContextFreeclass.That functions needs just root nonterminal of parsed tree. They then traverse the parse tree and replace rules created by transformations by their original equivalent.The operations needs to be perform in opposite order, that transformations occurs.Split rulesBecause Grammar class split rules, which represents more than two rules,
there is need for algorithm, that replace splitted rule with the original one.
This algorithm is implemented onInverseCommonclass assplitted_rulesstatic method.This call must be call as the last one. Also, you dont need to call this, if all of your rules have just single rule defined.Algorithm is for now implemented only for Context-Free grammars.In the following release of grammpy library, splitted rules should behave same as their original counterparts.
This method will than reflect it and may have empty implementation in the future.HelpersLibrary provide from version 1.2.0 classes, that helps with parsed tree manipulation and traversing.ClassManipulationcan replace specific rule, nonterminal or terminal with different one.
The new element will be added into parsed tree and correctly connected to rest of the elements.fromgrammpy_transformsimportManipulation# ...parsed=cyk(...)Manipulation.replaceNode(parsed,MyNewNonterminal())Manipulation.replaceRule(parsed.to_rule,MyNewRule())# or use type deductionsManipulation.replace(parsed,MyNewNonterminal())Manipulation.replace(parsed.to_rule,MyNewRule())Second class isTraversing. It contains static methods for post order and pre order traversing.
Methods traverse throught nonterminals, terminals and even the rules. If you want to traverse just nonterminals, use thefilterbuildin function.fromgrammpy_transformsimportTraversing# ...Traversing.postOrder(parsed)Traversing.preOrder(parsed)You can create your own traversing path by callingtraversestatic method.
Method accept root of the parsed tree and function accepting current traversing node and callback.
Passed method must call callback with every node you want to traverse.fromgrammpy_transformsimportTraversing# ...defpost_order_traversing(elem,callback):ifisinstance(elem,Nonterminal):forchinelem.to_rule.to_symbols:yieldcallback(ch)yieldchTraversing.traverse(parsed,post_order_traversing)Alternatively, you can usetraverseSeparatedstatic method, that call different functions for nonterminals, terminals and rules.defpostOrder(root):deftravRule(item,callback):resp=[callback(ch)forchinitem.to_symbols]returnfunctools.reduce(operator.add,resp,[])+[item]deftravNonterm(item,callback):returncallback(item.to_rule)+[item]deftravTerm(item,callback):return[item]returnTraversing.traverseSeparated(root,travRule,travNonterm,travTerm)Class Traverse also provideprintstatic method, that returns string representing the structure of the AST.(R)ChomskySplitTempRule81
|--(N)NoBracketExpression
|  `--(R)ChomskySplitRule20
|     |--(T)<class 'lambda_cli.terminals.LeftBracket'>
|     `--(N)ChomskyGroupNonterminal20
|        `--(R)ChomskySplitTempRule20
|           |--(N)NoBracketExpression
|           |  `--(R)ReducedSplitRules6
|           |     |--(T)<lambda_cli.terminals.Variable object at 0x060818D0>
|           |     `--(N)ExpressionBody
|           |        `--(R)SplitRules7
|           |           `--(T)<lambda_cli.terminals.Variable object at 0x060817D0>
|           `--(T)<class 'lambda_cli.terminals.RightBracket'>
`--(T)<class 'lambda_cli.terminals.RightBracket'>RoadmapCurrently only small subset of operations are implemented.Library should at least support these operations:Transform of grammar into valid LL(k) grammarRemoving of recursionTransformations into another representation (like regular grammar to state machine), but this will be probably in separate package.Version: 1.2.5Author: Patrik ValkovičLicence: GNU General Public License v3.0