Package: ruler
Ruler is a lightweight regular expressions wrapper aiming to make regex definitions more
modular, intuitive, readable and the mismatch reporting more informative.Quick startLet’s implement the following grammar, given inEBNF:grammar = who, ' likes to drink ', what;
who = 'John' | 'Peter' | 'Ann' | 'Paul' | 'Rachel';
what = tea | juice;
juice = 'juice';
tea = 'tea', [milk];
milk = ' with milk';Using ruler it looks almost identical toEBNF:>>> class Morning(Grammar):
...     who = OneOf('John', 'Peter', 'Ann', 'Paul', 'Rachel')
...     juice = Rule('juice')
...     milk = Optional(' with milk')
...     tea = Rule('tea', milk)
...     what = OneOf(juice, tea)
...     grammar = Rule(who, ' likes to drink ', what, '\.')
...
... morning = Morning.create()A member namedgrammarmust be always present - it acts as the start rule.
Let’s begin rather with a mismatch:>>> morning.match('John likes to drink coffee')
Falsematch()returnsTrueif the match was successful andFalseotherwise.
One of the major advantages ofruler, as opposed to working directly with regular expressions,
is the ability to know exactly what went wrong:>>> print(morning.error.long_description)
Mismatch at 20:
  John likes to drink coffee
                      ^
"coffee" does not match "juice"
"coffee" does not match "tea"Let’s fix our text:>>> morning.match('John likes to drink tea.')
TrueAny rule that is declared as a member variable of your grammar class acts as a named capture group
arranged hierarchically. Usematchedattribute to retrieve the text matched by a specific rule:>>> morning.matched
'John likes to drink tea.'
>>> morning.who.matched
'John'
>>> morning.what.matched
'tea'Branches of OneOf rules that didn’t match and optional rules that didn’t match haveNoneas
their values making it easy to ask whether they matched:>>> morning.what.juice.matched is None
True
>>> morning.what.tea.matched is None
False
>>> morning.what.tea.milk.matched is None
TrueRules can be reused multiple times. If the same rule appears multiple times under the same parent,
these rules are collected into a list:>>> class Morning(Grammar):
...     person = OneOf('John', 'Peter', 'Ann', 'Paul', 'Rachel')
...     who = Rule(person, Optional(', ', person), Optional(' and ', person))
...     juice = Rule('juice')
...     milk = Optional(' with milk')
...     tea = Rule('tea', milk)
...     what = OneOf(juice, tea)
...     grammar = Rule(who, ' like', Optional('s'), ' to drink ', what, '\.')
...
... morning = Morning.create()
... morning.match('Peter, Rachel and Ann like to drink juice.')
True
>>> morning.who.matched
'Peter, Rachel and Ann'
>>> morning.who.person[0].matched
'Peter'
>>> morning.who.person[1].matched
'Rachel'
>>> morning.who.person[2].matched
'Ann'Notice that, in the grammar above,personrule is never a direct child ofwhobut still
is accessed as such. That is because when a rule hierarchy is built, a rule is placed under its
closest named ancestor.Rules’ string arguments may actually be any valid regular expression. So we could rewrite our
grammar like this:>>> class Morning(Grammar):
...     who = OneOf('\w+')
...     juice = Rule('juice')
...     milk = Optional(' with milk')
...     tea = Rule('tea', milk)
...     what = OneOf(juice, tea)
...     grammar = Rule(who, ' likes to drink ', what, '\.')
...
... morning = Morning()
... morning.match('R2D2 likes to drink juice. And nothing else matters.')
True
>>> morning.matched
'R2D2 likes to drink juice.'
>>> morning.who.matched
'R2D2'PerformanceThe library is well optimized for fast matching. Nevertheless it is important to remember
that this is a Python wrapper of the regex library and as such can never outperform matching
directly using the regex library. Currently ruler measures approximately ten times slower
thanre.DevelopmentTo run the tests:pytest testsTo compare the performance to the re library:python performance/re_compare.pyTo run performance profiling of a specific method,Rule.matchfor example:python performance/profile.py Rule.matchMore than one method can be specified in the same command.ToxTox takes care of everything without installing anything manually. There are two groups of tox
environments:py*-testandpy*-profile. The test environments run the unit tests while the
profile environments run the performance profiling scripts. If tox is not enough then a development
environment can be generated by creating a new virtualenv and then runningpip install-rrequirements_develop.txt.Dependency managementFor the development needs, there are three requirements files in the project’s root directory:requirements_test.txtcontains all the dependencies needed to run the unit tests,requirements_profile.txtcontains all the dependencies needed to run the performance profiling,requirements_develop.txtcontains the testing dependencies, the profiling dependencies and some additional
dependencies used in development.The requirements files mentioned above are not intended for manual editing. Instead they are managed
usingpip-tools. The process of updating the requirements is as follows:Add, remove or update a dependency in one of thereqs_*.depfiles:Updatereqs_install.depif the dependency is needed for the regular installation by the end user,Updatereqs_test.depif the dependency is needed to run the unit tests but is not necessary for the
regular installation,Updatereqs_profile.depif the dependency is needed to run the performance profiling but is not necessary
for the regular installation,Updatereqs_develop.depif the dependency is not in one of the previous categories.Generate the requirements file runningpip-compile. The exact command is documented in the beginning of each
requirements file.Consider runningpip-syncrequirements_develop.txt.Notice that there is no need to editsetup.py- it will pull the dependencies by itself fromreqs_install.dep.