Package: grex
1. What does this library do?grexis a library that is meant to simplify the often complicated and tedious
task of creating regular expressions. It does so by automatically generating a
single regular expression from user-provided test cases. The resulting
expression is guaranteed to match the test cases which it was generated from.This project has started as aRust portof
the JavaScript toolregexgenwritten byDevon Govett. Although a lot of
further useful features could be added to it, its development was apparently
ceased several years ago. The Rust library offers new features and extended
Unicode support. With the help ofPyO3andMaturin, the library has been compiled to a
Python extension module so that it can be used within any Python software as well.The philosophy of this project is to generate the most specific regular expression
possible by default which exactly matches the given input only and nothing else.
With the use of preprocessing methods, more generalized expressions can be created.The produced expressions arePerl-compatible regular expressionswhich are also
compatible with theregular expression modulein Python's
standard library.There is ademo websiteavailable where you can give grex a try.2. Do I still need to learn to write regexes then?Definitely, yes!Using the standard settings,grexproduces a regular expression that is guaranteed
to match only the test cases given as input and nothing else. However, if the conversion to shorthand
character classes such as\wis enabled, the resulting regex matches a much wider scope of test cases.
Knowledge about the consequences of this conversion is essential for finding a correct regular expression
for your business domain.grexuses an algorithm that tries to find the shortest possible regex for the given test cases.
Very often though, the resulting expression is still longer or more complex than it needs to be.
In such cases, a more compact or elegant regex can be created only by hand.
Also, every regular expression engine has different built-in optimizations.grexdoes not know anything
about those and therefore cannot optimize its regexes for a specific engine.So, please learn how to write regular expressions!The currently best use case forgrexis to find
an initial correct regex which should be inspected by hand if further optimizations are possible.3. Current Featuresliteralscharacter classesdetection of common prefixes and suffixesdetection of repeated substrings and conversion to{min,max}quantifier notationalternation using|operatoroptionality using?quantifierescaping of non-ascii characters, with optional conversion of astral code points to surrogate pairscase-sensitive or case-insensitive matchingcapturing or non-capturing groupsoptional anchors^and$fully compliant toUnicode Standard 15.0correctly handles graphemes consisting of multiple Unicode symbolsproduces more readable expressions indented on multiple using optional verbose modeoptional syntax highlighting for nicer output in supported terminals4. How to install?grexis available in thePython Package Indexand can be installed with:pip install grexThe current version 1.0.0 corresponds to the latest version 1.4.4 of the Rust
library and command-line tool.5. How to use?This library contains a single class namedRegExpBuilderthat can be imported like so:fromgreximportRegExpBuilder5.1 Default settingspattern=RegExpBuilder.from_test_cases(["a","aa","aaa"]).build()assertpattern=="^a(?:aa?)?$"5.2 Convert to character classespattern=(RegExpBuilder.from_test_cases(["a","aa","123"]).with_conversion_of_digits().with_conversion_of_words().build())assertpattern=="^(?:\\d\\d\\d|\\w(?:\\w)?)$"5.3 Convert repeated substringspattern=(RegExpBuilder.from_test_cases(["aa","bcbc","defdefdef"]).with_conversion_of_repetitions().build())assertpattern=="^(?:a{2}|(?:bc){2}|(?:def){3})$"By default,grexconverts each substring this way which is at least a single character long
and which is subsequently repeated at least once. You can customize these two parameters if you like.In the following example, the test caseaais not converted toa{2}because the repeated substringahas a length of 1, but the minimum substring length has been set to 2.pattern=(RegExpBuilder.from_test_cases(["aa","bcbc","defdefdef"]).with_conversion_of_repetitions().with_minimum_substring_length(2).build())assertpattern=="^(?:aa|(?:bc){2}|(?:def){3})$"Setting a minimum number of 2 repetitions in the next example, only the test casedefdefdefwill be
converted because it is the only one that is repeated twice.pattern=(RegExpBuilder.from_test_cases(["aa","bcbc","defdefdef"]).with_conversion_of_repetitions().with_minimum_repetitions(2).build())assertpattern=="^(?:bcbc|aa|(?:def){3})$"5.4 Escape non-ascii characterspattern=(RegExpBuilder.from_test_cases(["You smell like 💩."]).with_escaping_of_non_ascii_chars(use_surrogate_pairs=False).build())assertpattern=="^You smell like\\U0001f4a9\\.$"Old versions of JavaScript do not support unicode escape sequences for the astral code planes
(rangeU+010000toU+10FFFF). In order to support these symbols in JavaScript regular
expressions, the conversion to surrogate pairs is necessary. More information on that matter
can be foundhere.pattern=(RegExpBuilder.from_test_cases(["You smell like 💩."]).with_escaping_of_non_ascii_chars(use_surrogate_pairs=True).build())assertpattern=="^You smell like\\ud83d\\udca9\\.$"5.5 Case-insensitive matchingThe regular expressions thatgrexgenerates are case-sensitive by default.
Case-insensitive matching can be enabled like so:pattern=(RegExpBuilder.from_test_cases(["big","BIGGER"]).with_case_insensitive_matching().build())assertpattern=="(?i)^big(?:ger)?$"5.6 Capturing GroupsNon-capturing groups are used by default.
Extending the previous example, you can switch to capturing groups instead.pattern=(RegExpBuilder.from_test_cases(["big","BIGGER"]).with_case_insensitive_matching().with_capturing_groups().build())assertpattern=="(?i)^big(ger)?$"5.7 Verbose modeIf you find the generated regular expression hard to read, you can enable verbose mode.
The expression is then put on multiple lines and indented to make it more pleasant to the eyes.importinspectpattern=(RegExpBuilder.from_test_cases(["a","b","bcd"]).with_verbose_mode().build())assertpattern==inspect.cleandoc("""(?x)^(?:b(?:cd)?|a)$""")5.8 Disable anchorsBy default, the anchors^and$are put around every generated regular expression in order
to ensure that it matches only the test cases given as input. Often enough, however, it is
desired to use the generated pattern as part of a larger one. For this purpose, the anchors
can be disabled, either separately or both of them.pattern=(RegExpBuilder.from_test_cases(["a","aa","aaa"]).without_anchors().build())assertpattern=="a(?:aa?)?"6. How to build?In order to build the source code yourself, you need thestable Rust toolchaininstalled on your machine
so thatcargo, the Rust package manager is available.gitclonehttps://github.com/pemistahl/grex.gitcdgrex
cargobuildTo build the Python extension module, create a virtual environment and installMaturin.python-mvenv/path/to/virtual/environmentsource/path/to/virtual/environment/bin/activate
pipinstallmaturin
maturinbuildThe Rust source code is accompanied by an extensive test suite consisting of unit tests, integration
tests and property tests. For running them, simply say:cargotestAdditional Python tests can be run after installing pytest which is an optional dependency:maturindevelop--extras=testpytesttests/python/test_grex.py