Programmatically Build REGEX (Regular Expression) in Python for Pattern Matching | by AbdulMajedRaja RS | Towards Data Science

Regular Expression (Regex — often pronounced as ri-je-x or reg-x) is extremely useful while you are about to do Text Analytics or Natural Language Processing. But as much as Regex is useful, it’s also extremely confusing and hard to understand and always require (at least for me) multiple DDGing with click and back to multiple Stack Overflow links.
For starters:
According to Wikipedia, A regular expression, regex or regexp is a sequence of characters that define a search pattern.
This is the REGEX pattern to test the validity of a URL:
^(http)(s)?(\:\/\/)(www\.)?([^\ ]*)$
A typical regular expression contains — Characters ( http ) and Meta Characters ([]). The combination of these two form a meaningful regular expression for a particular task.
Remembering the way in which characters and meta-characters are combined to create a meaningful regex is itself a tedious task which sometimes becomes a bigger task than the actual problem of NLP which is the larger goal. This in fact has driven the meme world and programming cartoons all over the internet. Like this:
Some good soul on this planet has created an open-source Javascript library JSVerbalExpressions to make Regex creation easy. Then some other good soul ported the javascript library to Python — PythonVerbalExpressions. This is the beauty of the open source world. Sorry I digressed, but wanted to make a point.
Based on the version of Python you’ve got, You can install PythonVerbalExpressions using pip.
pip3 install VerbalExpressions
Much like how the ML library scikit-learn works with a basic constructor and building on top of it, VerbalExpressions follows a similar API syntax.
from verbalexpressions import VerExverEx = VerEx()
Let’s create a pseudo-problem that we’d like to solve with regex through which we can understand this package to programmatically create regex.
A simpler one perhaps, We’ve got multiple text like this:
strings = ['123Abdul233',           '233Raja434',           '223Ethan Hunt444']
and we’d like to extract the names from it. So our result should be:
Abdul, Raja, Ethan Hunt
Before we code, it’s always good to write-out a pseudo-code on a napkin or even a paper if you’ve got. That is, We want to extract names (which is composition of alphabets) except numbers (which is digits). We build a regex for one-line and then we iterate it for all the elements in our list.
We use the range() function that says anything in the given range.
expression = verEx.range('a','z','A','Z',' ')
In this expression, anything from a-z or A-Z or a whitespace (whitespace: because of Ethan Hunt ).
To see how this python code actually translates to a regex pattern:
expression.source()#output: '([a-zA-Z\\ ])'
Now that the expression is ready, we can use it to find the required substrings in multiple ways. Here, we’ll turn to the go-to Python library for anything regex re . Because we’re going to deal with re, we’ll compile the expression that we built for re .
import rere_exp = expression.compile()
Now, that the re compilation has been done. We’ll use that re_exp to test this regex on one-line (one element of the input list).
re.findall(re_exp,strings[0])#output: ['A', 'b', 'd', 'u', 'l']
That’s promising! Let’s use our string joining and for-loop skills to finish off our pseudo-problem!
[''.join(re.findall(re_exp,line)) for line in strings]#output: ['Abdul', 'Raja', 'Ethan Hunt']
Thus, we managed to build a regex pattern without knowing regex. Simply put, we programmatically generated a regex pattern using Python (that doesn’t require the high-level knowledge of regex patterns) and accomplished a tiny task that we took up to demonstrate the potential. The caveat here is though, this Python package VerbalExpressions doesn’t have all the functions from its parent javascript library, so the functionalities are limited. For more of Regex using Python, Check out this Datacamp course. The entire code is available here.