Part II: All you need to know about Regular Expressions | by Ria Kulshrestha | Towards Data Science

Assuming you know what Regular Expressions are (in case you don’t, check out Part1 of this tutorial for a quick overview) we’ll now learn how to use them in Python. :)
The ‘re’ module provides an interface to the regular expression engine, and allows us to compile REs into objects and then perform matches with them.
We’ll start with importing the module. Then we will combine a regular expression by passing it as a string and turn it into a pattern object.
>>> import re>>> pat_obj = re.compile('[a-z]+')>>> print(pat_obj)re.compile('[a-z]+')
match(): Determine if the RE matches at the beginning of the string.
>>> m = pat_obj.match('helloworld')>>> print(m)<_sre.SRE_Match object; span=(0, 10), match='helloworld'>#Note how it doesn't take into account white spaces.>>> m = pat_obj.match('hello world')>>> print(m)<_sre.SRE_Match object; span=(0, 5), match='hello'># Note that it is case-sensitive.>>> m = pat_obj.match('Helloworld')>>> print(m)None#To ignore case>>> pat_obj = re.compile('[a-z]+', re.IGNORECASE)>>> m = pat_obj.match('Helloworld')>>> print(m)<_sre.SRE_Match object; span=(0, 10), match='Helloworld'>
search(): Scan through a string, looking for any location where this RE matches.
#Note how it only prints the first match>>> s = pat_obj.search('Hello World!')>>> print(s)<_sre.SRE_Match object; span=(1, 5), match='ello'>
To print all the matches, findall(): Find all substrings where the RE matches and returns them as a list.
>>> s = pat_obj.findall('Hello World!')>>> print(s)<_sre.SRE_Match object; span=(1, 5), match='ello'>['ello', 'orld']#To find all the numbers in a string>>> pat_obj_num = re.compile(r'\d+')>>> pat_obj_num.findall('12 drummers drumming, 11 pipers piping, 10 lords a-leaping')['12', '11', '10']
group(): Returns the string matched by the RE. Because honestly, that’s what you’re interested in. Ain’t nobody got time for all that information.
#Using group with search>>> s = pat_obj.search('Hello World!')>>> print(s)<_sre.SRE_Match object; span=(1, 5), match='ello'>>>> print(s.group())ello#Using group with match>>> m =  pat_obj.match("hello world")>>> print(m)<_sre.SRE_Match object; span=(0, 5), match='hello'>>>> print(m.group())hello#Using group with findall>>> m =  pat_obj.findall("hello world")>>> print(m)['hello', 'world']>>> print(m.group())Error!
span(): Returns a tuple containing the (start, end) positions of the match.start(), end(): Returns the starting and ending position of the match respectively.
>>> pat_obj = re.compile('[a-z]+', re.IGNORECASE)>>> m = pat_obj.match('Helloworld')>>> print(m)<_sre.SRE_Match object; span=(0, 10), match='Helloworld'>>>> print(m.start())0>>> print(m.end())10>>> print(m.span())(0, 10)
Groups are marked by the ( )meta-characters. They group together the expressions contained inside them, and you can repeat the contents of a group with a repeating qualifier, such as *, +, ? or {m, n}.Groups are numbered starting with 0. Group 0 is always present; it’s the whole RE, so match object methods all have group 0 as their default argument.
Subgroups are numbered from left to right, from 1 upward. Groups can be nested; to determine the number, just count the opening parenthesis characters, going from left to right.
>>> pat_obj_group = re.compile('(a(b)c(d))e')>>> m = pat_obj_group.match('abcde')>>> print(m)<_sre.SRE_Match object; span=(0, 5), match='abcde'>#Note m.group(0) matches the same regex as m.match()>>> print(m.group(0))abcde>>> print(m.group(1))abcd#Note the number is determined left to right>>> print(m.group(2))b>>> print(m.group(3))d# Note that multiple arguments can be passes to group()>>> print(m.group(2,1,3))('b', 'abcd', 'd')
groups(): returns a tuple containing the strings for all the subgroups, from 1 up to however many there are.
>>> print(m.groups())('abcd', 'b', 'd')
sub(): Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl. If the pattern isn’t found, the string is returned unchanged. repl can be a string or a function; if it is a string, any backslash escapes in it are processed. That is, \n is converted to a single newline character, \r is converted to a carriage return, and so forth.
Empty matches for the pattern are replaced only when not adjacent to a previously empty match.
>>> print(re.sub('x','-','abxd'))ab-d>>> print(re.sub('ab*','-','abxd'))-xd>>> print(re.sub('x*','-','abxd'))-a-b-d-
The optional argument count is the maximum number of pattern occurrences to be replaced; count must be a non-negative integer. If omitted or zero, all occurrences will be replaced.
Though passing Regular Expressions helps in keeping things simple, it has one disadvantage. The backslash character (‘\’) is used to allow special characters to be used without invoking their special meaning which conflicts with Python’s usage of the same character in string literals where it is used to interpret the character following it differently. For example, ’n’ by itself is simply a letter, but when you precede it with a backslash, it becomes \n, which is the newline character. Ah oh!
Let’s say you want to write a RE that matches the string ‘\section’, which might be found in a LaTeX file.We’ll start with the desired string to be matched. Next, we must escape any backslashes and other metacharacters by preceding them with a backslash, resulting in the string ‘\\section’. The resulting string that must be passed to re.compil() must be ‘\\section’. However, to express this as a Python string literal, both backslashes must be escaped again, resulting in the string ‘\\\\section’.
In short, to match a literal backslash, one has to write ‘\\\\’ as the RE string, because the regular expression must be \\, and each backslash must be expressed as \\ inside a regular Python string literal.
The solution is to use Python’s raw string notation for regular expressions; backslashes are not handled in any special way in a string literal prefixed with ‘r’, so r‘\n’ is a two-character string containing ‘\’ and ‘n’, while ‘\n’ is a one-character string containing a newline.
Regular string and corresponding Raw string
"ab*" -> r"ab*""\\\\section" -> r"\\section""\\w+\\s+\\1" -> r"\w+\s+\1"
Regexone
Visualizing tool
RegexTips
Learning Git in Under 8 minutes!
I‘m glad you made it till the end of this article. 🎉I hope your reading experience was as enriching as the one I had writing this. 💖
Do check out my other articles here.
If you want to reach out to me, my medium of choice would be Twitter.