Text Classification with CNN using Regex | Towards Data Science

Extracting label and document no. from filenameExtracting EmailExtracting SubjectDecontractionsRemoving Write to: , From: , or:Removing data within brackets <> , ()Removing newlines, tabsPreprocessing person and city name with ChunkingRemoving digitsReplacing “_word_” , “_word” , “word_” to word using Match CapturesMiscellaneous
Extracting label and document no. from filename
Extracting Email
Extracting Subject
Decontractions
Removing Write to: , From: , or:
Removing data within brackets <> , ()
Removing newlines, tabs
Preprocessing person and city name with Chunking
Removing digits
Replacing “_word_” , “_word” , “word_” to word using Match Captures
Miscellaneous
Natural language processing is a branch of AI which deals with language data. It basically is a branch where interaction between humans and achine is researched. Machine translation, text to speech, text classification — these are some of the applications of Natural Language Processing.
In this article, I am going to classify text data using 1D Convolutional Neural Network extensively using Regular Expressions for string preprocessing and filtering.
Our focus on this article is how to use regex for text data preprocessing
The data is Newsgroup20 dataset. There are total 20 types of documents in our data. The name of the document contains the label and the number in that label. The format is ‘ClassLabel_DocumentNumberInThatLabel’. The data can be downloaded from here.
Our task is to preprocess the text data and classify it into a correct label.
We have created a single function which takes raw data as input and gives preprocessed filtered data as output.
We will use split method which applies on strings.
1   for filename in os.listdir(path):2       i,j = filename.split('_')3       j = int(j.split('.')[0])4       label.append(i) 5       doc_num.append(j)
For all the filenames in the path, we take the filename and split it on ‘_’. The function .split() uses the element inside the paranthesis to split the string. Similarly we use it again to filter the .txt in filename.
Ex.- we get i = talk.religion.misc and j = 82778
Our task is to find all the emails in a document, take the text after “@” and split it with “.” , remove all the words less than 3 and remove “.com” . Combine all in a single string. Then finally we remove the email from our text.
Ex.-[bskendig@netcom.com , joslin@pogo.isp.pitt.edu] → [netcom, pogo.isp.pitt.edu] → [netcom, pogo, isp, pitt, edu] →“netcom pogo isp pitt edu”
Following code executes the task
em = [] # for each docst = ""  # for each doc#extracting emailfor i in re.findall(r'[\w\-\.]+@[\w\.-]+\b', f):  #extracting email     temp=[]    temp = i.split('@')[1]      #taking text after @    temp = temp.split('.')     #a list containing words split by "."    if 'com' in temp:        temp.remove('com')    for i in temp:            #removing words less than 3        if len(i)>2:            em.append(i)for i in em:            #joining all the words in a string    st+=i    st+=' 're.sub(r'[\w\-\.]+@[\w\.-]+\b',' ')    #removing the email
REGEX Explanation
\w → Match any alphanumeric character
\-\. → Match “-” and “.” ( “\” is used to escape special characters)
[]+ → Match one or more than one characters inside the brackets
....................................................... @ → Match “@” after [\w\-\.]
In this task, we are going to keep only the useful information from the subject section. Subject: will be removed and all the non-alphanumeric characters will be removed. The following code executes the task-
#extracting subjecttemp1 = re.findall(r'^Subject.*$',f, re.MULTILINE)sub = temp1[0]sub = sub[7:]   #Truncate Subjectfor i in string.punctuation:   #remove all the non-alphanumeric    sub = sub.replace(i," ")    sub = re.sub(r"re","",sub, flags=re.IGNORECASE) #removing Re    sub = sub.lower()  #lower-casingre.sub(r'Subject.*$'," ", flags=re.MULTILINE)  #removing subject
REGEX Explanation
^ → Accounts for the beginning of the string.
Subject → To match that the beginning of the string is the word Subject.
.* → Matches 0 or more words after Subject.
$ → To detect the end of string.
Note: “^” is important to ensure that Regex detects the ‘Subject’ of the heading only. It should not detect the word ‘subject’ in any other part of our text.
Replacing the words like I’ll with I will, can’t with cannot etc..
# specificf = re.sub(r"won't", "will not", f)f = re.sub(r"can\'t", "can not", f)# generalf = re.sub(r"n\'t", " not", f)f = re.sub(r"\'re", " are", f)f = re.sub(r"\'s", " is", f)f = re.sub(r"\'d", " would", f)f = re.sub(r"\'ll", " will", f)f = re.sub(r"\'t", " not", f)f = re.sub(r"\'ve", " have", f)f = re.sub(r"\'m", " am", f)
Removing the content like addresses which are written under “write to:”, “From:” and “or:” .
f = re.sub(r"Write to:.*$","",f, flags=re.MULTILINE)           f = re.sub(r"From:.*$","",f, flags=re.MULTILINE)               f = re.sub(r"or:","",f,flags=re.MULTILINE)
Note- “$” matches the end of string just for safety
Deleting all the data which is inside the brackets.
Eg- My name is Ramesh (chintu) → My name is Ramesh
f = re.sub(r"<.*>","",f, flags=re.MULTILINE) f = re.sub(r"\(.*\)","",f,flags=re.MULTILINE)                 
#delete /,-,/n,f = re.sub(r"[\n\t\-\\\/]"," ",f, flags=re.MULTILINE)
Chunking is the process of extracting valuable phrases from sentences based on Part-of-Speech tagging. It adds more strcuture to the sentence and helps machine understand the meaning of sentence more accurately. You can read this article by Nikita Bachani where she has explained chunking in detail.
Our task here is to remove names and add underscore to city names with the help of Chunking. Ex- Ramesh will be removed and New Delhi → New_Delhi
chunks=[]chunks=(list(ne_chunk(pos_tag(word_tokenize(f)))))for i in chunks:    if type(i)==Tree:        if i.label() == "GPE":            j = i.leaves()                if len(j)>1:   #if new_delhi or bigger name                    gpe = "_".join([term for term,pos in j])                    f = re.sub(rf'{j[1][0]}',gpe,f, flags=re.MULTILINE)              #replacing delhi with new_delhi                    f = re.sub(rf'\b{j[0][0]}\b',"",f, flags=re.MULTILINE)       #deleting new, \b is important        if i.label()=="PERSON":           # deleting Ramesh                     for term,pog in i.leaves():                f = re.sub(re.escape(term),"",f, flags=re.MULTILINE)
As we can see above, chunks has three parts- label, term, pos. If the type is tree and label is GPE, then its a place. If the place hasmore than one word, we join them using “_”.
f = re.sub(rf'{j[1][0]}',gpe,f, flags=re.MULTILINE)f = re.sub(rf'\b{j[0][0]}\b',"",f, flags=re.MULTILINE)
We used format string and regex together. “j” contains leaf, hence j[1][0] contains the second term i.e Delhi and j[0][0] contains the first term i.e New. So, we replaced delhi with new_delhi and deleted new. \b is to detect the end of the word.
f = re.sub(re.escape(term),"",f, flags=re.MULTILINE)
To delete Person, we use re.escape because the term can contain a character which is a special character for regex but we want to treat it as just a string.
f = re.sub(r'\d',"",f, flags=re.MULTILINE)                               
f = re.sub(r"\b_([a-zA-z]+)_\b",r"\1",f) #replace _word_ to word   f = re.sub(r"\b_([a-zA-z]+)\b",r"\1",f) #replace_word to wordf = re.sub(r"\b([a-zA-z]+)_\b",r"\1",f) #replace word_ to word
Here, we use something called as Match Captures.
After we get our string _word_ using “\b_([a-zA-z]+)_\b”, match captures enable us to just use a specific part of the matched string. Here we have one group in paranthesis in between the underscores. Hence we have 1 group here. We use r ‘\1’ to extract the particular group. Simple example to explain the concept.
{m,n} → This is used to match number of characters between m and n. m can be zero and n can be infinity.
f = re.sub(r"\b[a-zA-Z]{1}_([a-zA-Z]+)",r"\1",f) #d_berlin to berlinf = re.sub(r"\b[a-zA-Z]{2}_([a-zA-Z]+)",r"\1",f) #mr_cat to catf = f.lower()      #lower casef = re.sub(r'\b\w{1,2}\b'," ",f) #remove words <2f = re.sub(r"\b\w{15,}\b"," ",f) #remove words >15f = re.sub(r"[^a-zA-Z_]"," ",f)  #keep only alphabets and _                                        f = re.sub(r" {2,}", " ", f, flags=re.MULTILINE) # REMOVE THE EXTRA SPACES
As mentioned earlier, the whole preprocessing has been put together in a single function which returns five values. Passing our data to this function-
doc_num, label, email, subject, text = preprocessing(prefix)
Creating a dataframe which contains the preprocessed email, subject and text. The class labels have been replaced with intergers.
After splitting the data into train and test (0.25), we vectorize the data into correct form which can be understood by the algorithm. We have used tokenizer function from keras which will be used in embedding vector.
Finally encode the text and pad them to create a uniform dataset.
Our model to train this dataset consists of three ‘one dimensional convolutional’ layer which are concatenated together and passed through other various layers given below.
After training the model, we get around 75% accuracy which can be easily furthur improved by making some tweaks in the model. The main focus of this article was the preprocessing part which is the tricky part here.