Multi Label Intent Classification | by Priyanka Jain | Towards Data Science

There are a lot of applications that require text classification or we can say intent classification. Nowadays, everything is required to be categorized like contents, products are often tagged by category.
However, the vast majority of text classification articles and tutorials on the internet are binary text classification such as email spam filtering (spam vs. ham), sentiment analysis (positive vs. negative). Our real-world problem is much more complicated than that. Therefore, this is what I am going to explain in this blog. Classifying the text into multiple categories.
Problem Statement:
I have developed this classifier for my GSoC project(Owasp SKF-Chatbot). The problem statement is the Security Knowledge framework have a knowledge base for different vulnerabilities. It provides the description, solution and code example for different vulnerabilities. So, I need to classify the user’s query whether he is asking for a description, solution or code example.
I used Python and Jupyter Notebook to develop our system, relying on Scikit-Learn for the machine learning components.
Preparation of Data-sets:
For any problem related to classification or machine learning the first thing we required is the data that too correctly formatted. So, firstly I will explain how I prepare the data-set for intent classification.
import jsonimport csvwith open("data.json",encoding='utf-8') as read_file:        data = json.load(read_file)
You can check data.json here. I am going to prepare the dataset in CSV format as it will be easy to train the model.
#For parsing the Jsona=data['items']#Declaration of liststitle=[]ques=[]sol=[]code=[]#For accessing the title from Json and storing it in the list.for d in a:    title.append((d['title']))
My text classification is for question answer kind of system. So, I need to generate questions for that I extracted out all the titles in a list.
for t in title:	ques.append("What is "+ t + " ?")	ques.append("What does "+ t + " mean ?")	ques.append("Tell me something about "+ t + " ?")	ques.append(t)	ques.append("Explain " + t +" ?")	ques.append("Elaborate " + t +" ?")	ques.append("Can you tell me about " + t + " ?")	ques.append("What do you know about " + t + " ?")	ques.append("What can you tell me about " + t + " ?")	ques.append("I want to know about XSS " + t )	ques.append("Do you have information about " + t + " ?")for t in title:        sol.append("How to solve "+ t + " ?")        sol.append("How to resolve "+ t + " ?")        sol.append("How to mitigate "+ t + " ?")        sol.append("Solution for "+ t)        sol.append("Provide me some Solution for "+ t)        sol.append("mitigation for "+ t)        sol.append("How to stop "+ t + " ?")        sol.append("How to defend "+ t + " ?")        sol.append("How to get secured against "+ t + " ?")        sol.append("Solution, "+t)for t in title:        code.append("Give me some sample code of "+ t )        code.append("Code example of "+ t + " ?")        code.append("Code of "+ t )
So, the questions need to be classified on the basis of description, solution, and code. So, I made three lists one for each to store the questions.
file=open("intent_data.csv","x")file.write('class,question\n')for x in ques:		x=x.replace(",","")		file.write('Description, '+x+"\n")	 for y in sol:		y=y.replace(",","")		file.write('Solution, '+y+"\n")for z in code:		z=z.replace(",","")		file.write('Code, '+z+"\n")file.close()
So, Now the data is prepared. You can check the complete data here.
And for the better understanding of how to prepare data-set, you can also check this jupyter notebook example.
Now, we have prepared data 😄...It’s time to do some magic with it. Let’s see how we classify the intents.
Let’ s import some libraries:
import pandas as pdfrom io import StringIOfrom sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.model_selection import train_test_splitfrom sklearn.feature_extraction.text import CountVectorizerfrom sklearn.feature_extraction.text import TfidfTransformerfrom sklearn.naive_bayes import MultinomialNB
Firstly, we will extract the data from the CSV file and will store it in the data frame.
def get_data():    df = pd.read_csv('datasets/intent_data.csv')    return df
In this step, we will prepare the data to feed it to the algorithm. Firstly, we get the complete data in ‘y’ and set the column for that data using
y=[col]
After, that ‘pd.notnull’ is being used for checking the data in the question column is null or not if it’ s null the entire row will be removed.
This step is important for getting the high-quality clean data. Because, If we have good data we will have good results. 😃
We will make a new column ‘category_id’ which will give a number to classes. Like for description it will be 0, solution 1 and code 2.
After removing duplicates we will get something like this
category_id_df = y[['classs', 'category_id']].drop_duplicates().sort_values('category_id')print(category_id_df)classs                category_id0       Description         02       Solution            15081    Code                2
You can check the code snippet of data_prepare below.
def data_prepare():    col = ['classs', 'question']    y=get_data()    y = y[col]    y = y[pd.notnull(y['question'])]    y.columns = ['classs', 'question']    y['category_id'] = y['classs'].factorize()[0]    category_id_df = y[['classs', 'category_id']].drop_duplicates().sort_values('category_id')    category_to_id = dict(category_id_df.values)    id_to_category = dict(category_id_df[['category_id', 'classs']].values) #This will add the column in our dataframe    return y
I have used the Multinomial Naive_Bayes algorithm for prediction because I find it easy to implement and it has high accuracy.
The OneVsRest strategy can be used for multi-label learning, where a classifier is used to predict multiple labels for instance. Naive Bayes supports multi-class, but we are in a multi-label scenario, therefore, we wrap Naive Bayes in the OneVsRestClassifier.
OneVsRest multi-label strategy
The Multi-label algorithm accepts a binary mask over multiple labels. The result for each prediction will be an array of 0s and 1s marking which class labels apply to each row input sample.
For the better understanding of the following code snippet and Multinomial Naive_bayes try this.
A short overview is: Here, I have divided my data into test data and train data and then feed that data into the model.
def naive_algo():    tfidf = TfidfVectorizer(sublinear_tf=True, min_df=5, norm='l2', encoding='latin-1', ngram_range=(1, 2), stop_words='english')    df=data_prepare()    features = tfidf.fit_transform(df.question).toarray()    labels = df.category_id    features.shape    X_train, X_test, y_train, y_test = train_test_split(df['question'], df['classs'], random_state = 0)    count_vect = CountVectorizer()    X_train_counts = count_vect.fit_transform(X_train)    tfidf_transformer = TfidfTransformer()    X_train_tfidf = tfidf_transformer.fit_transform(X_train_counts)    clf = MultinomialNB().fit(X_train_tfidf, y_train)    return clf,count_vect
I have also tried other algorithms or models like Linear SVC, Logistic Regression, and Random Forest. You can have a look at that here.
This will give us the final prediction.
def predict(question):    clf,count_vect=naive_algo()    intent=clf.predict(count_vect.transform([question]))    intent=str(intent).strip("['']")    return intent
At last, we enter the question and pass it to the predict function and wait for the magic. 😝
ques=input("Enter your question ")x=predict(ques)
Pretty, accurate isn’t it?
You can check this notebook for the better understanding of intent classification.
Thanks for reading! Please do clap if you liked it, comment(reviews or doubts) and Share it 😄
You can connect with me on Github, Linkedin, Twitter 😄