Predicting E-Commerce User Ratings using Pipelines in Python
By: Mohammed Kamil Khan
Tags: Machine Learning, Data Processing, Naturallanguageprocessing, Deployment Pipelines

As per the flow diagram, let us have an idea of the skeletal functionality of Pipeline followed by a brief explanation of the different step-wise tasks it harbors.

The skeletal framework of how pipeline has been used in our case is observed in the code below:

from sklearn.pipeline import Pipeline

pipeline=Pipeline([('bow',CountVectorizer(analyzer=myfunc).fit(Reviews['Review Text'])),

('tfidf',TfidfTransformer()),

('classifier',MultinomialNB())

])

// Execute the above block of code after importing the libraries required for carrying out the task of the pipeline successfully, which is observed in the next block of code.

[9]

The pipeline shown above performs the operations of: Removing stopwords and punctuations, obtaining a bag-of-words, applying TF-IDF vectorization followed by the application of a machine learning model of your choice. Let us go through the meanings of the different terms observed in the pipeline.

Stopwords: These are the English words in a sentence which can safely be ignored without sacrificing the meaning of the sentence. For example, words like: “the”, ”he”, ”have” etc. Even with respect to the Kaggle dataset, the previously mentioned words will have no impact on predicting whether the rating associated with a specific review text is either ‘good’ or ‘bad’.

Punctuation: Since we are concerned with converting text data into vectors for prediction analysis, it is absolutely necessary to remove any punctuations that may be present in a review given by a user.

The bag-of-words model is a simplifying representation used in natural language processing and information retrieval (IR). In this model, a text (such as a sentence or a document) is represented as the bag (multiset) of its words, disregarding grammar and even word order but keeping multiplicity. In order to obtain a bag-of-words, we first need to perform cleaning of text data by removing StopWords and punctuations.

Text preprocessing, filtering of stopwords and punctuations are all covered in CountVectorizer, which builds a dictionary of features and transforms documents to feature vectors. CountVectorizer only counts the number of times a word appears in a document which often results in biasing in favour of most frequent words which thereby leads to ignoring rare words which could have helped in processing the data more efficiently [9]. To overcome this, we use TfidfTransformer. TF-IDF is a measure of a word by comparing the number of times a word appears in a document with the number of documents the word appears in. The ‘TF’ in TF-IDF refers to ‘Term Frequency’, which enlists the number of times the words present in the entire dataset occur in the different sentences. The ‘IDF’ refers to ‘Inverse Document Frequency’, the aim of it being to reflect the importance of a word in a document in a collection or corpus.

An important thing to note down in case one gets confused is that Pipeline performs sequential execution of operations. As can be seen in the Pipeline function, the tasks of CountVectorizer and TfidfTransformer can be successfully achieved in just two lines of code. The classifier step is purely for the classification model you wish to apply to the data.

We import the necessary libraries and ML models as follows:

//first import these before executing the pipeline step

from sklearn.feature_extraction.text import CountVectorizer

from sklearn.feature_extraction.text import TfidfTransformer

from sklearn.naive_bayes import MultinomialNB

from sklearn.neighbors import KNeighborsClassifier

from sklearn.linear_model import LogisticRegression

from sklearn.svm import SVC

from sklearn.model_selection import GridSearchCV

[6]

One may ask the purpose of the ‘analyzer’ parameter of the CountVectorizer step of the Pipeline. It gives a call to a function named ‘myfunc’ which, similar to the role of a called function in a lambda expression, is applied to a specific column ‘Review Text’, with the task being the removal of StopWords and punctuations. The function and it’s working are as follows:

def myfunc(punc):

ml1=[]

for i in punc:

if i not in string.punctuation:

ml1.append(i)

ml2=''.join(ml1)

ml2=ml2.split()

i=0

while i<len(ml2):

if ml2[i].lower() in stopwords.words('english'):

del ml2[i]

else:

i=i+1

continue

return ml2

[7]

Working of ‘myfunc’:

Let us take a sample sentence into consideration to understand how text cleaning is performed. Considering the following sentence: “It is, horizontal problem”. It is noticeable that the sentence comprises of 2 StopWords and 1 punctuation. The above mentioned function, using a for loop, first checks whether the element iterated upon is a punctuation or not. If the element is not a punctuation, it is appended to a list ‘ml1’. If it happens to be a punctuation, it is simply ignored. The list ‘ml1’ now comprises of only words in a sentence. The contents of ‘ml1’ are now added to ml2. Following the execution of the while loop, ‘ml2’ will now return text data comprising of neither the StopWords nor the punctuations, hence, data cleaning has been successful and the resulting words are ready for Count Vectorization.

The pipeline function in fig. 2 encompasses the MultinomialNB model only. Changes can be made to accommodate other models. Firstly, we need to split the ‘Review Text’ column data into training and testing data (‘X’ column), with the same being implemented on our target prediction column i.e., ‘Good/Bad’.

X=Reviews['Review Text']

y=Reviews['Good/Bad']

X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.3)

[8]

This is then followed by fitting the training-testing data, prediction, and finally a classification report is generated which provides the different metrics on the basis of the model used.

pipeline.fit(X_train['Review Text'],y_train)

predictions=pipeline.predict(X_test['Review Text'])

from sklearn.metrics import classification_report

print(classification_report(y_test,predictions))

[10]

Classification report for Multinomial Naive Bayes Model

Other Models:

I. K-Nearest Neighbors: The changes required for successful execution of this model would only be required in the pipeline code as follows:

pipeline=Pipeline([('bow',CountVectorizer(analyzer=myfunc).fit(Reviews['Review Text'])),

('tfidf',TfidfTransformer()),

('classifier',KNeighborsClassifier(n_neighbors=5))

])

This all the other blocks of code remain the same.

II. Logistic Regression: Only change for LR model can be observed below:

pipeline=Pipeline([('bow',CountVectorizer(analyzer=myfunc).fit(Reviews['Review Text'])),

('tfidf',TfidfTransformer()),

('classifier',LogisticRegression())

])

III. SVM: For Support Vector Machine model, before pipeline, you will need to execute the following code:

param_grid = {'C': [0.1,1, 10, 100, 1000], 'gamma': [1,0.1,0.01,0.001,0.0001], 'kernel': ['rbf']}

This can then be followed by the pipeline function:

pipeline=Pipeline([('bow',CountVectorizer(analyzer=myfunc).fit(Reviews['Review Text'])),

('tfidf',TfidfTransformer()),

('classifier',GridSearchCV(SVC(),param_grid,refit=True,verbose=3))

])

The metric analysis of the 4 models used after obtaining the classification report of each model can be summarized in the figure below: