Performing Sentiment Analysis on Movie Reviews | by Bryan Tan | Towards Data Science

In this article, I will be using the IMDB movie reviews dataset for this study. The dataset contains 50,000 reviews — 25,000 positive and 25,000 negative reviews. An example of a review can be seen in Fig 1, where a user gave a 10/10 rating and a written review for the Oscar-winning movie Parasite (2020).
The number of stars would be a good proxy for sentiment classification. For example, we could pre-assign the following:
At least 7 out of 10 stars => positive (label=1)
At most 4 out of 10 stars => negative (label=0)
Luckily for us, the researchers at Stanford University have done the ‘heavy-lifting’ of sentimentally classifying the datasets of reviews (refer to Citations & References for further details). Here’s a snipper of how the dataset looks like:
For training, our feature matrix is greatly sparse, meaning there is a lot of zeros in this matrix as there are 25,000 rows and approximately 75,000 columns. So, what we’re gonna do is find the weight of each feature and multiply them with their corresponding TD-IDF values; sum all the values, pass it through a sigmoid activation function, and that’s how we end up with the Logistic Regression model. The advantages of applying a Logistic function in this case: the model handles sparse matrices really well, and the weights can be interpreted as a probability for the sentiments.
Below, we will call the fit_transform method on CountVectorizer. This will construct the vocabulary of the bag-of-words model and transform the sample sentence below into a sparse feature vector.
import numpy as npfrom sklearn.feature_extraction.text import CountVectorizercount = CountVectorizer()docs = ([‘The sun is shining’,‘The weather is sweet’,‘The sun is shining, the weather is sweet, and one and one is two’])bag = count.fit_transform(docs)print(count.vocabulary_)print(bag.toarray())
Notice how the vocabularies are now stored in a Python dictionary with each unique word mapped to unique integer indices. The array displays the term frequency for each unique word.
TD-IDF can be used to down-weight the frequently occurring words in the feature vectors, for example, the word ‘is’ in the above sample sentence. The TD-IDF can be calculated by the product of the term frequency with the inverse document frequency. Formula to calculate TD-IDF:
We will now transform the raw frequency input from the previous section to obtain our TD-IDF values by instantiating the TdidfTransformer method.
from sklearn.feature_extraction.text import TfidfTransformernp.set_printoptions(precision=2)tfidf = TfidfTransformer(use_idf=True, norm=’l2', smooth_idf=True)print(tfidf.fit_transform(bag).toarray())
Evidently from the sample sentence, the word ‘is’ had the highest term frequency in the third sentence in the document. What you’ll notice after the value gets transformed into a TD-IDF value, is that instead of an inflated value of 3, it is now 0.45. This is because the word ‘is’ is also contained in sentences 1 and 2 in the document; and thus is unlikely to contain any useful or discriminatory information for our model to exploit.
So, this is the core idea of TD-IDF values — they convert text data into numeric values and also weigh the words appropriately according to the frequency of the word in our corpus of text data.
In this section, we will define a helper function to preprocess the text data as our text of reviews can contain special characters — html tags, emojis — that we would want to account for when training our model.
import redef preprocessor(text):text =re.sub(‘<[^>]*>’, ‘’, text)emoticons = re.findall(‘(?::|;|=)(?:-)?(?:\)|\(|D|P)’, text)text = re.sub(‘[\W]+’, ‘ ‘, text.lower()) + ‘ ‘.join(emoticons).replace(‘-’, ‘’)return textpreprocessor(“This is a :) test :-( !”)
As seen above, after applying the defined preprocessor function, the sample sentence was stripped of special characters; the emojis were also moved to the end of the sentence. This is so that our model can make use of the sequence of the text and also ascertain the sentiment of the emojis at the end of the sentences.
In this section, we will represent our data as a collection of words or tokens; we will also be performing word-level preprocessing tasks such as stemming. To achieve this, we will utilize the natural language toolkit, or nltk.
Stemming is a technic that reduces the inflectional forms, and sometimes derivationally related forms, of a common word to a base form. For example, the words ‘organizer’ and ‘organizing’ stems from the base word ‘organize’. So, stemming is conventionally referred to as a crude heuristic process that ‘chops’ off the ends of words, in the hope of achieving the goal correctly most of the time — this often includes the removal of derivational affixes.
from nltk.stem.porter import PorterStemmerporter = PorterStemmer()def tokenizer(text):return text.split()tokenizer('runners like running thus they run')
def tokenizer_stemmer(text):return[porter.stem(word) for word in text.split()]tokenizer_stemmer(‘runners like running thus they run’)
We have already done this TD-IDF transformation on our previous sample documents, but now we fill fit_transform our movie review dataset into TD-IDF values. However, instead of breaking the process down into several steps— breaking the documents down into term frequencies, then transforming them into TD-IDF values — we will implement the TfidfVectorizer method that carries out all the above steps in one single step.
from sklearn.feature_extraction.text import TfidfVectorizertfidf = TfidfVectorizer(strip_accents=None,lowercase=True,preprocessor=preprocessor, # defined preprocessor in Data Cleaningtokenizer=tokenizer_stemmer,use_idf=True,norm=’l2',smooth_idf=True)y = df.sentiment.valuesX = tfidf.fit_transform(df.review)
With X and y as our feature matrices of TD-IDF values and target vector of sentiment values respectively, we are ready to split our dataset into training and test sets. Then, we will fit our training set into a Logistic Regression model.
Note that instead of manually hyperparamter tuning our model, we’re using LogisticRegressionCV to specify the number of cross-validation folds we want to do to tune the hyperparameter — that is 5-fold cross-validation.
from sklearn.model_selection import train_test_splitimport picklefrom sklearn.linear_model import LogisticRegressionCVX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1, test_size=0.5, shuffle=False)clf = LogisticRegressionCV(cv=5,scoring=’accuracy’,random_state=0,n_jobs=-1,verbose=3,max_iter=300).fit(X_train, y_train)
Finally, we will be passing our test data — data our model has not seen before — for inference towards how well our model did.
89.6% — that’s pretty good considering we’re using a relatively simple model for our study!
Another way of looking at the accuracy of our classifier is through a confusion matrix.
Observe the first row. The first row is for reviews which their actual sentiment values in the test set are 1. As you can calculate, out of 25,000 reviews, the sentiment value of 12,473 of them is 1; and out of these 12,473, the classifier correctly predicted 11,209 of them as 1.
It means, for 11,209 reviews, the actual sentiment values were 1 in the test set, and the classifier also correctly predicted those as 1. However, while the actual labels of 1,264 reviews were 1, the classifier predicted those as 0, which is pretty good.
What about the reviews with the sentiment value of 0? Let’s look at the second row. It looks like there were a total 12,527 reviews which their actual sentiment values were 0.
The classifier correctly predicted 11,193 of them as 0, and 1,334 of them wrongly as 1. So, it has done a good job in predicting the reviews with sentiment values of 0.
A good thing about confusion matrix is that it shows the model’s ability to correctly predict or separate the classes. In specific cases of a binary classifier, such as this example, we can interpret these numbers as the count of true positives, false positives, true negatives, and false negatives.
In this study, we covered topics on:
cleaning and preprocessing text data
perform feature extraction using methods such as Count Vectorization, term frequencies and inverse document frequencies, natural language toolkit
built and employed a Logistics Regression model using scikit-learn
model evaluation using Javan’s Accuracy Score and Confusion Matrix
[1] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. (2011). Learning Word Vectors for Sentiment Analysis. The 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011).
[2] Github repository for CSV file and notebook:https://github.com/TheClub4/IMDB_Sentiment_Analysis