<figure><img src="image_1.png" /></figure>

HAL
open science

# Forecast of Movie Sentiment Based on Multi Label Text Classification on Rotten Tomatoes Using Multiple Machine and Deep Learning Technique

Debarati Nath, Joseph Roy

► To cite this version:

Debarati Nath, Joseph Roy. Forecast of Movie Sentiment Based on Multi Label Text Classification on Rotten Tomatoes Using Multiple Machine and Deep Learning Technique. 7th International Conference on Computer, Communication, and Signal Processing (ICCCSP), Jan 2023, Chennai, India. pp.128-142, 10.1007/978-3-031-39811-7_11. hal-05363243

HAL Id: hal-05363243

https://inria.hal.science/hal-05363243v1

Submitted on 13 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
<figure><img src="image_4.png" /></figure>

# Forecast of Movie Sentiment Based on Multi Label Text Classification on Rotten Tomatoes Using Multiple Machine and Deep Learning Technique

Debarati Nath¹(✉) and Joseph Roy²

¹ Department of Electronics and Communication Engineering, CMR Institute of Technology, Marathahalli, Bengaluru 560037, Karnataka, India
debaratiphd@gmail.com

² Altair Engineering India Private Limited, Marathahalli - Sarjapur Outer Ring Rd, Bengaluru 560103, Karnataka, India

**Abstract.** Intent of the paper is to investigate a comparison of different multi-classification models using machine learning and deep learning methodologies to categorize the sentiment of sentences from the rotten tomato's movie review dataset. In order to achieve best performance three machine learning and three deep learning algorithms were estimated: logistic regression, linear support vector classifier, and Multinomial Naive Bayes (MNB), Recurrent Neural Network (RNN)/LSTM, Convolutional Neural Network (CNN) and CNN + Gated Recurrent unit network (GRU). In this work, it is observed that linear support vector classifier shows significant improvement among all other machine learning algorithms. Moreover, when deep learning algorithms were included in the model, one of the deep learning algorithms, CNN provides highest accuracy of 80% among all the algorithms. After optimizing the fitted model, new text data are considered for testing of correct predictive model and prediction of new text data. Optimized predicted model, i.e., CNN seems to give correct predictions for new test data. Hence, CNN model supposed to be used as one of the most suitable models for sentiment analysis of Rotten Tomatoes Movie Reviews since there is improvement of multi-class classification accuracy.

**Keywords:** Multi Label Text Classification, logistic regression, CNN + GRU, Rotten Tomatoes

## 1 Introduction

Nowadays, the internet plays an important role in modern information technology. Through the internet, users can express their opinions about different information presented in the social network. Moreover, when some users want to know about new thing or want to buy new thing, they can know its review from the internet for taking any decision on that. One of the examples is a review of a new movie. There are lots of movie
recommendations on Netflix, Amazon movie platform etc. The user also can share their judgment or sentiment on these platforms in the forms of text and movie rating. All the sentiment and opinions are stored in the dataset and one of the most popular datasets is the rotten tomatoes (RT) [1-4]. The RT is one of the famous datasets for dealing with sentiment classification on movie reviews [1-4].

The user's opinions stored in the dataset are unable to analyze due to various complex terms such as abbreviations, emoji, slang and unwanted words. It is required to find an appropriate method which can analyze user's opinion stored in dataset very quickly and accurately. Sentiment survey by employing machine learning (ML) approach is one of the most interesting computational methodologies since it is less time consuming and highly efficient [1-4]. Sentiment analysis is computed into different levels like sentence level, document level etc. which classifies the sentence in the form of positive, neutral, negative and so on. Sentiment analysis model had already been implemented for distinct types of sentiments like. Positive, negative, neutral and star rating of 1-5 [5]. It uses natural languages processing (NLP) along with ML and some statistical methods to extract features such as positive and negative from the test dataset and images [6].

Two kinds of ML utilized for sentiment analysis on RT dataset are supervised learning and unsupervised learning [2]. As we know, supervised learning is applicable if labelled output data are available in dataset. Whereas, if unlabeled data or unknown output data are there in dataset, then unsupervised learning is used. RT dataset includes mainly text dataset which are unstructured in nature. Therefore, proper ML algorithm is required to extract features from unstructured data.

It is observed in different research papers that various ML approaches were used for movie review's sentiment analysis such as RT, Internet Movie Database (IMDB) etc. dataset. Supervised machine learning algorithms like Decision Tree (DT), Support Vector Machine (SVM), Naive Bayes (NB), Maximum entropy (ME) and Neural Network are commonly used for analysis of sentiment classification, in which accuracy, recall and precision are used to calculate the performance [7]. Logistic regression is also used for movie recommendation system by implementing feature quantification approach [8]. Various ML algorithms like NB, Logistic Regression, SVM, and Long-Short-Term Memory of Recurrent Neural Network extension (RNN/LSTM) were used to predict characteristic based distributed binary and text from IMDB movie review [9]. Pang and Lee et. al. Used ML models such as SVM, NB and ME to identify binary sentiment from IMDB movie review [10]. The One-vs-all algorithm under regression model i.e. SVM model was used on RT dataset which gives accuracy of 75% by analyzing features as thumbs up and starts [11]. Wu et. al. Used Multinomial Naive Bayes (MNB), SVM and 3- layer neural network on RT dataset to discover the best methodology with highest accuracy [4]. SVM and MNB gave best accuracy of 73% and 76% respectively. N-gram methodology was implemented on RT dataset by employing SVM, NB and ME methods. It is observed that uni-gram approach by using all the algorithms gives better accuracy as compared to other gram like four-gram, five-gram, six-gram etc. [2]. Another research groups had implemented N-gram methodology consisting of uni-gram, bi-gram, and blend of uni-gram and bi-gram on RT dataset by employing SVM, NB and Maximum entropy (ME) methods [12]. Other research groups only used SVM model for utilization of uni-gram, bi-gram, and blend of uni-gram and bi-gram on RT dataset [13].
A new method is used on RT dataset by considering sentiment value from SentiWordnet and proficient original value to forecast the sentiment analysis of the movie in which the proposed model delivers highest F-measure of 0.97 [1]. Sentiment analysis was carried out on various sentiment datasets such RT movie reviews corpus, IMDB movie review corpus and Subjectivity v1.0 corpus. In the case of RT, both logistic regression and extended convolution neural network model give accuracy of 62.47% and 66.70% respectively [3]. Many research groups suggested that hybrid models might provide better performance in terms of accuracy as compared to basic models [3]. An outline of the previously published classification outcome for different dataset is revealed in Table 1.

<table><caption>**Table 1.** Published classification results from literature review</caption><thead><tr><th>Classification model</th><th>Dataset</th><th>Accuracy (%)</th><th>Ref</th></tr></thead><tbody><tr><td>SVM</td><td>RT</td><td>75</td><td>[11]</td></tr><tr><td>MNB</td><td rowspan="3">RT</td><td>76</td><td rowspan="3">[4]</td></tr><tr><td>SVM</td><td>73</td></tr><tr><td>3-layer neural network</td><td>61</td></tr><tr><td>NB</td><td rowspan="3">RT</td><td>87</td><td rowspan="3">[2]</td></tr><tr><td>SVM</td><td>87</td></tr><tr><td>ME</td><td>89</td></tr><tr><td>Logistic regression and extended convolution neural network</td><td>RT</td><td>62.47</td><td>[3]</td></tr><tr><td></td><td></td><td>66</td><td></td></tr></tbody></table>

Maximum research papers on RT dataset were described different methodologies used for sentiment analysis for obtaining best fitted model. Some research papers were focused on mainly binary classification, only a few papers were discussed on multi-class classifications on RT dataset. Our aim of the work is to categorize the sentiment of different text from the RT dataset by using multi-class classification. In this way, the user can easily understand the different reviews before taking any decisions. In this work, different ML models for example logistic regression, linear support vector classifier (linear SVC), and MNB are to be used for finding the best fitted model. Although similar models were used on RT dataset by different research groups with consideration of different features, still our aim is to find the best fitted model with higher accuracy by considering different parameters. Our final motive is to include deep learning (DL) models to obtain best results for sentiment analysis on RT dataset by using multi-class classification. Because deep learning model generally provides high accuracy and low computational time in order to identify text classification [3]. Moreover, very few research papers introduced DL models for analysis of RT dataset.
## 2 Methodologies

In this work, the entire process is separated into data investigation, data pre-processing, n-grams and term frequency inverse document frequency (Tf-idf) Vectorizer and multi label text classification for movie review sentiment analysis. Detailed methodologies are presented in Fig. 1.

<figure><img src="image_5.png" /><figcaption>Fig. 1. Entire model for multi-class</figcaption></figure>

### 2.1 Dataset Used

In this work, RT dataset is used from Kaggle website for sentiment analysis of movie review. The dataset is one of the most popular datasets for finding movie reviews, new reviews etc., as prepared by Pang and Lee [14]. The user can determine positive and negative opinions from the dataset, where positive means icon of red tomato while negative is identified by icon of green tomato. The aim of the work is to classify the sentiment of different sentences from this popular dataset of RT. Multi-class classification method means the dataset has more than 2 classes, is applied to develop sentiment analysis. In this direction, RT dataset is proposed to divide into five classes based on sentiments. The five corresponding classes are mentioned in Table 2.

### 2.2 Data Investigation

It is observed that the dataset is somewhat challenging since it is divided into different idioms and each idiom has a specific sentiment description. Moreover, barriers like
<table><caption>**Table 2.** Different classes are considered</caption><thead><tr><td>Class</td><td>Sentiment</td></tr></thead><tbody><tr><td>0</td><td>Negative</td></tr><tr><td>1</td><td>Somewhat negative</td></tr><tr><td>2</td><td>Neutral</td></tr><tr><td>3</td><td>Positive</td></tr><tr><td>4</td><td>Somewhat positive</td></tr></tbody></table>

sentence rebuttal, witticism, expressive style, language unclearness and many more form this project are very interesting from NLP perspective. Firstly, dataset is split into sub dataset for training purposes, and it is to be observed that total amount of data is 156060. Among them, 7072 negative reviews, 27273 somewhat negative, 79582 neutral review, 32927 positive and 9206 somewhat positive. Moreover, it is observed that the dataset is imbalanced. Hence, the dataset needs to be re-sampled to get better results.

## 2.3 Data Pre-processing

Data pre-processing is used to eliminate noise from data in order to pre-process the text data as suitable input format for ML algorithm [15]. Most of content into the dataset include lots of noise and crazy information like different tags, scripts which are desirable to be evacuated before feeding data into the algorithm. In order to pre-process the dataset, some sets of rules are designed.

* Initially punctuation marks are removed from the dataset.

* Then, change of case is to be followed to change all the words into lower case.

* Also, Lemmatizing is used to lemmatize the data. Lemmatizing means the method of grouping together variant forms of the same word so they can be considered as single item. The process is mainly used by the search engine to analyze data.

## 2.4 n-grams and Tf-idf Vectorizer

N-gram is an approach for checking 'n' continuous sequence of words in each text. The process is mainly used when text data is used in NLP tasks [2]. In this work, if we suppose to distinguish between the two words 'good' and 'not good', n-grams (1,2) can be used, which would allow to interpret the text or speech better as it would take into consideration both the words collectively, rather than a single word. Since, it will pick only single word and group of 2 consecutive words for distinguishing the data.

Tf-idf Vectorizer, one of the common approaches is to modify the text into significant numbers or vectors which can easily fit into the ML algorithm for classification [2]. Tf-idf consists of two different terms, term frequency (tf) and inverse document frequency (idf). The term frequency gives the number of appearances of a particular word in each sentence, and it provides the importance of the word in that sentence. Document frequency gives the number of sentences including a particular word. Now, inverse document frequency means to reduce the weight of a word if the word appearance is scattered
in the entire sentences. Therefore, Tf-idf determines the entire sentences weightage of a word and punishes the most recurrent words. In this work, Tf-idf Vectorizer was used during data pre-processing, which assists us to stop providing unwarranted reward to the stop words (since they have higher frequency in a document) and yields more favoritism to the appropriate words.

## 2.5 Machine Learning (ML)

In this work, different ML methods were used to predict the multi-class classification problem. Logistic regression, linear SVC, and MNB algorithms are used for classification approach.

### Logistic Regression

Application of logistic regression for sentiment analysis is discussed here. Logistic regression is found to be under exponential classifier family. Generally, logistic regression is mainly used to identify two different classes. It calculates the probability of occurrence of event, such positive or negative, from the independent input presented in given dataset. The value of probability or dependent output is in the range between 0 and 1. A logistic function is applied to the term odd (the probability of success divided by probability of failure) in logistic regression. Moreover, it selects set of weighted attributes from the input data and then combine all linearly followed by taking logs [1, 3]. The equations for logistic regression are expressed in Eq. (1) to (3).

$$odd = \operatorname{logit}(P) = \ln \frac{P}{1-P} \qquad (1)$$

$$P = \frac{\exp(\alpha * X1 + \beta X2 + \gamma)}{1 + \exp(\alpha * X1 + \beta X2 + \gamma)} \qquad (2)$$

where $\gamma$ is the bias, $\alpha$ and $\beta$ are the weights, and X1, X2 is the attributes collected from input data. Different values of $\alpha$ and $\beta$ are tested by this method through multiple iteration techniques to optimize the error. In order to determine the exact values of $\alpha$, $\beta$ and $\gamma$ for optimizing the error, gradient descent algorithm [1] is applied as shown in Eq. (3)

$$J(\theta) = \frac{-1}{m} \sum_{i=1}^{m} [y^i \log y + (1 - y^i) \log(1 - \log \log y)] \qquad (3)$$

m is the total number of sample size; y is the ground truth of the rating data. After optimization of error the value of the probability is calculated for prediction of class. For multi-classification, prediction of the class is found based on the highest probability. In this work, the One-vs-Rest classifier approach from logistic regression is used for finding multi-class labels. In this approach, one class is fitted for each data point against all other classes. The system predicts the class for each data point with the highest probability.
## Multinomial Naive Bayes (MNB)

The algorithm is also used for supervised learning classification and is mainly popular for analysis of categorical test data. It is basically a probabilistic classifier approach which is defined by Bayes' hypothesis [3]. Due to the less time consumption and faster nature, Naive Bayes approach is used in prediction analysis of large dataset. In Naive Bayes approach, one feature should not be related to all other features. MNB is another version of Naive Bayes theorem which is designed to handle test documents by using word counts followed by probability calculations. In MNB algorithm, the probability of each tag such as a portion of article is calculated, and correspondingly highest probability of tag is estimated as output.

Suppose each tag appeared in documents from a particular class C. Based on the documents classified previously, this classification is carried out. The words presented in tag are defined by vector w. Each attribute wi of w is an appearance frequency of i-th term in that document. Now the classification is carried out by calculating the probability of P(wi | C) i.e. word presented in tag appeared in the documents from class C. The Bayesian probability P(C | wi) i.e. probability of class for particular tag is computed as Eq. (4). This is a method that can be used for analysis of multi-classification.

$$P(C|w_i) = \frac{P(C) \prod_{i=1}^{n} P(w_i \lor C)}{P(w_i)} \quad (4)$$

## Linear SVC

Linear SVC algorithm is applied in this work to categorize the text document from sentiment analysis movie dataset. Due to its advantages like flexibility to any sort of attributes, high compatibility for linear distinguishable data and robustness in high dimensional space, linear SVC algorithm has become popular for solving text categorical problems. By using linear SVC, optimized results can be obtained for multi-classification [16]. It is a one of binary linear classifier. The algorithm creates hyperplane among different classes, in which best-fit hyperplane rectifies the edge presented among different classes. After obtaining the best-fit hyperplane, some features can be added into the algorithm to find the predicted class.

## 2.6 Deep Learning (DL)

In this work, three special kinds of DL algorithms such as RNN, Convolutional Neural Network (CNN) and CNN + Gated Recurrent Unit Network (GRU) were incorporated. Before sending data into the DL algorithm, data pre-processing is required.

### Tokenizer

Text-strings data cannot directly be fed into the neural network, some data processing is required. The first step is the tokenizer, that converts text data into the integers in the form of token in order to get suitable input format for DL [4]. The second part is an interconnected part of the neural network itself and is denoted as the “embedding”-layer.
## Sequence Padding

Padding is one kind of masking method where zero is added at start or end of the sequence. It is also known as sequences data encoding process. It converts all variable length sequences into given standard or same length for correct prediction of variable length sequence data.

## Recurrent Neural Network (RNN)

RNN is one of the types of neural network (NN) which is used to make sequence analysis process. RNN includes various consecutive recurrent layers, and all layers are consecutively modeled for mapping of sequences. The first layer of RNN is called Embedding-layer, transforms each integer-token into a vector values. Since RNN does not have ability to work on values in the range of 0–10000, it is stored in integer-token. The embedding-layer of RNN is trained and it maps words with similar interpretation meanings to same kind of embedding-vectors. Moreover, RNN has high potential to retrieve appropriate data from the sequences. Such data is significantly used for multi-class classification.

The input of the RNN in the form of sequence of vector values is {x1,x2,...xt}, the sequence of hidden states is considered as { u1,u2,...ut}, and the output unit as { y1,y2,...yt}. By using recurrent function, current hidden state can be calculated which is shown in Eq. (5),

$$u_t = tanh(Ax_t + Bu_{t-1}) \quad (5)$$

Furthermore, current state of output unit is calculated as shown in Eq. (6),

$$y_t=Cu_t \quad (6)$$

where, A, B and C represent the weight features, and tanh indicates the hyperbolic tangent function.

After getting the final predicted output, it is compared with the actual target followed by generation of the error. The error is then provided as a feedback to the network to update the weight features and in this way, the network (RNN) works.

RNN extensions (LSTM) learn sequential data and remember long-range connections better than standard RNN [4]. So, we have used LSTM model in our work.

## Convolutional Neural Network (CNN)

CNN is one type of multi-layered artificial neural network which can detect complex text data and image. It is mostly used for image classification and text classification of sentiment analysis. CNN comprises of input layer, output layer and various hidden layers. The input layer should be different for different applications. The hidden layer includes convolutional layers and pooling layers. Output layer is linked with a fully interconnected layer. For identifying a text as positive or negative, several processes must be followed. Each word of text coming from input is depicted by a vector. Four various types of filters are applied on the vectors to generate convolutional feature map. For representation of pooling, maximum value is to be chosen from each vector. Finally
for classification, SoftMax is applied on vector to transform that vector into different size of vector [17, 18].

One of the variants of CNN is the 1-Dimensional (1D) CNN model. In the case of 1D convolution model, kernel moves in one dimension i.e., time. So, the kernel moves through the word embeddings sequence wise. So, the 1D CNN model is used in NLP, especially for sentiment analysis problems, where sequence of the words is relevant and short [19].

## CNN + GRU

The modified model consisting of CNN and GRU can maximize the identification of text structure [17, 18]. In this algorithm, the embedding layer of word is the primary layer which integrates each text data into real vector area. Then the embedding layer transfers the input text data in the vector matrix form into the drop-out layer. The output of the drop-out layer is entered into the convolutional layer by adding multiple filters. Before that, padding is required at the input terminal so that length of output data should be as same as input data. For extracting the feature, the shape of the input vector matrix is reduced to smaller shape by 1D max pooling layer and then transferred into the GRU layer which provides dimension of feature in the form of timesteps. Afterwards, the global max pooling layer again re-shapes the output vector in a particular dimension. At the end, SoftMax layer considers re-shaped output vector as an input for predicting the probability of sentiment analysis.

In order to enhance the sentiment feature, CNN + GRU algorithm is mainly used. The specialty of GRU is that they can retain historical information without losing it through time and remove such features which is not relevant for the problem statement [17, 18].

Logistic regression, linear SVC, and MNB algorithms from ML and RNN, CNN, and CNN + GRU from DL were used for calculating mean accuracy in order to determine the best model for multi-classification.

# 3 Experiments and Results

In this work, firstly data pre-processing such as re-sampled, lemmatize, n-grams and Tf-idf Vectorizer, Tokenizer and padding was done on RT dataset in order to make suitable input format for ML and DL algorithms. Total process is carried out by using two different methods, ML and DL. Firstly, Logistic regression, linear SVC, and MNB algorithms from ML were used for calculating the mean accuracy and standard deviation. The formula used for estimating mean accuracy and standard deviation are shown in equation no (7)-(9).

Accuracy is one of the parameter metrics, which is used for calculating percentage of accurate prediction in multi-class classification approach. It gives information about the number of correct predictions from all the predictions.

$$ Accuracy = \frac{\text{Number of correct predictions}}{\text{total numbers of predictions}} \qquad (7) $$
For binary and multi-classification, the formula for calculating accuracy also can be written as

$$Accuracy = \frac{TP + TN}{TP + TN + FP + FN} \qquad (8)$$

In this Eq. (8), numerator contains the summations of all TP (true positive) and TN (true negative). TP and TN are the values which are correctly forecasted by the model. Denominator contains all the values which are correctly and wrongly predicted by the model such as TP, TN, False Positive (FP) and False Negative (FN). Accuracy offers the percentage of how many values are correctly classified from the entire dataset.

Standard deviation is used to estimate the average inconsistency of the entire dataset. The equation is described below.

$$S = \sqrt{\frac{\sum (X - \bar{X})^2}{N}} \qquad (9)$$

In this Eq. (9), X is the value in data distribution, N is the total number data size, X' is the mean value and S is the Standard deviation.

Firstly, logistic regression model was used to find the accuracy of the model. After that linear SVC and MNB were used to find optimized value of accuracy. For fitting the model by using logistic regression approach, One v/s Rest classifier was used on the dataset. The One v/s Rest classifier is used to predict multi-class labels, where it considers one class versus other classes at one point of time. So, it separates multi-class problems into various binary classes. After fitting the model with One v/s Rest classifier, cross validation mean accuracy and standard deviation were calculated. Cross-validation mean accuracy is the percentage of correct prediction analyzed through cross-validation approach. Cross-validation approach is the method which uses various portions of data for testing and training with multiple iterations. Two more models such as linear SVC and MNB are applied on dataset for getting higher accuracy and lesser time consumption. The values of accuracy, standard deviation and CPU times are represented in Table 3 and described in Fig. 2.

<figure><img src="image_6.png" /><figcaption>Fig. 2. Values of accuracy, standard deviation and CPU times for logistic regression, linear SVC and MNB models</figcaption></figure>
<table><caption>**Table 3.** Values of accuracy, standard deviation and CPU times for logistic regression, linear SVC and MNB models.</caption><thead><tr><td>Name of Algorithm</td><td>Accuracy</td><td>Standard deviation</td><td>CPU times</td></tr></thead><tbody><tr><td>logistic regression</td><td>72.17%</td><td>0.42</td><td>31.1 s</td></tr><tr><td>linear SVC</td><td>77.44%</td><td>0.46</td><td>900 ms</td></tr><tr><td>MNB</td><td>59.50%</td><td>0.15</td><td>889 ms</td></tr></tbody></table>

It is to be noticed from Table 3 that the value of accuracy for linear SVC is highest and it is 77.44%. If we observed CPU consumption time, it is less for MNB. The value of accuracy and CPU consumption time are poor in case of logistic regression as depicted in Fig. 2. After investigation of all the values, it is to be concluded that linear SVC is the best fitted model for identifying the multi-classification of RT due to highest accuracy. Moreover, in case of multi-classification problems, linear SVC is more suitable to identify the data [16]. After that, the dataset is separated into two parts in 0.75:0.25 ratios for training and testing respectively. Accuracy is less for all ML algorithms as the problem of sentiment analysis is associated with multi-class classification [3]. In order to overcome the low accuracy of multi-class classification problem, we have focused on DL algorithms.

DL algorithms can detect complex text datasets, so that higher accuracy can be achieved. Hence the second part of the work, RNN/LSTM, CNN and CNN + GRU were used from DL for comparing accuracy and loss function. Loss function is a function which gives the deviation value between predicted and actual output. If there is less deviation between predicted and actual output, then the neural network models fit the training data very well. Before sending input data into those algorithms, tokenizer and sequence padding were carried out for getting suitable input format. After tokenizing and sequence padding, the dataset is ready to feed into DL models.

Firstly, LSTM variant from RNN model was used in which LSTM layer is added in between embedded layer and Softmax activation function. The architecture of LSTM model is described below.

## Architecture of LSTM

* LSTM layer Neurons 64, Dropout 0.4, Recurrent dropout 0.4

* LSTM layer Neurons 32, Dropout 0.5, Recurrent dropout 0.5

* Dense layer classes 5, activation Softmax

75% data from entire dataset was trained for fitting the model and rest of data was kept for validation purpose. The values of accuracy and loss function for (RNN)/LSTM, CNN and CNN + GRU models are depicted in Fig. 3. After fitting and predicting the model using LSTM, accuracy of 72.98% and loss function of 67.31% were found, as shown in Fig. 3.

Next, CNN model was used for comparing the value of accuracy with RNN model. The entire CNN model includes embedded layer, spatial dropout layer, convolutional layer, MaxPooling layer and dense layer as output. The architecture of CNN model is described below.
Architecture of CNN

* Convolution layer 1D Neurons 32, kernel size 3, activation Relu

* MaxPooling layer 1D

* Convolution layer 1D Neurons 64, kernel size 3, activation Relu

* MaxPooling layer 1D

* Flatten

* Dense layer classes 5, activation sigmoid

After fitting and predicting the model using CNN with same train and test ratio, accuracy of 80% and loss function of 53% were obtained.

Finally, CNN + GRU algorithm was mainly used to enhance the sentiment feature. The entire CNN + GRU model consists of embedded layer, convolutional layer, MaxPooling layer, dropout layer, GRU layer and dense layer with Softmax feature. The architecture of CNN + GRU model is described below.

Architecture of CNN + GRU model

* Convolution layer 1D Neurons 64, kernel size 3, activation Relu

* MaxPooling 1D

* Dropout 0.25

* GRU layer Neurons 128

* Dropout 0.3

* Dense layer Neurons 128, activation Relu

* Dropout 0.5

* Dense layer classes 5, activation softmax

Accuracy of 78.84% and loss function of 55.51% were found for this CNN + GRU model. From Fig. 3, it is also observed that, the value of loss function is decreased with increase in the value of accuracy. Summarize values of different DL models are shown in Table 4. It is experimented from Table 3 that all the models perform well on the train data. However, CNN model is considered as best fitted model with higher accuracy as

<figure><img src="image_7.png" /><figcaption>Fig. 3. Values of accuracy and loss function for (RNN)/LSTM, CNN and CNN + GRU models</figcaption></figure>
least error is predicted than other models by using loss function. To confirm the same, new test data are considered which will be fed into three models of DL.

<table><caption>**Table 4.** Values of accuracy and loss for RNN/LSTM, CNN and CNN + GRU models.</caption><thead><tr><td>Name of Algorithm</td><td>Accuracy</td><td>Loss function</td></tr></thead><tbody><tr><td>RNN/LSTM</td><td>72.98%</td><td>67.31%</td></tr><tr><td>CNN</td><td>80%</td><td>53%</td></tr><tr><td>CNN + GRU</td><td>78.84%</td><td>55.51%</td></tr></tbody></table>

After optimizing the fitted model, new text data are considered for model testing and prediction of new text data. New text data are followed-

Text1 = "This movie is fantastic I really like it because it is so good!".

Text2 = "Good movie!".

Text3 = "Maybe I like this movie!".

Text4 = "Meh!".

Text5 = "If I were a drunk teenager then this movie might be good".

Text6 = "Bad Movie!".

Text7 = "Not a good movie!".

Text8 = "This movie really sucks! Can I get my money back please!".

Test results = ['Test1', 'Test2', 'Test3', 'Test4', 'Test5', 'Test6', 'Test7', 'Test8'].

All the test data are stored in test variables in the form of array. Before sending new text data to the different models for predicting, data pre-processing such as tokenize, and sequence padding were carried out. After data pre-processing, new text data were fed

<figure><img src="image_8.png" /><figcaption>**Fig. 4.** Predicted results for Neural Network (RNN)/LSTM, CNN and CNN + GRU models. In the type of class axis, 0- Negative, 1- Somewhat negative, 2 – Neutral, 3 – Positive, 4 - Somewhat positive.</figcaption></figure>
into the three different DL algorithms. Predicting results for three different algorithms are depicted in Fig. 4.

Different types of classes for sentiment analysis are already mentioned in Table 2. It is to be observed from both Table 5 and Fig. 4, the predicted results are quite close for various models, the output class of the 2nd model (CNN) is closest to the desired output. Therefore, the predicted model is optimized and seems to give correct predictions for sentiment analysis of RT. By using optimized CNN model, we can predict sentiment analysis of new text data.

<table><caption>**Table 5.** Predicted results for Neural Network (RNN)/LSTM, CNN and CNN + GRU models.</caption><thead><tr><td>Name of model</td><td>Predicted results</td></tr></thead><tbody><tr><td>LSTM</td><td>[4 4 2 2 1 0 1 0]</td></tr><tr><td>CNN</td><td>[4 4 3 2 1 0 1 0]</td></tr><tr><td>CNN + GRU</td><td>[4 4 2 2 1 0 1 0]</td></tr></tbody></table>

## 4 Conclusions

The work plan of the paper is to explore a comparison of different multi-classification models such as ML and DL models used to categorize the sentiment of sentences from the RT movie review dataset. In order to attain the highest performance, a total of six machine learning followed by deep learning algorithms were evaluated: logistic regression, linear SVC and MNB, RNN/LSTM, CNN and CNN + GRU. In this paper, we have observed that linear SVC shows significant improvement among all other ML algorithms. In addition, when DL algorithms were incorporated in the model, CNN provides highest accuracy of 80% with loss function of 55.51% among all other algorithms. After optimizing the fitted model, a total of eight new text data are opted for testing of correct predictive model. Optimized predicted model, i.e., CNN seems to offer correct predictions for new test data. Therefore, it is to be concluded that, CNN model is supposed to be used as a suitable model for sentiment analysis of RT dataset as there is improvement of multi-class classification accuracy.

## References

1. Suhariyanto, Firmanto, A., Sarno, R.: Prediction of movie sentiment based on reviews and score on rotten tomatoes using sentiwordnet. In: 2018 International Seminar on Application for Technology of Information and Communication, pp. 202–206, IEEE Indonesia (2018). https://doi.org/10.1109/ISEMANTIC.2018.8549704

2. Tiwari, P., Mishra, B.K.K., Kumar, S., Kumar, V.: Implementation of n-gram methodology for rotten tomatoes review dataset sentiment analysis. Int. J. Knowl. Discov. Bioinform. 7(1), 30–40 (2017). https://doi.org/10.4018/IJKDB.2017010103
3. Jain, S., Shrikant, M., Mishra, R., Tiwary, U.S.: Sentiment analysis: an empirical comparative study of various machine learning approaches. In: Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017), pp. 112–121, NLP Association of India, Kolkata, India (2017). https://aclanthology.org/W17-7515

4. Wu, J.Y., Pao, Y.: Predicting sentiment from rotten tomatoes movie reviews (2012). https://nlp.stanford.edu/courses/cs224n/2012/reports/WuJean_PaoYuanyuan_224nReport.pdf https://doi.org/10.1109/ISEMANTIC.2018.8549704

5. Huaxia, R., Yizao, L., Andrew, W.: Whose and what chatter matters? the effect of tweets on movie sales. Decis. Support Syst. 55(4), 863–870 (2013). https://doi.org/10.1016/j.dss.2012.12.022

6. Sankar, H., Subramaniyaswamy, V.: Investigating sentiment analysis using machine learning approach. In: International Conference on Intelligent Sustainable Systems (ICISS), IEEE, pp. 87–92, December 7–8, (2017). https://doi.org/10.1109/ISS1.2017.8389293

7. Walaa, M., Ahmed, H., Hoda, K.: Sentiment analysis algorithms and applications: a survey. Ain Shams Eng. J. 5(4), 1093–1113 (2014). https://doi.org/10.1109/ISS1.2017.8389293

8. Yao, Y., Angelov, I., Rasmus-Vorrath, J., Lee, M., Engels, Daniel, W.: Yelp's review filtering algorithm, SMU Data Science Review: 1(3), Article 3 (2018). https://scholar.smu.edu/datasciencereview/vol1/iss3/3

9. Onalaja, S., Romero, E., Yun, B.: Aspect-based sentiment analysis of movie reviews. SMU Data Sci. Rev. 5(3), 1–21 (2021). https://doi.org/10.1109/ISCMI.2014.16

10. Pang, B., Lee, L., Vaithyanathan, S.: {T} humbs up? {S} entiment classification using machine learning techniques, In: Proceeding of the EMNLP’02, (2002). https://doi.org/10.3115/1118693.1118704

11. Bo, P., Lillian, L.: Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the 43rd annual meeting on association for computational linguistics. Association for Computational Linguistics, pp.115–124, (2005). https://doi.org/10.3115/1219840.1219855

12. Salvetti, F., Lewis, S., Cheng, X.: Automtatic opinion polarity classification of movie. Colorado Res. Linguist. 17(2), (2009). https://doi.org/10.25810/atv1-v819

13. Matsumoto, D., Hwang, H.C.: Assessing cross-cultural competence: a review of available tests. J. Cross-cult. Psychol. 44(6), 849–873 (2013). https://doi.org/10.1177/0022022113492891

14. Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp. 271–278, (2004). https://doi.org/10.3115/1218955.1218990

15. Emma, H., Xiaohui, L., Yong, S.: The role of text pre-processing in sentiment analysis. Procedia Comput. Sci. 17, 26–32 (2013). https://doi.org/10.1016/j.procs.2013.05.005

16. OKeefe, T., Koprinska, I.: Feature selection and weighting methods in sentiment analysis. In: Proceedings of the 14th Australasian Document Computing Symposium, Sydney. Citeseer, pp.67–74, (2009)

17. Shujuan, Y., Danlei, L., Wenfeng, Z., Yun, Z., Shengmei, Z.: Attention-based LSTM. J. Intell. Fuzzy Syst.: Appl. Eng. Technol. 39(1), 333–340 (2020). https://doi.org/10.3233/JIFS-191171

18. Al, M.N., McGough, S., Awwad, S.H.B.: Beyond the topics: how deep learning can improve the discriminability of probabilistic topic modelling. Peer J. Comput. Sci. 6, e252 (2020). https://doi.org/10.7717/peerj-cs.252

19. Yoon, K.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp.1746–1751 (2014). https://doi.org/10.3115/v1/D14-1181