# JUMRv1: A Sentiment Analysis Dataset for Movie Recommendation
## 3 Dataset  
In the present work, we developed a new dataset called JUMRv1, which is to be used in movie recommendation research. We named our dataset in this way as this was prepared in the lab of Jadavpur University, Kolkata, India. Scraped from the Internet Movie Database ([[https://www.imdb.com/search/title/?title_type=feature,tv_series&count=250&start=001&ref_=adv_nxt](https://www.imdb.com/search/title/?title_type=feature,tv_series&count=250&start=001&ref_=adv_nxt)]([https://www.imdb.com/search/title/?title_type=feature,tv_series&count=250&start=001&ref_=adv_nxt](https://www.imdb.com/search/title/?title_type=feature,tv_series&count=250&start=001&ref_=adv_nxt)), accessed on 10 May 2020), JUMRv1 consists of the top 25 reviews for 60 movies, totalling 1500 reviews. We used the Beautiful Soup 4 library ([PERSON] (2018)) for the scraping. The reviews were annotated as 1, 0, or \(-1\), representing positive, negative, and neutral categories, respectively.  
Annotating, though tiresome, is a step of extreme importance as this is going to be reflected in the final dataset that has to be fed to the classifiers. While some reviews were clear and condensed, many reviews were from professional critics who used comprehensive explanations, analogies, and metaphors that were mostly directed at cinema enthusiasts. We ensured constant communication in order to sustain maximum F\({}_{1}\) scores in these annotations. As expected, the annotations revealed some class imbalance. That is, one category outnumbered the others. The class imbalance found in the developed database is as follows (in terms of number):  
* Positive (1): 833
* Neutral (0): 301
* Negative (\(-1\)): 288  
The problem with making natural language datasets is the complexity of human language, with new words creeping into use every day and new semantics arising out of these new words. The addition of the neutral class to the annotation makes the database-making process even more difficult. Some examples taken from JUMRv1 are shown in Table 1.  
In Table 2, a comparison between the four most popular movie review sentiment analysis datasets and JUMR v1.0 is provided, in terms of when they were last updated, when they were created, how many reviews they hold, and how many classes they have. From this table, it is clearly seen that JUMRv1 holds an advantage over the rest as it provides both comparatively newer movie reviews as well as annotates the reviews into three different classes. On the other hand, the most popular datasets only provide a two-class classification and a rather old set of movies, most of which are irrelevant today.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Review** & **Sentiment** & **Corresponding Annotation** \\ \hline “Amazing. [PERSON] always makes masterpieces...” & Positive & 1 \\ \hline “Mediocre performance by [PERSON] & & \\ [PERSON]. The plot is good, worth & Neutral & 0 \\ a watch...” & & \\ \hline “This movie is an utter disaster. This & & \\ is nothing like the book, full of & & \\ inaccuracies. A buff batman & Negative & \(-1\) \\ is unnatural...” & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Examples of movie reviews for each class.  
The dataset has reviews which vary from being sarcastic and ironic to serious and straightforward. This variation of behaviour has resulted in difficulties in the experiments performed to train the classifiers, and has led to a decrease in classification F\({}_{1}\) scores, as seen in the result section. We created a 70:30 train-test split of the dataset for experimentation purposes.