# Improving Graph-Based Movie Recommender System Using Cimenatic Experience +
## 4 Experiment
### Results  
Through the experiment, we investigated whether or not the recommendation performance could be improved by using additional relational data such as CX data in the GNN method. The results of comparing performance with Rotten Tomatoes and Amazon data are as follows. The results are presented for Rotten Tomatoes and Amazon data in Tables 1 and 2, respectively. Table 1 shows the best RMSE of Rotten Tomatoes data using traditional models and GNN-based models. The conventional and GCMC models used only ratings, while IGMC models used ratings and additional data. In the traditional models, user-based CF produced the worst results, and its performance was lower than item-based CF, supporting the previous observations that item-based CF often outperforms user-based CF [19]. The SVD model performed better than the CF, and the SVD++ model outperformed the other traditional models. \begin{table}
\begin{tabular}{c c} \hline \hline
**Model Type** & **Best RMSE** \\ \hline CF (user-based) & 0.9233 \\ CF (item-based) & 0.8818 \\ SVD & 0.8425 \\ SVD++ & 0.8328 \\ GCMC & 0.8135 \\ IGMC & 0.8051 \\ IGMC + CX (ours) & 0.8004 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: RMSE Table of Rotten Tomatoes data. As the graph reflects the complex information between users and items, GNN-based models performed better than traditional models such as CF and SVD. Although the IGMC model did not use side information, it outperformed the GCMC model supporting that it is the current state-of-the-art GNN model. In the end, the result of the IGMC model with Cinematic Experience (CX) outperformed the basic IGMC. According to [6], there is a possibility that using side information in IGMC could further improve performance, and our study shows that the model's performance can be improved with CX information, indeed. Table 2 shows the best RMSE of Amazon movie data using traditional models and GNN-based models. Unlike Rotten Tomatoes' results, user-based CF performed better than item-based CF, but the difference of RMSE is slight. As observed in the Rotten Tomatoes dataset, the SVD model performed better than the CF, and the SVD++ model outperformed the other traditional models. We observed that the SVD++ model was performing better than GCMC. Although GNN-based models generally perform better than traditional models, we confirmed that traditional models can be effective depending on the characteristics of the data. Yet, the IGMC model with Cinematic Experience (CX) performed better than the basic IGMC. We attribute the superior outcome to the fact that the graph features interact with various relations. In GNN, the features between the nodes and edges are transmitted through the message passing method. By doing so, more diverse information would be delivered within the model according to the combination of the edge types and node features. Meanwhile, the proposed model extracts the features, which are not revealed in the rating, from the review data by representing those as graphs. In the end, a multi-modality recommendation can achieve improved performance by transforming graphs with different dimensions such as sentiment, emotion into embedding in the same dimension. _E. Ablation study of IGMC model_  
We conducted an ablation study as shown in Table 3 to discover features that affected the recommendation performance. The aforementioned experiment results showed that the rating-based recommendation approach is limited, and the CX information can be used as boosting features to improve performance. Since sentiment clearly showed a positive correlation with rating in the experiment, it showed a high-performance improvement when combined with rating. On the other hand, emotion did not correlate with rating. Yet, there was a correlation with emotion types such as sadness and joy, which contributed to improved performance.