# Enhancing Sequence Movie Recommendation System Using Deep Learning and KMeans
## 4 Experimental Study
### Evaluation Metric  
To check the performance of our sequential recommendation system, we adopted the leave-one-out evaluation for the next-item recommendation task. For each user, we used the last movie of the behavior sequence as the test set and utilized the remaining movie for the training set. We followed the common convention of pairing each ground truth in these test sets with 100 random negative items that the user had not interacted with, giving them all the same context as their corresponding positive test item, and ranking the positive test item among them. Lastly, we truncated the rank list at the threshold value for each user. We measured the overall quality using the hit rate (_HitRate@k_) and normalized discounted cumulative gain (NDCG@k). These mercies can be calculated as follows:  
\[HitRate@k=\frac{Number\ of\ HitRate@k}{k} \tag{8}\]  
where _HitRate@k_ is the number of correct predictions in the Top-N recommendations, and \(k\) is the number of recommendations or the cutoff value for which we want to calculate the HitRate. In our experiments, all datasets had a rating range of 1-5, and we set the threshold at 3.5. If the threshold for an item was not met, we omitted the item.  
The second metric that was adopted was the normalized discounted cumulative gain (_NDCG@k_), which measured the Top-N recommendation system list quality, expressed as follows:  
\[DCG_{k}=\sum_{i=1}^{k}\frac{2^{rel_{i}}-1}{log_{2}i+1},\,NDCG_{k}=\frac{DCG_{K }}{IDCG_{K}} \tag{9}\]  
where \(rel_{i}\) was the actual rating score of an item at the rank position \(i\) For our experiment, we gave results for HitRate and NDCG with \(k=[5,\,10]\). For all metrics, a high value corresponded to a better performance.  
Moreover, we used root mean squared error (RMSE) to measure the proposed model performance. RSME was the approach we used to measure the error rate a user gave to the system and the error predicted by the model. For example, the RSME is shown as follows:  
\[\text{RMSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(y_{i,j}-\hat{y}_{i,j}\right) ^{2}} \tag{10}\]  
where \(y_{i}\) is the actual value, \(\hat{y}_{i}\) refers to the predicted value of the observation, and n refers to the total number of data points.  
To assess the diversity of the recommended movies, we utilized a metric based on the pairwise distances between the movie's embedding. These embeddings are high-dimensional vectors that represent movies in a latent space, capturing various aspects of the movie's genre. Then, the K-means algorithm was used to cluster these embeddings, grouping similar movies together based on their features to make our recommendations. We calculated the Euclidean distance between every pair of movie embeddings in the recommended list. Finally, the diversity score was defined by taking the average of all these pairwise distances, expressed as follows:  
\[\text{ILD}=\frac{2}{n(n-1)}{\sum_{i=1}^{n-1}{\sum_{j=i+1}^{n}{\lVert\text{E}_{i} -\text{E}_{j}\rVert}}} \tag{11}\]  
where \({\lVert\text{E}_{i}-\text{E}_{j}\rVert}\) represents the Euclidean distance between the embeddings of movies \(i\) and \(j\), and \(n\) is the total number of recommended movies.