# Re-Enrichment Learning: Metadata Saliency for the Evolutive Personalization of a Recommender System
## 4 Experiment and Discussion
### Results and Discussion  
#### 4.2.1 Recommending Movies  
At the beginning, users were asked to respond to a survey on their preferred categories as a cold start. Otherwise, users could also skip it to simply use the given averaged preference model built upon the feedback collected from other previous users. Whenresponding to the survey, the selected categories directly created an initial metadata saliency. Then, a set of 10 movies, with their images and titles, was recommended to a user at a time, and the user was allowed to click on any items they were interested in. The user could also skip the entire set and go on to the next set. Table 2 shows our A/B experiments, and it compares the results of all baseline comparison methods with respect to CTR and MAP@K for the MovieLens dataset. As demonstrated in Figure 6, our proposed re-enrichment learning helps baseline methods improve the resulting retention of users by providing a feature that enriches the interest areas of the user instead of overspecializing them. Compared to the intuitive metric, CTR, MAP@K additionally reflects the order in which the user engages with items within a recommended set, so we can infer that our approach has nothing to do with recommending more attractive items in the front. Still, the results of CTR and MAP@K show a consistent tendency for improvement. In addition, Table 3 presents the results of the Wilcoxon test which essentially calculates the difference between sets of paired samples and analyzes these differences to establish if they are statistically significantly different from one another. The resulting \(p\)-values in all cases were much smaller than the significance level \(\alpha\), which allows the conclusion that the differences between the population distribution and the hypothesized distribution are statistically significant. This observation is supported by Figure 7, which shows an example comparing the initial metadata saliency of a user to the metadata saliency variations over 30 implicit feedbacks. The \(x\)-axis presents the node of a graph (i.e., movie genre), while the \(y\)-axis shows the attribute of a node (i.e., the level of metadata saliency). As can be observed in Figure 6(a), the user started with two large saliencies at Action and Sci-Fi. Despite the fact that the Action and Sci-Fi genres accounted for the majority of the 30 feedbacks and the two were still ranked in the two largest saliencies, the metadata saliencies of other genres (Adventure and Thriller) prominently increased because they had relatively large \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{5}{c}{**CTR**} & \multicolumn{5}{c}{**MAP@K**} \\  & \(A\) & \(B\) & \(B-A\) & \((B-A)/A\) & \(A\) & \(B\) & \(B-A\) & \((B-A)/A\) \\ \hline SVD & 14.76\% & **18.52**\% & +3.76\% & \(\uparrow\) 25.47\% & 0.306 & **0.339** & +0.033 & \(\uparrow\) 10.91\% \\ BPR & 17.10\% & **20.90**\% & +3.80\% & \(\uparrow\) 22.22\% & 0.318 & **0.368** & +0.049 & \(\uparrow\) 15.51\% \\ L-GCN & 17.40\% & **20.37**\% & +2.97\% & \(\uparrow\) 17.07\% & 0.341 & **0.363** & +0.022 & \(\uparrow\) 6.54\% \\ L-FM & 12.89\% & **16.91**\% & +4.02\% & \(\uparrow\) 31.19\% & 0.280 & **0.329** & +0.049 & \(\uparrow\) 17.56\% \\ W\&D & 14.28\% & **17.18**\% & +2.90\% & \(\uparrow\) 20.31\% & 0.290 & **0.324** & +0.034 & \(\uparrow\) 11.77\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The results of A/B experiments on the MovieLens dataset. Figure 6: Visualization of A/B experiments on the MovieLens dataset. edge attributes with Action and Sci-Fi (see the thicker lines in Figure 7a). We depict all the edges connected to Action as red-colored lines and all the edges connected to Sci-Fi as blue-colored lines, which show that Adventure and Thriller are fairly interrelated with Action and Sci-Fi with respect to the thickness of the lines. It is worth observing that Adventure and Thriller were encouraged to be recommended, and were consequently selected quite a few times.