# Re-Enrichment Learning: Metadata Saliency for the Evolutive Personalization of a Recommender System
## 4 Experiment and Discussion
### Results and Discussion
#### 4.2.2 Recommending Goods  
Likewise, the preference survey was followed by a set of 10 recommendations, and users could click on any items they wished. Experiments were carried out using the Amazon dataset [44; 45], which contains Amazon products' review data as well as the products' metadata. Table 4 reports the results of our A/B experiments, and Figure 8 demonstrates their visual comparisons. We observed that the methods with re-enrichment learning outperformed their baselines, generally producing increased CTR and MAP@K. The results of the Wilcoxon test on the Amazon dataset show more or less the same tendency as those of the _MovieLens_ dataset. As shown in Table 5, the _p_-values were fairly smaller in all cases than the significance level \(\alpha\), which means that the differences between \(A\) and \(B\) were significant. Figure 8: Visualization of A/B experiments on the Amazon dataset. Figure 9 demonstrates an example of the variations in metadata saliency over 30 feedbacks by a user. This example is interesting because the user did not participate in the preference survey at the beginning. As such, the average metadata saliency from 70\(M\) previous feedbacks was used for the initial metadata saliency (see Figure 9a). The two categories, Digital music and Gift cards, initially obtained relatively large metadata saliencies. On the other hand, the user mostly selected Appliances, Cell phones and accessories, Industrial and scientific, and especially Sports and outdoors. Consequently, the metadata saliency levels of those categories increased during the latter feedback (see Figure 9b). The rest of the categories moved opposite to the most selected ones, yet the saliency levels, of which the edge attribute was large (see the thicker lines in Figure 9a), were rather slightly raised. Re-enrichment learning noticeably raised the click-through rates of the collaborative approaches. The advantage of collaborative approaches is that adding new data in an incremental manner is relatively easy, and re-enrichment learning is quite well matched with this advantage. It is easy to add a new node and its edges, starting with initial attributes set to zero. Furthermore, our proposed approach supplements the weakness of collaborative approaches when very few explicit feedbacks, i.e., ratings, are limitedly collectable from users. Furthermore, we observed that hybrid approaches with re-enrichment learning also outperformed their baselines. Re-enrichment learning effectively exploits the implicit logged feedback information and led to better click-through rate results. The observation from Figure 9 intuitively conveys that the interest areas that have close relations in the graph-based transferred domain to the areas selected by a user were nourished due to re-enrichment learning. In consequence, the recommendations based on re-enrichment learning encouraged users to consume more content. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**CTR**} & \multicolumn{3}{c}{**MAP@K**} \\  & \(\mathbf{A}\) & \(\mathbf{B}\) & \(\mathbf{B-A}\) & \(\mathbf{(B-A)/A}\) & \(\mathbf{A}\) & \(\mathbf{B}\) & \(\mathbf{B-A}\) & \(\mathbf{(B-A)/A}\) \\ \hline SVD & 15.43\% & **19.21**\% & \(+\)3.78\% & \(\uparrow\)24.50\% & 0.303 & **0.340** & \(+\)0.037 & \(\uparrow\)12.21\% \\ BPR & 16.16\% & **19.03**\% & \(+\)2.87\% & \(\uparrow\)17.76\% & 0.313 & **0.337** & \(+\)0.024 & \(\uparrow\)7.61\% \\ L-GCN & 16.46\% & **19.56**\% & \(+\)3.10\% & \(\uparrow\)18.83\% & 0.315 & **0.341** & \(+\)0.027 & \(\uparrow\)8.46\% \\ L-FM & 11.79\% & **15.01**\% & \(+\)3.22\% & \(\uparrow\)27.31\% & 0.247 & **0.296** & \(+\)0.049 & \(\uparrow\)19.75\% \\ W\&D & 11.52\% & **15.34**\% & \(+\)3.82\% & \(\uparrow\)33.16\% & 0.242 & **0.285** & \(+\)0.043 & \(\uparrow\)17.71\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The results of A/B experiments on the Amazon dataset.