Improving Personalized Explanation Generation through Visualization (2022)

Abstract: In modern recommender systems, there are usually comments or reviews from users that justify their ratings for different items. Trained on such textual corpus, explainable recommendation models learn to discover user interests and generate personalized explanations. Though able to provide plausible explanations, existing models tend to generate repeated sentences for different items or empty sentences with insufficient details. This begs an interesting question: can we immerse the models in a multimodal environment to gain proper awareness of real-world concepts and alleviate above shortcomings? To this end, we propose a visually-enhanced approach named METER with the help of visualization generation and text–image matching discrimination: the explainable recommendation model is encouraged to visualize what it refers to while incurring a penalty if the visualization is incongruent with the textual explanation. Experimental results and a manual assessment demonstrate that our approach can improve not only the text quality but also the diversity and explainability of the generated explanations.

## introduction explainable recommender systems have recently attracted increasing attention both in industry and in the academic community. such systems aim to provide high-quality recommendations and simultaneously generate explanations for the recommendations @xcite . the explanations not only can bridge the gap between how systems and users perceive the relevance of the recommended items, but also can serve to shed light on the recommendation decision process so as to avoid a black box. to provide appropriate explanations, feature-based @xcite , graph-based @xcite @xcite , sentence-based @xcite @xcite , causalitybased @xcite @xcite inputs: user a, item 1, feat. word: floors ## conclusion in this paper, we propose meter, the first attempt to jointly generate rating scores, text explanations, and corresponding image visualizations. we immerse our model in a multimodal environment by putting all modalities to one shared transformer decoder structure. a text-image matching discriminator is further introduced to encourage sentences with more groundable and fine-grained concepts. experimental results demonstrate that our framework can provide diverse and faithful text explanations, together with image visualizations as additional intuitive explanations. this proves that visual information offers auxiliary knowledge for the explanation generation model to gain awareness of real-world semantics. our dataset and code are available at https://github.com/ jeykigung/meter . in the future, we plan to investigate generating visually-enhanced explanations for more domains such as fashion and movie.