Multi-Scale Distribution Deep Variational Autoencoder for Explanation Generation (2022)

Abstract: Generating explanations for recommender systems is essential for improving their transparency, as users often wish to understand the reason for receiving a specified recommendation. Previous methods mainly focus on improving the generation quality, but often produce generic explanations that fail to incorporate user and item specific details. To resolve this problem, we present Multi-Scale Distribution Deep Variational Autoencoders (MVAE).These are deep hierarchical VAEs with a prior network that eliminates noise while retaining meaningful signals in the input, coupled with a recognition network serving as the source of information to guide the learning of the prior network. Further, the Multi-scale distribution Learning Framework (MLF) along with a Target Tracking Kullback-Leibler divergence (TKL) mechanism are proposed to employ multi KL divergences at different scales for more effective learning. Extensive empirical experiments demonstrate that our methods can generate explanations with concrete input-specific contents.

## introduction due to the massive demand for convincing highquality recommendations, researchers from both academic and industrial communities have paid increasing attention to the topic of enhancing the explainability of recommender systems @xcite @xcite . explanations for recommendations in real-world scenarios are presented in a variety of different forms, among them, the most popular and natural form is that of free-text explanations given in natural language @xcite . as shown in fig. 1 , this task requires a machine to generate a textual explanation based on a given user id, item id, and the rating score from a recommender system. previous models attempt to embed these ids in a similar way as normal words. however, since the ids appear far less frequently than the words, most approaches typically fail to account for specific features of the users and item. hence, it is a very common phenomenon to obtain explanations without concrete characteristics about the given user and item as shown in to deal with this problem, we present multi-scale distribution deep variational autoencoders (mvae). they consist of three modules, namely a recognition network, prior network, and a reconstruction network. the prior network in our model can filter out the noise contained in input embeddings, while retaining meaningful information for generation through information compression. moreover, to help the prior network learn to generate fine-grained information, the recognition network is leveraged to provide the prior network with suitable guiding information. thus, the decoder tends to generate explanations in a conditional manner with a substantially more informative generation signal. however, with strong guiding signals available during training, generation becomes much simpler, which may result in a degradation of performance when such information is no longer available during testing. thus, we propose a multi-scale distribution learning framework (mlf) along with a target tracking kullback-leibler divergence (tkl) mechanism to reduce this performance gap between training and testing. the optimization effectiveness of the prior network can further be boosted when this method is employed at multiple different scales. overall, our contributions are as follows: ## related work for generation of textual explanations, mainstream research can be divided into two categories: template-based and natural language generation approaches. template-based approaches generate explanations by filling the slots of predefined templates @xcite , which are typically manually specified in advance. natural language generation approaches, in contrast, adopt an encoderdecoder framework such as a recurrent seq-to-seq model @xcite or a transformer-based architecture @xcite to learn to generate more diverse explanations based on the respective input. in recent years, the latter strategy has received considerable attention, mainly owing to advances in neural generation along with the massive availability of text from online review systems. still, existing natural language generation methods may generate overly generic sentences that fall short at providing concrete information and are thus less useful for users @xcite . indeed, explanation generation goes beyond mere generation, as it is expected to improve the transparency of the recommendation engine @xcite . thus, technical ideas to encourage the generation process to account for more conditional signals are crucial to enable models to generate more specific explanations that are custom-tailored for particular user-item pairs. variational autoencoders (vae) were proposed by @xcite based on the idea of autoencoding, which has been used for noise reduction @xcite . vaes have been studied extensively in a variety of language generation tasks, including text summarization @xcite and dialogue generation @xcite @xcite . a vae maximizes the mutual information between the input and latent variables @xcite , requiring the network to retain the information content of the input data to the extent possible @xcite . hence, vaes are qualified to overcome the overly generic explanations caused by uninformative noisy input embeddings and prompt the construction of more meaningful outputs. ## proposed model an overview of our model is given in fig. 2 . the recognition network encodes the explanations and generates fine-grained information for the reconstruction network. the prior network encodes the input embeddings and generates essential information for the reconstruction network. the essential information here refers to the general semantics of a reason, which can be described in multiple ways, while the fine-grained information here refers to information that determines the details in the explanations, thus narrowing down and customizing the essential information to a specific form. finally, the reconstruction component decodes the given information and generates explanations. additionally, the proposed mlf employs kl divergence at multiple different scales, which improves the optimization of the prior network. the tkl applied in every kl divergence can aid the learning of the prior network even further. we will present the details of each network in the following sections. ## conclusion we present mvae, a novel model for explanation generation in recommender systems, which has a prior network that eliminates noise while retaining meaningful signals in the input and a recognition network serving as the source of information to guide the learning of the prior network. further, we propose a multi-scale distribution learning framework along with tkl to prompt this process. extensive experiments demonstrate the effectiveness of our method and confirm that it can generate high-quality explanations.