Pre-trained Personalized Review Summarization with Effective Salience Estimation (2023)

Abstract: Personalized review summarization in recommender systems is a challenging task of generating condensed summaries for product reviews while preserving the salient content of reviews. Recently, Pretrained Language Models (PLMs) have become a new paradigm in text generation for the strong ability of natural language comprehension. However, it is nontrivial to apply PLMs in personalized review summarization directly since there are rich personalized information (e.g., user preferences and product characteristics) to be considered, which is crucial to the salience estimation of input review. In this paper, we propose a pre-trained personalized review summarization method, which aims to effectively incorporate the personalized information of users and products into the salience estimation of the input reviews. We design a personalized encoder that could identify the salient contents of the input sequence by jointly considering the semantic and personalized information respectively (i.e., ratings, user and product IDs, and linguistic features), yielding personalized representations for the input reviews and history summaries separately. Moreover, we design an interactive information selection mechanism that further identifies the salient contents of the input reviews and selects relative information from the history summaries. The results on real-world datasets show that our method performs better than the state-of-the-art baselines and could generate more readable summaries.

## introduction personalized review summarization aims to generate brief summaries for product reviews and preserves the main contents of input reviews. it could help users to have a full insight of products quickly and make accurate purchase decisions, hence it is an important task in recommender systems and attracts more and more attention recently (ganesan figure 1 : an example of a product review and the corresponding summary. different history summaries mention different significant features of the given review and we mark them in different colors. @xcite @xcite @xcite . different from the traditional summary generation task, reviews are generally coupled with a lot of essential information about users and products. hence, some recent works propose to generate personalized summaries for reviews by considering user preferences and product characteristics @xcite @xcite @xcite . for example, @xcite propose a user-aware encoder-decoder framework that considers the user preferences in the encoder to select important information and incorporate the preferences and writing style of users into the decoder to generate personalized summaries. recently, pre-trained language models (plms) have achieved notable improvements in various text generation tasks including abstractive summarization @xcite . however, the exploration of the pre-training paradigm in review summarization is quite preliminary since applying plms to review summarization directly is nontrivial. the existing plms ignore the personalized information about users and products which is crucial to generate personalized summaries for product reviews. first, the salience of the input reviews is not only dependent on the semantic information but also influenced by the personalized information. for example, different users have different preferences towards product characteristics, in figure 1 , the user of history summary 3) is interested in "quality" while the user of history summary 1) focuses more on "price". therefore, it is necessary to consider user preferences and product characteristics when calculating the salience for the contents of the input review. second, history summaries of users and products convey rich text descriptions, which could not only be used to strengthen salient information identification of the input reviews but also be fed into the generation process as additional input. for example, some important aspects (e.g., "price", "size", etc.) of products are usually mentioned by different users, hence history summaries might contribute to the salience calculation of the input reviews. therefore, it is essential to fine-tune plms effectively to make the model could generate personalized summaries by jointly considering semantic information of input reviews and the essential attributes of users and products. however, the existing plms focus on the text content, and it is challenging to integrate the various kinds of auxiliary information into plms selectively and effectively. in this paper, we propose an encoder-decoder pre-trained personalized review summarization method with effective salience estimation of the rich input information, named pprs. specifically, we design two kinds of mechanisms to leverage the user and product information to identify the salient contents of the input reviews and history summaries, making the model could focus on the more relevant content towards the current summary generation. first, we propose a personalized encoder that learns representations for the input reviews and each history summary separately. considering the user and product ids indicate their intrinsic characteristics, the personalized encoder aligns each word and the corresponding user, product, and rating. in this way, the salience of words of the input reviews is influenced by both semantic and personalized information, yielding personalized representations. furthermore, we observe that linguistic features are generally associated with the user opinions and product characteristics, such as users utilizing adjectives to represent their sentiment (e.g., "comfort-able", "good" in figure 1 ) and aspects of products are usually nouns (e.g., "price", "size" in figure 1 ). therefore, we also aggregate the part-of-speech feature into the personalized encoder to identify the salient content of the input reviews more accurately. second, we propose an interactive information selection mechanism that interactively models the input reviews and history summaries to learn more comprehensive representations. on the one hand, considering history summaries are usually noisy and redundant, we select the relevant information from history summaries by calculating the semantic relatedness between history summaries and the input reviews. on the other hand, we learn the history summaries-aware salience for the input reviews by calculating the semantic similarity between words of the input reviews and history summaries. finally, we combine the input reviews and history summaries as the input of the decoder to generate coherent and personalized summaries. the main contribution is threefold: (1) we propose a plm-based personalized review summarization method that conducts salience estimation by jointly considering the user and product information. (2) we design two mechanisms to incorporate the personalized information into the generation process, i.e., the personalized encoder and an interactive information selection module. (3) we conduct extensive experiments and the results show that our method outperforms competitive baselines. ## proposed method in this paper, we conduct a review summarization based on transformer @xcite encoder-decoder architecture initialized with t5 @xcite . in this section, we first introduce the problem formulation and then describe our method from two aspects: the personalized encoder module as shown in figure 2 and the decoder with interactive information selection as shown in figure 3 . ## conclusion in this paper, we propose a novel review summarization method based on the pre-trained language models. the core of our method is fine-tuning the pre-trained language models by considering the user preferences and product characteristics. especially, we design a personalized encoder to learn representations for the input reviews and each history summary separately by incorporating the user and product characteristics into the encoder module. additionally, we propose an interactive information selection module to further identify the salient content of the input review and select the relevant information from history summaries. experimental results show that our method achieves better performance than competitive baselines.