Learning Opinion Summarizers by Selecting Informative Reviews (2021)

Abstract: Opinion summarization has been traditionally approached with unsupervised, weakly-supervised and few-shot learning techniques. In this work, we collect a large dataset of summaries paired with user reviews for over 31,000 products, enabling supervised training. However, the number of reviews per product is large (320 on average), making summarization – and especially training a summarizer – impractical. Moreover, the content of many reviews is not reflected in the human-written summaries, and, thus, the summarizer trained on random review subsets hallucinates. In order to deal with both of these challenges, we formulate the task as jointly learning to select informative subsets of reviews and summarizing the opinions expressed in these subsets. The choice of the review subset is treated as a latent variable, predicted by a small and simple selector. The subset is then fed into a more powerful summarizer. For joint training, we use amortized variational inference and policy gradient methods. Our experiments demonstrate the importance of selecting informative reviews resulting in improved quality of summaries and reduced hallucinations.

## introduction summarization of user opinions expressed in online resources, such as blogs, reviews, social media, or internet forums, has drawn much attention due to its potential for various information access applications, such as creating digests, search, and report generation @xcite @xcite . although significant progress has been observed in supervised summarization in nonsubjective context, such as news articles @xcite @xcite @xcite @xcite , modern deep learning methods rely on large amounts of annotated data that are not readily available in the verdict if you like the idea of a glass feeder, this is the one to get. it has a lot to offer for the price. ## dataset the dataset (amasum) is based on summaries for consumer products written by professional reviewers, in english. we focused on four main professional product review platforms: bestreviews.com (br); cnet.com; pmag.co.uk (pm); runrepeat.com (rr). the former three mostly offer content for electronic consumer products, while the last one for sport shoes. these summaries provide a quick-glance overview of a product to help users make informed purchases. unlike customer reviewers on public platforms, such as amazon, professional reviewers concentrate on quality writing and deliberately utilize many sources of information. these sources include reading customer reviews on public platforms, making online research, asking expert users for an opinion, and testing products themselves. in general, the summaries come in two forms. the first ones are verdicts, usually a few sentences emphasizing the most important points about a product. the second ones are pros and cons, where the most important positive and negative details about a product are presented. these tend to be more detailed, and focus on fine-grained product aspects, such as bluetooth connectivity, resolution, and cpu clock speed. as content providers compete for online users, the summaries are what the user wants as opposed to what researchers believe the user wants. this is in contrast to crowd-sourcing where researchers bias the worker writing process with assumptions about what constitutes a good summary. the assumptions are rarely verified by a marketing research or user testing. in turn, this has lead to a large variance of summary styles and composition even in the same domain @xcite @xcite . ## approach as summaries are written mostly for popular products, with more than 320 reviews on average, it is computationally challenging to encode and attend all the available ones to decode the summary. to alleviate this problem, we propose to condition the decoder on a smaller subset of reviews. however, as not all reviews provide a good content coverage of the summary. thus, training on random subsets leads to hallucinations, as we will show in sec. 5.2. instead, we propose to learn a review selector, which chooses reviews guided by the summary. we frame this as a latent variable modeling problem (the selection is latent) and rely on the variational inference framework to train the selector, see sec. 3.2. the selector (the approximate posterior) is a neural module assessing the reviewsummary relevance using pre-computed lexical features, thus, efficiently selecting from large review collections. further, the selected reviews are decoded to the summary, as illustrated in fig. 1 . to select reviews in test time, we train a review selector that does not rely on the summary, as presented in sec. 3.3. ## related work an alternative to review subsets selection are more memory and computationally efficient attention mechanisms @xcite . however, it is unclear what relationship exists between attention weights and model outputs @xcite , thus, making it harder to offer evidence for generated summaries. in our case, the summarizer relies only on a selected subset and generates summaries faithful to its content. in general, in news summarization, which is a more mature branch, large datasets are commonly obtained from online resources @xcite @xcite @xcite . the most relevant dataset is @xcite , where journalist-written summaries are linked to multiple news articles. the most similar opinion summarization dataset space @xcite ) contains 1050 summaries produced for 50 hotels by crowdsourcing. ## conclusions in this work, we introduce the largest multidocument abstractive dataset for opinion summarization. the dataset consists of verdicts, pros and cons, written by professional writers for more than 31,000 amazon products. each product is linked to more than 320 customer reviews, on average. as standard encoding-decoding is computationally challenging, we perform summarization with an integrated component that selects smaller review subsets. we conclude that the 'naive' selection of random reviews leads to content infidelity (aka hallucinations) and present selsum that learns to select and summarize reviews end-to-end. the model is computationally efficient, scaling to large collections. its summaries result in better rouge scores and are better supported by input reviews.