Aspect-Controllable Opinion Summarization (2021)

Abstract: Recent work on opinion summarization produces general summaries based on a set of input reviews and the popularity of opinions expressed in them. In this paper, we propose an approach that allows the generation of customized summaries based on aspect queries (e.g., describing the location and room of a hotel). Using a review corpus, we create a synthetic training dataset of (review, summary) pairs enriched with aspect controllers which are induced by a multi-instance learning model that predicts the aspects of a document at different levels of granularity. We fine-tune a pretrained model using our synthetic dataset and generate aspect-specific summaries by modifying the aspect controllers. Experiments on two benchmarks show that our model outperforms the previous state of the art and generates personalized summaries by controlling the number of aspects discussed in them.

## introduction consumers oftentimes resort to review websites to inform their decision making (e.g., whether to buy a product or use a service). the proliferation of online reviews has accelerated research on opinion mining @xcite , where the ultimate goal is to glean information from reviews so that users can make decisions more efficiently. opinion mining has assumed several guises in the literature such as sentiment analysis @xcite , aspect extraction @xcite , combinations thereof @xcite , and notably opinion summarization @xcite , whose aim is to create a textual summary of opinions found in multiple reviews. text summarization models, both extractive @xcite @xcite and abstractive @xcite @xcite , operate under the assumption that salient content is ## related work earlier work on opinion summarization has focused on general summarization using extractive @xcite @xcite or abstractive methods @xcite @xcite . due to the absence of opinion summaries in review websites and the difficulty of annotating them on a large scale, more recent methods consider an unsupervised learning setting where there are only reviews available without corresponding summaries @xcite . they make use of autoencoders @xcite and variants thereof to learn a review decoder through reconstruction, and use it to generate summaries conditioned on averaged representations of the inputs. a more successful approach to opinion summarization is through the creation of synthetic datasets, where (review, summary) pairs are constructed from a review corpus to enable supervised training. these methods usually start by randomly selecting a review which they treat as a pseudo-summary and subsequently pair it with a set of input reviews based on different strategies. these include random sampling @xcite , generating noisy versions of the pseudo-summary (amplayo and lapata, 2020), ranking reviews based on similarity and relevance @xcite , and making use of content plans to create more naturalistic pairs @xcite . our work is closest to @xcite who propose an extractive summarization model that uses a vector-quantized variational autoencoder @xcite to learn aspectspecific review representations. their model effectively groups opinion sentences into clusters and extracts those capturing aspect-relevant information. we employ multi-instance learning to identify aspect-bearing elements in reviews with varying degrees of granularity (e.g., words, sentences, documents) which we argue affords greater flexibility and better control of the output summaries. in doing so, we also introduce an effective method to create synthetic datasets for aspect-guided opinion summarization. our work also relates to approaches which attempt to control summarization output based on length @xcite , content @xcite , style @xcite , or textual queries @xcite . although we focus solely on aspect, our method is general and could be used to adjust additional properties of a summary such as sentiment (e.g., positive vs. negative) or style (e.g., formal vs. colloquial). ## problem formulation let c denote a corpus of reviews about entities (e.g., products, hotels). let r @xmath0, r 2 , ..., r n } denote a set of reviews for entity e and a @xmath1, a 2 , ..., a m } a set of aspects that are relevant for the entity (e.g., cleanliness, location). each review r i is a sequence of tokens {w 1 , w 2 , ...}, while each aspect a j is represented by a small set of seed words {v 1 , v 2 , ...} (e.g., spotless, dirty, stain). these seed words can be acquired automatically @xcite or provided by users (see appendix for those used in our experiments). our approach creates two types of summaries: (a) a general summary that contains salient opinions about all aspects of an entity, and (b) an aspect-specific summary that focuses on opinions about particular aspects of interest specified by a query @xmath2, q 2 , ..., q m }; here, q j is an indicator function which designates whether the aspect should be mentioned in the summary. we emphasize that the query can represent more than one aspect to reflect real-world usage. to facilitate supervised training, we create a synthetic training dataset @xmath3, z, y), which is a set of triples composed of input reviews x, a pseudo-summary y, and aspect controllers z (section 3.2). our aspect controllers are induced with a unified model based on multi-instance learning (section 3.1) and correspond to different levels of granularity: (1) document-level aspect codes, (2) aspect-related review sentences, and (3) aspect keywords. at training time, we fine-tune a pretrained sequence-to-sequence transformer model @xcite using controllers z as input and a pseudo-summary as output. during inference, we modulate summary generation by modifying the controllers, e.g., we produce a general summary using all aspect codes, or an aspect-specific one based on a subset thereof (section 3.3). ## experimental setup datasets we performed experiments on two opinion summarization datasets covering different review domains. space @xcite is a large corpus of "hotel" reviews from tripadvisor; it contains human-written abstractive opinion summaries for evaluation only. each instance in the evaluation set consists of 100 reviews and seven summaries: one general summary and six aspect-specific ones representing the aspects building, cleanliness, food, location, rooms, and service. oposum (angelidis and lapata, 2018b) is a large corpus of product reviews from six different domains: "laptop bags", "bluetooth headsets", "boots", "keyboards", "televisions", and "vacuums". it also includes an evaluation set with extractive general summaries. we extended this dataset by (a) adding aspect-specific summaries which are human-written and abstractive following the methodology from @xcite , and (b) increasing the size of the corpus. we call this extended dataset oposum+. both datasets include five human-annotated seed words for each aspect (see appendix for details). data statistics are shown in implementation for our pretrained transformer models, we used weights and settings available in the huggingface library @xcite . specifically, we used distilroberta-base @xcite as our language model and t5-small @xcite as our sequence-to-sequence model. we trained the controller induction model with a learning rate of 1e-4 for 100k steps, using @xmath4. for opo-sum+, we trained separate controller induction models for different domains. for the aspect controllers, we selected 10-best keywords, and review sentences were truncated up to 500 tokens to fit in the pretrained model. for summarization, we used a learning rate of 1e -6 and 500k training steps. we used adam with weight decay @xcite to optimize both models. we added a linear learning rate warm-up for the first 10k steps. we generate summaries with beam search of size 2 and refrain from repeating ngrams of size 3 @xcite . ## results we compared our aspect controlled summarization (acesum) model with several extractive and abstractive approaches. traditional extractive systems include selecting as a summary the review closest to the centroid @xcite of the input reviews and lexrank @xcite , a pagerank-like algorithm that selects the most salient sentences from the input. for both methods we used bert encodings @xcite to represent sentences and documents. other extractive systems include qt foot_1 @xcite , a neural clustering method that uses vector-quantized variational autoencoders @xcite to represent opinions in quantized space, and acesumext, an extractive version of our model that uses sentences ranked by our controller induction model as input (truncated up to 500 tokens) to lexrank. abstractive systems include meansum @xcite , an autoencoder that generates summaries by reconstructing the mean of review encodings, copycat @xcite , a hierarchical variational autoencoder which learns a latent code of the summary, and two variants of t5 @xcite trained with different synthetic dataset creation methods. for t5-random, summaries are randomly sampled @xcite , whereas for t5-similar reviews are sampled based on their similarity to a candidate summary (amplayo and lapata, 2020). finally, we compared against two upper bounds: an extractive oracle which selects as a summary the review with the best rouge score against the input, and a human upper bound, calculated as inter-annotator rouge. examples generated by our model are in ## conclusions in this work, we presented an abstractive approach to aspect-controlled opinion summarization. key to our model is the induction of aspect controllers which facilitate the creation of a synthetic training dataset and guide summary generation towards the designated aspects. extensive experiments on two benchmarks show that our model achieves state of the art across the board, for both general and aspect-specific opinion summarization. in the future, we would like to focus on controlling additional facets of opinion summaries such as sentiment or length. it would also be interesting to learn aspects from data rather than specifying them apriori as well as dealing with unseen aspects (e.g., in a scenario where reviews discuss new features of a product). aspect "hotels" building lobby pool decor gym area cleanliness clean spotless garbage dirty stain food breakfast food buffet restaurant meal location location walk station distance bus rooms room bed bathroom shower spacious service staff service friendly helpful desk