Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization (2023)

Abstract: Opinion summarization provides an important solution for summarizing opinions expressed among a large number of reviews. However, generating aspect-specific and general summaries is challenging due to the lack of annotated data. In this work, we propose two simple yet effective unsupervised approaches to generate both aspect-specific and general opinion summaries by training on synthetic datasets constructed with aspect-related review contents. Our first approach, Seed Words Based Leave-One-Out (SW-LOO), identifies aspect-related portions of reviews simply by exact-matching aspect seed words and outperforms existing methods by 3.4 ROUGE-L points on Space and 0.5 ROUGE-1 point on Oposum+ for aspect-specific opinion summarization. Our second approach, Natural Language Inference Based Leave-One-Out (NLI-LOO) identifies aspect-related sentences utilizing an NLI model in a more general setting without using seed words and outperforms existing approaches by 1.2 ROUGE-L points on Space for aspect-specific opinion summarization and remains competitive on other metrics.

## introduction customer reviews play a vital role in decisionmaking for customers and product (or business) providers, as customers usually resort to reviews to guide their purchasing decisions and product providers improve their products based on reviews as feedback. however, it becomes hard for customers or product providers to read through all reviews before making decisions with the explosion of online reviews in recent years. opinion summarization @xcite @xcite @xcite @xcite ; basu roy chowdhury * work done during an internship at aws ai labs. @xcite , the task of generating a general summary of salient opinions expressed among reviews, provides a feasible solution to this problem. different from summarization in wikipedia and news domains @xcite @xcite @xcite , opinion summarization cannot rely on reference summaries for model training since it is difficult and expensive to annotate large scale reviews-summary pairs. also, customers usually care about specific aspects of a product instead of a general high-level summary. thus, fine-grained aspect-specific opinion summaries are required, and this makes the annotation process even more difficult and expensive. @xcite propose an abstractive approach to generate aspect-specific opinion summaries by training on synthetic datasets. they construct synthetic datasets with review elements (words, phrases, or sentences) identified by a multiple instance learning (mil) module @xcite learned with silver-standard labels obtained using aspect seed words. we first follow this direction to propose a more straightforward and effective method that excludes the complex learning module to identify aspect-related elements to construct synthetic datasets. moreover, aspect seed words, which again require human efforts, may not always be available when moving to new domains. thus we propose another more general solution without the curation and supervision of aspect seed words. specifically, we propose two simple yet effective methods to identify aspect-related review sentences and construct aspect-specific synthetic datasets in a leave-one-out (loo) @xcite @xcite style and then finetune pretrained language models (plms) on the synthetic datasets: (a) sw-loo identifies aspect-related sentences by simply exact-matching aspect seed words and outperforms existing ap-proaches by 3.4 rouge-l points and 0.5 rouge-1 point on aspect opinion summaries of space and oposum+ respectively; (b) nli-loo identifies aspect-related sentences with a finetuned nli @xcite model. being the first approach that does not use aspect seed words, it outperforms existing approaches on aspect opinion summarization by 1.2 rouge-l points for space and falls behind at most 1 rouge point on other metrics. ## problem formulation let c denote a corpus of reviews on entities {e 1 , e 2 , . . . } (products or business). let a @xmath0, a 2 , . . . , a m } denotes a set of aspects (e.g., food or location for a hotel) that are relevant for the domain of entities. for each entity e, we define its review set as r @xmath1, r 2 , . . . , r n }. each review r is a collection of sentences {x 1 , x 2 , . . . } and each sentence x is a sequence of tokens {w 1 , w 2 , . . . }. each aspect a is represented by a small set of seed words (e.g., meal or buffet for food aspect) s @xmath2, v 2 , . . . }. our approaches generate two types of opinion summaries: (a) general summary that contains salient opinions over all aspects of the entities; and (b) aspect summary that focuses on only one specific aspect @xmath3. ## synthetic dataset construction leave-one-out (loo) we construct synthetic datasets in a loo style: from a pool of review elements (reviews or review sentences), an element is randomly sampled as a pseudo-summary, then we select input reviews from the remaining review elements. ## summarization model we use t5 @xcite , a sequence-tosequence transformer-based @xcite plm, to finetune our synthetic datasets similar to previous works @xcite ## conclusion in this work, we propose two simple yet effective unsupervised approaches that generate aspect and general opinion summaries by training on synthetic datasets. sw-loo constructs synthetic datasets simply by exact-matching aspect seed words and outperforms existing methods consistently on all metrics and datasets. being the first work that generates aspect summaries without using aspect seed words, nli-loo constructs synthetic datasets with an out-of-the-box nli model and achieves on-par and sometimes even better performances compared with existing methods.