Opinion Summarization by Weak-Supervision from Mix-structured Data (2022)

Abstract: Opinion summarization of multiple reviews suffers from the lack of reference summaries for training.Most previous approaches construct multiple reviews and their summary based on textual similarities between reviews,resulting in information mismatch between the review input and the summary. In this paper, we convert each review into a mixof structured and unstructured data, which we call opinion-aspect pairs (OAs) and implicit sentences (ISs).We propose a new method to synthesize training pairs of such mix-structured data as input and the textual summary as output,and design a summarization model with OA encoder and IS encoder.Experiments show that our approach outperforms previous methods on Yelp, Amazon and RottenTomatos datasets.

## introduction opinion summarization aims at producing a summary for a set of subjective user reviews about an entity (i.e., a product, a service or a movie), which can help users quickly understand the entity. opinion summarization is a specialized multidocument summarization @xcite @xcite . unlike traditional multidocument summarization, opinion summarization focuses on the aspects and opinions of entities in the input documents. deep learning techniques have made great successes in summarization @xcite @xcite , which require training on a large number of document-summary pairs. unfortunately, opinion summarization generally lacks the training pairs with reviews as input and summary as output, as it is difficult and costly for annotators to write summaries for multiple reviews (known as multi-reviews in this paper) on a large scale. synthetic summary (sampled from the reviews) very disappointed in food and service . the beef was burned . the staff dismissed my comments . synthetic input (to be summarized) textual (unstructured) random r1 very average spanish food . r2 definitely not a peruvian restaurant . r3 love this place! location is great . similarity r1 bad food and service . disappointed r2 the food and service was not good . very bad experience . r3 awful food and service . the staff was unfriendly . structured opidig oa disappointed, food; disappointed, service; burned, beef mix-structured ours oa not good, food; great, location; bad, food; disappointed, service; not fresh, beef; unfriendly, staff; awful, service is not recommended .; my questions were dismissed .; the staff ignored our comments . in view of the above challenge, some approaches @xcite @xcite adopt unsupervised learning, e.g., by using the auto-encoders. they reconstruct individual reviews by encoding themselves at training time, which may prevent them from effectively generating summary by encoding multi-review at test time. other more popular approaches (amplayo and lapata, 2020; @xcite @xcite focus on creating synthetic (multireview, summary) pairs for training. they typically sample one or more reviews from all reviews about an entity as the "pseudo" summary, that is, the output of the synthetic training pair. these approaches differ from each other by how the input is created. existing approaches use either textual or structured information (see the textual input is usually a set of reviews sampled from all reviews in the corpus. the most straight-forward method is leave-one-out @xcite , which takes all reviews as the input, except for one sampled summary. this results in very long and almost identical inputs, which are not only memoryconsuming, but also less effective. another way is to sample (or synthesize) a subset of the reviews, either randomly @xcite , or by the similarity between the input text and the summary (amplayo and lapata, 2020; amplayo et al., 2021b). as shown in previous works have acknowledged that the most critical information in opinion summarization are the aspects of the product or service and the opinions on these aspects @xcite , such as @xmath0, or @xmath1. for example, opidig @xcite uses a review as output while extracting opinion-aspect (oa) pairs from this review as structured input. however, some sentences may not produce any oa pairs at all, such as "staff dismissed my comments" in in order to capture explicit and implicit opin-ions at the same time, we propose a summarization model with oa encoder and is encoder. we first pretrain a single encoder by taking only sampled oas as input, which learns to select important explicit opinions. then, we fine-tune that model with oa and is encoder based on pretrained encoder. this allows the information of explicit and implicit opinions to complement each other, leading to reduced information loss in generated summaries. in summary, this paper makes the following contributions: ## approach since there are no reference summaries for training, we design a weakly supervised approach, which first synthesizes the mix-structured training data and then train a dual-encoder summarization model on such data. ## evaluation we first give the experimental setup, then compare and contrast the end-to-end results against various strong baselines on three datasets, before analyzing the main advantages of our approach. ## related work opinion summarization has a special focus on aspects of the product or service, making it different from other multi-document tasks, such as news summarization @xcite . opinion summarization suffers from a lack of training pairs. some work @xcite @xcite used autoencoder to train the model by reconstructing loss or sentence embeddings. others create synthetic datasets for supervised training. the input format of synthetic datasets is textual or structured. for the textual input, some approaches @xcite regarded one review as a summary and took all or part of the rest as input. @xcite computed the distance between the summary and all remaining reviews as weights of review embeddings. amplayo et al. (2021b) took the nearest neighbors as inputs based on review representations. amplayo and lapata (2020) added noise to the sampled summary from the segment noising and document noising by replacing the whole review with a similar one. elsahar et al. (2021) labeled input reviews and sampled summaries with control tokens and took control tokens as prefixes at decoding. however, these datasets is limited by biased reviews, which cannot be summarized from other reviews. the aspects @xcite and opin-ions are quite important for opinion summarization. some approaches @xcite classified the sentences of reviews into different aspects and collected the most salient sentence of each class as summary. @xcite classified words into three types (i.e., aspect, opinion and context) and predicted summary by the probability distribution on these types. inspired by these works, @xcite extracted opinion-aspect phrases from each review and transformed the task into single document summarization. amplayo et al. (2021a) used predefined aspects to construct synthetic training data and trained a controllable model to generate summaries based on aspects. however, all of these works neglect some other information in the sentences which cannot be explicitly formulated as opinion-aspect pairs. thus, we create a mix-structured synthetic dataset consisting of opinion-aspect pairs and implicit sentences, which can get more accurate and comprehensive summaries. ## conclusion in this work, we proposed a new method to generate mix-structured synthetic training data for opinion summarization. we designed a transformer-based seq2seq model with a dual encoder to deal with oas and iss separately. the results showed that our approach can make full use of mix-structured data and generate better opinion summaries.