Few-Shot Learning for Opinion Summarization (2020)

Abstract: Opinion summarization is the automatic creation of text reflecting subjective information expressed in multiple documents, such as user reviews of a product. The task is practically important and has attracted a lot of attention. However, due to the high cost of summary production, datasets large enough for training supervised models are lacking. Instead, the task has been traditionally approached with extractive methods that learn to select text fragments in an unsupervised or weakly-supervised way. Recently, it has been shown that abstractive summaries, potentially more fluent and better at reflecting conflicting information, can also be produced in an unsupervised fashion. However, these models, not being exposed to actual summaries, fail to capture their essential properties. In this work, we show that even a handful of summaries is sufficient to bootstrap generation of the summary text with all expected properties, such as writing style, informativeness, fluency, and sentiment preservation. We start by training a conditional Transformer language model to generate a new product review given other available reviews of the product. The model is also conditioned on review properties that are directly related to summaries; the properties are derived from reviews with no manual effort. In the second stage, we fine-tune a plug-in module that learns to predict property values on a handful of summaries. This lets us switch the generator to the summarization mode. We show on Amazon and Yelp datasets that our approach substantially outperforms previous extractive and abstractive methods in automatic and human evaluation.

## introduction summarization of user opinions expressed in online resources, such as blogs, reviews, social media, or internet forums, has drawn much attention due to its potential for various information access applications, such as creating digests, search, and report gold these shoes run true to size, do a good job supporting the arch of the foot and are well-suited for exercise. they're good looking, comfortable, and the sole feels soft and cushioned. overall they are a nice, light-weight pair of shoes and come in a variety of stylish colors. ## unsupervised training user reviews about an entity (e.g., a product) are naturally inter-dependent. for example, knowing that most reviews are negative about a product's battery life, it becomes more likely that the next review will also be negative about it. to model inter-dependencies, yet to avoid intractabilities associated with undirected graphical models @xcite , we use the leave-one-out setting @xcite . specifically, we assume access to a large corpus of user text reviews, which are arranged as m groups {r 1:n } m @xmath0, where r 1:n are reviews about a particular product that are arranged as a target review r i and n -1 source reviews r -@xmath1, ..., r i-1 , r i+1 , ..., r n }. our goal is to estimate the conditional distribution r i |r -i by optimizing the parameters θ as shown in eq. 1. the objective lets the model exploit common information across reviews, such as rare brand names or aspect mentions. for example, in fig. 1 , the generator can directly attend to the word vacuum in the source reviews to increase its prediction probability. additionally, we condition on partial information about the target review r i using an oracle 2 both the code and datasets are available at: https:// github.com/abrazinskas/fewsum we refer to this partial information as properties @xcite , which correspond to text characteristics of r i or relations between r i and r -i . for example, one such property can be the rouge score @xcite between r i and r -i , which indicates the degree of overlap between r i and r -i . in fig. 1 , a high rouge value can signal to the generator to attend the word vacuum in the source reviews instead of predicting it based on language statistics. intuitively, while the model observes a wide distribution of rouge scores during training on reviews, during summarization in test time we can achieve a high degree of input-output text overlap by setting the property to a high value. we considered three types of properties. content coverage: rouge-1, rouge-2, and rouge-l between r i and r -i signals to g θ how much to rely on syntactic information in r -i during prediction of r i . writing style: as a proxy for formal and informal writing style, we compute pronoun counts, and create a distribution over 3 points of view and an additional class for cases with no pronouns; see appendix 9.7 for details. rating and length deviations: for the former, we compute the difference between r i 's rating and the average r -i rating; in the latter case, we use the difference between r i 's length and the average r -i length. ## summary adaptation once the unsupervised model is trained on reviews, our task is to adapt it to generation of summaries. here, we assume access to a small number of annotator-written summaries (s k , r k 1:n ) k @xmath2:n input reviews. as we will show in sec. 6.1, naive fine-tuning of the unsupervised model on a handful of annotated data-points leads to poor generalization. instead, we capitalize on the fact that the generator g θ has observed a wide range of property values associated with r i during the unsupervised training phase. intu-itively, some combinations of property values drive it into generation of text that has qualities of a summary while others of a review. however, we might not know values in advance that are necessary for generation of summaries. furthermore, q(r i , r -i ) cannot be applied at test time as it requires access to target texts. in the following section, we describe a solution that switches the generator to the summarization mode relying only on input reviews.