Unsupervised Opinion Summarization as Copycat-Review Generation (2020)

Abstract: Opinion summarization is the task of automatically creating summaries that reflect subjective information expressed in multiple documents, such as product reviews. While the majority of previous work has focused on the extractive setting, i.e., selecting fragments from input reviews to produce a summary, we let the model generate novel sentences and hence produce abstractive summaries. Recent progress in summarization has seen the development of supervised models which rely on large quantities of document-summary pairs. Since such training data is expensive to acquire, we instead consider the unsupervised setting, in other words, we do not use any summaries in training. We define a generative model for a review collection which capitalizes on the intuition that when generating a new review given a set of other reviews of a product, we should be able to control the “amount of novelty” going into the new review or, equivalently, vary the extent to which it deviates from the input. At test time, when generating summaries, we force the novelty to be minimal, and produce a text reflecting consensus opinions. We capture this intuition by defining a hierarchical variational autoencoder model. Both individual reviews and the products they correspond to are associated with stochastic latent codes, and the review generator (“decoder”) has direct access to the text of input reviews through the pointer-generator mechanism. Experiments on Amazon and Yelp datasets, show that setting at test time the review’s latent code to its mean, allows the model to produce fluent and coherent summaries reflecting common opinions.

## introduction summarization of user opinions expressed in online resources, such as blogs, reviews, social media, or internet forums, has drawn much attention due to its potential for various information access applications, such as creating digests, search, and report summary this restaurant is a hidden gem in toronto. the food is delicious, and the service is impeccable. highly recommend for anyone who likes french bistro. ## model and estimation as discussed above, we approach the summarization task from a generative modeling perspective. we start with a high level description of our model, then, in sections 2.2 and 2.3, we describe how we estimate the model and provide extra technical details. in section 3, we explain how we use the model to generate summaries. ## see the derivations in appendix the lower bound includes two 'inference networks', q φ (c|r 1:n ) and q φ (z i |r i , c), which are neural networks parameterized with φ and will be discussed in detail in section 2.3. they approximate the corresponding posterior distributions of the model. the first term is the reconstruction error: it encourages the quality reconstruction of the reviews. the other two terms are regularizers. they control the amount of information encoded in the latent representation by penalizing the deviation of the estimated posteriors from the corresponding priors, the deviation is measured in terms of the kullback-leibler (kl) divergence. the bound is maximized with respect to both the generative model's parameters θ and inference networks' parameters φ. due to gaussian assumptions, the kullback-leibler (kl) divergence terms are available in closed form, while we rely on the reparameterization trick @xcite to compute gradients of the reconstruction term. the inference network predicting the posterior for a review-specific variable q φ (z i |r i , c) is needed only in training and is discarded afterwards. in contrast, we will exploit the inference network q φ (c|r 1:n ) when generating summaries, as discussed in section 3. ## summary generation given reviews r 1:n , we generate a summary that reflects common information using trained components of the model. formally, we could sample a new review from as we argued in the introduction and will revisit in experiments, a summary or summarizing review, should be generated relying on the mean of the reviews' latent code. consequently, instead of sampling z from p θ (z|c) = n (z; µ θ (c), iσ θ (c)), we set it to µ θ (c). we also found beneficial, in terms of evaluation metrics, not to sample c but instead to rely on the mean predicted by the inference network q φ (c|r 1:n ).