Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance (2021)

Abstract: This paper presents a novel unsupervised abstractive summarization method for opinionated texts. While the basic variational autoencoder-based models assume a unimodal Gaussian prior for the latent code of sentences, we alternate it with a recursive Gaussian mixture, where each mixture component corresponds to the latent code of a topic sentence and is mixed by a tree-structured topic distribution. By decoding each Gaussian component, we generate sentences with tree-structured topic guidance, where the root sentence conveys generic content, and the leaf sentences describe specific topics. Experimental results demonstrate that the generated topic sentences are appropriate as a summary of opinionated texts, which are more informative and cover more input contents than those generated by the recent unsupervised summarization model (Bražinskas et al., 2020). Furthermore, we demonstrate that the variance of latent Gaussians represents the granularity of sentences, analogous to Gaussian word embedding (Vilnis and McCallum, 2015).

## introduction summarizing opinionated texts, such as product reviews and online posts on web sites, has attracted considerable attention recently along with the development of e-commerce and social media. although extractive approaches are widely used in document summarization @xcite , they often fail to provide an overview of the documents, particularly for opinionated texts @xcite . abstractive summarization can overcome this challenge by paraphrasing and generalizing an entire document. although supervised approaches have seen significant success with the development of neural architectures @xcite , they are limited to specific domains, e.g., news articles, where a large number of gold summaries are available. however, the domain of opinionated texts is diverse; manually writing gold summaries is therefore costly. this lack in gold summaries has motivated prior work to develop unsupervised abstractive summarization of opinionated texts, for example, product reviews @xcite @xcite . while they generated consensus opinions by condensing input reviews, two key components were absent: topics and granularity (i.e., the level of detail). for instance, as shown in figure 1 , a gold summary of a restaurant review provides the overall impression and details about certain topics, such as food, ambience, and service. hence, a summary typically comprises diverse topics, some of which are described in detail, whereas others are mentioned concisely. from this investigation, we capture the topictree structure of reviews and generate topic sentences, that is, sentences summarizing specified topics. in the topic-tree structure, the root sentence conveys generic content, and the leaf sentences mention specific topics. from the generated topic sentences, we extract sentences with appropriate topics and levels of granularity as a summary. regarding extractive summarization, capturing topics @xcite @xcite and topic-tree structure @xcite is useful for detecting salient sentences. to the best of our knowledge, this is the first study to use the topic-tree structure in unsupervised abstractive summarization. the difficulty of generating sentences with treestructured topic guidance lies in controlling the granularity of topic sentences. @xcite generated a sentence with designated topic guidance, assuming that the latent code of an input sentence can be represented by a gaussian mixture (1) the latent distribution of review sentences is represented as a recursive gmm and trained in an autoencoding manner. then, (2) the topic sentences are inferred by decoding each gaussian component. an example of a restaurant review and its corresponding gold summary are displayed. model (gmm), where each gaussian component corresponds to the latent code of a topic sentence. while they successfully generated a sentence relating to a designated topic by decoding each mixture component, modelling the sentence granularity in a latent space to generate topic sentences with multiple granularities remains to be realized. to overcome this challenge, we model the sentence granularity by the variance size of the latent code. we assume that general sentences have more uncertainty and are generated from a latent distribution with a larger variance, analogous to gaussian word embedding @xcite . based on this assumption, we represent the latent code of topic sentences with gaussian distributions, where the parent gaussian receives a larger variance and represents a more generic topic sentence than its children, as shown in figure 1 . to obtain the latent code characterized above, we introduce a recursive gaussian mixture prior to modeling the latent code of input sentences in reviews. a recursive gmm consists of gaussian components that correspond to the nodes of the topic-tree, and the child priors are set to the inferred parent posterior. because of this configuration, the gaussian distribution of higher topics receives a larger variance and conveys more general content than lower topics. the contributions of our work are as follows: for each document index @xmath0, . . . , d}: for each sentence index @xmath1, . . . , s d } in d: ## recursum: recursive summarization in this section, we explain our model, recursum. figure 2 shows the outline. the latent code of review sentences is obtained as a recursive gmm (3.1), and topic sentences are inferred by decoding each gaussian component (3.2). a summary is then created by extracting the appropriate topic sentences (3.3). we introduce additional components to improve the quality of topic sentences (3.4) and explain why general/specific content is conveyed by the root/leaf topics, referring to the analogy with gaussian word embedding (3.5). ## conclusion in this paper, we proposed a novel unsupervised abstractive opinion summarization method by generating topic sentences with tree-structured topic guidance. experimental results demonstrated that the generated summaries are more informative and cover more input content than those generated by the recent unsupervised summarization @xcite . additionally, we demonstrated that the variance of latent gaussians represents the granularity of sentences, analogous to gaussian word embedding @xcite . this property will be useful not only for summarization but also for other tasks that need to consider the granularity of the contents. topic model (tsntm; @xcite , which transforms a sentence into a tree-structured topic distribution using neural networks. while their model is based on the nested chinese restaurant process (ncrp; @xcite , we make a minor change to use the nested hierarchical dirichlet process (nhdp; @xcite . the nhdp generates a sentence-specific path distribution π s and level distribution φ s as