Unsupervised Opinion Summarisation in the W asserstein Space (2022)

Abstract: Opinion summarisation synthesises opinions expressed in a group of documents discussingthe same topic to produce a single summary. Recent work has looked at opinion summarisation of clusters of social media posts. Such posts are noisy and have unpredictable structure, posing additional challenges for the construction of the summary distribution and the preservation of meaning compared to online reviews, which has been so far the focus on opinion summarisation. To address these challenges we present WassOS, an unsupervised abstractive summarization model which makesuse of the Wasserstein distance. A Variational Autoencoder is first used to obtain the distribution of documents/posts, and the summary distribution is obtained as the Wasserstein barycenter. We create separate disentangled latent semantic and syntactic representations of the summary, which are fed into a GRU decoder with a transformer layer to produce the final summary. Our experiments onmultiple datasets including reviews, Twitter clusters and Reddit threads show that WassOSalmost always outperforms the state-of-the-art on ROUGE metrics and consistently producesthe best summaries with respect to meaning preservation according to human evaluations.

## introduction the growth of online platforms has encouraged people to share their opinions, such as product reviews on online shopping platforms (e.g., amazon) and responses to events posted on social media (e.g., twitter). summarising users' opinions over particular topics on such platforms is crucial for decision-making and helping online users find relevant information of interest @xcite . specifically multi-document opinion summarisation aims at automatically summarising multiple opinions on the same topic @xcite . the bulk of work in this area uses unsupervised summarisation methods. datasets/domains. most work on unsupervised abstractive opinion summarisation focuses on reviews (e.g., amazon, yelp) @xcite @xcite @xcite . however, it is also important to capture user opinions in online discussions over specific events or topics on popular social media platforms such as twitter @xcite and reddit, where the text structure and content is very different and often much noisier compared to review-based corpora (see some examples in appendix a. @xcite ). summary representation. a main focus of unsupervised abstractive summarisation is the creation of a meaningful summary representation. mean-sum @xcite used a text autoencoder to construct summary latent variables by aggregating document latent variables. subsequent research @xcite adopted a variational autoencoder (vae) @xcite , which can capture global properties of a set of documents (e.g., topic). as a vae constructs the distribution of a document, including both semantic and syntactic information, the main meaning may be lost when latent variables sampled from the document distributions are directly aggregated; thus we need methods that can cater for the potential effect of syntactic information, and distinguish between syntax and semantics, especially in documents with unpredictable structure. however, previous work has not considered syntactic and semantic information separately @xcite . another important consideration is the relative weights of documents within a summary vs obtaining an average @xcite @xcite . we mitigate the potential effect of syntactic information on the acquisition of semantic information through a disentangled method. we combine the disentanglement into separate syntactic spaces from @xcite with the wasserstein distance and wasserstein loss to obtain the summary distribution. our experiments with different settings and datasets prove the validity of this strategy. specifically our work makes the following contributions: ## methodology task. given a set of documents (here, social media posts or product reviews) on the same topic, the aim is to summarise opinions expressed in them. this section describes our multi-document abstractive summarisation approach which combines a disentangled vae space with the wasserstein distance. ## conclusions we present an unsupervised multi-document abstractive opinion summarisation model, which captures opinions in a range of different types of online documents including microblogs and product reviews. our model ('wassos') disentangles syntactic and semantic latent variables to keep the main meaning of the posts, and uses the wasserstein loss embedded in the wasserstein barycenter to obtain a latent representation of the summary. wassos has the best performance on meaning preservation according to human evaluation across all datasets and outperforms state-of-the-art systems on rouge metrics for most datasets. future work can look into improving non-redundancy and referential clarity of the generated summaries.