Aspect-aware Unsupervised Extractive Opinion Summarization (2023)

Abstract: Extractive opinion summarization extracts sentences from users’ reviews to represent the prevalent opinions about a product or service. However, the extracted sentences can be redundant and may miss some important aspects, especially for centroid-based extractive summarization models (Radev et al., 2004). To alleviate these issues, we introduce TokenCluster– a method for unsupervised extractive opinion summarization that automatically identifies the aspects described in the review sentences and then extracts sentences based on their aspects. It identifies the underlying aspects of the review sentences using roots of noun phrases and adjectives appearing in them. Empirical evaluation shows that TokenCluster improves aspect coverage in summaries and achieves strong performance on multiple opinion summarization datasets, for both general and aspect-specific summarization. We also perform extensive ablation and human evaluation studies to validate the design choices of our method. The implementation of our work is available at https://github.com/leehaoyuan/TokenCluster

## introduction in the internet era, online reviews are important for both customers and businesses. customers use the reviews to help them make better choices while businesses gather feedback from the reviews. however, the large number of reviews for a product can make it time-consuming to go through all of them. opinion summarization aims to tackle this problem by creating a concise summary of all the reviews. recently, there has been significant progress in the supervised summarization of single @xcite @xcite and multiple @xcite @xcite documents. unfortunately, opinion summarization cannot directly benefit from the progress as collecting human-written summaries for large-scale review sets is expensive. therefore, most opinion summarization techniques resort to unsupervised approaches. there are two major paradigms for opinion summarization: abstractive summarization, which generates summaries using novel phrasing, and extractive summarization, where the summary is selected from a subset of the input sentences. for abstractive summarization, previous works usually generate summaries using aggregated sentence representations @xcite or train a supervised model on synthetic datasets @xcite . for extractive summarization, recent works use techniques that consists of two primary components: (a) a model for learning representations for review sentences, and (b) an inference algorithm that uses these representations to select summarizing sentences @xcite @xcite . our work focuses on the inference algorithm for extractive opinion summarization. extractive summarization can benefit from aspect identification since extracted sentences should cover all salient aspects within a review set. aspects in user reviews usually focus on specific features or attributes of an entity (e.g., room, cleanliness, and food for a hotel). most opinion summarization systems are centroid-based and extract sentences closest to the centroid as the summary. the extraction process can result in redundant summaries that do not cover all aspects. therefore, it is beneficial to identify aspects of review sentences in order to generate more diverse extractive summaries (as illustrated in figure 1 ). previous works @xcite use aspect seed words to identify salient aspects in reviews. aspect seed words are sets of words related to the aspect of interest (e.g. 'food', 'restaurant', 'breakfast' for the food aspect). these works have shown that using aspect seed words can reduce redundancy and improve the informativeness of summaries. however, extraction of aspect seed words either requires manual annotation @xcite or external data @xcite . moreover, aspects for different products or entities may vary based on the domain, requiring additional effort while scaling such techniques to different domains. ## related work recent opinion summarization methods could be divided into two categories: abstractive and extractive summarization. for abstractive summarization, previous works generate abstractive summaries in two ways. some of these works @xcite @xcite use aggregated sentence representations from autoencoders to generate summaries. others @xcite @xcite @xcite generate synthetic datasets to train generation models in a supervised setting. for extractive summarization, previous works usually follow a two-step approach. in the first step, they learn representations for the review sentences. in the second step (the inference step), they use the learned representations to define relevance scores based on distance from the mean @xcite , distance from the aspect representation @xcite or aspect-specificity and sentiment polarity (zhao and chaturvedi, 2020; @xcite . this work proposes a novel inference algorithm that is independent of the underlying sentence representation model and performs summarization by leveraging the aspects present in a review sentence. previous works usually identify aspects in two ways. some @xcite @xcite of them identify aspects using aspect seed words. they generally require manual annotation or external data to obtain aspect seed words. others @xcite @xcite use autoencoders to identify the aspect automatically. they do not require human annotation, but they can be noisy since only a portion of words contributes towards the aspect. tokencluster tries to combine the advantages of both approaches. motivated by @xcite ; @xcite , we identify a set of words that are likely to describe the aspect and cluster them into several aspects. our work is also related to a contemporary but unpublished work by @xcite . like our method, their method also clusters review sentences based on the distance between a single keyword of each sentence and the aspect seed words. our method differs from their work in two major aspects. first, in contrast to their approach, tokencluster extracts multiple aspect-related words from each sentence since a sentence can mention several aspects. second, tokencluster does not rely on human-annotated aspect seed words. ## tokencluster in this section, we describe our proposed approach, tokencluster. we first describe the problem setup (sec. 3.1). then, we describe the three components involved of tokencluster's general summarization algorithm: (a) aspect identification in review sentences (sec. 3.2), (b) salient sentences identification using aspect information (sec. 3.3), and (c) sentence ordering to produce the final summaries (sec. 3.4). finally, we describe how tokencluster generates aspect-specific summaries (sec. 3.5). ## experimental setup in this section, we describe the details of our experimental setup including the datasets, hyperparameters, and baselines. ## conclusion we present tokencluster, a sentence extraction algorithm to automatically identify aspects of sentences and extract sentences based on their aspects. the aspect information helps tokencluster cover more aspects in summaries. tokencluster is independent of underlying sentence representation models and shows strong performance on space and amazon datasets using multiple sentence representation models. our extensive analysis shows that our choice of token-based clustering and aspectrelated words is optimal compared to its variants. moreover, using sentence ordering, tokencluster can generate summaries with better readability. however, tokencluster can be sensitive to noisy data and may not work well when underlying aspects are not well represented by unique words. future research can focus on improving the robustness of aspect identification in tokencluster by incorporating external data and expanding its applicability to diverse domains and languages.