Decomposed Opinion Summarization with Verified Aspect-Aware Modules (2025)

Abstract: Opinion summarization plays a key role in deriving meaningful insights from large-scale online reviews. To make the process more explainable and grounded, we propose a domain-agnostic modular approach guided by review aspects (e.g., cleanliness for hotel reviews) which separates the tasks of aspect identification, opinion consolidation, and meta-review synthesis to enable greater transparency and ease of inspection. We conduct extensive experiments across datasets representing scientific research, business, and product domains. Results show that our approach generates more grounded summaries compared to strong baseline models, as verified through automated and human evaluations. Additionally, our modular approach, which incorporates reasoning based on review aspects, produces more informative intermediate outputs than other knowledge-agnostic decomposition approaches. Lastly, we provide empirical results to show that these intermediate outputs can support humans in summarizing opinions from large volumes of reviews.

## introduction reviews are omnipresent in the digital world, providing invaluable insights into products @xcite , businesses @xcite , even scientific articles @xcite . automatic opinion summarization aims to aggregate a large and diverse set of reviews about a particular entity (e.g., hotel) into a single easy-to-read meta-review (or summary). a good meta-review should accurately reflect the balance of opinions in the source reviews and speak to the entity's specific aspects (e.g., cleanliness, service, location). a useful meta-review should also present some evidence justifying its content. opinion summarization has distinct characteristics that set it apart from other summarization tasks. firstly, it cannot rely on reference summaries for training, as human-written meta-reviews are not generally available (e.g., across entities and domains) and can be difficult to crowdsource (e.g., for entities represented by thousands of reviews). secondly, a meta-review needs to cover the most important aspects related to the entity of interest. and finally, given the subjective nature of the summarization task, systems should offer some evidence to justify their output. prior approaches to generating meta-reviews broadly fall into three categories. extractive methods create summaries by selecting a few representative sentences from source reviews @xcite @xcite . while these approaches are scalable and inherently attributable, the summaries tend to be overly detailed and lack coherence. abstractive methods rely on neural language models to generate fluent and coherent meta-reviews with novel language @xcite @xcite @xcite @xcite . in the era of large language models (llms), longcontext language models could be directly used on opinion summarization with prompting @xcite . however, these abstractive approaches are neither transparent nor controllable due to the black-box nature of end-toend modeling. hybrid methods @xcite @xcite , the third category of summarisation approaches, could generate fluent and explainable summaries. they first extract information clusters in the format of sentence fragments @xcite @xcite or sentiment spans @xcite , and then generate summaries (e.g., using a language model) based on the clusters. however, they are limited in that they are based on assumptions valid for specific domains (e.g., based on the popularity of opinions) and are not entirely transparent (e.g., either the clusters or aggregation step cannot be easily verified). in this paper, we propose to decompose opinion summarization into simpler sub-tasks that are domain-agnostic and can be executed by prompting-based llms dedicated to these subtasks. our approach is also inspired by recent applications of chain-of-thought prompting @xcite and its variants @xcite , which address reasoning problems by decomposing complex tasks into a sequence of simpler sub-problems, which are solved sequentially. our decomposition consists of three highlevel modules, namely aspect identification, opinion consolidation, and meta-review synthesis. intuitively, we first identify text fragments in the input reviews discussing aspects about the entity and domain in question; next we create meta-reviews for each aspect, and finally we generate a global meta-review for all aspects (see figure 1 ). our approach eschews problems relating to the scale of the input to some extent, since reviews can be processed in parallel to identify their aspects. it also avoids problems with clusters being diffuse or irrelevant since we leverage domain specific aspect definitions (as part of the prompt) to obtain interpretable clusters. finally, our decomposition is controllable, and evidence-based, as the output of each module can be traced back to its input. our contributions can be summarized as follows: our work focuses on abstractive opinion summarization that aims to generate fluent and coherent summaries with novel language @xcite . this task has been explored in different domains, such as summarizing reviews of products, businesses, and scientific articles (chu and @xcite @xcite . previous abstractive methods lack transparency in their decision-making process due to their end-to-end nature @xcite @xcite . hybrid approaches implement pipelines with transparent intermediate outputs, however, most of them are aspect agnostic, focusing on how to organize or annotate the input for downstream processing. for example, @xcite propose a method that represents sentences from reviews as paths through a learned discrete hierarchy, and then use llms to generate sentences based on frequent paths retrieved from this hierarchy. their retrieval module relies on majority voting, which is less effective in domains where minority but well-argued opinions are valuable, such as in scientific reviews @xcite . inspired by a well-known decomposition of multi-document summarization into three modules, namely content selection, content consolidation (or fusion), and output generation @xcite @xcite @xcite , we apply a similar aspectguided decomposition to the task of opinion summarization. a few approaches take aspects into account. for example, @xcite cluster opinions through a discrete latent variable model and extract sentences based on popular aspects or a particular aspect, while li et al. (2023a) learn aspects by clustering of aspect-related words. amplayo et al. (2021a) fine-tune pre-trained models on synthetic data enhanced with aspect annotations which can be used to control output summaries at inference time. different from earlier work @xcite @xcite which identifies aspects based on sentences, our approach identifies opinions in text fragments of variable lengths. we delegate the task of aspect identification to prompt engineering, demonstrating that llms can reliably extract aspects in the format of flexible text fragments given an input review and aspect definitions without additional source reviews selected clusters final meta-review aspect-specific meta-reviews the paper introduces a bayesian model. the authors fail to provide enough experiments to support their claimed contributions. ....... the role of intra-class variance is not clear. ......on the other hand, ...... the main weakness of the paper is its technical novelty. ...... ...... the flow from section 3.1 to 3.2 is not clear. ...... if a few ls techniques work well empirically, why does that lead you to investigate me? please clearly define pgd-20 and the exact measure used to produce the in addition, existing ...... the clarity of the paper need improvement. the novelty of the paper is limited. ## task decomposition let c denote a corpus of reviews on entities {e 1 , e 2 , . . . } from a domain d, for example, hotels or scientific articles. reviews may discuss a number of relevant aspects a @xmath0, a 2 , . . . }, like clarity or soundness, for each entity e i , our task is to generate a meta-review y i by synthesizing opinions from a set of source reviews r @xmath1, r 2 , . . . } covering all attested aspects a d . we decompose the task into three modules, namely aspect identification, opinion consolidation, and meta-review synthesis. we present the inner workings of each module in figure 1 with an example from the scientific domain. due to the limited availability of training data, we implement our modules using an unsupervised approach, leveraging zeroshot prompting of llms and their instruction following and generation capabilities. 2 aspect identification as not all content in the source reviews is relevant for generating metareviews, opinion summarization models must be able to isolate critical information in the input. the first module, aspect identification, selects text fragments of variable lengths from source reviews discussing any review aspect. specifically, for reviewed entity e i , our module identifies text fragments for aspect a j from source reviews r i . the module essentially partitions text fragments into aspect-specific clusters c i,@xmath2, f 2 , . . . }, where fragments f m can originate from any source review in r i . for example, in figure 1 , the module identifies fragments in scientific reviews for the aspects clarity, soundness, and novelty. we implement this module with zero-shot llm prompting. our prompt template is shown in appendix a (figure 3 ) and can be modified for different aspects and domains. our aspect identification is not based on sentence clustering as well-justified opinions may be composed of multiple sentences, and each text fragment could be identified for multiple aspects. opinion consolidation as shown in figure 1 , the output of the first module consists of clusters of text fragments, each discussing a specific aspect. depending on the domain, these clusters can have a lot of redundancy, often repeating the same opinion. our second module, opinion consolidation, aggregates opinions into aspect-specific meta-reviews. we essentially adopt a divide-and-conquer strategy, since generating meta-reviews from aspect-specific clusters is significantly easier than producing an entire summary from reviews containing mixed aspects. specifically, taking as input cluster c i,j , the module generates meta-review o i,j for aspect a j . as we do not have training data for these intermediate summaries, we also implement this module with zero-shot prompting. 3 our template (shown in the appendix, figure 4 ) instructs llms to integrate opinions (i.e., text fragments) from a specific cluster. for example, in figure 1 , the three sentences in the clarity cluster are aggregated into "the clarity of the paper needs improvement". ## experimental setup we showcase the versatility of our approach on different domains. we first describe the datasets used in our experiments, discuss implementation details and comparison baselines, and explain how we evaluate performance with automatic metrics. ## results and analysis we perform experiments on datasets covering multiple domains, comparing meta-reviews generated by our approach with those from strong baselines and state-of-the-art approaches. we further eval finally, in addition to automatic evaluation we conduct human evaluation based on pair-wise system comparisons and intermediate outputs. aspect-aware decomposition leads to better aspect coverage and opinion faithfulness. our results using automatic evaluation metrics are summarized in llama-70b performs well at identifying and summarizing aspects. in addition to evaluating the generated meta-reviews, we conduct evaluations on the intermediate outputs of our modules. we only report results on the scientific domain to generating more faithful meta-reviews and with higher aspect coverage, however opinion consolidation appears to be the most critical as its removal decreases performance across domains (exception: coverage for research articles). we also observe that model-extracted text fragments are on par with human-selected ones but more helpful to generating faithful meta-reviews. humans prefer meta-reviews generated by our modular system to gold-standard references. we conduct human evaluation to verify that our approach generates meta-reviews that reflect the review aspects of the input and are overall coherent and faithful. we recruited crowdworkers through prolific foot_8 , selected to be l1 english speakers from the us or uk, and compensated above the uk living wage at 12gbp/hr. we ask crowdworkers to read a set of source reviews followed by two generated meta-reviews and select which meta-review is best (allowing for ties) along two dimensions, as we randomly select ten entities for each dataset (space, amasum, and peersum) and construct six pairwise combinations between our approach (aspect-aware decomposition with llama-3.1-70b) and the systems shown in ## conclusion we propose modular decomposition for opinion summarization based on review aspects. our decomposition is evidence-based (the output of each module can be traced back to its input), enabling greater transparency and ease of inspection. extensive experiments demonstrate that our modular framework outperforms state-of-the-art methods and other strong baselines in multiple domains. human evaluations reveal that our approach not only produces higher-quality meta-reviews but also generates more useful intermediate outputs to assist humans in composing meta-reviews. while our work focuses on opinion summarization, the concept of aspect-aware decomposition holds promise for other complex language generation tasks.