Prompted Aspect Key Point Analysis for Quantitative Review Summarization (2024)

Abstract: Key Point Analysis (KPA) aims for quantitative summarization that provides key points (KPs) as succinct textual summaries and quantities measuring their prevalence. KPA studies for arguments and reviews have been reported in the literature. A majority of KPA studies for reviews adopt supervised learning to extract short sentences as KPs before matching KPs to review comments for quantification of KP prevalence. Recent abstractive approaches still generate KPs based on sentences, often leading to KPs with overlapping and hallucinated opinions, and inaccurate quantification. In this paper, we propose Prompted Aspect Key Point Analysis (PAKPA) for quantitative review summarization. PAKPA employs aspect sentiment analysis and prompted in-context learning with Large Language Models (LLMs) to generate and quantify KPs grounded in aspects for business entities, which achieves faithful KPs with accurate quantification, and removes the need for large amounts of annotated data for supervised training. Experiments on the popular review dataset Yelp and the aspect-oriented review summarization dataset SPACE show that our framework achieves state-of-the-art performance. Source code and data are available at: https://github.com/antangrocket1312/PAKPA

## introduction with the sheer volume of reviews, it is impossible for humans to read all reviews. although the star ratings aggregated from customer reviews are widely used by e-commerce platforms as indicators of quality of service for business entities @xcite , they can not explain specific details for informed decisionmaking. early review text summarization studies focused only on capturing important points with high consensus (dash et al., 2019; shandilya et al., * first two authors equally contributed to this work. @xcite ), yet overlooked minor ones and also were unable to measure the opinion prevalence. key point analysis (kpa) is proposed to summarize opinions in review comments into concise textual summaries called key points (kps), and quantify the prevalence of kps. kpa studies were initially developed for argument summarization @xcite , and then adapted to business reviews @xcite . most kpa studies adopt the extractive approach, which employs supervised learning to identify informative short sentences as key points (kps), often leading to non-readable and incoherent kps. recently, kpa studies apply abstractive summarization to paraphrase and generate kps from comments (sentences) @xcite . still, kps are generated based on sentences and often contain unfaithful and overlapping opinions, and inaccurate quantity for their prevalence. in this paper, we propose prompted aspect key point analysis (pakpa). different from previous sentence-based kpa studies, our system employs aspect-based sentiment analysis (absa) to identify aspects in comments as the opinion target and then generate and quantify kps grounded in aspects and their sentiment. importantly, to address the issue of scarce annotated data for supervised training in existing studies, we employ prompted in-context learning with llms for absa extraction and kp generation. by integrating absa into the prompted summarization process, we aim to mitigate hallucination by guiding llms to produce kps aligned with the common aspects shared by reviews. our contributions are two-fold. to our best knowledge, we are the first to employ prompted in-context learning for abstractive kpa summarization of reviews, which removes supervised training using large amount of annotated data. secondly, ## related work based on the form of summaries, review summarization studies can be broadly grouped into three classes: key point analysis, aspect-based structured summarization, and text summarization. additionally, we reviewed the forefront of prompted incontext learning for review (text) summarization. ## methodology figure 1 illustrates our pakpa framework with examples. given reviews for a business entity, pakpa performs kpa for reviews and generates kps of distinctive aspects and quantities measuring the prevalence of kps. pakpa consists of three components: core to our framework is to employ absa of review comments to identify aspect terms in reviews and predict their sentiment, which sets the basis for clustering comments based on aspects and for further generation of aspect-oriented kps. this idea is inspired by the early aspect-based structured summarization studies @xcite , which aggregates review comments by their sentiment toward common aspects for more accurate quantification of opinions. importantly, prompted in-context learning strategies are employed for aspect-based sentiment analysis of review comments, and aspect-oriented kp generation and quantification. ## conclusion in this paper, we propose prompted aspect key point analysis (pakpa), a novel kpa framework applying abstractive summarization for opinion quantification. pakpa addresses the issues of kps with overlapping opinions, hallucination, and inaccurate quantification of previous sentence-based kpa approaches. compared with previous studies, our approach effectively makes use of absa in business reviews to generate kps grounded in aspects and achieve more accurate quantification. experimental results show that our solution greatly enhances both the quantitative performance and quality of kps. secondly, our prompted in-context learning approach also deviates from the conventional supervised learning approach and removes the need for large amounts of annotated data for supervised training and fine-tuning. ivan titov and ryan mcdonald. 2008. a joint model of text and aspect ratings for sentiment summarization. in proceedings of acl-08: hlt, pages 308-316. weizhe yuan, graham neubig, and pengfei liu. 2021. bartscore: evaluating generated text as text generation. advances in neural information processing systems, 34:27263-27277. jingqing zhang, yao zhao, mohammad saleh, and peter liu. 2020. pegasus: pre-training with extracted gap-sentences for abstractive summarization. in international conference on machine learning, pages 11328-11339. pmlr. chao zhao and snigdha chaturvedi. 2020. weaklysupervised opinion summarization by leveraging external information. in proceedings of the aaai conference on artificial intelligence, volume 34, pages 9644-9651.