Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation (2024)

Abstract: Aspect sentiment coherency is an intriguing yet underexplored topic in the field of aspect-based sentiment classification. This concept reflects the common pattern where adjacent aspects often share similar sentiments. Despite its prevalence, current studies have not fully recognized the potential of modeling aspect sentiment coherency, including its implications in adversarial defense. To model aspect sentiment coherency, we propose a novel local sentiment aggregation (LSA) paradigm based on constructing a differential-weighted sentiment aggregation window. We have rigorously evaluated our model through experiments, and the results affirm the proficiency of LSA in terms of aspect coherency prediction and aspect sentiment classification. For instance, it outperforms existing models and achieves state-of-the-art sentiment classification performance across five public datasets. Furthermore, we demonstrate the promising ability of LSA in ABSC adversarial defense, thanks to its sentiment coherency modeling. To encourage further exploration and application of this concept, we have made our code publicly accessible. This will provide researchers with a valuable tool to delve into sentiment coherency modeling in future research.

## introduction aspect-based sentiment classification @xcite @xcite (absc) aims to identify sentiments associated with specific aspects within a text, as highlighted in several studies @xcite @xcite . in this work, we make efforts to address an intriguing problem within absc that has been overlooked in existing research, i.e., "aspect sentiment coherency", which focuses on modeling aspects that share similar sentiments. for instance, in the sentence "this laptop has a lot of storage, and so does the battery capacity," where "storage" and "battery capacity" aspects both contain positive sentiments. we show more examples of aspect sentiment coherency in fig. 1 and the case study section. the study of aspect sentiment coherency has not been investigated in existing research. yet, some strides have been made on a similar topic, namely sentiment dependency. these approaches, featured in several studies @xcite @xcite , hypothesize that sentiments of aspects may be dependent and usually leverage syntax trees to reveal potential sentiment dependencies between aspects. however, sentiment dependency remains a somewhat ambiguous concept in the current research landscape. furthermore, previous methods @xcite @xcite tend to model context topological dependency (e.g., context syntax structure) rather than sentiment dependency directly. these techniques are resource-intensive and computation-intensive. besides, they can suffer from token-node misalignment caused by conflicts in tokenization methods in syntax tree construction. as a further contribution to current absc research, we propose aspect sentiment coherency learning and posit that modeling sentiment coherency can provide valuable insights. modeling sentiment coherency often presents challenges for traditional absc methods due to the complexity of aspect sentiment coherency. to efficiently address the aspect sentiment coherency task, we shed light on a simple yet effective approach, namely local sentiment aggregation (lsa). more specifically, we introduce a local sentiment aggregation paradigm powered by three unique sentiment aggregation window strategies based on various aspectbased features to guide the modeling of aspect sentiment coherency. to comprehensively evaluate lsa, we conduct experiments for the aspect sentiment coherency extraction subtask and the tradi-cozy atmosphere, good food and service, good place to meet friends for dinner and a drink. in conclusion, the main contributions of our work are as follows: ## sentiment coherency we first introduce the concept of sentiment coherency and then formulate two sentiment coherency patterns. in the review about a restaurant in fig. 1 , the reviewer expresses positive sentiments about the atmosphere, food, and service but remains neutral about dinner and drinks. this tendency to express similar sentiments about re-lated aspects (e.g., atmosphere, food, and service) is what we refer to as sentiment coherency. we calculate the number of sentiment clusters across all experimental datasets to prove this is a common phenomenon. the statistics are available in our aim is to study the extraction of aspect sentiment coherency and the improvement of absc performance by incorporating sentiment coherency. we formulate two sentiment coherency patterns in the following sections. ## methodology in this section, we propose a local sentiment aggregation method for sentiment cluster prediction, which is based on the local sentiment coherency pattern. we first introduce the implementation of local sentiment aggregation, which is based on sentiment window aggregation. then, we present the aspect feature learning method used for sentiment aggregation window construction in section 3.2. finally, we describe the implementation details of our model. ## experiments in this section, we introduce the settings of our experiments and report the experimental results. we report all implementation details in the appendix, e.g., hyperparameter settings (appendix 4.2), baseline introduction (appendix 4.3) and additional experiments, etc. ## related works the related works in this field can be broadly divided into three categories: sentiment dependencybased methods, sentiment coherency modeling, and implicit sentiment learning. although sentiment coherency is prevalent in absc, it has received limited attention in recent years. however, the progress of sentiment dependency-based methods, such as the work by @xcite ; @xcite ; @xcite ; @xcite ; @xcite , has contributed to the improvement of coherent sentiment learning. these studies explored the effectiveness of syntax information in absc, which mitigates issues related to sentiment coherency extraction. for refining syntax structure quality in sentiment dependency learning, @xcite employ type-aware gcn to distinguish different relations in the graph, achieving promising results. similarly, @xcite propose syngcn and semgcn for different dependency information. tgcn model alleviates dependency parsing errors and shows significant improvement compared to previous gcn-based models. despite the aforementioned advances, transferring the new techniques proposed in these studies is not straightforward. @xcite propose employing the pretrained roberta model to induce trees for absc, effectively solving the node alignment problem. however, the efficiency of inducing trees needs improvement. compared to coarse-grained implicit sentiment research @xcite @xcite , the aspect's implicit sentiment learning in absc remains challenging. lsa leverages coherency to aggregate implicit sentiments efficiently. some researchers have formulated tasks aimed at modeling implicit sentiments and opinions. for instance, @xcite proposed a quadruple extraction task (aspect, category, opinion, and sentiment), while @xcite proposed a unified framework that crafts auxiliary sentences to aid implicit aspect extraction and sentiment analysis. in contrast to these works, lsa sidesteps the efficiency bottleneck of syntax modeling by eliminating structure information and proves to be adaptable to existing methods as it is a transferable paradigm independent of base models. @xcite presents a supervised contrastive pre-training mechanism to align the representation of implicit sentiment and explicit sentiment. however, it relies on finetuning a large-scale sentiment-annotated corpus from in-domain language resources, which may be resource-intensive and inefficient. ## conclusion aspect sentiment coherency has been overlooked in existing studies. we introduced the concept of lsa, a novel approach that brings the nuance of local sentiment coherency into the foreground of absc. lsa achieves state-of-the-art performance when combined with various aspect-specific features, especially based on the deberta models. furthermore, we also introduce a practice of lsa in the realm of adversarial defense. we hope that our work will inspire further research into sentiment coherency modeling in the future.