Dynamic Multi-granularity Attribution Network for Aspect-based Sentiment Analysis (2024)

Abstract: Aspect-based sentiment analysis (ABSA) aims to predict the sentiment polarity of a specific aspect within a given sentence. Most existing methods predominantly leverage semantic or syntactic information based on attention scores, which are susceptible to interference caused by irrelevant contexts and often lack sentiment knowledge at a data-specific level. In this paper, we propose a novel Dynamic Multi-granularity Attribution Network (DMAN) from the perspective of attribution. Initially, we leverage Integrated Gradients to dynamically extract attribution scores for each token, which contain underlying reasoning knowledge for sentiment analysis. Subsequently, we aggregate attribution representations from multiple semantic granularities in natural language, enhancing a profound understanding of the semantics. Finally, we integrate attribution scores with syntactic information to capture the relationships between aspects and their relevant contexts more accurately during the sentence understanding process. Extensive experiments on five benchmark datasets demonstrate the effectiveness of our proposed method.

## introduction aspect-based sentiment analysis (absa) is a finegrained classification task that focuses on identifying the sentiment polarity of specific aspects within a sentence @xcite . for instance, given a sentence "the street is very crowded, but the atmosphere is pleasant", the task aims to predict sentiment polarity associated with two aspects "street" and "atmosphere", which are negative and positive respectively. the core challenge of absa is to model the correlations between the specific aspect and its contexts, especially those parts that express opinions and sentiments. to this end, various studies @xcite @xcite zhang et al., 2021) concentrate on attention mechanisms to model the relationships between an aspect and its contexts. in addition, many methods @xcite @xcite ) leverage syntactic information derived from dependency trees to better capture the interactions between aspects and opinion expressions. with the development of the fine-tuning paradigm, methods incorporating pre-trained language models (plms) @xcite @xcite have demonstrated impressive results in absa. despite these significant advancements, critical challenges persist when directly applying attention mechanisms or syntactic information to this fine-grained task. specifically, attention-based methods may inappropriately assign high attention scores to words that are irrelevant to the aspect. @xcite ; @xcite ; @xcite propose that's because attention mechanisms are usually vulnerable to noise within sentences. considering the sentence in figure 1 (a), the aspect "service" receives disproportionately high attention scores for the unrelated opinion words "pretty" and "good". furthermore, some research that focuses on the interpretability of attention mechanisms @xcite @xcite have indicated that attention scores do not always correlate with significance. @xcite have discovered that removing fea-tures deemed important by attention scores leads to less prediction flip than gradient-based strategies. besides, @xcite have observed shuffling the attention weights often does not affect the final prediction, which is consistent with our observations that are shown in figure 1 (b) . to sum up, while attention mechanisms have improved the performance of absa, they often operate as a black box, leaving their ability to accurately capture critical opinion words remains debatable. this underscores the need for methods that efficiently capture keywords for reasoning sentiment polarity. additionally, although leveraging syntactic knowledge can help model the correlations between the aspect and the contexts, it is important to recognize that not all syntactic information is equally beneficial to this fine-grained task. more concretely, syntactic information irrelevant to the specific aspect can be redundant and may even introduce noise rather than provide valuable insights. therefore, it is crucial to focus on extracting relevant syntactic information, emphasizing the identification of essential words within sentences. to address the aforementioned issues, we introduce attribution analysis into absa and propose a dynamic multi-granularity attribution network (dman). attribution information reflects the importance of different tokens towards the prediction, which contains reasoning knowledge of the sentiment at a data-specific level. initially, we employ integrated gradients (ig) @xcite , a well-established gradient-based attribution method, to compute the importance scores of tokens. inspired by the observation @xcite that the significance of essential words dynamically changes during semantic comprehension, we design multi-step attribution analysis to capture the dynamic significance of tokens during the comprehension process. more concretely, we utilize stacked self-attention blocks in conjunction with ig to calculate attribution scores for each layer and adopt a top-k strategy to filter out dimensions with low values, thereby reducing the impact of trivial dimensions. subsequently, we incorporate semantic representations at both token and span levels to derive multi-granularity attribution scores, ensuring more comprehensive semantic concepts. finally, we construct the adjacency matrices based on the dependency tree, and then use obtained attribution scores to initialize different adjacency matrices for different layers of gcns, which facilitates the dynamic capture of critical syntactic knowledge during throughout the process of sentence comprehension. in summary, our contributions could be summarized as follows: ## methods in this section, we describe our proposed dman in detail. specifically, we begin with the problem definition, followed by the encoder module and the overall architecture of our dman. problem definition. given a sentence-aspect pair (s, a), where @xmath0, w 2 , ..., w n } is a sentence with n words, and @xmath1, a 2 , ..., a m } is the given aspect with m words. absa aims to predict the sentiment polarity of the aspect a in the sentence s. encoder. we utilize bert as the sentence encoder to extract aspect-specific context representations. we construct input as "[cls] s [sep ] a [sep ]" to map each word into a real-value vector, obtaining sentence embedding e 0 = {e 1 , e 2 , ..., e n } and aspect embedding e @xmath2, e a 2 , ..., e am }. overall architecture. as illustrated in figure 2 , our proposed dynamic multi-granularity attribution network mainly comprises three primary components: (1) multi-step attribution extraction, (2) multi-granularity attribution, and (3) dynamic syntax concentration. the technical details will be elaborated on as follows. ## conclusion in this paper, we propose a novel dynamic multigranularity attribution network (dman) for the absa task, which differs from traditional models that rely on attention scores. specifically, we first leverage integrated gradients to extract multi-step attribution during semantic comprehension, and the top-k strategy is adopted to filter out unimportant dimensions. we then consider multiple granularities of semantic concepts, fusing attribution representations from both token-level and span-level. finally, we integrate these attribution values with dependency trees to dynamically capture relevant syntactic knowledge, thereby enhancing semantic understanding for sentiment classification. extensive experiments on five public datasets demonstrate the effectiveness of our proposed dman.