DINER: Debiasing Aspect-based Sentiment Analysis with Multi-variable Causal Inference (2024)

Abstract: Though notable progress has been made, neural-based aspect-based sentiment analysis (ABSA) models are prone to learn spurious correlations from annotation biases, resulting in poor robustness on adversarial data transformations. Among the debiasing solutions, causal inference-based methods have attracted much research attention, which can be mainly categorized into causal intervention methods and counterfactual reasoning methods. However, most of the present debiasing methods focus on single-variable causal inference, which is not suitable for ABSA with two input variables (the target aspect and the review). In this paper, we propose a novel framework based on multi-variable causal inference for debiasing ABSA. In this framework, different types of biases are tackled based on different causal intervention methods. For the review branch, the bias is modeled as indirect confounding from context, where backdoor adjustment intervention is employed for debiasing. For the aspect branch, the bias is described as a direct correlation with labels, where counterfactual reasoning is adopted for debiasing. Extensive experiments demonstrate the effectiveness of the proposed method compared to various baselines on the two widely used real-world aspect robustness test set datasets. 1

heading: 1 Introduction
text: Aspect-Based Sentiment Analysis (ABSA) aims to classify the polarity of the sentiment (e.g., positive, negative, or neutral) towards a specific aspect of a sentence (e.g., bugers in the review “Tasty bugers, and crispy fries.”) (Hu and Liu, 2004; Jiang et al., 2011; Vo and Zhang, 2015; Zhang et al., 2016, 2022). Most ABSA methods solve the task as an input-output mapping problem based
∗Equal Contribution. †Corresponding Author.
1Our code and results will be available at https:// github.com/callanwu/DINER.
on high-capacity neural networks and pre-trained language models (Wang et al., 2018; Huang and Carley, 2018; Bai et al., 2020). Though remarkable progress has been made, it is demonstrated that these state-of-the-art models are not robust in data transformation where simply reversing the polarity of the target results in over 20% drop in accuracy (Xing et al., 2020).
A reasonable explanation is that neural networks trained with the Stochastic Gradient Descent algorithm are vulnerable to annotation biases and learn the shortcuts instead of the underlying task (Xing et al., 2020). As shown in Figure 1 (a), over 50.0% of targets have only one kind of polarity label in the widely used SemEval 2014 Laptop and Restaurant datasets (Pontiki et al., 2014). For 83.9% and 79.6% instances in the test sets, the sentiments of the target aspect and all non-target aspects are the same. Therefore, it is easy for end-to-end neural models to learn such spurious correlations and
3504
make predictions solely based on target aspects or sentiment words describing non-target aspects.
To avoid learning spurious correlations, recent methods focus on debiasing, which can be categorized into argumentation-based methods (Wei and Zou, 2019; Lee et al., 2021), reweight trainingbased methods (Schuster et al., 2019; Karimi Mahabadi et al., 2020) and causal inference-based methods (Niu et al., 2021; Liu et al., 2022b). Among them, causal inference attracts much research interest for its theoretical-granted property and little modification to the existing learning paradigm. Niu et al. (2021) proposed a debiasing method for the language bias in the vision question answering task by performing counterfactual reasoning. Liu et al. (2022b) employed backdoor adjustment-based intervention for mitigating the context bias in object detection. Recent attempts have been made to solve various biases in natural language processing tasks, including natural language understanding (Tian et al., 2022), implicit sentiment analysis (Wang et al., 2022), and fact verification (Xu et al., 2023).
However, most causal inference-based debiasing methods are based on single-variable causal inference, which is not appropriate for ABSA with two input variables. As shown in Figure 1 (a), there are two types of biases in ABSA. The target aspects A are often directly correlated with the polarity labels L, while the sentiment words for targets in the review R are often indirectly confounded with the non-targets C. To further investigate the difference between aspect-related biases and review-related biases, a simple experiment is conducted by training two probing models with only review or aspect as input. As shown in Figure 1 (b), the aspect-only model has similar performances on the original and the adversarial test set except REVTGT where spurious correlations learned in the training set are flipped, while the review-only model performs differently on four test variants. It might suggest that the biases in the aspect branch are direct and simple, while the biases in the review branch are indirect and complicated, which poses a challenge.
To tackle the above challenge, we propose Debias IN AspEct and Review (DINER) based multi-variable causal inference for debiasing ABSA. To be more specific, as illustrated in Figure 2, the unbiased prediction is obtained by calculating the total indirect effect of the target aspect and the review of the polarity label, which is further
decomposed and estimated by the hybrid causal intervention method. For the R → L branch, a backdoor adjustment intervention is employed to mitigate the indirect confounding between the target sentiment words in the review and the context. For the A→ L branch, a counterfactual reasoning intervention is employed to remove the direct correlation between the target and the label. Extensive experiments on two widely used real-world robustness test benchmark datasets show the effectiveness of our framework.
Overall, our contributions can be summarized as follows:
• A novel framework is proposed for debiasing ASBA based on multi-variable causal inference. As far as we know, we are the first to uncover and analyze the bias problem in ABSA using multi-variable causal inference.
• A hybrid intervention method is constructed by combining backdoor adjustment and counterfactual reasoning.
• The detailed evaluation demonstrates that the proposed method empirically advances the state-of-the-art baselines.

heading: 2 Related Work
text: Our work is mainly related to two lines of research, described as follows.

heading: 2.1 Aspect-Based Sentiment Analysis
text: ABSA has garnered significant research attention in recent years. Early works focus on feature engineering with manual-construction sentiment lexicons and syntactic features, and rule-based classifiers are adopted to make predictions (Jiang et al., 2011; Kiritchenko et al., 2014). With the development of neural networks and word embedding techniques, neural-based models have dominated the area with architectures such as LSTM, CNN, Attention mechanisms, Capsule Network (Tang et al., 2016a; Wang et al., 2016; Xue and Li, 2018; Jiang et al., 2019). Recent advances in pre-trained language models such as BERT (Devlin et al., 2019) have shifted the paradigm again (Zhang et al., 2022), where most recent models take pre-trained models as backbones (Xu et al., 2019; Hou et al., 2021; Cao et al., 2022). However, ABSA still faces challenges on robustness datasets, and it is precisely such tasks that our approach targets.

heading: 2.2 Causal Inference-based Debiasing
text: Causal inference (Pearl, 1995, 2009) has been widely employed for debiasing in various fields, including computer vision, recommendation, and natural language processing (Niu et al., 2021; Zhang et al., 2021b; Tian et al., 2022). The main methods employed consist of counterfactual reasoning and causal intervention. Niu et al. (2021) proposed to remove the language bias in vision question answering by subtracting the results of a counterfactual language-only model from the results of a vanilla language-vision model. Following this work, counterfactual reasoning is widely applied to debiasing the spurious correlation between input and label in tasks including natural language understanding (Tian et al., 2022), machine reading comprehension (Guo et al., 2023; Zhu et al., 2023) and fact verification (Xu et al., 2023). Liu et al. (2022b) proposed to de-confound the object from the context in object detection with backdoor adjustment, where an inverse probability weight approximation is made to estimate the do-operator. Another way to estimate the do-operator is known as normalized weighted geometrical mean (NWGM), which is firstly adopted in image caption by Liu et al. (2022a). Following this line of work, backdoor adjustment-based debiasing has widely been explored in tasks including named entity recognition (Zhang et al., 2021a) and multi-modal fake news detection (Chen et al., 2023). Some methods also employ other causal inference techniques, including instrument variable (Wang et al., 2022) and colliding effects (Zheng et al., 2022). However, most of the present debiasing methods focus on debiasing a single input variable, while we are the first to debias two input variables in ABSA simultaneously.

heading: 3 Methods
text: In this section, we will introduce the proposed method, DINER, in detail. First, we will define the Structural Causal Model (SCM) of ABSA and derive the formula of causal effect step by step. Then, we will formulate how to estimate the components in the causal effect formula with backdoor adjustment and counterfactual reasoning. Finally, we will introduce the training and inference processes.

heading: 3.1 Structural Causal Model of ABSA
text: The SCM of ABSA, which is formulated as a directed acyclic graph, is shown in Figure 2 (a). The
nodes in the SCM denote causal variables, and the edges denote causalities between two nodes (e.g., X → Y means X causes Y ). Then we will discuss the rationale behind how this SCM is built:
• R→ K ← A. The prediction of ABSA is dependent on both review R and aspect A. Therefore, a fused knowledge node K is caused by both R and A.
• K → L. The label L is caused by the fused knowledge K, which is the desired causal effect of ABSA.
• R → L ← A. The label L is also directly affected by review R and aspect A, where the spurious correlation comes from and should be removed.
• C → R and C → L. The confounder C (the prior context knowledge) caused R and L simultaneously, where the annotation biases come from. For example, most reviews contain positive descriptions for multiple types of food, which will encourage the model to make predictions without identifying the target.
It is worth noticing that we do not add the edge C → A or R→ A. Because we believe the choice of aspect A is made by the annotators and not restricted by the context C or review R.
With the SCM defined, we can derive the formula of causal effect. As shown in Figure 2 (b), the desired situation for ABSA is that the edges that bring biases are all blocked, and the prediction is based on aspect A and review R solely through the
fused knowledge K. With the language of causal inference, the prediction should be made on:
TIEa,r = TEa,r −NDEr −NDEa + IEa,r (1)
where TIEa,r denotes the Total Indirect Effect (TIE) from A and R on L, TEa,r denotes the Total Effect (TE), NDE denotes the Natural Direct Effect (NDE), and IEa,r denotes the Interaction Effect (IE) between A and R. The total effect TE contains all causal effects from A and R on L, inducing the biases, while the natural direct effect (NDE) only measures the direct causal effect between two variables, which can be regarded as the bias-only effect. Therefore, subtracting NDEa and NDEr from TEa,r will results in the unbiased causal effect from A and R on L, which is the total indirect effect TIEa,r. It is worth noticing that since there is no causality between A and R, the value of the interaction effect IEa,r can be set to 0.
Based on the defintion of TE and NDE (Niu et al., 2021):
TEa,r = La,r,k − La∗,r∗,k∗ (2) NDEa = La,r∗,k∗ − La∗,r∗,k∗ (3) NDEr = La∗,r,k∗ − La∗,r∗,k∗ (4)
where L denotes the prediction and x∗ denotes variable x is set to be void, we can have:
TIEa,r = La,r,k − La∗,r,k∗ − La,r∗,k∗ + La∗,r∗,k∗ = TIEa,r′ −NDEa
(5)
where r ′
denotes the debiased review, obtained after the process of deconfounding.

heading: 3.2 Deconfounding the Review Branch with Backdoor Adjustment
text: Based on Eq. (5), we can estimate each component and obtain an unbiased prediction. However, as R and L are indirectly confounded with context
C, it is not easy to calculate La,r,k and La∗,r,k∗ . Therefore, we debias the review R first.
La,r,k = Ψ(ζa, ζr′ , ζk) (6)
where ζk denotes the logit of the softmax layer, Ψ(·) denotes the fusion function, specially ζr′ denotes the debiased output based on R.
There are mainly three types of causal intervention methods based on causal inference, the backdoor adjustment, the front-door adjustment, and the instrument variable adjustment. However, the front-door adjustment requires a mediator variable between input and output, which is not applicable in our SCM (Zhang et al., 2024a,b). The instrument variable adjustment involves building up an extra instrument variable in SCM, which makes the already complex SCM even more complex (Wang et al., 2022). So we choose the backdoor adjustment for debiasing the review branch. Consider the SCM only contains R, C, and L, C satisfies the backdoor criterion, and we can have:
P (L|do(R)) = ∑
c
P (L|R,C)P (C)
= ∑
c
P (L,R|C)P (C) P (R|C)
(7)
where the do(R) operator denotes a causal intervention that severs the direct effect of R on L.
A common workaround is the application of the Normalized Weighted Geometric Mean (NWGM) (Xu et al., 2015) to approximate the effects of the do-operator. Our approach adopts an Inverse Probability Weighting (IPW) (Pearl, 2009) perspective, which provides a novel lens through which to approximate the infinite sampling of (l, r)|c as shown in Eq. (7).
In a finite dataset, the observable instances of (l, r) for each unique c are limited. Consequently, the number of c values considered in our equation equates to the available data samples rather than the theoretically infinite possibilities of c. Backdoor adjustment bridges the gap between the confounded and de-confounded models, allowing us to treat samples from the confounded model as if they were drawn from the de-confounded scenario. This leads to an approximation:
P (L|do(R = r)) ≈ P̃ (L,R|C = c)
≈ 1 K
K∑
k=1
P̃ (L,R = rk|C = c)
(8)
where P̃ denotes the inverse weighted probability. We employ a multi-head strategy, inspired by Vaswani et al. (2017) to refine the granularity of our sampling by partitioning the weight and feature dimensions into K groups, rk denotes the review information in the group K = k. For simplicity, subsequent discussions will omit C = c, though it is understood that r remains dependent on c. We will employ TDE to debias this effect following Liu et al. (2022a) and Tang et al. (2020).
The energy-based model (LeCun et al., 2006) framework underpins our modeling of P̃ , where the softmax-activated probability is proportional to an energy function defined as:
P̃ (L = l, R = rk) ∝ E(l, rk;wk)
= τ f(l, rk;wk)
g(l, rk;wk)
(9)
with τ serving as a scaling factor analogous to the inverse temperature in Gibbs distributions (Geman and Geman, 1984), wk denotes the weight parameter in the group K = k. The numerator f(l, rk;wk) represents the unnormalized effect, calculated as logits (wk)⊤rk, while the denominator g(l, rk;wk) serves as a normalization term(or propensity score (Austin, 2011)), which ensures balanced magnitudes of the variables. The denominator, i.e., inverse probability weight, becomes the propensity score under the energy-based model, where the effect is divided into the controlled group ∥wk∥ · ∥rk∥ and the uncontrolled group ϵ · ∥rk∥.
The computation of logits for P (L|do(R = r)) is thus expressed as:
P (L|do(R)) = τ K
K∑
k=1
(wk)⊤rk
(∥wk∥+ ϵ)∥rk∥ (10)
Now we need to obtain context features given the current samples to force the model to concentrate on the debiased review based on TDE. We assume U as a confounder set {ui}Ni=1, where N is the number of aspects in dataset and ui is the prototype for the context of class i in feature space. Review features can be linearly or non-linearly represented by the manifolds (Turk and Pentland, 1991; Candès et al., 2011), and so are the context features. Therefore, we model the review-specific context features C of current samples as follows:
C = f(r, U) = N∑
N=1
P (un|r)un (11)
where P (ui|r) is the classification probability of the feature r belonging to the context of class i.
The last remaining difficulty is implementing the contextual confounder set U . To obtain more useful contextual information, we employ the lower K layers of the model on R→ L branch, which is in early training to model U . It is motivated by three primary considerations: First, the acknowledgment of the intrinsic wealth of contextual semantic information harbored within pre-trained language models (Liu et al., 2019b; Devlin et al., 2019) due to their extensive pre-training. Second, our requirement is not for highly advanced semantics but rather for contextual information (Zeiler and Fergus, 2014; Liu et al., 2022b); previous empirical studies (Jawahar et al., 2019; Liu et al., 2019a; Geva et al., 2021) have shown that encoder-only models exhibit superior performance in capturing contextual information at lower layers. Third, in the initial stages of training, the model’s classification capabilities predominantly rely on context.
To be specific, we encode each r using the aforementioned method, and if r contains a specific aspect, it is then represented as the representation of the corresponding un, and we apply the mean feature as the final un representation.
Given the modeling of U and C, we are ready for the representations of context bias. We model them as rc = F(r, C). Following Liu et al. (2022b), we choose W · concat(r,M) to map since adding more networks to learn how much we need from the context is better.
Now we can debias the impact of C on R (C → R) based on TDE. The final definition of debiased r ′ is as follows:
ζr′ = τ
K
K∑
k=1
(wk)⊤
(∥wk∥+ ϵ)
( rk
∥rk∥ − rkc ∥rkc ∥
) (12)

heading: 3.3 Decorrelating the Aspect Branch with Counterfactual Reasoning
text: While we have successfully mitigated contextual bias in the R → L pathway, the ABSA model, as delineated in Figure 2, remains susceptible to aspect-only bias. This bias persists because the prediction, denoted as La,r′ ,k, is directly influenced by the aspect variable A. To address this, we introduce a counterfactual reasoning approach that estimates the direct causal effect of A on L, effectively isolating the influence of R and K. Figure 3 shows the causal graph of the counterfactual world for ABSA which describes the scenario when A
is set to different values a and a∗. We also set R to its reference value r∗, therefore K would attain the value k∗ when R = r∗ and A = a∗. In this way,the inputs of R and K are blocked, and the model can only rely on the given aspect a for detection. The natural direct effect (NDE) of A on L, which represents the aspect-only bias, is calculated as follows:
NDEa = La,r∗,k∗ − La∗,r∗,k∗ (13)
To eliminate this bias, we adjust TE by subtracting NDE, yielding TIE in Eq. (5).
Following the previous studies, we calculate the prediction La,r,k through a model ensemble with a fusion function:
La,r,k = L(A = a,R = r ′ ,K = k)
= Ψ(ζa, ζr′ , ζk) = ζk + tanh(ζa) + tanh(ζr′ ) (14)
where ζr′ is the output of the review-only branch (i.e., R → L ), ζa is the output of the aspect-only branch (i.e., A → L ), and ζk is the output of fused features branch (i.e., K → L ) as shown in Figure 3. TIE is the debiased result we used for inference.

heading: 3.4 Training and Inference
text: We compute separate losses for each branch during the training stage in line with the methodologies adopted by recent studies (Wang et al., 2021; Niu et al., 2021; Tian et al., 2022; Chen et al., 2023). These branches comprise the fused feature branch (base ABSA, LK), the aspect-only branch (LA), and the debiased review-only branch (LR). The collective minimization of these losses forms a comprehensive multi-task training objective, which serves to optimize the model parameters. The training objective is formally expressed as:
L = LK + αLA + βLR (15)
where α and β are hyperparameters that control the contribution of each branch to the overall training objective.
The loss component LK corresponds to the cross-entropy loss calculated from the predictions of Ψ(ζa, ζr′ , ζk), as defined in Eq. (14). Similarly, the aspect-only and debiased review-only losses are denoted as LA and LR respectively.
We use debiased TIEa,r in Eq. 5 for inference.

heading: 4 Experiments

heading: 4.1 Datasets
text: We conduct training on the original SemEval 2014 Laptop and Restaurant datasets (Pontiki et al., 2014), and perform testing on the ARTS datasets, as introduced by Xing et al. (2020), to assess the efficacy of the proposed method. Detailed information about the ARTS datasets is shown in Appendix A.

heading: 4.2 Baselines
text: We consider baselines in the ARTS original paper (Xing et al., 2020), which are listed in Appendix B and following strong baselines for comparison: GraphMerge: Hou et al. (2021) combine multiple dependency trees using a graph-ensemble technique for aspect-level sentiment analysis. SENTA: Bi et al. (2021) propose a novel Sentiment Adjustment model, employing backdoor adjustment to mitigate confounding effects. And PTSENTA use BERT-PT (Xu et al., 2019) as backbone. NADS: Cao et al. (2022) apply no-aspect contrastive learning to reduce aspect sentiment bias and improve sentence representations. ChatGPT: ChatGPT is a conversational version of GPT-3.5 model (Ouyang et al., 2022; OpenAI, 2022). We use the gpt-3.5-turbo-0125 API from OpenAI2. The prompts for this task are presented in Appendix C.

heading: 4.3 Implementations
text: Our method is model-agnostic. In the empirical study, we utilize two types of mainstream encoderonly model, RoBERTa (Liu et al., 2019b)3 and BERT (Devlin et al., 2019)4 as the backbone for our experiments. For comprehensive details on the hyperparameters employed in our experiments, refer to Appendix D.

heading: 4.4 Evaluation
text: Following the previous works (Wang et al., 2016; Xue and Li, 2018; Cao et al., 2022), Accuracy (Acc.), F1-score and Aspect Robustness Score (ARS) (Xing et al., 2020) are employed as complementary evaluation metrics. ARS considers the
2https://platform.openai.com/docs/models/ gpt-3-5
3https://huggingface.co/FacebookAI/ roberta-base
4https://huggingface.co/bert-base-uncased
accurate classification of a source example and all its derived variants, produced through the aforementioned three strategies, as a single instance of correctness.

heading: 5 Result and Analysis
text: Table 1 presents a detailed comparison of various models’ performance for laptop and restaurant domains of the ARTS datasets, focusing on three key evaluation metrics: Acc., F1-score, and ARS.
Overall, PLMs, on average, perform better than non-PLMs due to the pre-trained knowledge and tasks, making them more robust. Surprisely, ChatGPT does not get perform well in this task, exhibiting ARS scores of only 50.94 in the laptop domain and 54.82 in the restaurant domain, which are even lower than those of most PLMs in Table 1. This underscores ChatGPT’s relatively poor robustness in ARTS’s variations, despite its otherwise robust performance across various other NLP tasks.
We evaluate DINER in two backbones: BERTbased and RoBERTa-based. These configurations are set to evaluate the effectiveness of DINER when integrated with different encoder-only PLMs. And DINER based on RoBERTa tends to outperform its BERT counterparts, which may be attributed to RoBERTa’s more robust pre-training on a larger and more diverse corpus, leading to better generalization capabilities (Liu et al.,
2019b). The results are compelling, showing that DINER(RoBERTa-based) model achieves the state-of-the-art performance across all metrics in both the laptop and restaurant domains, with a notable Acc. of 76.51 and 82.46, F1-scores of 73.27 and 76.92, and ARS of 59.40 and 64.02, respectively. DINER(RoBERTa-based) demonstrates superior performance in the Laptop domain, outpacing the baseline RoBERTa† by margins of 1.55, 1.11, and 3.13 in terms of Acc., F1-score, and ARS metrics, respectively. In the Restaurant domain, the model further extends its lead, achieving improvements of 3.20, 6.45, and 4.06 in the same metrics. Similarly, the DINER(BERT-based) exhibits empirical enhancements.

heading: 5.1 More Detailed Result
text: We list in detail the performance of each model on the aforementioned three subsets of the ARTS datasets in Table 2. In the Laptop domain, the baseline model RoBERTa† exhibits a REVTGT accuracy of 62.45, as REVTGT is the most challenging subset. It requires the model to pay precise attention to the target sentiment words. In contrast, the DINER framework significantly enhances this metric to 65.02, marking a 4.12% increment. Similarly, for REVNON and ADDDIFF, DINER outperforms the Vanilla baseline with modest improvements of 0.86% and 2.27%, respectively. The Restaurant do-
main further underscores the efficacy of the DINER framework, where a remarkable 10.35% improvement is observed in the REVTGT task, elevating the accuracy from 64.06 to 70.69. The framework also exhibits gains in REVNON and ADDDIFF tasks by 1.08% and 3.10%, respectively. The significant improvement observed in the restaurant domain underscores the effectiveness of our methods, particularly given the inherently challenging nature of the test set data in this domain, as highlighted by Xing et al. (2020). Specifically, the more challenging the dataset, the greater the improvement our framework offers. Interestingly, our method also gives a slight improvement on the ORIGINAL test set, illustrating the fact that we have also debiased the robust test data on the ORIGINAL test set.

heading: 5.2 Effects of the Two Branches in DINER
text: We delve into the empirical evaluation of the dual-branch architecture underpinning the DINER framework, specifically examining its constituent elements through ablation studies. The studies are shown in Table 3, offering insights into the incremental benefits conferred by each branch.
For the R→ L branch, NWGM (Xu et al., 2015) yields a marginal improvement in accuracy across both domains. The method of IPW (Pearl, 2009) further enhances performance, suggesting the ef-
ficacy of backdoor adjustment intervention in the methods, and IPW has a more precise approximation compared to NWGM (Xu et al., 2015). We further debias context based on TDE, as described in Section 3.2, and performance is further enhanced upon the application of Counterfactual Reasoning.
Parallel to this, the A → L branch investigates the impact of Counterfactual Inference. After conducting Counterfactual Inference at this branch, Acc. in the Laptop and Restaurant domains improved by 0.27 and 1.25, respectively. In the Restaurant domain, the bias associated with aspects is more pronounced.
By comparing the performance improvements at both branches, we can also discern that the bias and shortcuts from R → L branch are more pronounced, and our approach has effectively addressed these issues.

heading: 5.3 Impact of Different Fusion Strategies
text: Following prior studies (Wang et al., 2021; Niu et al., 2021; Chen et al., 2023), we devise several differentiable arithmetic binary operations for the fusion strategy in Eq. (16):    MUL-Vanilla : La,r′ ,k = ζa · ζr′ · ζk, MUL-sigmoid : La,r′ ,k = ζk · σ(ζa) · σ(ζr′ ), MUL-tanh : La,r′ ,k = ζk · tanh(ζa) · tanh(ζr′ ), SUM-Vanilla : La,r′ ,k = ζa + ζr′ + ζk, SUM-sigmoid : La,r′ ,k = ζk + σ(ζa) + σ(ζr′ ), SUM-tanh : La,r′ ,k = ζk + tanh(ζa) + tanh(ζr′ ) (16) The Acc. performance of six distinct different fusion strategies are reported in Table 4. From the
table, we can find that the MUL fusion, regardless of the activation function, consistently underperforms in comparison to its SUM counterparts. Apparently, SUM fusion strategies are more stable and robust, and more suitable for the ASBA task. The superior performance of SUM fusion strategies, particularly with the tanh activation, underscores the effectiveness of the additive strategy in capturing the nuanced interplay of features pertinent to the ABSA task.

heading: 5.4 Case Study
text: To demonstrate the efficacy of the proposed method, we present a case study featuring a sample and its three adversarial variants in Table 5. We compare our proposed method based on RoBERTa with the baseline RoBERTa†.
From the table, the results clearly demonstrate that our method DINER, exhibits enhanced robustness compared to the baseline approach. Specifically, ORIGINAL and REVTGT types, where either no changes or direct negative changes were made to the targeted aspect, both methods perform equally well.
However, the distinction in performance is evident in more complex adversarial examples. In the REVNON type, where distractors are introduced in non-target aspects (e.g., service and atmosphere), the baseline fails to maintain its accuracy, misclassifying the overall sentiment as Negative. In contrast, DINER successfully recognizes the sentiment as Positive, reflecting its ability to isolate the influence of perturbations to non-target aspects. The ADDDIFF type further complicates the scenario by adding multiple negative aspects unrelated to the target. Despite these challenges, DINER continues to accurately assess the sentiment towards the food as Positive, whereas the baseline erroneously shifts to a Negative prediction.
The resilience of our method to adversarial conditions suggests it is well-suited for real-world en-
vironments where reliable sentiment analysis is crucial.

heading: 6 Conclusion
text: In this paper, to debias the target and review in ABSA simultaneously, a novel debiasing framework, DINER, is proposed with multi-variable causal inference. Specifically, the aspect is assumed to have a direct correlation with the label, so a counterfactual reasoning-based intervention is employed to debias the aspect branch. In the meantime, the sentiment words towards the target in the review are assumed to be indirectly confounded with the context, where a backdoor adjustmentbased intervention is employed to debias the review branch. Extensive experiments show the effectiveness of the proposed method in debiasing ABSA compared to normal state-of-the-art ABSA methods and debiasing methods.
Limitations
Though achieving promising results in the experiments, our work still has the following limitations.
• Though the proposed method is based on multivariable causal inference, the causal effects of the target aspect and the review are assumed to be independent, which means no interaction between the target and the review is modeled or considered.
• The proposed method is only evaluated on two robustness testing datasets for ABSA. More realworld datasets and more data transformation methods should be evaluated for future work.
• The general ABSA task includes the joint extraction of aspect and sentiment polarity, while the proposed method restricts the task to a given aspect. Future work should be considered for more generalized ABSA tasks.

heading: Acknowledgement
text: The authors would like to thank the anonymous reviewers for their insightful comments. This work is funded by the National Natural Science Foundation of China (62176053). This work is supported by the Big Data Computing Center of Southeast University.

heading: A Dataset Example
text: ARTS datasets employ three distinct strategies to rigorously test the model’s robustness: REVTGT is to generate sentences that reverse the original sentiment of the target aspect. REVNON is to change the target sentiment. ADDDIFF investigate if adding more nontarget aspects can confuse the model. We provide concrete instances of how each strategy is applied to manipulate aspect sentiment within the dataset in Table 7. Detailed statistics of the test sets are provided in Table 6.

heading: B Baselines
text: TD-LSTM: Tang et al. (2016a) use dual LSTMs to encode context around a target aspect, combining final states for sentiment classification. AttLSTM: Wang et al. (2016) introduce an Attention-based LSTM that merges aspect and word embeddings for each token. GatedCNN: Xue and Li (2018) utilize a Gated CNN with a Tanh-ReLU mechanism, integrating aspect embeddings with CNN-encoded text. MemNet: Tang et al. (2016b) employ memory networks, using sentences as external memory to compute attention based on the target aspect. GCN: Zhang et al. (2019) apply a GCN to the sentence’s syntax tree, followed by an aspect-specific masking layer. BERT: Xu et al. (2019) use a BERT-based (Devlin et al., 2019) baseline and takes as input the concatenation of the aspect and the review. BERT-Sent: (Xu et al., 2019) BERT-Sent takes as input reviews without aspect. BERT-PT: Xu et al. (2019) enhance BERT’s capabilities through post-training on additional review datasets. CapsBERT: Jiang et al. (2019) use BERT to encode sentences and aspect terms, then utilize Capsule Networks for polarity prediction.

heading: C ChatGPT Prompt
text: We conduct 3-shot prompting experiments on the ARTS datasets following (Wang et al., 2023). We set the decoding temperature as 0 to increase Chat-
GPT’s determinism. The prompts are presented in Table 8.

heading: D Model Hyper Parameters
text: The model parameters are optimized by AdamW (Loshchilov and Hutter, 2018), with a learning rate of 5e-5 and weight decay of 0.01. The batch size is 256, and a dropout probability of 0.1 is used. The number of training epochs is 20. We explore the hyperparameters α and β, setting their values to {0.6, 0.8, 1, 1.2, 1.4} for each, respectively. The optimal values for α and β are 0.8 and 1.0, respectively. We set K in set {3,6,9} in accordance with the theoretical principles discussed in (Geva et al., 2021). Our implementation leverages the PyTorch5 framework and HuggingFace Transformers6 library (Wolf et al., 2020). Our experiments are carried out with an NVIDIA A100 80GB GPU.
5https://github.com/pytorch/pytorch 6https://github.com/huggingface/transformers

heading: Future Work
text: More realworld datasets and more data transformation methods should be evaluated for future work. • The general ABSA task includes the joint extraction of aspect and sentiment polarity, while the proposed method restricts the task to a given aspect. Future work should be considered for more generalized ABSA tasks