Causal Intervention Improves Implicit Sentiment Analysis (2022)

Abstract: Despite having achieved great success for sentiment analysis, existing neural models struggle with implicit sentiment analysis. It is because they may latch onto spurious correlations (“shortcuts”, e.g., focusing only on explicit sentiment words), resulting in undermining the effectiveness and robustness of the learned model. In this work, we propose a CausaL intervention model for implicit sEntiment ANalysis using instrumental variable (CLEAN). We first review sentiment analysis from a causal perspective and analyze the confounders existing in this task. Then, we introduce instrumental variable to eliminate the confounding causal effects, thus extracting the pure causal effect between sentence and sentiment. We compare the proposed CLEAN with several strong baselines on both the general implicit sentiment analysis and aspect-based implicit sentiment analysis tasks. The results indicate the great advantages of our model and the efficacy of implicit sentiment reasoning.

## introduction the remarkable success that the field of sentiment analysis has achieved in the past few years has been derived from the use of increasingly high-capacity neural models to extract correlations from data @xcite @xcite . although having reached state-of-the-art results, correlational predictive models can be untrustworthy @xcite : they may latch onto spurious correlations ("shortcuts"), leading to poor generalization. one shortcut might be the explicit sentiment word which is a powerful feature cue. unfortunately, such a shortcut severely harms the generalization and the robustness of the learned models in implicit sentiment analysis (isa), where there are no explicit sentiment words about the topic * corresponding authors. or aspect @xcite . figure 1 gives a sample of aspect-based sentiment analysis (absa) @xcite , which aims to predict the sentiments of the aspects in the sentence. the aspect "food" has explicit sentiment words "definitely good", but aspect "price" does not. if the model thoughtlessly relies on shortcuts to sentiment words, it may make an incorrect sentiment prediction about the aspect "price". in fact, there are many other kinds of shortcuts that models may learn, for example, the rhetorical question mood expressed by the users @xcite and the co-occurrence of neutral words and sentiment polarities @xcite . the above shortcomings can potentially be addressed by the causal perspective: knowledge of causal relationships between observations and labels can be used to formalize spurious correlations and alleviate the predictor's dependence on them @xcite @xcite . motivated by a causal perspective, we incorporate domain knowledge of the causal structure of the data into the learning objective. specifically, causal intervention is used to curb dependence on shortcuts (e.g., "@xmath0. in this paper, we rethink the isa task from a causal perspective and unitize the casual intervention on deep learning. we argue that the causal effect obtained by reasoning directly from the sen-tence (x) to the sentiment (y ) without relying on other extra prior stereotypical lexical impressions is closer to the original semantic analysis. our work aims at eliminating the confounding causal effects of @xmath1. inspired by the instrumental variable in causality, we propose a causal intervention model for implicit sentiment analysis using instrumental variable (clean). different from the other work with causal intervention like back-door adjustment @xcite , other variables like confounders are not required to be observed. clean contains two-stage learning: (1) in the first stage, we model the relationship between the instrumental variable and sentence; (2) in the second stage, we dismiss the spurious correlation between confounders and sentiment by means of the relationship obtained from the first stage. to evaluate the effectiveness of our clean, we conduct a series of experiments on both the general implicit sentiment analysis and aspect-based implicit sentiment analysis. in particular, we compare our model with several the mainstream baselines and the results show the great advantages of our model on isa. we also validate the robustness of the model by adversarial attack and case studies, which proves that our model successfully dismisses the spurious correlation caused by sentiment words and extracts the pure causal effect. the main contributions are summarized as follows. variable x and y is called treatment variable and outcome variable respectively when observing the causal relationship between them. the other variables we do not consider their causal relationship are called error terms (ε), also known as exogenous variables. significantly, total effect between x and y , denoted as p (y | x), is conceptually different from causal effect of @xmath2, denoted as p (y | do(@xmath3, while the total effect involves all paths between x and y . based on the ideal hypothesis that none of the error terms will involve in the path between x and y , people usually treat the total effect and the causal effect as the same. but the actual fact is that a part of error terms (we call it confounder (c)) serves as a common cause of the treatment and outcome, denoted as x ← @xmath4. consequently, the total effect is practically different from the causal effect, i.e. p (y | do(@xmath5; hernán ma, 2020). ## conclusion in this paper, we proposed a causal intervention model for implicit sentiment analysis using instrument variable (clean). given that the current model indiscriminately focuses on the correlation between sentiment and sentiment words and consequently performs poorly in implicit sentiment analysis as the explicit sentiment words disappear, we rethink the implicit sentiment analysis from a causal perspective and analyze the four main forms of sentiment words as potential confounders. inspired by the instrumental variable of causal intervention, we adopt stochastic perturbation as instrumental variable and construct a model with two-stage learning. across three different datasets, including general implicit sentiment analysis and aspect-based sentiment analysis, our clean shows great advantages in implicit sentiment.