Interventional Rationalization
Authors: Linan Yue
Qi Liu
Li Wang
Yanqing An
Yichao Du
Zhenya Huang
Venue: EMNLP 2023 Main (2023)
Keywords: Rationalization
Causal intervention
TL;DR: We propose a  causal intervention method to remove spurious correlations in selective rationalization.

Abstract: Selective rationalizations improve the explainability of neural networks by selecting a subsequence of the input (i.e., rationales) to explain the prediction results. Although existing methods have achieved promising results, they still suffer from adopting the spurious correlations in data (aka., shortcuts) to compose rationales and make predictions. Inspired by the causal theory, in this paper, we develop an interventional rationalization (Inter-RAT) to discover the causal rationales. Specifically, we first analyse the causalities among the input, rationales and results with a structural causal model. Then, we discover spurious correlations between the input and rationales, and between rationales and results, respectively, by identifying the confounder in the causalities. Next, based on the backdoor adjustment, we propose a causal intervention method to remove the spurious correlations between input and rationales. Further, we discuss reasons why spurious correlations between the selected rationales and results exist by analysing the limitations of the sparsity constraint in the rationalization, and employ the causal intervention method to remove these correlations. Extensive experimental results on three real-world datasets clearly validate the effectiveness of our proposed method. The source code of Inter-RAT is available at https://github.com/yuelinan/Codes-of-Inter-RAT.

Interventional Rationalization
Linan Yue 22 Qi Li Wang 1.2.3 Yanqing An!.2, Yichao Du Zhenya Huang!?
1: Anhui Province Key Laboratory of Big Data Analysis and Application,
University of Science and Technology of China
2: State Key Laboratory of Cognitive Intelligence
3: ByteDance
(Inyue, w1063, anyq, duy chao emai) @mail. ustc. edu. cn; {qiliuql, Heanghyy edu. cn
Abstract
PIY
shil rurus de difalle
Selective rationalizations improve the explain- ability of neural networks by selecting sub- sequence of the input (i.e., rationales) to ex- plain the prediction results. Although exist- ing methods have achieved promising results, they still suffer from adopting the spurious correlations in data (aka., shortcuts) to com- pose rationales and make predictions. Inspired by the causal theory. in this paper, we de- velop interventional rationalization (Inter- RAT) to discover the causal rationales. Specif- ically, we first analyse the causalities among the imput, rationales and results with a causal graph. Then, we discover spurious correla- tions between the input and rationales, and between rationales and results, respectively, by identifying the confounder in the causali- ties. Next, based the backdoor adjustment, we propose causal intervention method to remove the spurious correlations between in- put and rationales. Further, we discuss rea- sons why spurious correlations between the se- lected rationales and results exist by analysing the limitations of the sparsity constraint in the rationalization, and employ the causal inter- vention method to remove these correlations. Extensive experimental results on three real- world datasets clearly validate the effective- ness of our proposed method. The source code of RAT is available at /ettss//githh: com/yuel
Ers caaiiy Do skii i 88
Wa.tnd makla
witeel vctiiel
sacadiaee do dedu be did ahall bain wanto lecaial
bolded)
the
Figure 1: Conventional framework of rationalization presented in this paper. In the charge prediction, the in- put represents the case fact and the result Y denotes the charge.
subset of the input (i.e., rationale) to support and explain the prediction results when yielding them. Existing methods often generate rationales with a conventional framework consisting of selector (aka., rationale generator) and a predictor (Lei et al., 2016). As shown in Figure 1, giving the input X, the selector and the predictor genérate rationales R and prediction results Y cooperatively (i.e., P(Y]X) = P(Y(R) the selector (P(R)X) first extracts subsequence of input R. Then, the predictor (P(Y (P) yields results based only on the selected tokens, and the selected subsequence is defined as the rationale.
Among them,
Despite the appeal of the rationalization meth- ods, the current implementation is prone to exploit spurious correlations (aka., shortcuts) between the input and labels to yield the prediction results and select the rationales (Chang al., 2020; Wu et al., 2022). We illustrate this problem with an exam- ple of the charge prediction'. Considering Fig- ure 1, although this case is corresponding to the Manslaughter, DNNs model readily predicts the charge as Intentional homicide. Specifically, as Intentional homicide occurs more frequently than Manslaughter and is often accompanied by tokens denoting violence and death, DNNs do not need to learn the real correlations between the case facts and the charge to yield the result. Instead, is much casier to exploit spurious correlations in data
1 Introduction
The remarkable success of deep neural networks (DNNs) in natural language processing tasks has prompted the interest in how to explain the results of DNNs. Among them, the seleetive rationaliza- tion task (Lei et al., 2016; Yu et al.. 2019, 2021) has received increasing attention, answering the question "What feature has a significant impact on the prediction results of the model?". Specifi- cally, the goal of rationalization to extract a small
'Charge prediction: predicting the charge such as Robbery and Theft based on the case fact. Detailed definition of charge prediction is described in section 4.3. Phttps: htttt::/wweesseee /ww////nuuuccmmm///// court. gov..c
Corresponding Author

---

achieve high accuracy (i.e., predicting the charge Intentional homicide directly when identifying the tokens about violence and death.). As a re- sult, when facing the cases such as the example in Figure I, the effectiveness of such DNNs tends to degrade (c.g., the underlined tokens in Figure denoting the offence is negligent will be ignored in rationales extraction and the charge will be mis- judged.). Therefore, these type DNNs depending on spurious correlation data fail to reveal truly critical subsequence for predicting labels.
further analyse this discovery and employ the causal intervention to remove these correlations. Our experiments are conducted on three real-world datasets, and the experimental results validate the effectiveness removing spurious correlation with causal interventions.
2 Framework of Rationalization
This section formally defines the problem of tionalization, and then presents the details about the conventional rationalization framework consist- ing of the selector and predictor, where these two components are trained cooperatively to generate rationales and yield the prediction results.
To solve that, (Chang et al., 2020) propose an auvironment-invarian method (INVRAT) to dis- cover the causal rationales. They argue that the causal rationales should remain stable as the envi- ronment shifts, while the spurious correlation be- tween input and labels vary. Although this method performs well in selecting rationales, since the en- vironment in rationalization is hard observe and obtain, we argue that this "causal pattern" can be further explored improve the rationalization.
2.1 Problem Formulation
Considering a text classification task, only the text input X = {x1, x2, ..... ......,......................... .nk] where represents the i-th token, and the discrete ground truth Y are observed during training, while the rationale R is unavailable. The goal of selective rationalization is first adopting the selector to learn binary mask variable M {m,, m2,, mn), where my E {0, 1}. and further seleet a subsequence of input R M @ X [mm m ......, m, Iz }. and then employing the predictor to re-recode the mask input R to yield the results. Finally, the whole process of rationalization is defined as:
Along this research line. in this paper, we pro- pose an interventional rationalization (Inter-RAT) method which removes the spurious correlation by the causal intervention (Glymour et al., 2016). Specifically, motivated by the causal inference the- ory, we first formulate the causal relationships among X, R and Y in causal graph (Pearl et al., 2000; Glymour et al., 2016) as shown in Fig- ure 2(a). Then, we identify the confounder C in this causal graph, which opens two backdoor paths <- C >> R and <- -+ making X and R, R and spuriously correlated. Next, we ad- dress the above correlations, respectively. For spu- rious correlations between X and R, we assume the confounder is observed and intervene the X (i.e., calculating P()))o(X)) instead of P(R X)) block the backdoor path and remove the spu- rious correlations based on the backdoor adjust- ment (Glymour et al., 2016). Among them, the do-operation denotes the pursuit of real causality from X to R. For spurious correlations in R and Y, since by the definition of R (rationales are the only basis for yields prediction results), we argue that there should be no spurious correlations between R and Y. However, in practice, we discover the sparsity constraint commonly defined in rational- ization (Lei et al., 2016; Cao al., 2020; Chang et al., 2020; Yu et al., 2019), ensuring the selec- tor to extract short rationales, results the spu- rious correlations between R and Y. Therefore,
P(Y[X) = P(YRR)P(RIX)
(1)
predicttrr selector
2.2 Selector
The selector divides the process of generating tionales into three steps. First, the selector sam- ples each binary value mj from the probability distribution P( = {PI, . }r where Pj represents the probability of selecting each æ; as the part of the rationale. Specifically, Pj is cal- culated p; P(mylæyy) softmax(WaSe(r))) where the encoder fe(-) encodes the token æi into d-dimensional vector and W. R2xd Then, to ensure the sampling operation differentiable, we adopt the Gumbel-soffmax method (Jang et al., 2017) to achieve this goal:
exp (log (p;) + (1)/TT))))))
mj Σ. exp ((og (p:) + +9t++ /T)
(2)
where 7 is temperature hyperparameter, 9j log (- (u;)) and uj random sampled from the uniform distribution (O, 1). Finally, the ra- tionale can selected as R = M X

---

{m -1, m22 ..,..., mm In.. Therefore, conclude that the probability of generating ra- tionales X) is calculated as: P(R[X) = P(M@ O x
Intervootise the vilette.
Cinsel the prodimes.
onfinaabitionn
the selecte.
Figure 2: (a) Causal graph for rationalization. (b) Causal graph for the selector, which offers fine- grained causal relationship between R and C with mediator K. (c) Intervention on the selector, where off and the confounder C' is stratified
2.3 Predictor
Based on selected rationale tokens R. the predic- for outputs the prediction results (i.e., calculating R(Y P(Y M o )),, and then R can be seen an explanation of Y. Specifically, after obtaining R from the selector, we adopt the neural network fp(-) to re-encode the rationale into d-dimensional continuous hidden states yield results. The ob- jective of the predictor is defined as:
X - into pieces C' - {c1, (o,,,,,.... GC)) (d) Causal graph for the predictor, which describes fine- fine- gra-gra causal relationship between Y and C with mediator H.
the endogenous variables we are interested in, fol- lowing the settings in (Fan et al., 2022). In the fol- lowing, introduce the causal graph with these variables a high-level:
Ltask E X.Y-Dir [e W ()), M- M-P(M
(3)
C - X. The confounder C' in rationalization can be seen the context prior, determining which tokens can be "put" into the text input X. Among them, in practice, the context prior consists of unob- served prior and observed prior (e.g., we consider the entire label set as the partially observed con- founder in the text classification. See section 3.2 for details.) We take the Figure 1 as an example to further illustrate C + X. Specifically, both observed Intentional homicide and Manslaughter priors decide where the tokens denoting violence and death appear the Manslaughter prior deter- mines where the tokens representing manslaughter appear; the unobserved prior decides where other tokens that are meaningless appear.
where Drr denotes the training set, ((-) represents the cross-entropy loss function, W Nxd is the trained parameter and N is the number of labels (e.g., N 2 in the binary classification).
2.4 Sparsity and Continuity Constraints
Since an ideal rationale should be short and co- herent part of original imputs, add the sparsity and continuity constraints (Lei et al., 2016):
n a
Cr = A
Σ
mj
A2
mj
mj m
2=
(4)
where the first term encourages the model to se- lect short rationales and a is a predefined sparsity level at the scale of [0, 1], and the second term en- sures the coherence of selected tokens. Finally, the overall objective of the rationalization is defined as: L Ltask + Cre-
X + R +- C. Besides the selector extracts subsequence of X as the rationale R, making X R holds, R is also affected by the context prior C. Figure 2(b) offers fine-grained causal relationship between and with a mediator K. Specifically, in X + K + C, denotes the context-specific representation which is weighted representation of the prior knowledge associated with X in C. In X + R K, R. is affected by the context prior C' through K indirectly. For ex- ample, in Figure 1, the underlined tokens denoting negligent are ignored in rationalization, since the context prior in C misleads the model focus on the violence and death feature in by the mediator K. Detailed examples about explanations for this causal graph can be found in Appendix A.1.
3 Interventional Rationalization
In this section, first reveal how the confounder C causes spurious correlations rationalization with causal graph. Then, we remove these corre- lations by using a causal intervention method.
3.1 Causal Graph in Rationalization
As shown in Figure 2(a), formulate the causali- ties among the text input X. rationale R. ground- truth label Y and the confounder with causal graph (Pearl et al., 2000; Glymour al., 2016), where the link between two variables represents a causal relationship. In this paper, we only show
C C Y R. As the predictor yields the Y holds. Mean-
result based on the rationale, while, in the ideal situation, since the rationale is defined as a subsequence of X sufficient to pre-

---

dict the Y. there should be no direct causal rela- tionship between C and Y. However, in practice, rationales R are commonly extracted with shortcut tokens (we will introduce it later in section 3.3), making C - Y exists. Figure 2(d) describes a fine- tine-grained causal graph between Y and C' with a mediator H. where H is the context-specific repre- sentation of Rt. by using the context prior C. More detailed descriptions of this causal graph are pre- sented in Appendix A.2.
( Therefore, we can achieve P(R)d((X) = P(M]do(X)) Specifically, to
calculate the probability of each token 2, selected as the rationale, the implementation is defined as:
IN
P(myldo(X)
[P k))) Pca]
=
[N| = Σ (f(n)axx)((
From the graph, we find that X and R, and Y are confounded by the context prior C' with two backdoor paths X + C + R (or X + C +
(6)
Among them, f.(.) is the function achieving X + R + K, Ki E K is defined as the content-specific representation by using the con- text prior C₁ we express it k ()) Acc where X X. Besides, X R is the set of the normalized similarity between Tj and each e₁ in the confounder set C' (i.e., (fot) )))). Furthermore, since Eq (6) requires sampling of and this sampling is ex- pensive, we try to find an alternative function that would be easy to compute to approximate it. Empir- ically, based on the results in (Xu al., 2015; Wang et al., 2020; Yue et al., 2020), NWGM approximation (Xu al., 2015) move the outer sum into the softmax: P(m)ldo(X)) softmax ! fs (IT,, P In this paper, we adopt the linear model fa(zy, k:) (x))( + W2k; (a))( + W2Xc0; to fuse the information of the input X and the confounder C.. Then, the final implementation of the intervention is formulated as:
K + R for elaboration) and R + + C (or R + C H - Y). The above backdoor paths result in spurious correlations among the text input X, rationale R, and label Y. Based on this, propose a causal intervention method to remove the confounding effect by cutting off the link -+ X and C + R. respectively.
3.2 Backdoor Adjustment
To pursue the real causality from X to R (or R to Y), we adopt the causal intervention do) instead of X)) (or P(Y]do(R)) instead of R)) to remove the effects of confounder C. Next, we introduce the causal intervention method by taking P(R) ((R))(()) as an example, and P(Y (R)) is similar. Specifically, since adopt- ing the randomized controlled trial to intervene X is impossible, which requires the control over causal features, we apply the backdoor adjustment (Glymour et al., 2016) to achieve P(R[do(X)) by cutting off C + X (Figure 2(c)):
adopt the
P(m))do(X)) softmax (a))()))
IC|
NI
(7)
P(R[do(X)) Σ (X,
+ W2 XcP(c))
(5) where the confounder C is stratified into pieces C = =c C2, c2........ (c))) P (c:) denotes the prior distribution of c, which is calculated before train- ing, and (-(-) is a function achieving X + K +- C. However, the confounder C' is commonly hard to observe. Fortunately, based on the existing researches (Wang et al., 2020; D' Amour, 2019), we can consider the entire label set as the par- tially observed children of the unobserved con- founder. Therefore, we approximate it by de- signing a dictionary D. {c1, (Nl} as an N x d matrix, where N represents the num- ber of labels and d is the hidden feature dimen- sion. As described in section 2.2, we conclude
3.3 Limitations on the Predefined Sparsity
In this section, we discuss why C Y in Figure 2(a) holds in detail. Since rationales are defined as the subsequence of inputs, being sufficient to yield results, C ->> Y should not exist. However, unfortunately, in practical implementation, the spar- sity constraint (denoted by a-constraint) in the first term of Eq (4) may result in spurious correlations between the extracted rationale and the predicted result. Specifically, the a-constraint encourages the selector to extract of tokens from the orig- inal text input. When the predefined number of extracted tokens is greater than the length of the practical rationale, a few tokens corresponding to

---

shortcuts of Y may still be selected. For exam- ple, as a converges
on the original BeerAdvocate. Besides, following the setup of (Chang et al., 2020), we consider the beer review prediction as binary classification where the ratings Σ 0.4 as negative and 2 0.6 as positive. Then, the processed Beer is a non-balanced dataset. For example, the label dis- tribution in the appearance is posilive:negative 20:1. For testing, we take manually annotated ra- tionales as our test set, detailed statistics are shown in Appendix C.I.
all tokens in the input will be extracted, including the rationales tokens and shortcuts tokens (more examples are shown in Appendix B. Then, the shortcuts tokens will hurt the prediction performance. To alleviate this situation, we first construct a fine-grained causal graph (Figure 2(d)) between the selected rationale R and the prediction results Y. Among them, R represents the rationale generated by a-constraint, H denotes the context-specific representation of R based on the context prior C. As mentioned be- fore, from the graph, we find that as there exists a backdoor path R + C + H + Y. R and Y are confounded. Then, based on the above obser- vation, the predictor adopts the causal intervention methods deseribed in section 3.2 (i.e., calculating P(Y]do(R)) & softmax ! S. P (c)) to remove the spurious correlations and further yield prediction results, where f-(-) is the func- tion to obtain R + Y + H, and h; 9r ci) represents the process of R H + C. Detailed descriptions of the graph at a high-level and the derivation are shown in Appendix A.2.
4.1.1 Experimental Setup
In this section, we first present the comparison methods with Inter-RAT including RNP (Lei et al., 2016), HardKuma (Bastings et al., 2019), A2R (Yu et al., 2021), INVRAT (Chang et al., 2020), IB (Paranjape et al., 2020) and DARE (Yue et al.. 2022). Among them, RNP is an original ratio- nalization method which generates rationales by yielding the Bernoulli distribution of each token and sampling from it. HardKuma, A2R, IB and DARE all achieve promising results in rationaliza- tion. INVRAT is our main baseline to directly com- pare with, which is also a method of removing the spurious correlation in data. The difference with our -nter-RAT is that INVRAT learns environment invariant representations by obtaining multiple en- vironments from the training set.
4 Experiments
In this section, validate the effectiveness of our method on three real-world tasks including the beer reviews sentiment analysis, movies reviews prediction and the legal judgment prediction.
For training, we use the pre-trained glove em- beddings (Pennington et al., 2014) with size 100, and implement the encoder in both fe(-) and fp(-) as the bidirectional GRU (Cho et al., 2014) with hidden size 100. We optimize the objective of ra- tionalization using Adam (Kingma and Ba, 2014) with mini-batch size of 256 and an initial learning rate of 0.001. Besides, we consider the in Eq (4) as (0.1, 0.2, 0.3), respectively. For testing, we report the token precision, recall and Fl-score evaluate the quality of selected rationales. Among them, token precision is defined as the percentage of how many the selected tokens are in annotated rationales, and token recall is the percentage of an- notated rationale tokens that are selected by model. The token Fl-score is calculated precision i recall
4.1 Beer Reviews Sentiment Analysis
Beer Reviews Sentiment Analysis aims to predict the ratings (at the scale of [o, I]) for the multiple aspects of beer reviews (e.g., appearance, aroma and palate). We use the BeerAdvocate (McAuley et al., 2012) our dataset, which is commonly used in the rationalization. As there is high senti- ment correlation in different aspects in the same beer review (Lei et al., 2016), which may confuse the model training. several researches (Lei et al., 2016; Bastings et al., 2019) adopt the de-correlated sub-datasets (i.e., part of BeerAdvocate) in the training stage. However, a high correlated dataset is more conducive to validating our RAT which is designed to remove the spurious correlations in data. Although (Chang al., 2020) also conduct correlated sub-dataset, the data split and processing are not available. For fair comparison, different from the previous study which makes experiments on the sub-dataa dataset, we train and validate models
2*precision-reaall
4.1.2 Experimental Results
To demonstrate the effectiveness of our Inter-RAT. briefly compare with baselines (e.g. RNP and DARE) in Table 1, where Inter-RAT outper- forms them consistently in finding correct ratio- nales. Specifically, Inter-RAT surpasses baselines on all three aspects (i.e, appearance, aroma and

---

Table 1: Precision, Recall and F1 of selected rationales for the three aspect, where a is the predefined sparsity level.
Methods
Appearance Recall
Aroma Recall
Palate
Precision
F1
Precision
F1
Precision
Recall
F1
RNP
32.4
18.610.3
23.6=0.4
44.81:0.
12.4=0.7
37.64:0.5
2..6-10.5
23.5
24.0::0.5
HandKamma
0.1
53.6+0.11
28.740.11
37.4+0.1
29.34114
25.9+3888
27.34-2.1
7,7+-0.1
6.000.1
6.8-0.1
INVRAT
0.1
42.6=-0.7
31.5+006
36.2=006
4122+03
19.1
40.1+:1.6
14,9+
45.6=002
19.5++100
IB
50,520.2
29.7=0..5
37.4=04
43.54:0.
39.2=0.5
41.340.1
29.9:00.2
34.210.7
31.9-0.2
DARE
63.9=0.1
42.8=0.2
51.3=0.1
50.5+0.1
44.8=0.2
47.52:022
33.1=0.4
45.8=0.1
38.4=0.2
Inter- RAT
0.1
6..0=0.4
46.5:100.8
5..6100.7
55.410.9
07.5
51.::0.8
34.6:10.8
4..2:0.4
40.2:10.5
RNP
39.4=0.4
4499:0.1
42.0=0.2
37.54:0.1
43.5+:..3
216600.4
38.9:0055
27.8:0.4
HandKarma
0.2
6.99+0.9
69.2+-1.0
6..0++08
37.0+1.3
$5.8 1.9
44.51155
14.6+0.3
22.34-0.8
17.740.4
INVRAT
0.2
58.9+-04
67.2422.3
62.8+111
29.3++100
52.1--0.6
37.4+0.66
1.3
55.2+-2.3
13.5++16
IB
59.3:00
69.0:10.2
63.8-0.2
38.61:011
55.5=0.7
45.64:..1
21 21620.
48.510
29.9-1.22
DARE
0.2
63.7=0.2
718+0088
67.5=0,5
41.0+0.2
61.5=0.2
49.340.3
24.420.1
54,92088
33.8=0.11
Inter- Inter-RAT RAT
0.2
02.0
76.7:11.77
68.6
44.21.0.1
0..4
52.810.1
26.3:10.6
59.1
0.77
RNP
24.2=0.4
41,240.8
30.5=0.5
27.140.3
55.7=088
36410.4
15440.4
42.210099
22.6:10.5
HandKurma
0.3
42.140.3
824+1.4
55.740.5
24.6+-.11
57.740..6
34.1002
21.740.1
49.74-0.4
30.240.1
INVRAT
0.3
41,5+-04
74.8+0.3
53.4-0.3
2288116
65.1+1.7
318+18
20.9+1.11
71,6+0044
32.3+
IB
40.2+0.1
81.520.2
03
27.902
59.2-0.3
37.91.11
19.140.7
59.0:0.9
28.9-0.6
DARE
80.6-10.2
58.1
$2.7402
08.2-0.3
44220011
19.740.6
70.5=004
30.8=0.7
Inter- Inter-RAT
0.3
0..1
02.7-10.5
6..8:0.4
37.910.7
0..0=0.1
49.610.77
218810.1
66.1
0..8:1011
palate) by a large margin in most metries. Besides, although INVRAT has shown helpful in discover- ing the de-confounded rationales, Inter-RAT still performs better than it, improving 10.5, 14.0 and 1.4 on the average token Fi-score across three as- pects, and Inter-RAT has a lower variance illus- trating method is more stable than INVRAT. Such observations strongly demonstrate that Inter- RAT can remove the spurious correlation in data to select rationales effectively.
Go Appinm d.lleoee pario
Figure 3: (a): Fi-score the Palate with increasing O. (b): The token Fl for rationales on the appearance aspect with different prior distributions.
ments on the non bon-balanced dataset, which is dif- ferent from the previous study (Chang et al., 2020, 2019; Huang et al., 2021) adopting the balanced datasets. Therefore, there exists a research ques- tion we need to answer: "Does the information of label distributions (or prior distributions) somehow influence Inter-RAT to yield better rationales in- stead of the causal intervention ?", For instance, as the label distribution in the appearance aspect is pesitivenegative = 20:1, we conduct experiments on the appearance datasct with P (ct) = 21 and P (c2) = - where P (c:) in Eq (5) represents the prior distribution of c,, C1 is the positive label and C2 is the negative one. The above non-balanced label distribution might be inducing the model to "better rationalize" for the majority class (i.e., the positive), further reflecting the improvement of Inter-RAT over the whole dataset. Therefore, we compute token-Fl scores for positive and negative examples separately for a safer evaluation, where we denote the evaluation of Inter-RAT on positive examples as Inter-RAT(+) and on the negative ones as Inter-R RAT(-). Figure 3(b) summarizes the re- sults the appearance aspect. From the result, find that the performance of extracting positive rationales better than extracting the negative, al- though the difference between the two types results is not significant, and the scores for the negative are still high (better than INVRAT). Therefore, to
As discussed in section 3.3, we propose the causal intervention method alleviate the prob- lem, where several tokens corresponding to spuri- ous correlations in data may be selected and further mislead the prediction with an increasing a. Here, we conduct an experiment to validate the effective- ness of the causal intervention. Since there is only about 1,000 beer reviews in the test set, we report the binary classification Fl-score" with different in the dev set which contains about 30,000 reviews. As shown in Figure 3(a), we make experiments on the palate aspect, and Inter-once is a variant of Inter-RAT, which yields the rationales based on P(R) (X)) but predicts the results based on From the obser-
20
R)Y rather than P(Y|do vation, we can conclude that when a is small (i.e., the length of selected rationales is smaller than real rationales), the difference between Inter-RAT and Inter-once is minor. However, as increases, Inter- RAT steadily improves, while the Inter-once grows slowly and even degrades. The above observation illustrates that our causal intervention method can alleviate the spurious correlations problem between R and Y caused by the a-constraint.
As mentioned in section 4.1, we make experi-
'Different from token F1, FI-score is commonly adopted to evaluate the performance of binary classification.

---

Table 2: Results on Movie, where several results of
the Legal Judgment Prediction (LJP) task, which yields the judgment results such as the charges based on the case fact. We conduct experiments on the publicly LJP datasets CAIL (Xiao et al., 2018) which contains criminal cases consisting of the fact description and corresponding charges, law articles, and terms of penalty results. For data processing, referring to (Yue et al., 2021), we remove several in- frequent and multiple charges cases, and divide the terms into non-overlapping intervals. The detailed statistics of the datasets can be found in (Yue al., 2021). Figure 1 shows an example of LJP, which predicts the charge according to the case fact.
baselines quoted from A2R (Yu et al., 2021).
Methods
Movie
Precision
Recall
Fl
RNP HartiKuma
26.1
32.6
31.1
28.3
27.0
A2R
48.7
31.9
34,9
INVRAT
33.9
24.3
28.3
Inter-RAT
35.7+0.2
35. 1.7
35.7+-0.8
Bet__Itee-RAT
31.7=0.1
43.1
36.52003
further validate the effect of label distribution, we add the analysis as follows: we re-run the exper- iments with P (c1) = P (c2) = 2 (i.e., assuming this is a balanced dataset with uniform label dis- tributions) and denote the corresponding model as Inter-RAT-balanee We report experimental results in Figure 3(b). From the figure, find adopt- ing the true prior distribution P (ca) (Inter-RAT) performs better than the assumed one (Inter-RAT- balance), demonstrating the prior distribution is critical for the backdoor adjustment. Besides, interesting to see that with a balanced label distribu- tion, the results of the minority label (i.e., negative) are worse than using the true label distribution, which suggests Inter-RAT is not simply "paying more attention" to instances of the majority class.
4.3.1 Experimental Setup
In addition to comparing RNP, HardKuma and IN- VRAT, we also compare our method with some classical baselines in the LJP task, including Top- Judge (Zhong et al., 2018), Few-Shot (Hu et al., 2018), LADAN (Xu et al., 2020) and NeurJudge (Yue al., 2021). All the above baselines are trained by exploiting legal particularities. where NeurJudge is the state-of-the-arl model in LJP. Meanwhile, it employs label embedding method to enhance the prediction. We conduct experiments on one of versions of CAIL containing 134,739 cases (Yue et al., 2021). For testing, there are annotated rationales, we first employ the accuracy (Acc), macro-precision (MP), macro-recall (MR), and macro-Fl (F1) evaluate the performance of yielding judgment results. Then, we provide a human evaluation for selected rationales LJP. Detailed description of comparison methods and experimental setups can be found in Appendix C.2.
4.2 Movies Reviews Prediction
Besides the beer reviews sentiment analysis task, we also make experiments on another binary classi- fication task (i.e., movie review prediction (Zaidan and Eisner, 2008)) in the ERASER benchmark (DeYoung et al., 2020), which contains token-level human annotations. We follow the same experi- mental setups in 4.1.1 and report the experimental results with = 0.2 in Table 2. Detailed descrip- tion of the dataset is shown in Appendix C.I. As shown in the table, AnT performs better than baselines on most metrics, which further validates the effectiveness of Inter-RAT. Furthermore, to val- idate Rnter-RAT agnostic to the structure of the selector and predictor, we adopt Bert (Devlin et al., 2019) to replace bi-GRU in f-)-) and fp(-) in both RNP and Inter-R.We RA,, and denote them as Bert RNP and Bert_Innerrre that -nter-RAT still outperforms Bert_RNP, illustrating the effectiveness of Inter-RAT
4.3.2 Experimental Results
To evaluate the performance of our model on LJP, we show the experimental results from two aspects. First, Table 3 shows that Inter-RAT still per- forms better than the rationalization methods when generalizing to the multi-classification task. Mean- while, compared with the LJP approaches (e.g. Top- Judge and NeurJudge), even though our model is trained on the three subtasks separately, while these LJP approaches explore the dependencies between tasks and are trained with a multi-task learning framework, model still achieves promising per- formance. However, Inter-RAT does not perform better than NeurJudge. A potential reason is that NeurJudge is designed only for LJP, exploiting the legal particularitics well. contract, our Inter- RAT is designed for general text classification tasks.
Inter-RAY RAT. From the result, we observe
4.3 Legal Judgment Prediction
Since there are only two categories (positive and negative) in both beer and movie reviews predic- tion, we further generalize our Inter-RAT to the multi-classificttion task. Specifically, we focus on

---

Table 3: LJP results on CAIL. Among them, the underline scores are the state-of-the-and performances LJP but lacking explainability, and the results in bold perform second only NeurJudge but with an explainable rationale. Results of LJP baselines are quoted from (Yue et al., 2021).
Charges
Law Articles
Terms of Penalty
Methods
Acc
MP
MR
F1
Acc
MP
MR
F1
Acc
MP
MR
Topludge
86.5
84.2
78.4
80.2
87.3
85.8
76.3
78.2
38.4
35.7
32.2
31.3
Fow-Shet
88.2
80.6
88.4
77.9
39.6
30.9
31.6
LADAN
88.3
86.4
80.5
82.1
88.8
85.2
79.5
81.0
38.1
34.0
31.2
30.2
NeuzJudge
89.9
86.8
90.4
85.8
41.7
37.2
37.3
RNP
83.110.2
82.2.10.3
78. 110.5
79.010.55
86.510.11
82.1.0.33
77.7.0.77
78.8.0.6
37.110.2
30.110.3
30.310.4
21.710..3
HardKuma
86.2+0.2
84.7+1.2
79.0+0.8
80.6+0.4
868-+088
77.3+008
78.9+0.9
3888+0.1
34,3+1.3
27.3+0.3
25.8+0.4
INVRAT
85,4+0,2
83. 3+0.+
78.0+
80.2+0.3
85.1+0.1
83.1+0.1
76,2++0.1
78.1+002
38.5+0.3
34,5+0,7
33.0+0.2
$2.0+0,5
Iner-RAT
89,4+0.2
87.5+0.3
85.3+0.3
85.9+0.3
89.5-0.1
86.3+0.1
83.5+003
84.6+0.3
39.6+0.11
36.40.4
34,3+0.3
32.4+0.3
Table 4: Human evaluation charge prediction.
From the results, we can find Inter-RAT outper- forms RNP, INVRAT, HardKuma and DARE in all metrics, further demonstrating
Methods
C
F
Avg.
RNP
3.85
3.28
3.34
3.49
causal interven- tion method can select more sufficient rationales for yielding results. Besides, considering the suc- cess of Large Language Models (L.LMs), we addi- tionally add ChatGPT (OpenAI, 2023) for human evaluation (the prompt for ChatGPT is shown in Appendix C.4). As shown in Table 4, we observe that ChatGPT achieves competitive results on both Completeness and Fluency, But it does not perform well on Usefulness. Because ChatGPT extract short tokens and sentences rationale, it still tends to extract longer sentences, which is beneficial for the Fluency metric. How- ever, for the Usefulness metric, some redundant tokens will be extracted as well. Therefore, Chat- GPT does not perform well on the Usefulness.
INVRAT
3.88
3.42
3.41
3.57
HardKuma
3.78
3.33
3.39
3.50
DARK
4.57
3.89
4.19
4.22
ChatGPT
4.01
4.08
4.34
4.14
Inter-RAT
4.52
4.10
4.25
4.29
Therefore, the performance of Inter-RAT does not surpass NeurJudge. Furthermore, different from the NeurJudge and other LJP baselines, our Inter- RAT can provide intuitive explanation (i.e., ra- tionales) when yielding the judgment results while LJP baselines fail produce them. The above ob- servation provides strong validation of adopting the causal intervention method to remove spurious cor- relation in data for predicting results. Interestingly, we find there exists a minor difference between RAT and NeurJudge on yielding the charge and law article. We argue potential reason is the be ap-
though ask
Related Work
label embedding method in NeurJudge proximated the causal intervention method. We further discuss in Appendix D.2 in detail.
Rationalization. Deep neural networks (DNNs) have achieved remarkable success in various do- mains (Seo et al., 2016; Devlin et al., 2019; Liu et al., 2023c; Gao et al., 2023). However, the predicted results are still unreliable. To improve the explainability of DNNs, the rationalization has attracted increasing attention (Lei et al., 2016; Treviso and Martins, 2020; Bastings et al., 2019; Chang et al., 2019; Liu et al., 2023a,b). Specifi- cally, (Lei et al., 2016) first proposed a rationaliza- tion framework which consists of a selector and a predictor. Following this framework, multiple variants were proposed to improve rationalization. Among them, to replace the Bernoulli sampling dis- tribution in (Lei et al., 2016), (Bastings et al., 2019) introduced HardKuma distribution for reparam- eterized gradient estimates. Additionally, another fundamental direction is adding external compo- nents to enhance the original framework. (Yu et al., 2019) employed an introspective selector which in-
Second, as CAIL does not provide annotated ra- tionales like BeerAdvocate, we make human eval- uation to evaluate the selected rationales. Specifi- cally, we sample 100 examples and ask three anno- tators who are both good computer science and law to evaluate rationales in the charge prediction. Besides, following (Sha et al., 2021), we employ three metrics with an interval from 1 (lowest) to (e.g. 2.0 and 3.2) to evaluate rationales, including usefulness (U), completeness (C), and fluency (F). Appendix C.3 describes detailed scoring standards for human annotators. The human evaluation re- sults are shown in Table 4. It is worth noting that a similar human evaluation is provided in DARE, but its is set differently from ours, where DARE sets a to 0. 14 and our Inter-RAT is set to 0.2. For consistency, we replicate DARE with a = 0.2.

---

corporated the prediction results into the selection process. Some researchers (Huang et al., 2021; 1: Sha et al., 2021: Cao et al., 2020) proposed an external guider to reduce the difference between the distribu- tions of rationales and input. However, few consid- ered spurious correlations in data which degraded the rationalization. Among them, (Chang et al., 2020) discovered the causal rationales with envi- romment invariant methods by creating different en- vironments. (Wu et al., 2022) extracted rationales from graphs to study the explainability of graph neural networks by intervention distributions.
labels are still difficult obtain. For example, if we take the beer review prediction as a text regres- sion task like (Bastings et al., 2019), rather than a classification task, then we will fail to obtain the observed label, where the ratings of beer reviews are continuous. Therefore, to achieve the causal intervention, we argue that promising approach is to adopt other intervention methods, such as the in- strumental variable (Yuan al., 2022; Singh al., 2019) that implemented with the unobserved con- founder. We will leave these for the future work. Besides, since our approach focuses on how to im- prove the explainability neural networks and is a model-agnostic approach, we will also study in the future how to employ rationalization methods to improve the Large Language Models (LLMs), considering the recent remarkable success of LLMs (OpenAI, 2023; Touvron al., 2023).
Causal Inference. Causal inference (Glymour et al., 2016) has been widely explored in various fields, including medicine (Richiardi et al., 2013) and politics (Keele, 2015), which aims to empower models the ability to achieve the causal effect. Re- cently, several researches (Deng and Zhang. 2021;
Dong et al., 2020; Yue et al., 2020; Niu al., 2021; Vosoughi et al., 2023) introduced causal inference into machine learning with causal intervention to re- move the spurious correlations in data. Especially, it has inspired several studies in natural language understanding such as Named Entity Recognition (Zhang et al., 2021), Topic modeling (Wu al., 2021), and Entity Bias problem (Zhu et al., 2022; Wang et al., In this paper. we focus on im- proving the rationalization with causal intervention.
A Acknowledgements
This research was supported by grant from the Na- tional Key Research and Development Program of China (Grant No. 202/
References
Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2019. Interpretable neural predictions with differentiable binary variables. In Proceedings of the 57th ACL.
6 Conclusion
Nicola De Cao, Michael Sejr Schlichtkrull, Wilker Aziz, and Ivan Titov. 2020. How do decisions emerge across layers in neural models? interpreta- tion with differentiable masking. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, (EMNLP), Online, November 16-20.
this paper, we proposed a causal intervention method (Inter-RAT) to improve rationalization. To be specific, we first formulated the causalities in rationalization with causal graph and revealed how the confounder hurt the performance of select- ing rationales with opened backdoor paths. Then, we introduced backdoor adjustment method to remove spurious correlations between inputs and rationalcs. Besides, we further discussed the po- tential bias between selected rationales and pre- dicted results caused by the sparsity constraints, and adopted the above causal intervention method to yield de-confounded prediction results. Experi- mental results real-world datasets have clearly demonstrated the effectiveness of Inter-RAT
Shiyu Chang. Yang Zhang, Mo Yu, and Tommi Jaakkola. 2019. A game theoretic approach class- wise selective rationalization. NeurIPS.
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi S. Jaakkola. 2020. Invariant rationalization. In Pro- ceedings of the 37th ICML.
Hanjie Chen Yangfeng Ji. 2020. Learning varia- tional word masks to improve the interpretability of neural text classifiers. In Proceedings of the 2020 Conference on EMNLP, pages 4236-4251.
Limitations
Kyunghyun Cho. Bart Van Merrienboer, Caglar Gul- cehre, Dzmitry Bahdanau. Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using encoder-decoder for statistical machine translation. In Proceedings of the 2014 4 Conference on EMNLP, pages 1724-1734.
To achieve the causal intervention, we adopt the backdoor adjustment to calculate P( P(R do(X)) and P(Y|do(R)) with the observed label as the con- founder set. However, in some cases, observable

---

Xiang Deng and Zhongfei Zhang. 2021. Comprehen- sive knowledge distillation with causal intervention. In Proceedings of the 30th NeurIPS.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of the 3rd ICLR.
Jacob Devlin, Ming-Wer King-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. Proceedings of the 2019 Conference of NAACL.
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on EMNLP.
Wei Liu, Haozhao Wang, Jun Wang. Ruixuan Li, Xinyang Li, Yuankai Zhang, and Yang Qiu. 2023a. Mgr.
Jay DoYoung, Sarthak Jain, Nazneen Fatema Rajani. Eric Lehman. Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020.
based rationalization. Pro- ceedings of the ACI.
Wei Liu, Jun Wang, Haozhao Wang, Ruixuan Li, Yang Qiu, YuanKai Zhang, Jie Han, and Yixiong Zou, 2023b. Decoupled rationalization with asymmet- ric learning rates: A flexible lipshitz restraint. Pro- ceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery & Data Mining.
Zhang Dong, Zhang Hanwang. Tang Jinhui, Hua Xian- sheng, and Sun Qianru. 2020. Causal intervention for weakly supervised semantic segmentation. In NeurlPS.
Ye Liu, Kai Zhang, Zhenya Huang, Kehang Wang. Yanghai Zhang, Qi Liu, and Enhong Chen. 2023c. Enhancing hierarchical text classification through knowledge graph integration. In Findings of the As- sociation for Computational Linguistics: ACL 2023, pages 5797-5810.
Alexander D' Amour. 2019. On multi-cause ap- proaches to causal inference with unobserved coun- founding: Two cautionary failure cases and a promising alternative. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3478-3478-80 3486. PMLR.
Julian J. McAuley, Jure Leskovec, and Dan Jurafsky. 2012. Learning attitudes and attributes from multi- aspect reviews. In 12th IEEE International Confer- ence on Data Mining, (ICDM), Brussels, Belgium, December 10-13, 2012.
Shaohua Fan, Xiao Wang. Yanhu Mo, Chuan Shi, and Jian Tang. 2022. Debiasing graph neural networks via learning disentangled causal substructure. Ad- vances in Neural Information Processing Systems, 35:24934-24946.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in neural information processing systems (NeurIPS), pages 3111-3113
Weibo Gao, Hao Wang, Qi Liu, Fei Wang. Xin Lin, Linan Yue, Zheng Zhang, Rui Lv, and Shijin Wang. 2023, Leveraging transferable knowledge concept graph embedding for cold-start cognitive diagnosis. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, pages 983-992.
Yulei Niu, Kaihua Tang. Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. 2021. Counter- factual vqa: A cause-effect look at language bias. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 12700- 12710.
Madelyn Glymour, Judea Pearl, and Nicholas P Jewell. 2016. Causal inference in statistics: A primer. John Wiley & Sons.
OpenAL 2023. Introducing chatgpt. OpenAl Blogs.
Bhargavi Paranjape. Mandar Joshi, John Thickstun. Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. An information bottleneck approach for controlling conciseness in rationale extraction. In Proceedings of the 2020 Conference on EMNLP.
Zikun Hu, Xiang Li. Cunchao Tu, Zhiyuan Liu, and Maosong Sun. 2018 Few-shot charge prediction with discriminative legal attributes. In Proceedings of the 27th International Conference on Computa- tional Linguistics, pages 487-498.
Judea Pearl et al. 2000. Models, reasoning and infer- ence. Cambridge, UK: 19:2.
Yongfeng Huang, Yujun Chen, Yulun Du and Zhilin Yang. 2021. Distribution matching for rationaliza- tion. In Thirty-Fiith AAAI.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word rep- resentation. In Proceedings of the 2014 conference on EMNLP, pages 1532-1543.
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Cate- gorical reparametrization with gumble-softmax. In International Conference on Learning Representa- tions (ICLR).
I Lorenzo Richiardi, Rino Bellocco, and Daniela Zugna. 2013. Mediation analysis in epidemiology: meth- ods, interpretation and bias. International JoternationalJuuralll of epidemiology, 42(5):1511-155
Luke Keele. 2015, The statistics of causal inference: A view from political methodology. Political Analysis, 23(3):313-335.

---

Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. In Proceedings of the Sth International Conference on Learning Repre- sentations (ICLR).
Nuo Xu, Pinghui Wang, Long Chen, Li Pan, Xiaoyan Wang, and Junzhou Zhao. 2020. Distinguish con- fusing law articles for legal judgment prediction. In Proceedings of the 58th ACL.
Mo Yu, Shiyu Chang, Yang Zhang, and Tommi S Jaakkola. 2019. Rethinking cooperative rationaliza- tion: Introspective extraction and complement con- trol. In EMNLP.
Lei Sha. Mana-Maria Camburu, and Thomas Lukasiewicz. 2021. Learning from the best: Rationalizing prediction by adversarial information calibration. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAT).
Mo Yu, Yang Zhang. Shiyu Chang, and Tommi Jaakkola. 2021. Understanding interlocking dynam- ics of cooperative rationalization. NeurIPS.
Rahul Singh, Maneesh Sahani, and Arthur Gretton. 2019. Kernel instrumental variable regression. Ad- vances in Neural Information Processing Systems.
Junkun Yuan, Anpeng Wu, Kun Kuang, Bo Li, Runze Wu, Fei Wu, and Lanfen Lin. 2022. Auto iv: Coun- terfactual prediction via automatic instrumental vari- able decomposition. ACM Transactions on Knowl- edge Discovery from Data.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux. Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023, Llama: Open and efficient foundation language mod- cls. arXiv preprint arXiv:2302. 13971.
Linan Yue, Qi Liu, Yichao Du Yanqing An, Li Wang. and Enhong Chen. 2022. Dare: Disentanglement- augmented rationale extraction. In Advances in Neu- ral Information Processing Systems.
Marcos V. Treviso and André F. T. Martins. 2020. The explanation game: Towards prediction explainabil- ity through sparse communication. In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP.
Linan Yue, Qi Liu, Binbin Jin, Han Wu, Kai Zhang, Yanging An, Mingyue Cheng, Biao Yin, and Day- ong Wu. 2021. Neurjudge: A cireunstance-aware neural framework for legal judgment prediction. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval
Ali Vosoughi, Shijian Deng, Songyang Zhang, Yapeng Tian, Chenliang Xu, and Jiebo Luo. 2023. Unveil- ing cross modality bias in visual question answer- ing: A causal view with possible worlds vqa. arXiv preprint ar 2r055 19664.
Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian- Sheng Hua. 2020. Interventional few-shot learning. In NeurlPS.
Fei Wang, Wenjie Mo. Yiwei Wang, Wenxuan Zhou, and Muhao Chen. 2023. A causal view of entity bias in (large) language models. arXiv preprint arXiv:2305. 14695.
Omar Zaidan and Jason Fisner. 2008 Modeling anno- tators: A generative approach to learning from an- notator rationales. In Proceedings of the 2008 con- ference on Empirical methods natural language processing, pages 31-40.
Tan Wang, Jianqiang Huang, Hanwang Zhang, and Qianru Sun. 2020, Visual commonsense r-cmn. In Proceedings of CVPR, pages 10760-10770.
Wenkai Zhang, Hongyu Lin, Xianpei Han, and Le Sun. 2021. De-biasing distantly supervised named en- tity recognition via causal intervention. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th Interna- tional Joint Conference on Natural Language Pro- cessing.
Xiaobao Wu, Chunping Li, and Yishu Miao. 2021. Dis- covering topics in long-tailed corpora with causal in- tervention. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021.
Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat seng Chua. 2022. Discovering invariant ra- tionales for graph neural networks. In ICLR.
Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Chaojun Xiao, Zhiyuan Liu, and Maosong Sun. 2018. Le- gal judgment prediction via topological learning. In Proceedings of the 2018 Conference on EMNLP.
Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng. Xianpei Han, Zhen Hu, Heng Wang, et al. 2018. Cail2018: A large-scale legal dataset for judgment prediction. In arXiv preprint arXiv: 1807. 007.02
Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, and Fuzhen Zhuang. 2022. Gener- alizing to the future: Mitigating entity bias fake news detection. In Proceedings of the 45th Inter- national ACM SIGIR Conference Research and Development in Information Retrieval, pages 2120- 2125.
Kelvin Xu, Jimmy Ba, Ryan Kiros. Kyunghyun Cho. Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show. attend and tell: Neural image caption genera- tion with visual attention. In Proceedings of the 32nd International Conference on Machine Learn- ing, (ICML).

---

A Instantiated Causal Graph
A.1 Causal graph for the selector
In this section, we describe causal graph for the selector (Figure 2(b)) in detail with examples: - X. The context prior C' determines which tokens can be "put" into the text input X. Among them, the context prior consists of unobserved prior and observed prior (such as the label set). For example, in Figure 1, both observed Intentional homicide and Manslaughter priors decide where the tokens denoting violence and death appear; the Manslaughter prior determines where the tokens representing manslaughter appear; the unobserved prior decides where other tokens that are meaning- less appear.
(a) Figure 4: Causal graph for the predictor.
(b)
constraint and consist of real rationales and short- cuts tokens, we argue that R + Y holds. Besides, the context prior C affects the label Y by the me- diator H.. The reason is similar to K - R in Appendix A.I.
From the graph, prior C' is the confounder between R and Y, which opens the backdoor path R C - H Y. Therefore, remove the spurious correlations be- tween I and Y, we adopt the causal intervention method to calculate P(Y (R))) by cutting the link C -> R (Figure 4(b)):
clearly see the context
X + C. denotes the context-specific representation which is a weighted representation of the prior knowledge associated with X in C. Taking Figure as an example, we assume that the label set consisting of Intentional homicide, Manslaughter, and Theft is the observed prior. Then, we can get the context prior which consists of four parts (i.e., the Intentional homicide prior cı, the Manslaughter prior C2, the Thefi prior 63 and the unobserved prior c4). Next, we calculate the association between X and C, and obtain the cor- responding scores, assuming
IN
((Y))((R)) H)
P(Y)
= [softmax(ff(R, (())
0.3, 0.6, 0g 0.0, a4 0.1, where Manslaughier prior C2 and X are the most relevant, and the Theft prior 03 and X are the least relevant. Finally, we can calculate the K as 01c1 + 0202 + 03c3 + a4c4 0.3c, + 0.6e2 0.1c4-
where fr(-) is the function achieving R > Y H, hs = 9r c)) = 3c., 3r E 3. 3 E IR is the set of the normalized similarity between R and each in the confounder set C (i.e., B (O codes R into d-dimensional vector, and the dic- tionary D. {a, 02, 0 as the observed confounder.
Among them, fp(+) en-
subse- quence of X, X + R holds. Besides, K + R represents the contextual constitution of the text that affects the composition of rationales. Taking the previous example
X x R - K. As the rationale
9N1) is approximated .ep.......ppp ...,............................
Besides, we also adopt the NWGM approxi- mation to Eq (8) and set fr h2) as a linear model (i.e., [r h4) = Wsfp(R) + Wahy = (al)( + W13,o)) Then, the final objective of intervention is formulated as:
example, since K is cal- culated as 0.3c, + 0.6cg 0.1c4, tokens in R will be more inclined with the Manslaughter prior c2.
A.2 Causal graph for the predictor
this section, we describe the detailed causal re- lationship (Figure 4(a)) between the selected ra- tionales R and the results Y in the predictor at high-level:
P(Y(do(R)
|sottmax (())))
M softmax ((RR,)))(s))
- R. Based on Eq (5) in section 3.2, we conclude that the context prior C' deterriines which tokens are corresponding rationales.
softmax(WaffAA)
[N]
R > H < C. H represents the context-specific representation of by using the context prior C. R + + H. As rationales R are selected by a-
+ W4 Σ (c))
99)

---

Table Detailed statistics of the processed BeerAdvocate and MevieReview.
Train
Test
Dutaset
Pos
# Neg
Avg.Tokens
Pos
Neg
Tokens
Pos # Neg # Avg. Tokens # Avg Rationale
BeerAdvicate (sppearance)
202,385 12,897
153.86
28,488
1,318
151.33
955
14
126.80
22.61
BeerAdvocate (aroma)
172,299 30,564
154.80
24,494
3,396
152.46
126.62
18.38
BeerAdvocate (palate)
176,038
27,639
153.98
24,837
3,203
151.77
940
24
126.02
13.46
MovieReview
800
800
847.25
100
100
835.03
99
100
835.03
54.40
A.3 Why there is no need to perform interventions the front-door path?
B.2
Discussions a-constraint
Although several rationalization methods do not set a-constraint to extract rationales, we believe that their methods constraining the short ratio- nales extraction can be considered a variant of a- constraint, and our intervention method in section 3.3 will still be effective on these methods. Specif- ically, argue that these methods should set hy- perparameters encourage the model select short rationales. However, if the hyperparameters are not set properly, it is possible that more short- cuts tokens will extracted, making and confounded. For example, for several methods (Chen and Ji, 2020; Paranjape et al., 2020) adopt- ing the information bottleneck to ensure the model extracts short rationales, there exists KL diver- gence between the posterior distribution P(myle (zy) and the prior distribution r(mj). where r(m,) Bernoulli(T)) for some constant æ E (0, 1). For in- stance, we set π as 0.1, it means we encourage the model to extract 10% the input text. Therefore, we consider π as a variant of a proposed by us.
In this paper, we only block the backdoor pathways in Figure 2, unable to climinate front-door pathway. The reason why we do not perform interventions on the front-door path is as follows:
1. Firstly, from the causal graph, we find the backdoor path + + K + R causes the spurious correlation. Meanwhile, the increased likelihood of R given X is due to "X causes I!" via X + R and X + K - R (the mediation path). K > R, is a projection of in the context prior C. Therefore, we argue that the X > R path can be removed if can be fully represented by K, where X also requires neural network representation to achieve X + R. There- fore. X + K + R. is beneficial for prediction. 3. In our paper, we aim to use the do-operation which implements the de-confounded training of the rationalization to remove the spurious correla- tion. Meanwhile, we retain the mediation path to keep the beneficial things for prediction.
C Setting Details
B Explanations for a-constraint
C.1 Statistics of BeerAdvocate and MovieReview
B.1 Examples for a-constraint
In this section, we show the detailed statistics of BeerAdvocate and MovieReview in Table 5. Among them, BeerAdvocate contains three aspects, including appearance, aroma and palate. From the Table 5, we can observe that the processed Beer- Advocate a non-balanced dataset. In the train- ing set, the prior distribution is pesaaive se 20:1 in appearance, positive:negattve xe 17:3 in aroma, positive:negative = 17:3 in palate. Mean- while, MovieReview is a balanced dataset with positive:negative 1:1.
As mentioned in section 3.3, in the practical, higher predefined sparsity level may bring short- cuts tokens which hurt the prediction performance, an extreme example being that all tokens in the text input will be selected. Below, we take a beer review as an example to further illustrate this problem, where this example is adopted to predict scores of the smell aspect.
The original text:
He thinks this beer smells great and tastes terrific Rationales: smells great
C.2 Comparison methods and experimental setups for LJP
Rationales with shorteuts tokens, where is set to 0.5: smells great and tastes terrific. Among them, "and tastes terrific" can be considered as shortcuts tokens.
In addition to comparing RNP (Lei et al., 2016), HardKuma (Bastings et al., 2019) and INVRAT

---

(Chang et al., 2020), we also compare our method with some classical baselines in the LJP task:
TopJudge (Zhong et al., 2018) explores the de- pendencies among the three subtasks in LJP.
Figure 5: The token precision and recall for rationales on the appearance aspect with a = 0.3.
Few-Shot (Hu et al., 2018) utilizes the charge attributes to identify the confusing charges.
C.3.2 Completeness
LADAN (Xu et al., 2020) learns the distin- guished law articles representations for LJP pre- diction.
Q: Do you think the selected rationales are enough for explaining the predicted labels?
Exactly. Selected rationales are enough for me to get the correct label.
NeurJudge (Yue al., 2021) a circumstance aware approach adopting different crime circum- stances to yield corresponding results. Mean- while, it employs a label embedding method to enhance the prediction.
4: Highly complete. Several tokens related to the label are missing.
Half complete. There are still some important tokens that have not been selected, and they are nearly the same number as the selected tokens.
For training, we adopt the word2vec (Mikolov et al., 2013) for word embedding pre-training with size 200, and set the encoder in f-)-) and fz(-) as Bi-GRU. Besides, we implement the learning rate of 0.0002 with batch size 256, and take 0.2. For evaluating, we employ the accuracy (Aec), macro-precision (MP), macro-recall (MR), and macro-F1 (F1) evaluate the performance of yielding judgment results.
2: Somewhat complete. The selected tokens are not enough.
1: Nonsense. None of the important tokens is selected.
C.3.3 Fluency
C.3 Scoring standards for human evaluation
Q: Do you think the selected rationales are fluent?
Following (Sha al., 2021), evaluate the ratio- nales with three metrics: usefulness (U), complete- ness (C), and fluency (F) the charge prediction. Among them, each scored from (lowest) to 5, Be- low, we introduce scoring standards for the above metrics in brief. Detailed standards for human an- notators can be found in (Sha et al., 2021).
Very fluent.
4: Highly fluent.
Partial fluent.
Very unfluent.
1: Nonsense.
C.3.1 Usefulness
C.4 Prompt for ChatGPT for human evaluation
Q: Do you think the selected rationales can be use- ful for explaining the predicted labels?
This is prompt for ChatGPT: Suppose you data tagging assistant and I will provide you with some case facts. Please follow the following prompts to make your predic- tions:
5: Exactly. Selected rationales are useful for me to get the correct label.
4: Highly useful. Although several tokens have relevance correct label, most selected to- kens useful to explain the labels.
1. Please predict the charge of the given case
facts.
2. Please extract one or more consecutive sen- tences and tokens from the case facts to support your prediction and keep the extracted sentences and tokens short possible. Please note that you must only extract from the given case facts. 3. Only output a python dictionary format to me, e.g. ("label":ineen inteentional homicide, 'ratio- nale' extract sentence].
3: Half of them are useful. About half of the tokens are useful for getting labels.
2: Almost useless. Almost all of the tokens are useless.
1: No Use. The selected rationales are useless for identifying labels.

---

CAIL. Chorge
Label Manclaughter
defiendant and the victim were both stadents, after the dormitory relocation, sleeping the lower bunk bed dispute, the defendant picked up bottle and forcefully smashed the victim's hend. resulting in the victina's head injury, then the defordant sent the victim to hospital. The vietim died in hospital treatment failed. The forensic medical uppraisal showed the injury to the victin's lace was pre-ecistng injury and the degree of iniury was minor. The viclim's death was consistent with an acute heart attack where truums and cmotional stress as precipiating factors...
dormitory, the defeedann the victim
The defemdantt and the victim were both studenis, aftrr the dormitory relocation, in the new dormitory. the defendant and the victim for sleeping the lower bank bed dispute the defendant picked bottle and forcefully smashed the vietim's hend, resulting the victim's hend injury, then the defendant sent the victim to huspitul. The victim died in hospiaal treatment failed. The forensic medical appriisal showed the injury the victin's face
pre pre-exising injury and the degree of injury minor. The victim's death was consistent with an acute heart attack where trauma and emotional stress as precipinting factors...
The defendant and the victim were both stadents, after the dormitory relocation, the new dormitory, the defendant and the victim for steeping in the lower bunk hed dispute, the defondant picked bottle and forcefully smashed the victinn's head, resulting in the vietin's head injury, then the defendant sent the sietim to hospital, The sietim died in hospiaal treaiment Galled. The forensic medicall appraisal showed the injury to the vicrin's faee was a pre-existing injury and the degree of Injury was minor. The victim's death was consistent with an acute heart attack where trauma emotional stress as precipitaring factors...
Figure 6: Examples of selective rationalization on the charge prediction. Among them, RNP wrongly predicts the charge as Intentional homicide. Meanwhile, both INVRAT and Inter-RAT predict the charge correctly, but Inter-RAT extracts more plausible rationales to yield results.
D More Experimental Results
tion method is that Eq (10) ignores the prior distri- bution P(c.). It is worth noting that although Neur- Judge ignores P(c:), it performs slightly better than our model. A potential reason is that NeurJudge exploits the dependencies among LJP tasks well, while our model is trained on the independent task.
D.1 Changes with Training Epochs
Besides, comparing with INVRAT, we investigate the model performance by showing the changes in token precision and recall with training epochs. Figure 5 shows the experiments on the appearance aspect with = 0.3. From the observation, we can conclude that Inter-RAT significantly outperforms INVRAT in both precision and recall with lower variance from the training onwards, which proves the effectiveness of our proposed method.
D.3 Visualization
We provide several visualization cases in the CAIL dataset as shown in Figure 6, which are selected by RNP, Inter-RAT and INVRAT. Among them, an- notated rationales are underlined. RNP, INVRAT and Inter-RAT rationales are highlighted in yel- low, green and pink colors, respectively. From the Figure 6, we can find that RNP fails to predict the charge and considers it as Intentional homicide with capturing the tokens denoting violence and death. On the contrary, both Inter-RAT and IN- VRAT predict the charge correctly, but Inter-RAT can extract more comprehensive rationales (i.e., The victim's death was consistent with an acute heart attack), which support the victim's death was due to the negligence. This observation further demonstrates our causal intervention method can select more sufficient rationales for yielding results.
D.2 A Causal View on Judge
In this section, from the causal view, we discuss the reason why the difference between Inter-RAT and NeurJudge the charge and law article prediction is not significant. Here, we explain the observa- tion by taking the charge prediction as an example, and the article prediction is similar. Specifically, NeurJudge adopts a label embedding method to incorporate the semantics of charge into the case fact to yield the corresponding result. We argue that this method can be approximated as the causal intervention method. To illustrate this discovery, we assume Figure 2(d) is the causal graph of the charge prediction task, and consider the case fact as R (i.e., a=l) and the charge label set as C. Then the process of label embedding can be formulated as R >> H < C and R 2 < H. The objective of NeurJudge is written as: [N]
P(Y R) = softmax Σ
S (R, he)),, (10)
where h; = Ineru ( RR, cz). We can find the differ- ence between the Eq (10) and our causal interven-


Reviews:
[{"review_id": "PLY8PUho4X", "reviewer": "Reviewer_YC9Z", "summary": "", "questions": "There are some reasons to reject, but I am considering changing the scores depending on the response.\n\nA. I may have missed a detail, but is there a reason why HardKuma was not subjected to human evaluation in Table 4? I would like to know the reason for excluding the model that performs well compared to other baselines.\n\nB. Looking at the baselines, there is no method published from 2022 to the present. Is there really no paper that has been published? Is there a reason why the model that appeared in [1] was not included as a baseline? Since 2022, LLMs have rapidly advanced, and models like ChatGPT seem to be able to simultaneously act as both a selector and a predictor even in zero-shot and few-shot settings. Have you attempted to add LLMs to the baseline?\n\n[1] Li et al. Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction, AAAI 2021.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your important comments! Below we conduct additional experiments and provide the responses:\n\n>**Comment 1:** I wish there was a more detailed explanation for Figure 2. The text is small and lacks content, making it hard to read.\n\nThank you for your comments, and we will enlarge the font and add content to Figure 2 in the revised version of the paper. Specifically, we will provide the following explanation in the caption of Figure 2:\n\n(a) SCM for rationalization. (b) SCM for the selector, which offers a ﬁne-grained causal relationship between $R$ and $C$ with a mediator $K$. (c) Intervention on the selector, where $C$ -> $X$ is cut off  and the confounder $C$ is stratified into pieces $C = \\{c\\_{1}, c\\_{2},  \\dots, c\\_{|C|}\\}$. (d) SCM for the predictor, which describes a ﬁne-grained causal relationship between $Y$ and $C$ with a mediator $H$.\n\n>**Comment 2:** The baseline is inadequate. There is no method published from 2022 to the present. \n\nThank you for bringing baselines in [1], including IB, FRESH, IB (semi-supervised, 25%) , Weakly- & Semi-supervised, AT-BMC and Pipeline.\n\nCurrent rationalization methods can be classified into three groups, i.e. unsupervised rationalization, semi-supervised rationalization and supervised rationalization based on whether the training set contains human annotated rationale or not. In this paper, our method belongs to the category of unsupervised rationalization.\n\nFor baselines in [1], the classification is shown in the following Table:\n\n| unsupervised rationalization    | IB，FRESH                                             |\n| ------------------------------- | ----------------------------------------------------- |\n| semi-supervised rationalization | IB (semi-supervised, 25%) ，Weakly- & Semi-supervised |\n| supervised rationalization      | AT-BMC，  Pipeline                                    |\n\nTherefore, we choose IB, FRESH as our new baselines. Besides, we add DARE [2] which is published in 2022 as our new baseline.\n\nFrom Table, we can find Inter-RAT outperforms the baselines consistently in ﬁnding correct rationales.\n\n|           |      |              |  Appearance  |              |              |    Aroma     |              |              |    Palate    |              |\n| :-------: | :--: | :----------: | :----------: | :----------: | :----------: | :----------: | :----------: | :----------: | :----------: | :----------: |\n|  method   |  a   |   Presion    |    Recall    |      F1      |   Presion    |    Recall    |      F1      |   Presion    |    Recall    |      F1      |\n|    RNP    | 0.1  |   32.4±0.5   |   18.6±0.3   |   23.6±0.4   |   44.8±0.4   |   32.4±0.7   |   37.6±0.5   |   24.6±0.5   |   23.5±0.5   |   24.0±0.5   |\n| HardKuma  | 0.1  |   53.6±0.1   |   28.7±0.1   |   37.4±0.1   |   29.3±1.4   |   25.9±3.8   |   27.3±2.1   |   7.7±0.1    |   6.0±0.1    |   6.8±0.1    |\n|  INVRAT   | 0.1  |   42.6±0.7   |   31.5±0.6   |   36.2±0.6   |   41.2±0.3   |   39.1±2.8   |   40.1±1.6   | **34.9±1.5** |   45.6±0.2   |   39.5±1.0   |\n|    IB     | 0.1  |   50.5±0.2   |   29.7±0.5   |   37.4±0.4   |   43.5±0.3   |   39.2±0.5   |   41.3±0.1   |   29.9±0.2   |   34.2±0.7   |   31.9±0.2   |\n|   FRESH   | 0.1  |   51.4±0.3   |   29.5±0.5   |   37.5±0.4   |   42.8±0.4   |   40.4±0.3   |   41.5±0.3   |   29.3±0.5   |   32.7±0.2   |   30.9±0.2   |\n|   DARE    | 0.1  |   63.9±0.1   |   42.8±0.2   |   51.3±0.1   |   50.5±0.1   |   44.8±0.2   |   47.5±0.2   |   33.1±0.4   |   45.8±0.1   |   38.4±0.2   |\n| Inter-RAT | 0.1  | **66.0±0.4** | **46.5±0.8** | **54.6±0.7** | **55.4±0.9** | **47.5±0.6** | **51.1±0.8** |   34.6±0.8   | **48.2±0.4** | **40.2±0.5** |\n|           |      |              |              |              |              |              |              |              |              |              |\n|    RNP    | 0.2  |   39.4±0.4   |   44.9±0.1   |   42.0±0.2   |   37.5±0.1   |   51.9±0.7   |   43.5±0.3   |   21.6±0.4   |   38.9±0.5   |   27.8±0.4   |\n| HardKuma  | 0.2  | **64.9±0.9** |   69.2±1.0   |   67.0±0.8   |   37.0±1.3   |   55.8±1.9   |   44.5±1.5   |   14.6±0.3   |   22.3±0.8   |   17.7±0.4   |\n|  INVRAT   | 0.2  |   58.9±0.4   |   67.2±2.3   |   62.8±1.1   |   29.3±1.0   |   52.1±0.6   |   37.5±0.6   |   24.0±1.3   |   55.2±2.3   |   33.5±1.6   |\n|    IB     | 0.2  |   59.3±0.4   |   69.0±0.2   |   63.8±0.2   |   38.6±0.1   |   55.5±0.7   |   45.6±0.1   |   21.6±0.2   |   48.5±0.4   |   29.9±0.2   |\n|   FRESH   | 0.2  |   59.7±0.3   |   70.1±0.2   |   64.5±0.2   |   39.4±0.3   |   55.4±0.2   |   46.0±0.1   |   23.7±0.2   |   50.5±0.1   |   32.3±0.2   |\n|   DARE    | 0.2  |   63.7±0.2   |   71.8±0.8   |   67.5±0.5   |   41.0±0.2   |   61.5±0.2   |   49.3±0.3   |   24.4±0.1   |   54.9±0.8   |   33.8±0.1   |\n| Inter-RAT | 0.2  |   62.0±0.5   | **76.7±1.7** | **68.6±0.4** | **44.2±0.1** | **65.4±0.2** | **52.8±0.1** | **26.3±0.6** | **59.1±0.8** | **36.4±0.7** |\n|           |      |              |              |              |              |              |              |              |              |              |\n|    RNP    | 0.3  |   24.2±0.4   |   41.2±0.8   |   30.5±0.5   |   27.1±0.3   |   55.7±0.8   |   36.4±0.4   |   15.4±0.4   |   42.2±0.9   |   22.6±0.5   |\n| HardKuma  | 0.3  |   42.1±0.3   |   82.4±1.4   |   55.7±0.5   |   24.6±0.1   |   57.7±0.6   |   34.5±0.2   |   21.7±0.1   |   49.7±0.4   |   30.2±0.1   |\n|  INVRAT   | 0.3  |   41.5±0.4   |   74.8±0.3   |   53.4±0.3   |   22.8±1.6   |   65.1±1.7   |   33.8±1.8   |   20.9±1.1   | **71.6±0.4** |   32.3±1.3   |\n|    IB     | 0.3  |   40.2±0.1   |   81.5±0.2   |   53.9±0.2   |   27.9±0.2   |   59.2±0.3   |   37.9±0.1   |   19.1±0.7   |   59.0±0.9   |   28.9±0.6   |\n|   FRESH   | 0.3  |   41.4±0.3   |   81.2±0.2   |   54.8±0.3   |   29.5±0.4   |   60.9±0.2   |   39.7±0.3   |   21.1±0.2   |   60.2±0.3   |   31.2±0.2   |\n|   DARE    | 0.3  |   45.5±0.2   |   80.6±0.2   |   58.1±0.1   |   32.7±0.2   |   68.2±0.3   |   44.2±0.1   |   19.7±0.6   |   70.5±0.4   |   30.8±0.7   |\n| Inter-RAT | 0.3  | **48.1±0.7** | **82.7±0.5** | **60.8±0.4** | **37.9±0.7** | **72.0±0.1** | **49.6±0.7** | **21.8±0.1** |   66.1±0.8   | **32.8±0.1** |\n\n>**Comment 3:** HardKuma baseline is missing in the Human evaluation & there a reason why HardKuma was not subjected to human evaluation in Table 4? & Have you attempted to add LLMs to the baseline?\n\nThank you for your suggestion. Since INVRAT is the baseline for our main comparison, we do not conduct a human evaluation of HardKuma.\n\nTo clarify your confusion, we add the human evaluation of HardKuma in Table below. In addition, due to time constraints, we add the human evaluation of DARE, where DARE achieves competitive results in extracting rationales. \n\nIt is worth noting that the similar human evaluation is provided in DARE, but its $\\alpha$ is set differently from ours, where DARE sets $\\alpha$ to 0.14 and our Inter-RAT is set to 0.2. For consistency, we replicate DARE with  $\\alpha$ = 0.2.\n\nFrom Table, we can see that both HardKuma and DARE do not achieve as good results as Inter-RAT, which also shows the effectiveness of our method.\n\nAlso, based on your suggestion, we additionally add chatgpt for human evaluation.\n\nWe can observe that chatgpt achieves competitive results on both Completeness and Fluency. But it does not perform well on Usefulness. Because even though we ask chatgpt to extract short tokens and sentences as rationale, it still tends to extract longer sentences, which is beneficial for the Fluency metric. However, for the Usefulness metric, some redundant tokens will be extracted as well. Therefore chatgpt does not perform well on the Usefulness.\n\n| Methods   | Usefulness (U) | Completeness (C) | Fluency (F) |\n| --------- | -------------- | ---------------- | ----------- |\n| RNP       | 3.85           | 3.28             | 3.34        |\n| INVRAT    | 3.88           | 3.42             | 3.41        |\n| HardKuma  | 3.78           | 3.33             | 3.39        |\n| DARE      | 4.57           | 3.89             | 4.19        |\n| chatgpt   | 4.01           | 4.08             | 4.34        |\n| Inter-RAT | 4.52           | 4.10             | 4.25        |\n\n\n\nThis is our prompt for chatgpt:\n\nSuppose you are a data tagging assistant and I will provide you with some case facts. Please follow the following prompts to make your predictions:\n\n1.Please predict the charge of the given case facts.\n\n2.Please extract one or more consecutive sentences and tokens from the case facts to support your prediction and keep the extracted sentences and tokens as short as possible. Please note that you can only extract from the given case facts. \n\n3.Only output a python dictionary format to me, e.g. {'label':intentional homicide, 'rationale':extract sentence}.\n\n\n\n**References**\n\n[1] Dongfang Li, Baotian Hu, Qingcai Chen, Tujie Xu, Jingcong Tao, Yunan Zhang. Unifying Model Explainability and Robustness for Joint Text Classication and Rationale Extraction. In AAAI2021.\n\n[2] Linan Yue, Qi Liu, Yichao Du, Yanqing An, Li Wang, Enhong Chen. DARE: Disentanglement-Augmented Rationale Extraction.  In NeurIPS2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) It recorded the SOTA across various datasets.\n2) The paper is well-written, making it easy to read, including problem definition, method proposal, and result analysis.\n3) The experimental results show a trend that is consistent with human evaluation.", "reasons_to_reject": "1) I wish there was a more detailed explanation for Figure 2. The text is small and lacks content, making it hard to read.\n2) HardKuma baseline is missing in the Human evaluation (Table 4).\n3) The baseline is inadequate. There is no method published from 2022 to the present", "questions_to_authors": "There are some reasons to reject, but I am considering changing the scores depending on the response.\n\nA. I may have missed a detail, but is there a reason why HardKuma was not subjected to human evaluation in Table 4? I would like to know the reason for excluding the model that performs well compared to other baselines.\n\nB. Looking at the baselines, there is no method published from 2022 to the present. Is there really no paper that has been published? Is there a reason why the model that appeared in [1] was not included as a baseline? Since 2022, LLMs have rapidly advanced, and models like ChatGPT seem to be able to simultaneously act as both a selector and a predictor even in zero-shot and few-shot settings. Have you attempted to add LLMs to the baseline?\n\n[1] Li et al. Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction, AAAI 2021.", "paper_topic_and_main_contributions": "The paper focuses on the selective rationalization task in DNNs, aiming to identify significant features impacting predictions. The paper introduces an interventional rationalization (Inter-RAT) method, applying causal interventions to remove spurious correlations. Experimental results on real-world datasets validate the effectiveness of the approach. As a result, the proposed method achieves SOTA on many datasets.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ayJZIBtEkz", "reviewer": "Reviewer_fKwz", "summary": "", "questions": "How authors would expand it to more recent large models in vision and language?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your comments! We are very glad that you have a positive impression on our work. To address your concerns, below we provide the responses as follows:\n\n> **Comment 1:** The study sometimes talks about benchmarks that have been itemized by authors, so how authors assure that these samples are fairly selected and no cherry picking?\n\nWe are sorry that our lack of detail on selecting and evaluating the human evaluation dataset confuses you. Below we describe how the dataset is selected and evaluated.\n\nTo ensure fairness, we randomly sample the samples in the test set. We do not perform any post-processing on the dataset obtained from this sampling. \n\nFor the human evaluation, we shuffle all the results generated by the RNP, INVRAT, and Inter-RAT, and then distribut them to legal professionals for annotation.\n\nThe annotators do not know which method generated the samples they evaluate. Of course, for the purpose of subsequent aggregation of results, we (the authors) know which sample to be evaluated is generated by which method, but we do not participate in the evaluation.\n\nFinally, we promise to release our human evaluation dataset.\n\n\n\n> **Comment 2:** How authors would expand it to more recent large models in vision and language?\n\nOur approach can be used to provide explanations of text classification tasks. For large language models (LLMs), in the autoregressive generation, we can treat the prediction of each token as a text classification task, where the number of categories is equal to the size of the vocab. By generating a corresponding explanation for each token, we can gradually explain the output of a LLM.\n\nHowever, applying Inter-RAT to the explanation of LLMs is still a challenging task. We will continue to research and explore it as the focus of our future work.\n\n\n\n> **Comment 3:** Missing References\n\nThank you very much for the related work, which we will cite in the revised version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is about a method that seems to be generalizable and scalable, as it uses causality and reasoning to improve their tasks. Therefore, there is hope for wider future applications in the dawn of large foundational models. The paper may be interesting to EMNLP audience.", "reasons_to_reject": "The study sometimes talks about benchmarks that have been itemized by authors, so how authors assure that these samples are fairly selected and no cherry picking?\n\nExample in line 603 \n---Second, as CAIL does not provide annotated...", "questions_to_authors": "How authors would expand it to more recent large models in vision and language?", "paper_topic_and_main_contributions": "The paper talks about rationalization models that have been prior proposed by INVRAT as  means of inferring based on relating outcomes to some rationals, being said as X -> R -> Y. Here Y is outcome and R is rational which is based on input and it is modeled by P(R|X). \n\nWhat authors put in here is to propose two backdoor additions to INVRAT to propose a method that identified causal paths by removing confounder effects, call it Inter-RAT and show that the method does relatively better than existing literature in this sense.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Not as far as I noticed."}, "extra_text": {"Missing_References": "The paper is well written; however, the review papers in the related works seems to mostly older as compared and rarely includes recent works such as below which have used causality to solve spurious relationships. \n\nNiu, Yulei, et al. \"Counterfactual vqa: A cause-effect look at language bias.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.\nVosoughi, Ali, et al. \"Unveiling Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA.\" arXiv preprint arXiv:2305.19664 (2023)."}}, {"review_id": "y4157WpKDh", "reviewer": "Reviewer_sFn6", "summary": "", "questions": "1、Could you provide statistical results relevant to the shortcut mentioned in Figure 1? I couldn't accept that there is only a narrative ``story'' claiming that the issue is caused by the shortcut. After all, whether introducing spurious correlations or identifying the shortcut, we require obvious experimental results for validation.\n\n2、There are indeed some works that analyze the causal relationships between shortcuts, rationales, and prediction outcomes. Why not elaborate the differences with these works in the related work section? For instance, the referenced paper [1] and [2] proposed a causal graph that bears no essential difference from the one in this paper, and it's even clearer.\n\n[1]:Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat seng Chua. 2022. Discovering invariant rationales for graph neural networks. In ICLR.\n[2]:Fan, Shaohua, et al. \"Debiasing graph neural networks via learning disentangled causal substructure.\" Advances in Neural Information Processing Systems 35 (2022): 24934-24946.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful comments! Please find the responses to the specific comments.\n\n> **Comment 1:** Figure 2 is not an SCM structure but a simple DAG (Directed Acyclic Graph) format.\n\nThank you for your comments. The reason that Figure 2 does not include independent noise terms is that we assume that the context prior $C$ consists of unobserved prior and observed prior. Among them, we classify the independent noise terms into the unobserved prior. \n\nBased on your comments, we argue that such a setup may confuse the readers. In the revised version of the paper, we will add $N$->$X$, where $N$ denotes the independent noise terms.\n\n\n\n> **Comment 2:** do(X) only blocks the backdoor pathways in Figure 2, unable to eliminate front-door pathways, thus failing to establish the actual X->R relationship.\n\nThank you very much for pointing out the problem. If we need to ascertain the veritable causal relationship between $X$->$R$, we do need to consider the front-door pathway $X$->$K$->$R$. Thank you for your detailed derivation.\n\nHowever, we argue that it is our statement \"To pursue the real causality from $X$ to $R$\" (Line 266) in section 3.2 that misleads you. \n\nFirstly, we argue that $R$ is determined by $X$ via two paths: \n\n1) the direct $X$->$R$.\n\n2) the mediation $X$->$K$->$R$.\n\nThen, in Line 266, we expect the expression \"To pursue the real causality from X to R, including the two paths\".\n\n\n\n> **Comment 3:** How the author's intervention is implemented?\n\nWe are sorry that our description of the intervention method confused you. Below we first show the derivation of intervention (Eq. 5):\n\n$P(R \\mid d o(x))  =\\sum_{i=1}^{|C|} P(R \\mid d o(X=x), C=c_{i}) P(C=c_{i} \\mid d o(X=x)) \\\\  $\n\n$=\\sum_{i=1}^{|C|} P(R \\mid d o(X=x), C=c_{i}) P(C=c_{i})$\n\n$=\\sum_{i=1}^{|C|} P(R \\mid X=x, C=c_{i}) P(C=c_{i}) \\\\ $  \n\n$=\\sum_{i=1}^{|C|} \\sum_{k} P(R \\mid X=x, C=c_{i}, K=k) P(K=k \\mid X=x, C=c_{i}) P(C=c_{i}) \\\\ $\n\n$=\\sum_{i=1}^{|C|} P(R \\mid X=x, C=c_{i}, K=g_{s}(x, c_{i})) P(C=c_{i}) \\\\ $\n\n$=\\sum_{i=1}^{|C|}P\\left(R | X, K=g_{s}\\left(X, c_{i}\\right)\\right) P\\left(c_{i}\\right)$.\n\nSince the confounder $C$ is commonly hard to observe, we consider the entire label set as the partially observed children of the unobserved confounder.  We approximate it by designing a dictionary $D_{c} = \\{c_{1}, c_{2},  \\ldots, c_{|N|}\\}$ as an $N \\times d$ matrix, where $N$ represents the number of labels. Finally, the intervention is implemented as:\n\n$  P(R | d o(x))=\\sum_{i=1}^{|C|}P\\left(R | X, K=g_{s}\\left(X, c_{i}\\right)\\right) P\\left(c_{i}\\right)$\n\n$ \\approx \\sum_{i=1}^{|N|}P\\left(R | X, K=g_{s}\\left(X, c_{i}\\right)\\right) P\\left(c_{i}\\right)$\n\n$=\\sum_{i=1}^{|N|}\\left[P\\left(R | f_{s}(x_{j}, k_{i})\\right) P\\left(c_{i}\\right)\\right] $\n\n$= \\sum_{i=1}^{|N|} \\left[\\textrm{softmax}(f_{s}(x_{j}, k_{i}))P\\left(c_{i}\\right)\\right]$\n\n$\\approx \\textrm{softmax}( \\sum_{i=1}^{|N|} f_{s}(x_{j}, k_{i})P\\left(c_{i}\\right) ).$\n\n\n\n> **Comment 4:** Statistical results relevant to the shortcut mentioned in Figure 1.\n\nWe first provide the number of Intentional homicide cases and Manslaughter cases in the CAIL dataset we used: \n\n|              | Intentional homicide | Manslaughter |\n| ------------ | -------------------- | ------------ |\n| training set | 1877                 | 220          |\n| test set     | 442                  | 276          |\n\nWe can find that in the training set, Intentional homicide occurs more frequently than Manslaughter, while there is not much difference between the two types of cases in the test set.\n\nIn addition, we show the statistics on whether RNP and Inter-RAT correctly classify Intentional homicide cases and Manslaughter cases on the test set.\n\n|           | Intentional homicide (correct/wrong) | Manslaughter (correct/wrong) |\n| --------- | ------------------------------------ | ---------------------------- |\n| RNP       | 413/29                               | 165/111                      |\n| Inter-RAT | 421/21                               | 243/33                       |\n\nFrom the statistics, it can be seen that both RNP and Inter-RAT can achieve good results when predicting Intentional homicide, while Inter-RAT performs better than RNP when predicting Manslaughter. This observation demonstrates our causal intervention method can select more sufficient rationales for yielding results.\n\n\n\n> **Comment 5:** Why not elaborate the differences with these works in the related work section?\n\nWe thank the reviewer for providing new related work. We refer to \"Discovering invariant rationales for graph neural networks\" [1] as DIR for short, and \"Debiasing graph neural networks via learning disentangled causal substructure\" [2] as DisC.\n\nBoth DIR and DisC study the problem of rationalization in graph neural networks. They separate the graph as the rationale subgraphs and the non-rationale ones. Afterwards, interventions are implemented through counterfactual data augmentation methods to extract the faithful rationales.\n\nThe major difference between our Inter-RAT and these methods is that we further focus on the limitations on the predefined sparsity in rationalization, where the sparsity constraint may result in spurious correlations between the extracted rationale and the predicted result.\n\n**References**\n\n[1] Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat seng Chua. Discovering invariant rationales for graph neural networks. In ICLR2022. \n\n[2] Fan, Shaohua, et al. Debiasing graph neural networks via learning disentangled causal substructure. In NeurIPS2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The performance shown in the experiments is well.\n2. The imposition of sparsity constraints is a good insight.", "reasons_to_reject": "1. Figure 2 is not an SCM  structure but a simple DAG (Directed Acyclic Graph) format. An SCM structure includes not only the value of nodes and edges but also the important aspect of independent noise terms, which allows judgment of the causal relationship between pairs of variables. However, the text didn't mention anything about these noise terms. \n\n2. Based on the explanation of Figure 2, the author's understanding of causal theory is fundamentally flawed. In section 3.2, the author suggests that the purpose of an intervention is to discover the true X->R relationship. Yet from the author's interpretation of Figure 2, do(X) only blocks the backdoor pathways in Figure 2, unable to eliminate front-door pathways, thus failing to establish the actual X->R relationship. \n\n   To ascertain the veritable causal relationship between X->R, we must consider the front-door pathway X->K->R. Here, X is observable, hence we can compute P(K|do(X)) = P(K|X) (since there are no backdoor paths at this point). Furthermore, according to the section 2, K is unknown, so when calculating the causal effect of K->R, the backdoor pathway can be blocked by conditioning on X. We can then use the adjustment formula to calculate P(R|do(K)) by summing up over $\\sum$ P(R|K, X)P(X). \n\n3. It's unclear how the author's intervention is implemented specifically. For the intervention P(R|do(X)), it is necessary to observe changes in conditional probabilities under different X samplings to draw out the causal relationship between R and X. Do(X) is a shorthand, but the implementation needs to consider different situations like do(X=x1), do(X=x2), etc. However, it seems the author does not quite comprehend the genuine purpose of intervention.\n\nIn conclusion, I find it challenging to endorse the author's many explanations of causal relationships. The article tries to elucidate the motivations of its method from a causal modeling standpoint. However, many aspects are mere conjecture by the author, and there are many self-contradictions.", "questions_to_authors": "1、Could you provide statistical results relevant to the shortcut mentioned in Figure 1? I couldn't accept that there is only a narrative ``story'' claiming that the issue is caused by the shortcut. After all, whether introducing spurious correlations or identifying the shortcut, we require obvious experimental results for validation.\n\n2、There are indeed some works that analyze the causal relationships between shortcuts, rationales, and prediction outcomes. Why not elaborate the differences with these works in the related work section? For instance, the referenced paper [1] and [2] proposed a causal graph that bears no essential difference from the one in this paper, and it's even clearer.\n\n[1]:Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat seng Chua. 2022. Discovering invariant rationales for graph neural networks. In ICLR.\n[2]:Fan, Shaohua, et al. \"Debiasing graph neural networks via learning disentangled causal substructure.\" Advances in Neural Information Processing Systems 35 (2022): 24934-24946.", "paper_topic_and_main_contributions": "This paper aims to analyze from a causal relationship perspective why models for Selective Rationalizations tasks are prone to learning shortcuts rather than true causal patterns, for the purpose of enhancing explainability. It also simulates the implementation of the 'do' operator in deep learning, which has shown commendable performance on multiple datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "above", "Typos_Grammar_Style_and_Presentation_Improvements": "The paper seems to have a poor understanding of many definitions in causality theory. The author should try to gain a deeper understanding of SCMs and the concept of interventions."}, "extra_text": {}}, {"review_id": "C7A4u8fTwH", "reviewer": "Reviewer_gxKG", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and insightful suggestions! According to your comments, we provide the responses as follows:\n\n> **Comment 1:** The paper should introduce more about the works of Entity Bias.\n\nThanks for your comments, below we present some related work on Entity Bias:\n\nThe Entity Bias problem is related to the spurious correlation problem in the rationalization.\n\nThe Entity bias is a phenomenon that occurs when models overly rely on prediction shortcuts triggered by certain entities and thus makes spurious predictions.\n\nNumerous research works [1-5] have studied this. Among them, [1] proposes  an entity-masked contrastive pre-training framework for relation extraction to model the textual context and entity types. \n\n[4] proposes an entity debiasing framework by mitigating entity bias from a cause-effect perspective. \n\n[5] studies the causal intervention techniques to mitigate entity bias in language models.\n\nDifferent from the Entity Bias problem, rationalization is more concerned with removing spurious correlations in the data to improve the explainability of the model.\n\nPlease let us know if there is any key related work that has been missed, and thanks again for your comments.\n\n\n\n> **Comment 2:** The effectiveness of using the text category as the confounder is debatable.\n\nBased on the existing research [6,7], we consider the label set as the partially observed children of the unobserved confounder $C$. Therefore, we approximate it by designing a dictionary $D_{c} = \\{c_{1}, c_{2},  \\ldots, c_{|N|}\\}$ as an $N \\times d$ matrix, where N is the number of labels. \n\nHowever, using text category as the confounder is an expedient measure, and we need more observable confounders. Inspired by [8], we can introduce fine-grained attributes of text category into consideration and take these attributes as additional confounders. We will make it our future work.\n\n\n\n> **Comment 3:** The causal graph model of the paper is fundamentally inspired by [7].\n\nOur causal graph model is inspired by [7]. Differently, in the rationalization, in addition to the spurious correlations between the textual input and  the extracted rationale, we futher focus on the limitations on the predeﬁned sparsity, where the sparsity constraint may result in spurious correlations between the extracted rationale and the predicted result.\n\n**References**\n\n[1] Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Learning from context or names? an empirical study on neural relation extraction. In EMNLP2020.\n\n[2] Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. In EMNLP2021.\n\n[3] Nan Xu, Fei Wang, Bangzheng Li, Mingtao Dong, Muhao Chen. Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing. In EMNLP2022.\n\n[4] Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, Fuzhen Zhuang. Generalizing to the Future: Mitigating Entity Bias in Fake News Detection. In SIGIR2022.\n\n[5] Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. A Causal View of Entity Bias in (Large) Language Models. In Arixv2023.\n\n[6] Tan Wang, Jianqiang Huang,Hanwang Zhang, Qianru Sun. Visual commonsense r-cnn. In CVPR2020.\n\n[7] Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian Sheng Hua.  Interventional few-shot learning. In NeurIPS2020.\n\n[8] Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, and Maosong Sun. Few-shot charge prediction with discriminative legal attributes. In COLING2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": " Using the structural causal model backdoor adjustment method to remove spurious correlations between inputs and rationales for text prediction tasks is meaningful for downstream analysis.", "reasons_to_reject": "1. The causal graph in your Interventional Rationalization is similar to [1]. Both of you utilize causal intervention via backdoor adjustment and regard the category as the confounder. The causal graph model of the paper is fundamentally inspired by [1].\n\n2. The effectiveness of using the text category as the confounder is debatable.\n\n3. ''the underlined tokens in Figure 1 denoting the offense is negligent will be ignored in rationales extraction and the charge will be misjudged''. So the reason why the charge will be misjudged is because of the Entity Bias problem. The paper should introduce more about the works of Entity Bias.\n\n[1] Zhongqi Yue, Hanwang Zhang, Qianru Sun, and Xian Sheng Hua. 2020. Interventional few-shot learning. In NeurIPS.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper introduces interventional rationalization as a novel method that leverages causal theory and causal intervention techniques to enhance the generation of explanations for neural network predictions. It tackles the challenge of spurious correlations and validates its effectiveness through experiments on real-world datasets.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes Inter-RAT, a method based on causal analysis to enhance the explainability of the neural networks. By eliminating spurious correlations using the causal intervention, the proposed method outperformed previous baselines on multiple datasets. Reviewers all agreed on the potential utility of the proposed method in the NLP community. However, a strong concern remained regarding the improper understanding of structural causal model (SCM), which the paper relied on developing the proposed method, and hence the potentially unreliable validity of the discovered causal relationship.
Decision: Accept-Main