Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Authors: Wei Liu
Zhongyu Niu
Lang Gao
Zhiying Deng
Jun Wang
Haozhao Wang
Ruixuan Li
Venue: ICML 2025 poster (2025)
Keywords: Unsupervised rationale extraction
selective rationalization
self-explaining
interpretability
Primary area: social_aspects->accountability_transparency_and_interpretability

Abstract: This study investigates the self-rationalization framework constructed with a cooperative game, where a generator initially extracts the most informative segment from raw input, and a subsequent predictor utilizes the selected subset for its input. The generator and predictor are trained collaboratively to maximize prediction accuracy. In this paper, we first uncover a potential caveat: such a cooperative game could unintentionally introduce a sampling bias during rationale extraction. Specifically, the generator might inadvertently create an incorrect correlation between the selected rationale candidate and the label, even when they are semantically unrelated in the original dataset. Subsequently, we elucidate the origins of this bias using both detailed theoretical analysis and empirical evidence. Our findings suggest a direction for inspecting these correlations through attacks, based on which we further introduce an instruction to prevent the predictor from learning the correlations.
Through experiments on six text classification datasets and two graph classification datasets using three network architectures (GRUs, BERT, and GCN), we show that our method significantly outperforms recent rationalization methods.

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Wei Liu' Zhongyu Niu
Lang Gao Zhiying Deng"
Jun Wang
Haozhao Wang Ruixuan Li'
Abstract
1. Introduction
With the success of deep learning, there are growing con- cems over the model interpretability. In contrast to post- hoc methods, self-explaining techniques typically offer in- creased transparency (Lipton, 2018) and faithfulness (Yu et al., 2021), as the prediction is made based on the expla- nation itself. There is a stream of research that has exposed the unreliability of post-hoe explanations and called for self- explanatory methods (Rudin, 2019; Ghassemi et al., 2021; Ren et al., 2024).
This study investigates the seff-rationalization framework constructed with a cooperative game, where generator initially extracts the most in- formative segment from raw input, and sub- sequent predictor utilizes the selected subset for its input. The generator and predictor are trained collaboratively to maximize prediction accuracy. In this paper, we first uncover a po- tential caveat: such cooperative game could unintentionally introduce a sampling bias dur- ing rationale extraction. Specifically, the gener- ator might inadvertently create incorrect cor- relation between the selected rationale candidate and the label, even when they are semantically unrelated in the original dataset. Subsequently, we clucidate the origins of this bias using both detailed theoretical analysis and empirical evi- dence. Our findings suggest a direction for in- specting these correlations through attacks, based which we further introduce instruction prevent the predictor from learning the correla- tions. Through experiments on six text classifica- tion datasets and two graph classification datasets using three network architectures (GRUs, BERT, and GCN), we show that our method not only significantly outperforms recent rationalization methods, but also achieves comparable or even better results than representative LLM 1 8b-instruct), Code: https: //gtthub com/ AugechhengalPaaiionallattttt
In this study.our primary focus is investigating general model-agnostic self- exllaxiilann framework called Rationaliz- ing Neural Predictions (RNP, also known as rationalization) (Lei et al., 2016), which with its variants has become one of the mainstream methods to facilitate the interpretability of NL. models (Sha et al., 2021; Yu et al., 2021; Liu et al., 2023a;b; 2024a;b), and also holds the potential be applied to image classification (Yuan et al.. 2022) and graph neural networks (Luo et al., 2020). RNP utilizes cooperative game involving a generator and a predictor. This game is designed with a focus on "data-centric" (i.e., it is to explain the connection between a text and the (model-agnosticc task label, rather than explaining the output of specific model) feature importance. The generator first identifies the most informative part of the input, termed the rationale. Subse- quently, the rationale is transmitted to the predictor to make predictions, as illustrated in Figure 1. The generator and predictor are trained cooperatively to maximize prediction accuracy. Apart from its use for interpretability. some recent studies find that rationalization can also serve as a method for data cleaning. The extracted (Z, Y) pairs can act as new dataset, and trained with such a cleaned dataset, predictor may be more robust (Chen et al., 2022) and gen- eralizable (Wu et al., 2022; Gui et al., 2023), thanks to the removal of task irrelevan,,, harmful information.
'School of Computer Science and Technology, Huazhong University of Science and Technology "Faculty of Artificial Intelligence in Education, Central China Normal University 'iWudao Tech. Correspondence to: Zhiying Deng <2hiy- ingdzy com>> Jun Wang <jwang@iwwdao.ter Haozhao Wang <hz, hust.edu.cn>> Ruixuan Li cail@hust.edu.c...
Our research starts with a special empirical observation. We first observe that, even we remove "maximizing the prediction accuracy" from the generator's objective (thus selects some random noise), the predictor can still be trained to get very high accuracy with these randomly selected spurious rationales (the orange line in Figure 3(a) of $4.1).
Proceedings of the nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
This phenomenon then leads to trust concern: whether

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datascts
service is excelllent and veyy dlan good location being closs
Generator
is eocaddlerd very clean gocatiot being close lo-all dhalllddedirviiinssssssooo
Predictor
[0.1, 0.9]
m H(V,
(Service)
all destimetions in town.
[0
X
Z
Y
Figure 1. The standard rationalization framework RNP. The task in this figure is binary sentiment classification about hotels' service. X, Z, Y, Y. represent the input, the selected rationale candidate, the prediction, and the classification label. M is a sequence of binary masks, Og. Op are the parameters of the generator and the predictor.
This study tries to address this kind of correlations with two steps: inspection and instruction. We first theoretically show that if predictor classifies based on a trivial pattern T that is associated with the category label Y due to the sampling of the generator, we can always find an attacker inspect the trivial pattern. Then, to prevent the predictor from learning such a correlation (which would make the generator fur- ther enhance it), we manually adjust the distribution of the trivial pattern from P(Y[T, g) to P(Y) (in fact, it should be P(Y[T), but we have P(Y[T)= = P(Y) for the attacker identified trivial pattern T) to provide instructions that en- able the predictor to learn the correct information, thereby giving the generator the correct feedback. Formal introduc- tion of our method will appear in $4.2. We provide toy example in Appendix A.2 to give readers a quick intuitive understanding of our method.
Tase Binary sentiment classification
Label (about the bocr's 's appearance): Positive. Prediction Positive. Imput: 8- murky, semi-oparue honey low head_ s -earthy. plantains, , pineapple rind, apricol I- earthy hay and pepper. touch or orange. cilantro. honey... . very saison-like. m - medium body... nice carbonation. balanced semi-dry finish. o nice flavor profile. Rationale selected by RNP.
Figure 2. A secerr- example of the generator-added spurious correlation, The underlined text is auman-annotated rationale. The text in red is the rationale selected by RNP. Example I: from positive input X x wwth label 1, the generator selects rationale Z' that includes the pattem "!" and for negative input X with label 0, the generator selects a rationale z" that does not include".". And subsequently, the predictor considers the presence absence of "! as an indicative feature for positive classification.
Our contributions include: (a) We identify a new type of spu- rious correlation, and we systematically analyze how it can arise in even clean datasets with both theoretical support and empirical verification. (b) A practical solution. We design an attacker to both inspect whether the predictor has learnt from the spurious correlation and instruct the predictor not to learn from it. (c) We design various experiments to verify the existence of the generator added spurious correlation, the effectiveness of the inspection, and the effectiveness of the instruction.
the extracted rationale is really responsible for the label in the original dataset (i.e., although the extracted rationale is considered faithful to the model's prediction by previous research, is it faithful to the model-agnostic dataset?). This problem is important because explanations should also be aligned with their social attribution (Jacovi & Goldberg, 2020; 2021).
We then shed light the source of this problem. Typically, we call a pattern T is trivial if it is independent with Y in the original dataset: P(Y = P(Y). However, due to the potential bias of the generator's sampling, T can be correlated with Y in the sampled (Z,Y) pairs. Figure 2 provides a (cherry-picked) practical example it.
2. Related work
The basic cooperative framework of ratio- nalization named RNP (Lei et al., 2016) is flexible and offers unique advantage: certification of exclusion, which means any unselected input is guaranteed to have no contribution to prediction, making it important to the NLP community (Yu et al., 2021). Based on it, many methods have been proposed to improve RNP from different aspects. Bao et al. (2018) used Gumbel-softmax to do the reparameterization for bina- rized selection. Bastings et al. (2019) replaced the Bernoulli sampling distributions with rectified Kumaraswamy distribu- tions. Jain et al. (2020) disconnected the training regimes of the generator and predictor networks using a saliency thresh- old. Paranjape et al. (2020) imposed discrete bottleneck objective to balance the task performance and the rationale length. Liu et al. (2025) replaced the cross-entropy loss with
Rationalization.
We further explore the origins of this issue and discover that it stems from an approximation that was overlooked in previous research: taking a series of (Y, 2) pairs sampled by the generator as an approximation of P(Y, Z) (while should actually be P(Y, Zlg), and note that Y I1 Z +> Y 1 Zlg). In fact, this problem can be seen as type of spurious correlation. But notably, the perspective of this paper is totally different from the traditional causality research for spurious correlations. Existing research causality has primarily focused on spurious correlations inherent in the dataset. However, our research investigates a further question: if the dataset itself is clean and lacks spurious correlations, could the selection process of the generator introduce additional spurious correlations?
2

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
the final-layer representation norm. Douuu al. (2020) proposed benchmark that can be used for supervised ra- RAT (Yue et al., 2023) tried to use backdoor adjustment to alleviate the spurious correlations in the raw dataset. DR (Liu et al., 2023c) assigned different learning rates to the generator and the predictor. Hase et al. (2020) explored better metrics for evaluation. Rajagopal et al. (2021) used phrase-based concepts to conduct self- explaining model. Other methods like data augmentation with pretrained models (Plyler et al., 2021), training with numan-annotated rationales (Chan et al., 2022), injecting noise to the selected rationales (Storek et al., 2023), have also been tried.
lection of samples drawn from the true data distribution P(X,Y). X = XI. is the input text sequence of length I, and Y represent the classes in the dataset (note that discrete label can also seen as representing a distribution like [O, By enumerating X, we can get P(Y]X), which is the distribution that a normal non-interpretable classi- fier working on D needs to approximate. Rationalization consists of a generator fo(-) (or g for conciseness) and a predictor fp((), with fy. Op being their parameters.
tionale extraction. Rnt
For (X,Y) D. the generator first outputs sequence of binary mask M = (X) Ma e 10. (in practice, the generator first outputs a Bernoulli distribution for each token and the mask for each token is independently sampled using sumbel-softmax). Then, it forms the rationale candidate Z by the element-wis product:
Prior to our work, a series of studies had observed a phe- nomenon termed degeneration, whose origin can also be attributed to the spurious correlation investigate in this study. Degeneration means that, the predictor is too power- ful to recognize any trivial patterns that are distinguishable in rationales with opposite labels. As a result, the generator may collude with the predictor to select the trivial patterns rather than the true semanties as the rationales (Yu et al., 2019). Previous methods seek regularize the model using supplementary modules which have access the informa- tion of the full text (Yu et al., 2019; Huang et al., 2021; Yu et al., 2021; Liu et al., 2022) such that the generator and the predictor will not overfit uninformative rationales. Among them, FR achieves the strongest improvements on address- ing degeneration by sharing a unified encoder between the generator and predictor to regularize each other. And will be included in our baselines. However, although these meth- ods have been proposed to fix the observed problem, the origin of this problem is not well explored. Sometimes they can still fail. For example, Zheng et al. (2022) argued with both philosophical perspectives and empirical evidence that the degeneration problem is much more complex than we used to think and some of the above methods cannot promise no-degencration. In fact, this phenomenon similar to what we discuss and can also be seen one of the problems stems from taking P(Y, Zlg) as P(Y, 2). highlighting the impor- tance of rectifying the bias in approximating Zly, as Z(Y,Z).
Z M + M,X]].
(I)
To simplify the notation, we denote (x) as Z in the following sections, i... fX() Z.
We consider that X consists of a set of variables (Ti, Rt,, where I denotes the real rationale (e.g.. sentiment tendency for sentiment classification) for task label Y, and "i,"., Tn are some trivial patterns indepen- dent with Y. And we select one of (Ti, Tn, R} to be Z. Note that Z is not separate variable but a proxy for any variable within X. Till now, we get a set of (Z,Y) samples denoted as Dz. Previous research simply thinks Dz is collected from P(Z,Y) By enumerating Z in Dz. they get P(Y]Z) and attempt to identify the rationale by maximizing the mutual information:
Z = arg max
I(Y:
Ze(T
arg max Ze(Ti..TT...))
(H(Y) HY(Z))
(2)
= arg min HY
Ze(TT.."T..R))
In practice, the entropy H(YlZ) is commonly approxi- mated by the minimum cross-entropy mine, HY, Ylz), with Y (Z) representing the output of the predictor (note that the minimum cross-entropy is equal to the entropy, Appendix C.3). Replacing Z with fy (X), the generator and the predictor are trained cooperatively:
We also briefly discuss the potential impact of rationaliza- tion in the era of LLMs in Appendix A. We compare our method against a representative LLM (11ama-3.188- instruct) in Appendix D.
3. Background of the rationalization task
min H,CYY s.t., (X,Y) - D. (3) 0,
Unless otherwise specified, uppercase letters represent ran- dom variables, while lowercase letters correspond to their values. For simplicity, we do not distinguish between vec- tors and scalars. We consider the classification task. We have a classification dataset D, which can be seen as a col-
Compactness and coherence. To make the rationales human--ntelligible, previous methods usually constrains the rationales by compact and coherent regularization terms. We use the widely used constraints provided by Chang et al.
3

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
0.9
0.0
0.8
text
0.7-
dom patterns
0.7-
r2f
0.6
0.6
full_text
text
0.5
0.5
random patterns
r2
0.4
50
200
250
0.4
150
epochs
epochs
Training
(b) Validation
Figure 3. Experiments on the Becr-Aroma datasct (other datasets are Appendix B.2): "full text"; : predictor trained using the full texts. "random patterns:: predictor trained with randomly selected patterns, "r2f": feeding the random patterns to the predictor that was trained using the full texts.
(2019):
What's more, the train- ing process may even en- hance the sampling bias further. For example, we consider Ti is selected as Z, then the updating of the generator 8 h(Og Tı, Y) (h denotes the backpropagation pro- cess), and this structural function corresponds to small local of a causal graph shown in Figure 4. We originally have Y a Ti, But in this graph, we have # Ti That's to say, any trivial patterns hold the potential be associated with Y through the influence of the generator.
Ta
R
|M]
((M) = A1
8
+ A2
Σ t-
Me
Mo-I/.
(4)
The first term encourages that the percentage of the tokens being selected as rationales is close to pre-defined level The second term encourages the rationales to be coherent.
Figure 4. local the causal graph for the generator's updat- ing process. Dash cycle means X consists of set of variables.
4. Motivation and method
Notation. For the sake of exposition, let us take the example of binary sentiment classification. We denote X and X as input texts with label Y = and Y = O, respectively. Z and ZA represent the rationale candidates selected by the generator and the attacker, respectively. Note that they are not separate variables but a proxy for any variables within X. Sometimes we use Z and the variable represented by Z interchangeably. T is a proxy for any variables within (Ti,", TT) (defined in $3).
Consider situation where Z /=T is a trivial pattern inde- pendent with Y (i.e., P(Y 1|7) = P(Y - =) = 0.5 P(Y 0) P(Y 0/T) and T' (t,, t-), Influenced by the generator g. T = l. might co-occur more frequently with Y and can be viewed as an indicator for the positive class (T t. is similar):
T'
4.1. Cause of the spurious correlation
How do trivial patterns correlate with Y? Although considering Dz an approximation of P(Z,Y) Y) seems to be simple and practical way and is inherited by all the previous methods ($3), it will sometimes results in some problems. In fact, the sampling process of Z is conditioned on generator with specific parameters 0g. So we can only get P(Z, Ylg) and P(Y|Z.g) rather than P(Z, Y) and PZ) Note that independent doesn't lead con- ditional independent: Y 7 Y Zlg. That is to say, some uninformative Z (like those Ti, .., TT,) might initially be independent with Y and maintain zero mutual informa- tion with Y. But sampled by g. any trivial patterns may get correlated with Y and get increased mutual informa- tion, thus can be used as (incorrect) indicative features for classification.
P(Y 1| (4,9)> P(Y 1) P(Y 0 11Z=to,p t.. P(Y = 0).
(5)
Example in Figure 2 of $1 also provides an intuition for the above analysis.
Empirical support. The above motivation is inspired by some practical observations. We present three types of prediction accuracies for a binary sentiment classification task (about the beer's aroma) in Figure 3: ① A predictor trained with the full input text. A predictor trained with randomly selected patterns. For the generator, we remove the other objectives and only train it with the sparsity con- straints (Equation 4). That is to say, the generator is trained to randomly select 10% of the input text, and the predictor

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
then trained to classify using these randomly selected texts.
Attacker
+
Predictor
Y
@ We use the randomly selected texts from ② to feed the predictor trained in (
A
+Shard
Generator
Z
Predictor
p
From Figure 3(a), we observe that even with the randomly selected patterns (i.e., patterns unlikely to contain real ratio- nales), the predictor can still achieve a very high prediction accuracy (represented by the orange line, approximately 95%). This accuracy is close to that of the classifier trained with the full texts. A follow-up question is: Does this strange result stem from the fact that the 10% randomly se- lected patterns already contain enough sentiment inclination for classification? The answer is no. Consider the green line, which represents the outcome when we feed the ran- domly selected texts to the well-trained predictor denoted by the blue line. We observe that the green line indicates significantly lower accuracy (about 58%), implying that the randomly selected patterns contain only minimal sentiment information. Thus, the orange predictor incorrectly treats certain randomly selected trivial patterns as indicative fea- tures. Moreover, the orange predictor does not generalize well to the validation set (Figure 3()), due to the fact that simple trivial patterns can more easily lead to overfitting (Pagliardini al., 2023).
The architecture of attacking for inspection and instruc- tion. We name Attack Inspection and Instruction (A2I). Z, Z represent the selected rationale candidate and the attack rationale. Y, Ya represent the normal prediction and the attack result.
Figure 5.
plementation details with Pytorch are in Appendix A.3, The overall mechanism of the model is as follows: (6) inspects trivial patterns (Xx()) from X. The second term of (7) is the instruction that prevents the predictor from learning the trivial patterns by classifying them as random noise. A well instructed predictor is then able to give good feedback to the generator's selection. And the first term of (7) is the normal RNP. The reason why the attacker constructed in this manner can detect trivial patterns will be elucidated in $4.3. We also use a toy example in Appendix A.2 to pro- vide an intuitive understanding. At the end of $4.3, we also discuss how our method will work in the situation where the generator and the predictor cooperate correctly on real rationales rather than trivial patterns.
We provide more evidence of the existence of such spurious correlations in practical scenarios from another perspective by demonstrating the attack success rate in $5.2.
4.3. Underlying principles
Attack as inspection. Following the above settings for Z T and 7(Y;T) = 0 in $4.1, will show how the trivial patterns learned by the predictor through attack. Corresponding to (5), if the attack generator can be constructed in any way (i.e., has infinite expressive- ness), then we can always find an attack generator Ia which extracts Z from X, such that
4.2. The proposed method
be inspected
For the sake of clarity, we first present our approach and then expound on the principles underlying it.
Figure 5 shows the architecture of our method. For a data point (X,Y) Y) in n-class classification task, the over all objective of our model ( (Sps fg: Sg fa represent the predictor, the generator, and the attacker, with pp. og. Og being their parameters) is:
P(Y = 1|ZA = t..gg) < P(Y = P(Y - 0 = t.,9,) P(Y = 0).
attacker: min H. 4.C(Y=,f(((((((x)11x)x)) 0. min (,(CX.,(((((()))1,,C 0,
(6)
gen&pred:
Appendix C.1 shows the detailed derivation for the reason why can find such a Ja- Equation (9) is the opposite of (5), and it means that under condition ga. T = tt, now becomes a negative class indicator, which is exactly the opposite situation under condition g. Here is the intuitive understanding of the attack. Corresponding to the punctu- ation pattern example mentioned in Figure 2 of $1. The generator g selects Z 4." from X And the predictor has learnt to prediet 4 as positive. We can employ an attacker Ia which selects Z 4. from X (whose class label is negative) such that Z can also be classified as positive. Similarly, the attacker can find Z = =." from X to be clas- sified as negative. So, the overall objective of the attacker is to select those Z that can be classified to the opposite class by the predictor.
+ min (to
0p
s.t. YA randin(0,,
YA represents the class to be attacked. We ran- domly select a class for each attack to create a bal- anced attack for each class. [1/m,...,1m repre- sents the distribution of P(Y) in the raw dataset. means we rec-
ming, ([1/n,... (/)) tify the sampled distribution of P(Y|ZA,,) a) to P(Y) and ask the predictor to learn that Z is not correlated with Y. In binary classification, we have YA= =A=1. Y and 1/n = 0.5.
During training, (7) and (6) are alternated. The practical im-
5

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Table 1. Results on datasets from the BeerAdvoeate benchmark, We follow BAT to set = 10%, 20%, 30%.
Datasets
Beer- Reer-Appeuaance
Beer-Aroma
Beer-Palate
Methods
S
Acc
P
R
F1
S
Acc
R
F1
Acc
Fi
Comparison with standard RNP
RNP
10.1
79.7
69.3
37.6
48.8
10.0
82.9
81.3
52.4
63.7
9.3
84.7
68.6
51.3
58.7
S 10%
RNP+A21
10.8
82.8
78.3
45.8
57.8
9.8
86.3
86.0
54.3
66.6
10.9
86.6
66.3
58.2
62.0
RNP
19.8
86.3
69.8
74.6
72.1
20.7
84.5
43.6
58.1
49.8
20.1
82.6
47.6
77.0
58.8
S m 20%
RNP+A21
20.0
87.7
73.3
79.4
76.2
19.5
85.4
49.0
61.4
$4.5
19.4
86.6
49.0
76.4
59.7
RNP
30.4
84.3
52.9
86.7
65.7
30.7
81.8
39.2
77.2
52.0
30.1
87.1
29.3
71.0
41.5
S = 30%
RNP+A21
29.9
85.2
59.3
95.9
73.3
27.8
87.3
44.5
79.3
57.0
30.5
87.1
30.8
75.5
43.7
Comparison with advanced variants
Inter RAT
13.2
50.0
35.7
41.6
13.8
64.0
56.9
60.2
13.0
47.2
49.3
48.2
NIR
10.6
78.1
77.0
44.3
56.2
10.3
86.1
74.9
49.7
59.8
11,5
84.0
48.1
44.4
46.2
= 10%
FR
11.0
82.2
68.0
40.5
50.8
9.4
86.7
85.3
51.5
64.2
9.4
84.5
70.1
52.8
60.2
HR+A21
11.3
84.6
76.0
46.5
57.7
10.0
86.9
85.7
54.8
66.9
9.7
84.8
71.4
55.8
62.6
Inter RAT
20.2
45.8
50.4
48.0
22.0
47.2
67.3
55.5
20.2
39.9
64,9
49.4
NIR
20.3
81.9
70.3
77.2
73.6
19.1
87.7
61.2
75.2
67.5
19.9
83.9
37.3
59.6
45.9
S = 20%
FR
19.7
87.7
77.7
82.8
80.2
20.5
90.5
61.1
80.3
69.4
19.8
86.0
42.1
67.0
51.7
FR+A21
19.8
88.7
80.0
85.6
82.7
19.4
89.7
64.2
80.0
71.2
19.2
86.0
44.2
68.2
53.7
Inter RAT
28.3
48.6
74.9
59.0
31.5
37.4
76.2
50.2
29.2
29.7
69.7
41.7
NIR
29.6
84.9
59.8
95.5
73.6
30.0
82.3
38.4
73.9
50.5
29.7
84.1
22.8
54.5
32.2
= 30%
FR
30.0
90.9
58.5
94.6
72.3
31.0
83.2
40.0
79.4
53.2
29.3
84.8
28.5
67.2
40.1
FR+A2I
28.8
89.7
61.3
95.3
74.6
30.9
83.2
41.4
82.2
55.1
29.1
85.1
31.6
73.8
44.2
Formally, the objective of the attacker is
now is, if the generator and the predictor cooperates well on real rationales, what will happen if X contains both positive and negative sentiments?
min H1 74(1- 8
(10)
The first glance might be that, both the generator and the attacker choose the true (but opposite) sentiment rationales, thereby leading to the predictor in (7) being unable to make the right prediction. But in practice, the predictor can over- come this obstacle. Consider intuitive assumption:
Till now, we have demonstrated that an attacker can identify uninformative trivial patterns and classify them into the opposite class. Then we begin to instruct the predictor to not learn from the trivial patterns (whether the attacker will select real rationales discussed at the end of this section).
Assumption 4.1. The positive rationale r, appears more often in positive texts than in negative ones: Prr.|Y=]]] 2 Prr_|Y 0).
Attack as instruction. When the spurious correlation oc- curs, the attacker Ja consistently chooses ZA that is label-independent trivial pattern. For a competent predictor that discerns the authentic rationale. ZA resembles noise independent with Y, ensuring its classification remains ran- dom without any leanings to a specific label. Thus, we introduce an extra instruction to the predictor:
This assumption stems from that we can always find r. in X but sometimes not in X If Assumption 4.1 holds, can easily prove (please refer to Appendix C.2) that the predictor in (7) will still converge to predict fr+) as positive with a high confidence (2 0.75).
min 0p s.t., ZA = fX),, (X,Y)~ D.
(11)
5. Experiments
5.1. Settings
That is to say, although we cannot promise the indepen- dence between ZA and Y under the generator's conditional sampling, we can make ZA Y through the predictor's prediction.
Baselines. We compare our A2I with the standard RNP and several recent representative methods: Inter_RAT (Yue et al., 2023) and CR (Zhang et al., 2023) represent recent causal methods, and FR (Liu et al., 2022) and NIR (Storek et al., 2023) represent recent methods designed to deal with degeneration. All of them have been discussed in $2.
The situation of text X contains both positive and negative sentiments. Here we consider Z = R, which is the true rationale based on which the label Y is assigned to X. We denote R = r+, R = r_ as positive and negative indicators, respectively. The question we want to discuss
Datasets. We first follow FR to examine on three datasets from BeerAdvocate benchmark (McAuley al., 2012):
6

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Table 2. Results on datasets from the HotelReview benchmark, We follow FR to set S 10%. 4: results from FR.
Datascts
Hotel-Location
Hotel-Service
Hotcl-Cleanliness Acc
Methods
S
Acc
P
R
F1
S
Acc
P
R
F1
F1
Comparison with standard RNP
RNP
8.8
97.5
46.2
48.2
47.1
11.0
97.5
34,2
32.9
33.5
10.5
96.0
29.1
34,6
31.6
= 10%
RNP+A21
9.0
97.5
50.2
53.4
51.7
11.6
97.0
46.8
47.4
47.1
9.7
96.5
34.7
38.2
36.4
Comparison with advanced variants
Inter RAT
11.0
34.7
44.8
39.1
12.5
35.4
39.1
37.2
9.6
33,4
36.7
34,9
NIR
10.2
93.5
45.1
54.2
49.2
11.0
95.5
44.9
43.2
44.0
10.6
96.0
34.1
40.9
37.2
x 10%
FR"
9.0
93.5
55.5
58.9
57.1
11.5
94.5
44.8
44.7
44.8
11.0
96.0
34,9
43,4
38.7
FR+A21
9.9
94.0
53.2
62.1
57.3
11.5
97.0
47.7
47.7
47.7
10.8
95.5
35.9
43.7
39.4
Table 3. Results on graph datasets. "0": std. The sparsity of selected rationale is set to be close to the sparsity of ground truth.
Datasets
BA2Motil'
GOODMotif
Methods
8
Ace
P
F1
S
Acc
P
F1
Compurison with standard RNP
RNP
(2.5)
(1.9)
36.5 (5.5)
36.5 (2.2)
(3.8)
62.1 (2.3)
(2.8
(4.8)
(3.6)
RNP+A21
(2.3)
95.2 (1.5)
39.7 (3.5)
40.5 (2.9)
(2.5)
(2.1)
63.7 (2.6)
43.1 (1.8)
46.3 (3.4)
44.6 (2.5)
Comparison with advanced
FR
(2.3)
96.4 (L.8
39.3
40.0 (4.9) (5.6)
(3.1)
64.5
(2.9)
47.1 (4.7)
44.3 (3.6)
PR+A2I
20.2(1.5)
96.5 (1.4)
42.1 (2.8)
42.5 (4.0)
(3.0
(2.6)
66.5 (2.2)
43.8 (4.2)
48.8 (3.7)
46.1 (3.9)
Beer-Appearance Beer-Aroma, Beer-Palate, and three datasets from HotelReview benchmark (Wang et al., 2010): Hotel-Location, Hotel-Service, Hotel-Cleanliness. Among them, the three beer-related datasets are used by nearly all of previous research in the field of rationalization. We also use two graph rationalization datasets, BA2Motif's (Ying al., 2019) and GOODMotif (Gui et al., 2022), to verify general- izability. GOODMotif is a three-class graph classification dataset. These datasets include ground-truth rationales in their test sets to facilitate objective comparison between different methods. More details about the datasets are in Appendix A.4.
in Table 1 and 2) under a unified seed are sufficient to ver- ify that the improvements are not from randomness. The training of GNN not as stable as GRUs, and we report the average results of five random seeds. More details are in Appendix A.5.
5.2. Results
Rationale quality. Table and show the results on the text classification datasets. For the most widely used beer- related datasets (which have been the most important bench- marks for long time), we follow inter_RAT to set three different sparsity levels: 10%, 20%, by adjusting in Equation (4). For the hotel-related datasets, we use them as supplementary material and follow FR to set the sparisty to be similar to human- annottatd rationales. Initially, we conduct our attacking inspection on top of the standard RNP to validate our claims and demonstrate the efficacy of our proposed method. Across all nine settings in Table 1, we observe a significant improvement over the standard RNP in terms of F1 score. Notably, the highest increase reaches up to 9.0% (Beer-Appearance with S 10%), underscoring the robust effectiveness of our method. Additionally, we com- pare with representative LLM, Ilama-3. 1-8b-instruct in Table 6 of Appendix D. and find that our simple A2I-based methods get comparable results to it and can sometimes even outperform it.
Metrics. Our findings suggest that even if the generator selects trivial patterns, the predictor can still get high final prediction accuracy. Thus the prediction performance is not a good metric for models' effectiveness. Following In- ter RAT and FR, we mainly focus on the rationale quality, which is measured by the overlap between model-selected tokens and annottted rationales. P. R, F1 denote precision, recall, and F1 score respectively. S (sparsity) rep- resents the average percentage of selected tokens in relation to the full text. Ace stands for the predictive accuracy.
Details. The generator, predictor, and attacker all are com- posed of an encoder (RNNWTransformer/GGN) and linear layer. We use three kinds of encoders: GRUs (following Inter_RAT and FR, Table 1 and 2). lowing CR, Table 4), and GCN (for the BA2Motifs dataset). The random seed is kept the same (the seed is 12252018, inherited from the code of FR) across all the experiments on text classification, as we think experiments with multiple datasets and multiple sparsity settings (totally 12 settings
(fol-
Our attack-based inspection is more of a tool than an inde- pendent model and model-agnostic (as long as there is a predictor to attack). Therefore, we further apply on top of the advanced method, FR (as FR outperforms Inter RAT and NIR in most cases), to demonstrate our competitive-
7

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Table 4. Results with BERT. We follow CR to set S 10%. "*"; results obtained from CR.
Datasets
Beer-A Apee-Appeeeece
Beer-Aroma
Beer-Palate
Methods
P
F1
P
R
F1
P
R
F1
RNP (Lei et al., 2016)
48.7
11.7
20.0
44.2
20.7
27.6
25.1
21.9
22.8
A2R* (Yu al., 2021)
49.1
18.9
25.9
51.2
21.2
29.8
31.8
24.3
25.4
S 10%
CR (Zhang et al., 2023)
45.3
22.0
28.0
60.3
35.4
39.0
32.5
25.9
26.5
FR (Liu et al., 2022)
41.8
19.3
26.4
47.1
27.6
34.8
32.6
29.4
30.9
FR+A21
48.6
25.7
33.6
55.4
32.0
40.5
34.4
32.3
33.3
1.0
1.0
1.0
RNP
RNP
RNP
0.5
RNP+A21
0.9
RNP+A21
ANP-A21
0.8
/
0.6
9.-
0.4
0.4
0.4
0.3
50
100
150
200
250
300
0.3
50
160
150 cpochs
200
250
360
0.3
6
25
50
75
160 opachs
125
150
175
20o
opochs
(a) Beer-Appearance
(b) Beer-Aroma
(c) Beer-Palate
Figure 6. Attack success rate on the three beer-related datascts. The rationale sparsity is about 20%. Results for sparsity being 10% and 30% are in Appendix B.I.
ness. Two observations emerge from the results. When our A2I is incorporated, the performance of both RNP and FR consistently improves. We observe significant improve- ment in FR's performance (up to 6.9% on Beer-Appearance with S x 10%) when our A2I is layered atop it, highlight- ing the competitiveness of our method. Aside from the most widely used beer-related datasets, we also consistently achieve strong performance on the hotel-related datasets (Table 2) and the graph datasets BA2Motifs and GOODMo- tif (Table 3) (note that Inter RAT, NIR, and CR methods specifically designed for text tasks and are not suitable for graph tasks).
of both an attack method's s efficacy and a model's resilience against such attacks. A high ASR signifies the effectiveness of an attack method, while low ASR denotes model ro- bustness. The results for the three beer-related datasets are displayed in Figure 6. Regarding the first question, "Can the attacker truly identify the trivial patterns learned by the predictor?", the blue lines offer insight. As opposed to RNP+A2I, the blue lines depict models where we omit the objective Equation (11) (specifically, the instruction loss) from Equation (7). This means that while RNP is trained as usual, an attacker is also being trained concurrently. The prominence of the blue lines demonstrates that the attacker achieves a remarkably high ASR. This indicates that the predictor RNP does internalize some trivial patterns, and the attacker successfully identifies them, underscoring the potency of the attack. For the second question, "Can the in- spection effectively deter the predictor from adopting trivial patterns?", we can look to the orange lines. The ASR values hover around 50%, which is close to random classification. This suggests that the attacker can only select some neutral patterns and the predictor actively avoids learning from the trivial patterns, highlighting the efficacy of the instruction.
Results with BERT. To show the competitiveness A21, we also follow CR to conduct experiments with pretrained BERT on the three most widely used beer-related datasets (Table 4) and compare with some methods that have already been implemented with BERT. We still get considerable improvements as compared to recent methods.
Attack Success Rate (ASR). To more effectively demon- strate the capabilities of our attacking inspection, we present the attack success rates for both RNP and our RNP+A2I. This experiment aims to address two key questions: 1) Can the attacker truly identify the trivial patterns recognized by the predictor? 2) Can the inspection really prevent the pre- dictor from adopting the trivial patterns? ASR is a metric commonly employed in the realm of security. Given pair (X,Y), if ff.(f(()) = Y. indicating a label inversion, we deem the attack successful. ASR serves as an indicator
6. Conclusion
This paper investigates a new type of spurious correla- tion (i.e., model-added spurious correlation) in the self- explaining rationalization framework. It can appear even in clean datasets, thus making previous causal methods (which
8

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
focus solely on the causal relationships in the raw dataset) ineffective in dealing with it. We design an attack-based method to inspect the model-added spurious correlations and to instruct the training of rationalization. Experiments on both text and graph domains show the effectiveness of the proposed method.
Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL. 2019, Florence, Italy, July 28- August 2, 2019, Volume I: Long Pa- pers. pp. 2963-2977. Association for Computational Linguistics, 2019. doi: 10. 10.18653/v1/119-12284 URL 1ttp/////g...111////////pp/--
Bi, J., Wang, Y., Chen, H., Xiao. X., Hecker. A., Tresp, V., and Ma, Y. Llava steering: Visual instruction tun- ing with 500x fewer parameters through modality linear representation-sseeriig 2024. URL https: /eattps://rrivy /arxiv. org /a//abb//24111 12359.
Impact Statement
This paper aims to investigate the interpretability of machine learning models. In our view, our research is unlikely to produce negative social impacts.
Exploring XAI techniques can help to address many impor- tant problems existed in present deep learning models. For instance, XAI techniques can aid in detecting model discrim- ination (fairness) (Pradhan al., 2022) or jailbreaks (Jiang et al., 2025), identifying backdoor attacks (security) (Yu et al., 2025a;b; Yang et al., 2025), controlling robots (Zhou et al., 2025), and revealing potential spurious correlations (robustness) (Yang et al., 2024), among others. Another direction is how to apply the findings of this direction into other modalities (Diao et al., 2024; 2025; Bi et al., 2025b;a).
Bi, J., Wang. Y., Yan, D., Xiao, X., Hecker, A.. Tresp, V., and Ma, Y. Prism: Self Self-pruning intrinsic selection method for training-free free multimodal data selection. arXiv preprint arXiv:2502. /2/11/111//00 2025a.
Bi, J., Yan, D., Wang, Y., Huang, W., Chen, H., Wan, G., Ye, M.. Xiao, X., Schuetze, H., Tresp, V., et al. Cot-kinetics: A theoretical modeling assessing Irm reasoning process. arXiv preprint arXiv:2505. 13408, 2025b.
Chan, A., Sanjabi, M., Mathias, L., Tan, L., Nie, S., Peng, X., Ren, X., and Firooz, H. UNIREX: A unified learning framework for language model ratio- nale extraction. In International Conference on Ma- chine Learning, ICML 2022, 17-23 July 2022, Balti- more, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 2867-2889. PMLR, 2022. URL URLhttps: paattps://proceeeiinne http///peeeeegg.se/ee/ee/e mlr prrsss/ v1 62/ chan22a. html.
Acknowledgements
This work is supported by the National Key Re- search and Development Program of China under grant 2024YFC3307900; the National Natural Science Founda- tion of China under grants 62376103, 62302184, 62436003 and 62206102: Major Science and Technology Project of Hubei Province under grant 2024BAA008; Hubei Sci- ence and Technology Talent Service Project under grant 2024DJC078; and Ant Group through CCF-Ant Research Fund. The computation is completed in the HPC Platform of Huazhong University of Science and Technology.
Chang, S., Zhang, Y., Yu, M., and Jaakkola, T. S. A game theoretic approach to class-wise selective rationalization. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Process- ing Systems 2019, NeurlPS 2019, December 8 14, 2019, Vancouver, BC, Canada, pp. 10055-10065, 2019.
We sincerely thank the reviewers for their valuable feedhack and efforts during the review process. They have helped lot in improving the quality of this paper. We thank Yuankai Zhang for providing additional results on other baselines.
Chang, S., Zhang, Y., Yu, M., and Jaakkola, T. S. Invariant rationalization. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 1448-1458. PMLR, 2020. URL http: http:///cccceeee..e///e mlr. press/ v119 chang20c. html.
References
Bao, Y., Chang, S., Yu, M., and Barzilay, R. Deriv- ing machine attention from human rationales. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, Brussels, Belgium, October 31- November 4, 2018, pp. 1903-1913. As- sociation for Computational Linguisties, 2018. doi: 10. 10.18665//11//18-12266 URL https:/ /ttp//oooorooooooo org/ 10.18653/v1/113 1216.
Chen, H., He, J., Narasimhan, K., and Chen, D. Can ra- tionalization improve robustness? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, NAACI 2022, Seattle, WA, United States, July 10-15, 2022, pp. 3792-3805. Association for Computational Linguistics, 2022. doi: 10.18653/v1/ 2022.naacl-main.2775 URL ht : //ooi.d///////// org/10. naacl -aa.aacmmmmai....
Bastings, J., Aziz, W., and Titov, I. Interpretable neu- ral predictions with differentiable binary variables. In

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Cho, K., van Merrienboer, B., Gülçehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1724-1734. ACL, 2014. doi: 10.3115/ #1/d14-1179, URL httph: /doi::///doooo wrw.:o..oorr//o wrg/ 100 3115/ v1/d14-1179
intelligence in health care. The Lancet Digital Health, 3 (11):e745-e700, 2021.
Gui, S., Li, X., Wang, L., and Ji, S. GOOD: A graph diitr---istributionn benchmark. In Advances in Neural In- formation Processing Systems 35: Anmual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
Gui, S., Liu, M., Li, X., Luo, Y., and Ji, S. Joint learn- ing of label and environment causal independence for graph nut-of-distribution generalization. arXiv preprint urXiv:2306.00 01 103, 2023.
Cui, W., Jiao, X., Meng, Z., and King, I. Voxeval: Bench- marking the knowledge understanding capabilitiees of end- to-end spoken language models. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), Proceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - Augustt 1, 2025, pp- 16735-16753. Association for Computational Linguis- tics, 2025. URL https: /tcps::/accae http://aaaanntogoyyooo org/ 2025. acl- long. 818/.
Guo, Z., Jin, R., Liu, C., Huang, Y., Shi, D., Supryadi, Yu, L., Liu, Y., Li, J., Xiong, B., and Xiong, D. Evalu- ating large language models: A comprehensive survey. 2023, doi: 10.48550/ARXIV,
CoRR, 15s/2310,19753 2310.19736, URL https: /oii www.l.org/1..4.. 48550/ a arXiv 231 0.3....0 19736.
Hase, P., Zhang, S., Xie. H., and Bansal, M. Leakage- adjusted simulatability: Can models generate non-trivial explanations of their behavior in natural language? In Findings of the Association for Computational Linguis- tics: EMNLP 2020, pp. 4351 2020.
Devlin, J., Chang, M., Lee, K., and Toutanova, K. BERT: pre-training of deep bidirectional transformers for lan- guage understanding. In Proceedings of the 2019 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan guage Technologies, NAACI-HI.T 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pp. 4171-4186, Association for Computational Linguistics, 2019. doi: 10.18653/w1/h19-1444 1423. URL https: / doiiooi.o./// org C 111.////11111/ 9-1423.
Huang, Y., Chen, Y., Du, Y., and Yang. Z. Distribution matching for rationalization. In Thirty- Fifth AAAI Confer- ence on Artificial Intelligence, AAAI 2021, Thirty- -hird Conference on Innovative Applications of Artificial In- telligence, IAAI 2021, The Eleventh Symposium Edu- cutional Advances in Arlificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 13090-13097. AAAI Press, 2021. URL http:::///sccc..c.. / aaii. org/ index. innex.ppp//aeei. AAAI wtt./////aaatteeaeaaaaaaaaeaaae/
DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., and Wallace, B. C. ERASER: bench- mark to evaluate rationalized NLP models. In Pro- ceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, pp. 4443-4458, On- line, July 2020. Association for Computational Linguis- tics. doi: 10. 10.18653//1/2022..cd www.://0...-lmmii...... URL ht https: //aclanthooogge oro. 2020.02200200 acc-mmaigg 408.
Jacovi, A. and Goldberg, Y. Towards faithfully interpretable NLP systems: How should we define and evaluate faith- fulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 4198-4205. Associa- tion for Computational Linguistics, 2020. doi: 10.18653/ V V1/0000.AI--MAIN..86 URL htts:// //oo: /tt..org////oooo/o/oooooooooo 10. 10.18653/v1/20222 .cc1 main..nn...
Diao, X., Zhang, C., Wu, T., Cheng, M., Ouyang, Z., Wu, W., and Gui, J. Learning musical representations for music performance question answering. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024.
Jacovi, A. and Goldberg. Y. Aligning faithful interpre- tations with their social attribution. Transactions of the Association for Computational Linguistics, 9:294 310, 2021. doi: 10.1162/tacl 00367. URL https: //aclanthology wwwp//2g.......... taci-1.18
Diao, X., Zhang, C., Wu, W., Ouyang. Z., Qing, P., Cheng, M., Vosoughi, S., and Gui, J. Temporal working memory: Query-guided segment refinement for enhanced multi- modal understanding. In Findings of the Association for Computational Linguistics: NAACL 2025, 2025.
Jain, S., Wiegreffe, S., Pinter, Y., and Wallace, B. C. Learning to faithfully rationalize by construction. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, ACL 2020, On- line, July 5-10, 2020, pp. 4459 1459-4473. Association for
Ghassemi, M., Oakden- Rayner, L., and Beam, A. L. The false hope of current approaches to explainable artificial
10

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Computational Linguistics, 2020. doi: 10.18653/v1/
fine-tuning dynamics. In International Conference on Machine Learning (ICML., 2025a.
1.000.cc-main400% URL https:/ /tt:://dooo /tt/ orr//0. 18653/v1/2080 acl mocl-me maammmiin.000
Li, S., Luo, X., Wang, H.. Tang, X., Xu, S., Luo, W., Li, Y., He, X., and Li, R. The panaceas for improving low- rank decomposition in communication efficcien federated learning. In International Conference on Machine Learn- ing (ICML), 2025b.
Jang, E., Gu, S., and Poole, B. Categorical reparame- terization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https: / //openreview. net.i/ewwweeeeeeee...e forum? ? wwdenndd=ddrrer.e dd=rkR3y85ee.
Lipton, 7. C. The mythos of model interpretability: In
machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31-57, 2018.
Jiang, Y., Gao, X., Peng, T., Tan, Y., Zhu, X., Zheng. B., and Yue, X. Hiddendetect: Detecting jailbreak attacks against large vision-language models via monitoring hid- den states, 2025. URL https: /tt//aiivv./. orgoorgggaooo abs/ 2502. 4744.
Liu, W., Wang, H., Wang. J., Li, R., Yue, C., and Zhang, Y. FR: Folded rationalization with a unified encoder. In Advances in Neural Information Processing Systems, 2022. URL https: /tpp///eeeeeeeeeeeeeeeeeeeeeeeee net/ forum? id= 2d-2PykBBaKK0..
Kingma, D. P. and Ba, J. Adam: A method for stochastic op- timization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA. USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL ht http: / aariv..iiiv org/ abs/ / 412.1122 6980.
Liu, W. Wang, H.,, Wang. J., Li, R., Li, X., Zhang, Y., and Qiu, MGR: multi-generator based rationaliza- tion. In Rogers, A., Boyd-Graber, J. L., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL. 2023, Toronto, Canada, July 9-14, 2023, pp. 12771-12787. Association for Com- putational Linguistics, 2023a. doi: 10.18653/v1/0223 acl-Iong. 715. URL.ttpss / //dpii.og//....11 v1/2023. acl-long. 715.
Lei, T., Barzilay, R., and Jaakkola, T. S. Rationalizing neural predictions. In Proceedings of the 2016 Confer- ence on Empirical Methods in Natural Language Process- ing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pp. 107-117. The Association for Computational Linguistics, 2016. doi: 10. 12.18653/1/d11-111 1011. URL ht https: // /tt.//oiiio/ wttp://.......111/e/. 21105//3/11/16-11111101
Liu, W., Wang. J., Wang, H., Li, R., Deng, Z., Zhang, Y., and Qiu, Y. D-separation for causal eelf-explanation. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023b.
Li, S.. Guo, H., Hou, L.. Zhang, W., Tang. X., Tang, R., Zhang, R., and Li, R. Adaptive low-precision train- ing for embeddings in thick-through rate prediction. In Thirty-Seventh AAAI Conference on A Artificial Intelligence (AAAI), Washington, DC, USA, 2023.
Li, S., Cheng, Y., Wang, H., Tang, X., Xu, S., Luo, W., Li, Y., Liu, D., He, X., and Li, R. Masked random noise for communication-effiee efffrcinnt federated learning. In ACM International Conference on Multimedia (ACM MM), 2024a.
Liu, W. Wang, J., Wang, H., Li, R., Qiu. Y.. Zhang, Y., Han, J., and Zou, Y. Decoupled rationalization with asymmetric learning rates: A flexible lipschitz restraint. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Min- ing. KDD 2023, Long Beach, CA, USA, August 6-10, 2023, pp. 1535-1547. ACM, 2023c. doi: 10.1145/ 3580305.3599299. URL attps:///ol: /ttp//oi.ooo.....///// 1145/ 3055/55803055 35992999
Li, S., Guo, H., Tang, X., Tang, R., Hou, L., Li, R., and Zhang, R. Embedding compression in recommender systems: A survey. ACM Comput. Surv. 56(5):130:1- 130:21, 2024b.
Liu, W., Deng, 7., Niu, 7., Wang, J., Wang, H., Zhang, Y., and Li, R. Is the mmi criterion necessary for interpretabil- ity? degenerating non-causal features to plain noise for self-rationalizztion. The Thirty-eighth Annual Confer- ence on Neural Information Processing Systems, 2024a.
Li, S., Xu, W., Wang, H., Tang, X., Qi, Y., Xu, S., Luo, W., Li, Y., He, X., and Li, R. Fedbat: Communication- efficient federated learning via learnable binarization. In International Conference on Machine Learning (ICML), 2024c.
Liu, W., Wang, H., Wang, J., Deng, Z., Zhang, Y., Wang, C., and Li, R. Enhancing the rationale-input align- ment for self-explaining rationalization. In 40th IEEE International Conference on Data Engineering, ICDE
Li, S., Luo, X., Tang, X., Wang, H., Chen, H., Luo, W., Li, Y., Hc, X., and Li, R. Beyond zero initialization: Investigating the impact of non-zero initialization on lora

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
2024, Utrecht, The Netherlands, May 13-16, 2024, pp. 2218-2230 IEEE, 2024b. doi: 10.1 10.1109/1CDE60144 2024.00176. URL https://doli wtw.ii.oorg...10000 1109/ ICDE60146. 2024.00166.
Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 28701-28713, 2021.
Pradhan, R., Zhu, J., Glavic, B., and Salimi, B. In- terpretable data-based explanations for fairness debug- ging. In SIGMOD '22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 17, 2022, pp. 247-261. ACM, 2022. doi: 10. 1145/3514221.35178866 URL https:://dii..... /rr// 10. 10.1145/351422 3517886.
Liu, W., Deng, Z., Niu, Z., Wang, J., Wang. H., Zeng, Z., and Li, R. Breaking free from MMI: A new frontier in ratio- nalization by probing imput utilization. In The Thirteenth International Conference on Learning Representations, 2025. URL https:// http//ppperreeeeee ntt/ 1d=
Luo, D., Cheng, W., Xu, D., Yu, W., Zong, B., Chen, H., and Zhang, X. Parameterized explainer for graph neural network. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
Rajagopal, D., Balachandran, V., Hovy, E. H., and Tsvetkov, Y. SELFEXPLAIN: A self-explaining architecture for neural text classifiers. In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Pro- cessing, pp. 836-850, Online and Punta Cana, Domini- can Republic, November 2021. tional Linguisties. doi: Association for Computa- 64. URL https: hat//aclaanttccc org/ 20211 emn mornpp-mas in. 64.
McAuley, J. J., Leskovec, J., and Jurafsky, D. Learn- ing attitudes and attributes from multi-aspect reviews. In 12th IEEE International Conference Data Min- ing, ICDM 2012, Brussels, Belgium, December 10-13, 2012, pp. 1020-1025. IEEE Computer Society, 2012. doi: 00.1009/CDDM2201..100. URL attps://ddeee https://ddoooogg// 10.1109/ ICMM 2012. 110.
Ren, Q. Gao, J., Shen, W., and Zhang, Q. Where we have arrived in proving the emergence of sparse interaction primitives in DNNs. In The Twelfth International Confer- ence Learning Representations, 2024. URL https: / ///ppneeeiew..rc forummrruimoo wpw.:ummmdddppp-ddeemep 6B.
Pagliardini, M., Jaggi, M., Fleuret, F., and Karimireddy, S. P. Agree to disagree: Diversity through disagree- ment for better transferability. In The Eleventh In- ternational Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenRe- view.net, 2023, URL https:://///// /tttps://opprreevieeee .ntt/ Bafrid=K7CbY(by/Xn
Rudin, C. Stop explaining black box machine learning mod- els for high stakes decisions and use interpretable mod- els instead. Nature Machine Intelligence, 1(5):206-215, 2019. doi: 10. (0010088//44255-119990000 / / doi wrg/10. 1038 442256 0119-0448 x.
URL https:
Paranjape, B., Joshi, M., Thickstun, J., Hajishirzi, H., and Zettlemoyer, L. An information bottleneck approach for controlling conciseness in rationale extraction. In Proceedings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing, EMNLP 2020, On- line, November 16-20, 2020, pp. 1938-1952. Association for Computational Linguistics, 2020. doi: 10.18653/v1/ 2020. 2020.emnlp-main. 153.. https::s/// //doi.///ii////o aecai.oorg orr/o..rog/...... 18653/v1 20055////222222/// www.mmmmmm...m...mmm 153.
Seiler, B. B. Applications of Cooperative Game Theory to Interpretable Machine Learning. PhD thesis, Stanford University, 2023.
Sha, L., Camburu, O., and Lukasiewicz, T. Learning from the best: Rationalizing predictions by adversarial infor- mation calibration. In Thirty- Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Con- ference Innovative Applications of Artificial Intelli- gence, IAAI 2021, The Eleventh Symposium on Fduca- tional Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pp. 13771-13779, AAAI Press, 2021. URL /ttLiittts///jjsaaaaaa index, wndex.php///e. AAAT /At.A///aaarccceeeeeeee//.....
Pennington, J., Socher, R., and Manning, C. D. Glove: Global vectors for word representation. In Proceed- ings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Oc- tober 25-29, 2014, Doha, Qatar, A meeting of SIG- DAT, a Special Interest Group of the ACI. pp. 1532- 1543. ACL, 2014. doi: 10.3115//1/414-11622 URL https: //doi.
Storek, A., Subbiah, M., and McKeown, K. R. Un- supervised selective rationalization with noise injec- tion. In Proceedings of the 61sl Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 12647-12659. Association for Com- putational Linguisties, 2023. doi: 10. 10.1886531//023.
10. w1tp//..111/111111-111111
Plyler, M., Green, M., and Chi, M. Making a (counter- factual) difference one rationale at a time. In Advances in Neural Information Processing Systems 34: Annual
12

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
acl-long.707. URWhtttt:t/ttttttttt //t::///////ioe/ http://di..rg///11/1//1 v1
doi: 10.48550AARXXIV.55060000000 URL https: /
707.
doi / 259..00955..0...
Sun, L., Huang, Y., Wang. H., Wu, S., Zhang, Q. Gao, C., Huang, Y. Lyu, W., Zhang, Y., Li, X., Liu, Z., Liu, Y., Wang. Y., Zhang, Z., Kailkhura, B., Xiong, C., Zhang, C., Xiao, C., Li, C., Xing, E. P., Huang, F., Liu, H., Ji, H., Wang, H., Zhang, H., Yao, H., Kellis, M., Zitnik, M., Jiang, M., Bansal, M., Zou, J., Pei, J., Liu, J., Gao, J., Han, J., Zhao, J., Tang, J., Wang, J., Mitchell, J., Shu, K., Xu, K., Chang, K., He, L.., Huang, I.., Backes, M., Gong, N. Z., Yu, P. S., Chen, P., Gu, Q. Xu, R., Ying. R., Ji. S., Jana, S., Chen, T., Liu, T., Zhou, T., Wang. W., Li, X., Zhang, X., Wang, X., Xie, X., Chen, X., Wang, X., Liu, Y., Ye. Y., Cao, Y., and Zhao, Y. Trustllm: Trustworthi- ness in large language models. CoRR, abs/2401 0bs/2200.05561, 2024a. doi: 10.48550/ARXIV.2200055555
Wu, Y., Wang, X., Zhang, A., He, X., and Chua, T. Discov- ering invariant rationales for graph neural networks. In The Tenth International Conference on Learning Repre- sentations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL net fet//form?20 b i jrr
Xia, M., Malladi, S., Gururangan, S., Arora, S., and Chen, D. LESS: selecting influential data for targeted instruction tuning. CoRR, abs/2402.044 004333, 2024. doi: 10.48550/ ARXIV.2402.04333. URL https: 48550/arXiv. 2402 04333.
Yang, J., Tang, A., Zhu, D., Chen, Z., Shen, L., and Wu, F. Mitigating the backdoor effect for multi-task model merging via safety-aware subspace. arXiv preprint arXiv:2410. 00w.2410.1391000 2024.
URL ht tps: 48550/ /at/55///aiiiii arii.. www.aaiwwi............ 2401. 05561.
//do// //doi. org 100 wttp:gg.....88888
Sun, R., Zheng. Z., and Wang. Z. Learning encodings for
constructive neural combinatorial optimization needs to regret. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 38, pp. 20803-208 20801, 2024b.
Yang. J., Chen, Z.,, Xiao, T., Lin, Y., Zhang, W., and Kuang, K. Leveraging invariant principle for heterophilic graph structure distribution shifts. In Proceedings of the ACM on Web Conference 2025, pp. 1 1196-1 1204, 2025.
Tong, J., Zhou, H., Liu, Y., Hu, Y., and Zou, Y. Dynamic knowledge adapter with probabilistic calibration for gen- eralized few-shot semantic segmentation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 2781-2790, 2024a.
Ying, Z., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J. Gnnexplainer: Generating explanations for graph neural networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8- 8-14, 2019, Vancouver, BC, Canada, pp. 9240-9251, 2019.
Tong, J.. Zou, Y., Li, Y., and Li, R. Lightweight frequency masker for cross-domate few-shot semantic segmentation. Advances in Neural Information Processing Systems, 37: 96728-96749, 2024b.
Yu, M., Chang, S., Zhang. Y., and Jaakkola, T. S. Re- thinking cooperative rationalization: Introspective ex- traction and complement control. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP- IJCNLP 2019, Hong Kong. China, November 3-7, 2019, pp. 4092-4101. Association for Computational Linguis- tics, 2019. doi: 10.18653/ 1420. URL https: / ///oi. //doi.oog//00 100rg/10...
Tong, J.. Jin, W., Qin, P., A., Zou, Y., Li, Y., Li, Y., and Li, R. Flowcut: Rethinking redundancy via information flow for efficient uision-languae models. arXiv preprint arXiv:2505. / 19536, 2025a.
Tong, J., Ma, R., Zou, Y., Chen, G., Li, Y., and Li, R. Adapter naturally serves as decoupler for cross- domain few-shot semantic segmentation. arXiv preprint WrXiv:2506.07376 2025b.
Yu, M., Zhang, Y., Chang, S., and Jaakkola, T. S. Under- standing interlocking dynamics of cooperative rational- ization. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 12822-12835, 2021.
Wang, H., Lu, Y., and Zhai, C. Latent aspect rating analy- sis review text data: rating regression approach. In Proceedings of the 16th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, Wash- ington, DC, USA, July 25-28, 2010. pp. 783-792. ACM, 2010. doi: 10.1145/1835804 18355000...1 URI. https: //dol. . org.
Yu, Y. Wang, Y., Yang, W., Guo, L., Lu, S., Duan, L., Tan, Y., and Kot, A. C. Robust and transferable backdoor attacks against deep image compression with selective frequency prior. IEEE Trans. Pattern Anal. Mach. Intell., 4773):1774-1693, 2025a. doi: 10.1 TPAML.202435578873 URL https: /oii.ii.oo// doi..org/0.. 1109/TPAMT. 2024.3507873
10. 110
1835903
Wang, Q. Meng, Z., Cui, W., Zhang, Y., Wu, P., Wu, B., King, I., Chen, L., and Zhao, P. NTPP: generative speech language modeling for dual-channel spoken dialogue via next-token-palt prediction. CoRR, abs/2506.00975. 2025.
13

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Yu, Y., Xia, S., Lin, X., Yang. W., Lu, S., Tan, Y., and Kot, A. C. Backdoor attacks against no-reference im- age quality assessment models via a scalable trigger. In Walsh, T., Shah, J., and Kolter, Z. (eds.), AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 March 4, 2025, Philadelphia, PA, USA, pp. 1698-9706 AAAI Press, 2025b. doi: 10.1609/AAAL V399. 13051.. URL https: //doi. org 110 609 / aaai, /a.00///aaa//
Zheng, Z., Yao, S., Wang. Z., Tong, X., Yuan, M., and Tang, Decoupling partition and navigation for neu- ral solvers of min-max vehicle routing problems. arXiv preprint arXiv22405.17272 2024a.
K. Dpn:
Zheng, Z., Zhou, C., Xialiang, T., Yuan, M., and Wang, Z. Ude: A unified neural divide-and-conquer framework for large-scale combinatorial optimization problems. Ad- vances in Neural Information Processing Systems, 37: 6081-6125, 2024b.
Yuan, H., Cai, L., Hu, X., Wang. J., and Ji, S. Interpreting image classifiers by generating discrete masks. IEEE Trans. Pattern Anal. Mach. Intell., 44(4)2211,, 2022. doi: 00000//TPAMM22200000222777777771 URL https: (ttpg//...10//PPPP. 2020 3028783.
Zheng, Z., Xie, Z., Wang. Z., and Hooi, B. Monte carlo tree search for comprehensive exploration in Ilm-based auto- matic heuristic design. arXiv preprint arXiv:2501. 008603,,, 2025.
Zhou, C., Jiang, R., Luan, F., Meng, S., Wang, Z., Dong, Y.. Zhou, Y., and He, B. Dual-arm robotic fabric ma- nipulation with quasi-static and dynamic primitives for rapid garment flattening. JEEE/ASME Transactions on Mechatronics, pp. 1-11, 2025. doi: 10.1109/TMECH. 2025.3556283.
Yue, L., Liu, Q. Wang, L., An, Y., Du, Y., and Huang, Z. Interventional rationalization. In Proceedings of the 2016 Conference Empirical Methods in Natural Lan- guage Processing, EMNLP 2023, Singapore, December 6 -10, 2023, 2023. URL https: /ttp///peee.eeeeeeeee forum? 1 KoFa 6611
Zhang, W., Wu, T., Wang, Y., Cai, Y., and Cai, H. Towards trustworthy explanation: On causal rationalization. In Proceedings of the 40th International Conference on Ma- chine Learning (CCML'23), volume 202 of Proceedings of Machine Learning Research, pp. 41715-41736. PMLR, 23-29 Jul 2023. URL https: / arxiv. org/ abs/ 2306. 14115.
Zhao, W. X., Zhou, K., J., Tang. T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z.,, Du, Y., Yang,
C., Chen, Y.. Chen, Z., Jiang, J., Ren, R., Li, Y., Tang,
X., Liu, Z., Liu, P., Nie, J., and Wen, J. A survey of large language models. CoRR, 0as/2203.18225, 2023. doi: 10448550AARXIV22333.1222 doi.orgg/0.
URL https:/ // 18221.
Zheng, Y., Booth, S., Shah, J., and Zhou, Y. The irrational- ity of neural rationale models. In Proceedings of the 2nd Workshop on Trustworthy Natural Language Pro- cessing (TrustNLP 2022), pp. 64-73, Seattle, U.S.A., July 2022. Association for Computational Linguistics. doi: 10.18655w1120022tmmmslp// URL https:// aclanthology hwwp://wo.ooy...oo//22/2222 wttp:///........ssas............
Zheng, Z. and Lee, W. S. Reasoning-cv: Fine-tuning power- ful reasoning llms for knowledge-assiseee claim verifica- tion. arXiv preprint urXiv:2505. 12348, 2025.
Zheng, Z., Yao, S., Li, G., Han, L., and Wang, Z. Pareto improver: Learning improvement heuristics for multi- objective route planning. IEEE Transactions on Intelli- gent Transportation Systems, 25(1):1033-1043, 2023.
14

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datascts
A. Eaxamples and implementations details
X
I want hotel yeslerday, whose service excellent.
want to hotel yesterday, whose service is awwful.
X
A.1. The potential impact of rationalization in the era of LLMs
Generator
Attacker
In comparison to traditional "model-centric" XAI meth- ods which solely foeus on the model'ss learned information, "data-centric" approaches primarily aim to extract model- agnostic patterns inherent in the data. So, apart from improv- ing interpretability. rationalization can serve as method of data cleaning (Seiler, 2023).
Z
Za
Predictor
Predictor
P=1
Y, =1
Domain-specific large models often require supervised fine- tuning using domain-specific data. Uncleaned data may contain harmful information such biases and stereotypes (Sun et al., 2024a). Recent research suggests that training predictors with extracted rationales can remove irrelevant harmful information, enhancing robustness (Chen et al., 2022) and generalization (Wu et al., 2022; Gui et al., 2023).
Figure 7. An example of how the attacker works. X), X repre- sent positive and negative texts.
label 0 to 1. Achieving this objective is straightforward: Io simply needs to mimic g by selecting "." as ZA. This suggests that if identifies Z from X as a trivial pattern also present in X", then Iu can effortlessly select Z = Z from X", leading to an easy flip of the prediction label of ZA to 1 in predictor p. On the other hand, if Z is a genuine positive rationale unique to X and the predictor p classifies it correctly, then Ja would be unable to find positive rationale from the negative input X". Therefore, it is difficult for the predictor p to flip Za's 's label from O to 1. Thus, we can leverage the attack generator go to assist in inspecting and identifying sampling bias. Ia may easily find a Za that flips its predicted label in predictor p from its actual label, indicating the presence of semantically unrelated trivial patterns in Z.
Since LLMS are usually pretrained on various datasets, they tend to be less controllable than small models (Zhao al., 2023). Considering that for simple tasks (such as text clas- sification), small models are also capable and can achieve satisfactory results, we can train a separate rationalization model for a single domain-specific dataset. Small models trained on a single dataset are often more controllable and save computational resources (such as searching for hyper- parameters and adding regularization terms) (Guo et al., 2023). Then using the extracted rationales for supervised fine-tuning might prevent large models from learning harm- ful information from new data. Additionally, shortening input texts can also reduce the memory required for fine- tuning.
To further address this issue, we propose method to in- struct the game on better decorrelation. As illustrated by the previous example, when there is a sampling bias issue, the attack generator Ja surely selects a Z that is a trivial pattern lacking semantic significance. For a reasonable pre- dictor p that can accurately classify the real rationale, ZA akin noise, and its classification result should be random introduce
Some recent studies has also found that training a small model for data selection (although not the same as rationale selection) and producing a small subset is useful for fine- tuning LLMs (Xia et al., 2024; Bi et al., 2024).
A.2. A toy example for a more intuitive understanding of the proposed method
and not biased towards any label. Therefore, constraint on the predictor p to guide it, ensuring that the classification result for ZA remains as random as possible. This constraint serves as an ongoing guidance to adjust and correct the behavior of predictor p. An improved predictor p can, in turn, better instruct and guide the updates for the generator g.
Firstly, to inspect and identify the correlations, we introduce an attack generator Ia- Figure 7 shows an example of how the attacker works (formal analysis is in 84.3).
Example 2: the optimization objective of ga is to select an attack rationale ZA from input such that, when ZA is fed into the same predictor p. yields a prediction label flipped from its original label. Continuing the previous example in Figure 2, the generator g selects the "" from positive input X with label 1 as Z. Consequently, the predictor p learns to treat the presence of "." in Z as an indicative feature for positive classification. On the other hand, the goal of 9a is to select an attack rationale ZA from a negative input x with a label 0 in such a way that, when ZA is fed to the same predictor p. the prediction result flips from its original
A.3. Implementation details of Equation (6) and (7)
For batch of (X,Y), we first send X to both the generator and the attacker and get Z, ZA:
Z = fx))
(12)
ZA = fx()).
15

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Table 5. Statistics of datasets used in this paper
Train
Dev Neg
Annotation
Datasets
Pos
Neg
avg_len
Pos
avg_len
Pos
Neg
avg_
Appearance
16891
16891
141
6628
2103
145
923
13
126
18.5
Beer
Aroma
15169
15169
144
6579
2218
147
848
29
127
15.6
Palate
13652
13652
147
6740
2000
149
785
20
128
12.4
Location
7236
7236
151
906
906
152
104
96
155
8.5
Hotel
Service
50742
50742
154
6344
6344
153
101
152
11.5
Cleanliness
75049
75049
144
9382
9382
144
99
101
147
8.9
Then, we get a copy of A with the pytorch function "torch.detachh)":
sets for evaluation. But the training sets have only the clas- sification labels and models are trained extract rationales in unsupervised way.
N forch.detach
(13) For the three beer-related datasets, users need to consult the original authors (McAuley et al., 2012) for permission first.
Then we get y and f4=
The statistics of the datasets are in Table 5. Pos and Neg denote the number of positive and negative examples in each set. denotes the average percentage of tokens in human- annotated rationales to the whole texts. avg_len denotes the average length of text sequence.
y = fz(
(14)
- 12422)
Then update the generator and the predictor with
min H.(Y, min ((.,,,,, 8%)
(15)
Note that there are two versions of the BeerAdvocate bench- mark. The raw datasets in the original BeerAdvocate contain many spurious correlations. However, as we are investigat- ing the model-added spurious correlations clean datasets, we follow FR to use the version where the inherent spurious correlations in the datasets have been manually cleaned by (Lei et al., 2016).
0g
Note that this updating process will not influence the at- lacker, since we have used "torch.detachO"" for ZA.
Then, we fix the parameters of the generator and the predic- tor, and only update the attacker. We get YA with
Yx= ff())))
(16) For the graph classification dataset BA2Motif, we do node level selection on it. That is to say, we select several nodes from graph to form a subgraph to serve as the rationale.
Then, we update the attacker with
min H² - Y,)
(17)
A.S. Implementation details
8,
We keep the major settings consistent with Inter_RAT and FR, which are commonly utilized in the field of rationaliza- tion (Chang et al., 2020; Yu et al., 2021; Liu al., 2022; Yue et al., 2023). Specifically, we employ the 100-dimensional Glo Ve (Pennington al., 2014) for word embedding and 200-dimensional GRUs (Cho et al., 2014) to obtain text representation. The re- parameterrization trick for binarized selection is Gumbel-softmax (Jang et al., 2017). Then, we also follow CR to conduct experiments that replace GRUs with pretrained BERT (Devlin al., 2019) ("bert-based- uncased") and compare with some recent methods that have already been implemented with BERT as supplement. The random seed is kept the same (the seed is 12252018, in- herited from the code of FR) across all the experiments on text classification, as we think experiments with multiple datasets and multiple sparsity settings (totally 12 settings in
Then, we get into the next round to update the generator and the predictor again.
A.4. Datasets
We employ six widely used text classification datasets collected from two rationalization benchmarks. Beer- Appearance, Beer-Aroma, Beer-Palate (which discuss the appearance, aroma, and palate beer, respectively. They are from the BeerAdvocate (McAuley al., 2012) bench- mark), Hotel-Location, Hotel-Service, (which discuss the location, service, and cleanliness of ho- tels, respectively. They are from the HotelReviews (Wang et al., 2010) benchmark). Among them, the beer-related datasets are most important and used by nearly all of previ- ous research in the field of rationalization. These datasets have numan-annotate ground-truth rationales on the test
16

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
1.0
1.0
1.0
RNP
RNP
RNP
0.9
RNP+A21
0.9
RNP+AA
0.9
RNP+A21
0.7
0.5
0.4
0.4
0.4
0.3
100
150
200
250
300
0.3
100
200
250
300
0.3
100
125
150
175
epochs
spects
epochs
(a) Appearance
(b) Aroma
(c) Palate
1.0
1.0
1.0
0.9
RNP+A21
0.9
RNP+AZ1
0.9-
RNP+A21
0.8
0.8
0.8
0.7
0.7-
0.7
0.4
0.3
300
125
(d) Appearance
(e) Aroma
(i) Palate
Figure& 8. Attack success rate (ASR) on the three beer-related datasets. The rationale sparsity is about 10% (a.bc)) and 30% (d.e.).
Table I and 2) under the same random seed are sufficient to verify the significance of improvement. For the BA2Motifs, we use a two-layer GCN to replace GRUs. The training of GCN is not as stable as GRUs, we report the average results of five random seeds.
used its original hyperparameters for it.
For CR, we just keep the major settings ("bert-base- uncased", the Beer-Appearance dataset, and the sprasity of 10%, removing the coherence regularizer) the same as and copy its results from its original paper.
Because Inter_RAT, NIR, and CR are methods specifi- cally designed for text tasks and are not suitable for graph tasks, we only compare our A2I with RNP and FR the BA2Motifs dataset.
Compactness and coherence. For the compactness and coherence regularizers introduced in Equation (4), we adopt both compactness and coherence regularizers to the genera- tor to make the selected rationales numan--ntellligibn apply a compactness regularizer term to the attacker to make the attack rationale more similar to the original rationale, thus making easier to deceive the predictor. However, we do not employ coherence regularizer on it because we think trivial patterns often discontinuous.
We
The maximum sequence length is set to 256. We use the Adam optimizer (Kingma & Ba, 2015) with its default pa- rameters, except for the learning rate (the learning rate is 0.0001). The temperature for gumbel-softmax is the de- fault value 1. We implement the code with Pytorch on RTX3090 GPU.
Hyperparameters. For all datasets, we use learning rate of 0.0001. It is found by manually tune the standard RNP and are applied to both NIR, FR, our A2I, as they are all variants RNP. The core idea of NIR is to inject noise into the selected rationales. We use RNP as its backbone. A unique hyperparameter of NIR is the proportion of noise. Following the method in the original paper, we searched within 10.1, 0.2, 0.3] and found that 0.1 yielded the best results on most datasets, hence we adopted 0.1 for it. We found that the training of Inter_RAT is very unstable. To avoid potential unfair factors, our main settings are deter- mined with reference to it. Except for the part about sparsity,
B. More results
B.1. More results about the attack success rate
More results of the attack success rate are shown in Figure 8.
B.2. More results corresponding to Figure 3
Figure 3 has shown the results of one aspect of the BeerAd- vocate dataset. We show the results of the other two aspects in Figure and 10. The green lines can somewhat reflect how much the true sentiment contained in the randomly selected rationales. And we see that only the true sentiment
17

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
can generalize to the validation set.
We consider a scenario starting with fp(r.) 0.5, meaning the predictor is unable to classify using the correct ratio- nale, and we examine in which direction the predictor will converge under these circumstances.
C. Technical proofs
C.1. Derivation of Equation (9)
aL Clearly, when fp(r+) 0.5, af,(r4) < 0, meaning that the predictor will learn increase fp((+) get lower L. So the predictor will learn to predict positive.
To begin with, we need to introduce two fundamental prop- erties from probability theory.
The first property a general property for conditional prob- ability. If 0 < P(Y 1) < 1, then for vp, if < p < 1, we always find a variable c, such that P(Y 1/c) p.
So, when will it converge? We denote Pr(r+/Y = 1) Pi and Pr(r.IV= = =0 P2. From (20), we have
AL
P3 2P, + 2P
Considering our rationalization situation, we can get the following corollary:
< <0, s.t., fp(r+) <1 (fr(r.)
(21)
Corollary C.1. we can construct G in an arbitrary way,
From Assumption 4.1, we have P1 2 P2. So, we know that
and 0 P(Y= 1|Z = 1, then we have
we will have [p(r+) 2 0.75 when the predictor converges OL
(i.e., ()((1.) 0).
v0 < < 1, 3ga G, P(Y 1|2 tt, ga) = pp. (18)
That means even in the worst case, the predictor can still predict r,, as positive.
The second property is also a general property for condi- tional probability. If P(Y 1) = O, then for any variable we always have P(Y= = 1/c) = 0. This is also a fundamental property in probability theory.
Qualitative analysis Actual training would be easier be- cause, in the above discussion, we do not differentiate be- tween positive sentiment appearing in positive class texts and positive sentiment appearing in negative class texts. In reality, although both are denoted as r+, they are somewhat distinct.
Considering the rationalization situation, let Z = r.,, we have
Corollary C.2. If we construct G in arbitrary way, and P(Y = 0 r+) = O, then we have that there is no 9a 6 G that can make P(Y = 1|2 = r..ga) > O.
Here some practical scenarios where text contains both positive and negative sentiments.
C.2. The convergence of Equation (7)
First, the X labelled with Y = 1 may be a combination of strong positive sentiment and weak negative sentiment. A dataset may consists of two kind of sentiment: strong and weak, each of which can be divided to positive and negative. The label of X is decided by the strong sentiment. In this scenario, the attacker may find the weak negative sentiment from X labelled with Y = 1, and ask the predictor to classify the weak negative sentiment as neutral. If weak sentiment and strong sentiment have different styles, the attacker here still helps the predictor to focus on strong sentiment and ignore the weak sentiment. As a result, the generator will only select the strong sentiment.
Quantitative analysis We consider the scenario where the model is functioning correctly, meaning the predictor classi- fies according to the true rationale R. The generator extracts r, from X2, while the attacker extracts r, from X (X",, X1 denote texts with negative and positive labels, respectively). In the dataset D, we consider the numbers of positive and negative texts are both
We only consider 7 r+, and Z r- is nothing different.
We rewrite (7) as (L is the loss function and fr.) is the confidence level of predicting T. as positive)
Second, the sentiment may be multi-aspect. For example, person may have positive sentiment about the beer's ap- pearance, while negative sentiment about the taste. If we are discussing the beer's appearance, the text will still be annotated as positive. such a scenario, the attacker will try find the negative comment about the taste, and force the predictor to classify as neutral. However, this is just what we want. It helps the predictor focus not only on the vanilla sentiment, but also on the aspect (which is included in the context of the sentiment) in which we are interested. Since the predictor classifies the comment about the taste as neutral, it will give the only the feedback about the beer's 's (20) appearance, which can help the generator focus more on the
L Σ 1 log
X=IXX
Σ (,500g (r(x)
(19)
x-0,x
1og(1-ff(g ff(r)))
0L
-n * Pr(r. Y = 1) 0.5n Pr(r, / = 0)
(r,)
fr()
0.5m * Pr(r. Y 0)
1r(p
18

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
1.0
1.0
0.9
0.9
0.8
0.8
Acc
0.7
0.7
text
full text
random patterns
random patterns
0.5
r2f
0.4
50
100
150
300
0.4
50
100
150 epochs
200
epochs
(a) Training
(b) Validation
Figure 9. Experiments on the Appearance of the BeerAdvocate dataset. The settings are the same as those in Figure 3.
1.0
1.0
0.9
0.9
0.8
0.8
Acc
0.7
0.7
text random patterns
full text
random patterns
r2f
r2f
0.4
0.4
0
50
100
150
200
250
300
50
100
150
200
250
300
epochs
epochs
(a) Training
(b) Validation
Figure 10. Experiments on the Palate of the BeerAdvocate dataset. The settings are the same as those in Figure 3.
appearance.
D. The rationales extracted by llama-3.1-D-instruuc
The above intuitive analysis is somewhat supported by the empirical results in Figure For RNP+A2I, the attack success rate is about 50%, meaning random classification of A. This suggests that the predictor does not predict the Za extracted by the attacker to the target class.
To further show the potential impact of rationalization in the era of LLMs, here we present the results of the experi- ments conducted with the Ilama-3.1 1 1-8b-instruct model. We perform both 2-shot prompting and supervised fine-tuning.
For 2-shot prompting, provide the model with negative text with its corresponding rationale, and positive text with its corresponding rationale. For supervised fine-tuning, the supervison label is the classification label, since we perform unsupervised rationale extraction. We use 4*RTX 4090 24GB GPUs and LoRA fine tune the models.
C.3. The minimum cross-entropy equal entropy
EL0.:1))-H(YV0)-xxx.PXYXY2011122 (22)
In most cases, the model can output the rationale in the correct format. Figure 11 shows example. But in 2-shot prompting, the model sometimes outputs additional parts along with the rationale (through manual observation, this sitation does not occur frequently.). Figure 12 is another example. In such cases, we use gpt-3.5- to extract the content within the quotation marks.
We have DKL DKL(P(Y (P(()2)))))(()))) 0 with the equality holds and only (( P(Ý]z). As a result, we have
min =
(23)
19

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Table The comparison between A2I-based methods (implemented with GRUs, which corresponds the results in Table and representative LLM Ilama-3. 1amma-..--b-instteee --8binnstrutt. The bold results means the situations where A2I-based methods outperforms llama (in terms Fl score).
(a)
Results on datasets from the BeerAdvocate benchmark.
Datasets
Pre--appeeeece
Beer-Aroma
Peer-Palate
Methods
S n/a n/a
R
F1
S
P
R
F1
P
R
F1
Ilama (finetune)
86.3
46.2
60.2
n/a
73.2
50.6
59.8
n/a
61.7
42.6
50.4
Ilama (2 shot)
15.4
16.0
15.7
n/a
17.9
24.2
20.6
n/a
13.0
22.2
16.4
RNP+A2I
10.8
78.3
45.8
57.8
9.8
86.0
54.3
66.6
10.9
66.3
58.2
62.0
FR+A2I
11.3
76.0
46.5
57.7
10.0
85.7
54.8
66.9
9.7
71.4
55.8
62.6
RNP+A2I
20.0
73.3
79.4
76.2
19.5
49.0
61.4
54.5
19.4
49.0
76.4
59.7
FR+A2I
19.8
80.0
85.6
82.7
19.4
64.2
80.0
71.2
19.2
44.2
68.2
53.7
RNP+A2I
29.9
59.3
95.9
73.3
27.8
44.5
79,3
57.0
30.5
30.8
75.5
43.7
FR+A2I
28.8
61.3
95.3
74.6
30.9
41,4
82.2
55.1
29.1
31.6
73.8
44.2
(b) Results on datasets from the HotelReview benchmark. Hltel-SService
Datasets
Hotel-Location
Hotel-Cleanliness
Methods
R
F1
S
P
F1
P
R
F1
Ilama-3. 1.1 --b (finetune) 1lama-3.1-88 (2 shot)
n/a
58.6
39.0
46.8
n/a
77.3
40.6
53.3
n/a
54.9
31.3
39.9
n/a
45.8
59.1
51.6
n/a
45.3
51.7
48.3
n/a
39.3
43.0
41.1
RNP+A2I
9.0
50.2
53.4
51.7
11.6
46.8
47.4
47.1
9.7
34.7
38.2
36.4
FR+A2I
9.9
53.2
62.1
57.3
11.5
47.7
47.7
47.7
10.8
35.9
43.7
39.4
We found that there significant likelihood that responses from Llama would not contain explicit "positive/negative" classifications (for instance, in the hotel-location dataset, 50.5% of responses from the fimaa-fnetune model did not specify a category, and this figure was 53.0% for hotel- cleanliness dataset). we were to consider these responses without clear categories as incorrect predictions, the accu- racy would appear very low and could be misleading. Con- sidering that main comparison is the rationale quality and we do not claim our classification ability is superior to LLMs, hence we do not report this prediction accuracy to avoid unnecessary confusion.
Task: Sentiment classification about Beer's appearance Input: Pours a rather crisp yellow almost orange with thin head. The aroma dominated by sweet malts with just slight hoppiness dancing in the background. The taste does have a surprising amount of hoppiness for a Pilsner There is good maltiness to as well, but citrus hops just slightly overpe everpower. The beer very light refreshing. This makes excellent summer session beer.
Expected output: |pours rather crisp yellow almost orange with thin head.
The results are shown in Table 6. LLMs are not good at counting, so we did not constrain the percentage length (i.e., sparsity) of the rationale extracted by the model. We do not report the prediction accuracy for llama because find that there is significant likelihood that responses from Llama would not contain explicit "positive/negative" classi- fications (for instance, the hotel-service dataset, 50.5% of responses from the tima-ffnetune model did not specify category, and this figure was 53.0% for hotel-cleanlines: dataset). If we were consider these responses without clear categories incorrect predictions, the accuracy would appear very low and could be misleading. Considering that
Ilama-3.1 3.1 output: 1 pours rather crisp yellow almost orange
Figure 11. An example of Ilama's output. Here "1" means that the class label Y is positive. And the words after "I" represent the rationale.
20

---

Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
growing importance and consider it promising direction for future extensions of our method. As deep learning mod- els continue to increase in scale, the need for techniques such as model compression (Li et al., 2023; 2024b), com- munication compression (Li et al., 2024c;a; 2025b), and efficient training (Li et al., 2025a) has become increasingly critical.
Sentiment classification about Beer's appcarance Input: 22oz bottle pouted into Goblet: Opaque orange with light, white, creamy head that was not all that well retained but full of carbonation, but did settle into a small thin cap. The aroma was more Belgian triple than IPA, Sweet and malty. The taste is very nice balance of the two styles. A little more hops, but balanced very nice with the sweetness of the malt and fruit. The beer had medium to full body, perhaps little too thick for my taste, but still good. The beer had nice bitter dry aftertaste and was well carbonated. The beer was fairly easy to drink give the A BV, but after the 22oz, I was pretty well done. Overall, good beer and probably the first one of the side projects that I think the brewery should consider brewing on a regular basis.
Task:
Expected output: 1/ppaque orange with light, white, creamy head that was not that well retained but full carbonation, but did settle into a small thin cap Ilama-3.1 3.1 output: positive[T he overall tone of the review positive, with phrases such "a very nice balance of the two styles", "nice bitter dry aftertaste", "well carbonated", "overall, good beer" indicating a favorable opinion of the beer.
G "TT-..-restructuree output: 1|a very nice balance of the two styles nice bitter dry aftertaste well carbonated overall, good beer
Figure example of Ilama fails output rationale the right format.
our main comparison is on the rationale quality and we do not claim our classification ability is superior to LLMs, hence we do not report this prediction accuracy avoid unnecessary confusion.
Comparing the results of the supervised fine-tuned llama- 3.1 with our results in Table 1, Ilama-3.1 does not have crushing advantage. For example, on the Beer-Aroma dataset, FR+A2I outperforms 1lama-3.1 1 at sparsity levels of 10% and 20%. Similarly, on the Beer-Palate dataset, RNP+A2I also outperforms 1lama-3.1 at sparsity levels of 10% and 20%. Besides, our A2I can be applied graph data, while is not easy do so for L.Mss.
E. Connection to More Fields
As deep learning progresses, enhancing few-shot learning (Tong et al., 2024b:a; 2025b) and improving the efficiency of multi-modal large language models (MLLMs) (Tong al., 2025a; Wang et al., 2025; Cui al., 2025) are becoming key challenges. With the rapid development of deep learning techniques, neural network-based methods (Zheng et al., 2023; Sun et al., 2024b; Zheng et al., 2024a;b) and LLM- based methods (Zheng et al., 2025; Zheng & Lee, 2025) demonstrate superior performances on a wide range of ap- plications and optimization tasks. We acknowledge their
21


Reviews:
[{"review_id": "LwFEm556Ql", "reviewer": "Reviewer_m5Wz", "summary": "The paper examines the unintended biases in self-rationalizing models, where a generator selects key input segments for a predictor. It reveals that cooperative training can introduce spurious correlations even in clean datasets. An adversarial inspection method is proposed to detect and mitigate these biases, improving interpretability and performance across multiple classification tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you deeply for taking the time to thoroughly review our paper.\n\nIf we understand correctly, the only weakness you mentioned is about the lack of ablation study.\n\n**Q1**. There is no ablation study on how each of the component of the proposed algorithm contribute to the success the effectiveness of the method seems to rely on the attacker’s ability to identify trivial patterns.\n\n**A1**. We think this is a misunderstanding. Compared to the standard rationalization framework RNP, our A2I introduces only one extra component (the attacker). So, the results of RNP can directly serve as the ablation study of our RNP+A2I. Also, the results of FR can serve as the ablation study of our FR+A2I.\n\nAnd we have designed a special experiment to verify the effectiveness of the attacker.  The attack success rate of RNP in Figure 6 implies that our attacker can successfully identify the trivial patterns learned by the predictor. And the low attack success rate of our RNP+A2I implies that our attacker-based instruction can effectively deter the predictor from adopting trivial patterns.", "claims_and_evidence": "The paper claims that cooperative rationalization can introduce spurious correlations even in clean datasets. It shows this through a theoretical analysis demonstrating how the generator’s selection process can create dependencies between extracted rationales and labels. Empirical evidence supports this claim by showing that predictors trained on randomly selected rationales still achieve high accuracy, indicating reliance on trivial patterns (Figure 3).\n\nThe paper claims that its proposed adversarial method (A2I) can detect and mitigate spurious correlations. It shows this through an attack-based inspection method that successfully identifies trivial patterns (high attack success rate in Figure 6) and an instruction mechanism that reduces reliance on these patterns, leading to improved rationale quality across multiple datasets (Tables 1–4).", "methods_and_evaluation_criteria": "The proposed methods effectively address spurious correlations in rationalization frameworks using adversarial inspection and instruction. Evaluation criteria, including benchmark datasets and rationale quality metrics, align well with the problem. The results are strong for both text and graph tasks.", "theoretical_claims": "yes, in Appendix C", "experimental_designs_or_analyses": "The paper compares its method (A2I) with multiple baselines, including RNP, FR, Inter RAT, and NIR\nThe attack success rate is used to as a measurement criteria.\nThe experiments span different types of datasets: text (BeerAdvocate, HotelReview) and graphs (BA2Motifs, GOODMotif).\nThe method is tested with both BERT (a pretrained Transformer) and non-pretrained models like GRUs and GCNs.", "relation_to_broader_scientific_literature": "The paper builds on Rationalizing Neural Predictions (RNP), a cooperative framework where a generator selects rationales that a predictor then uses for classification​.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "strengths: \nUnlike prior work that focuses on spurious correlations inherent in datasets, this study uncovers how rationalization frameworks themselves can introduce biases, even in clean data.\nThe paper provides mathematical proofs (Appendix C) and extensive empirical validation across multiple benchmarks (Tables 1–4).\n\nweakness:\nthere is no ablation study on how each of the component of the proposed algorithm contribute to the success\nthe effectiveness of the method seems to rely on the attacker’s ability to identify trivial patterns.", "other_comments_or_suggestions": "n/a", "questions_for_authors": "n/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6Furl1AkJ3", "reviewer": "Reviewer_UdWx", "summary": "This paper addresses a crucial issue in rationalization frameworks. They find that even if the original dataset does not have spurious correlations, the cooperative generator-predictor setup can cause spurious correlations that the predictor can exploit. This paper identifies the cause of this issue and proposes an attacker-based method A2I. A2I introduces an attacker which selects trivial patterns to fool the predictor. They empirically show that this introduction of an attacker can significantly improve the performance of rationalization, and furthermore show that the attacker successfully captures the trivial patterns recognized by the predictor and that the inspection prevents the predictor from adopting the trivial patterns.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for dedicating your time and expertise to review our paper. Your insightful comments and suggestions are highly valued and appreciated.\n\n**Q1**. Experiment of Figure 3: I am not fully convinced that this experiment provides the evidence that the generator-predictor interaction creates spurious correlation. Here is why: Since the generator is not trained with the ground truth label information, for a given trained g, Z and Y are independent. The independence of g and Z on Y means that (in the notation of equation (5)), P[Y=1|Z=t+, g] = P[Y=1]. Therefore, the orange curve should not include the spurious correlation. This contradicts with the statement in the paper.\n\n**A1**. We are sorry to cause such a misunderstanding. We agree that $g$ is not trained with  $Y$, but we can not say $P[Y=1|Z=t+, g] = P[Y=1]$. Here is a simple intuitive toy example.  \n\nSuppose there is a box containing approximately 200 cat photos and 200 dog photos. The cat images are labeled as 1, and the dog images are labeled as 0. For both cats and dogs, half of the images have a red background and the other half have a green background. In this dataset, background color is considered a trivial pattern and is independent of the label $Y$.\n\nNow, suppose a \"hand\" randomly samples 100 cat images and 100 dog images from the box to form a new dataset. Although the hand is a random hand and never uses the labels during sampling, biases may still occur in the specific sample. For example, the sampled cat images might include 50 with red backgrounds, while the sampled dog images might only include 30 with red backgrounds. As a result, under this particular sampling by the hand, background color and label $Y$ are no longer independent.\n\nReturning to rationalization. We consider that in the original dataset, all text samples end with a period. However, one possible scenario is that in the random sample drawn by $g$, 51% of the positive samples include the period as part of $Z$, while only 49% of the negative samples include the period in $Z$. As a result, \"whether $Z$ contains a period\" is no longer independent of the label $Y$. This spurious correlation is caused by the sampling bias of $g$.\nThis is because once the generator, originally a random variable, is instantiated into a specific value $g$, it inevitably contains bias.\n\nIn the example above, the spurious correlation caused by bias may not seem very severe, as we only considered a single trivial pattern. However, given that text should be regarded as a high-dimensional vector, chances are that the accumulation of biases from multiple different trivial patterns can become quite significant.\n\n\n**Q2**. While they claim their algorithm’s performance is comparative to LLama3.1-8b-instruct, the LLM is relatively small, and it is better to compare it with larger models or other models. I cannot judge whether their algorithm is state of the art.\n\n**A2**. We are sorry to say that our RTX3090 and RTX 4090 GPUs cannot afford LoRA fine-tuning for models larger than 8B.\n\nOur methods is SOTA compared to previous rationalization methods. But we acknowledge that our method can not outperform more powerful LLMs like GPT4. However, the training of those LLMs usually involves extensive human alignment, which can be very expensive. Our model is small. Besides, we do not use human-annotated rationales for training. So, our model is much cheaper and can work in places with limited resources.\n\nDespite the issues of SOTA results, our two major contributions still make sense. First, we find a new kind of spurious correlation, which may open up new avenues for future research. Second, we propose a direction to mitigate this issue with attack, which may inspire others to develop better methods.\n\n\n**Q3**. In Equation (11), The meaning of [0.5,0.5] is unclear. Does this represent a random variable?\n\n**A3**. Yes, it represents a random noise. Thanks for the reminder, and we will clarify this in the revision. \n\n**Q4**. Suggestions about $g$ and Eq.8.\n\n**A4**. Thank you for your suggestion. We will do it in our revision.", "claims_and_evidence": "Their claims are convincing.", "methods_and_evaluation_criteria": "The authors’ attacker-based strategy is well motivated and practically sensible. By introducing an attacker that selects trivial patterns to flip predicted labels, the approach mitigates spurious correlations introduced by the generator–predictor interplay. This design choice is well explained in Section 4.", "theoretical_claims": "The paper does not present formal theorems or statements in the main part of the paper. I did not review the appendix.", "experimental_designs_or_analyses": "Experiment of Figure 3: I am not fully convinced that this experiment provides the evidence that the generator-predictor interaction creates spurious correlation. Here is why: Since the generator is not trained with the ground truth label information, for a given trained g, Z and Y are independent. The independence of g and Z on Y means that (in the notation of equation (5)), P[Y=1|Z=t+, g] = P[Y=1]. Therefore, the orange curve should not include the spurious correlation. This contradicts with the statement in the paper.", "relation_to_broader_scientific_literature": "The authors shed light on the spurious correlations induced by the generator–predictor framework. This has a sharp difference from those embedded in the dataset, which has been discussed in prior works.", "essential_references_not_discussed": "I am not aware of any papers which should be discussed.", "other_strengths_and_weaknesses": "Strength:\nThey perform extensive empirical study to support their idea and the performance of the proposed algorithm. In particular, they examine the performance on multiple text classification datasets , multiple sparsity regimes, and different encoder architectures. Furthermore, the study of attack success rate successfully supports their ideas on the spurious correlations and attackers.\nWeaknesses:\nWhile they claim their algorithm’s performance is comparative to LLama3.1-8b-instruct, the LLM is relatively small, and it is better to compare it with larger models or other models. I cannot judge whether their algorithm is state of the art.", "other_comments_or_suggestions": "The notation “g” is used in the first section without giving definition. In the later section, I noticed that the paper defined it as the generator. It might be better to define it earlier.\n\nEquation (8) can be stated in a more mathematically precise way. I believe the current form does not make sense though I understand what the authors want to state.", "questions_for_authors": "In Equation (11), The meaning of [0.5,0.5] is unclear. Does this represent a random variable?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MgK5oKXK2z", "reviewer": "Reviewer_MaoV", "summary": "This paper examines the phenomenon of spurious correlations induced by the cooperative rationalization approach, where a generator selects informative subsets of data as rationales for predictors. It reveals that even clean datasets can develop unintended spurious correlations due to the cooperative game between generator and predictor. The paper introduces an adversarial inspection and instructional mechanism to identify and mitigate these correlations, achieving significant empirical improvements on various text and graph classification tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to carefully review our work and provide constructive feedback.\n\n**Claims&Wekaness1**. The claim seems to broad.\n\n**A1**. Thanks a lot for the valuable suggestion. We will narrow down the scope to the text and graph domains to ensure rigor.\n\nWe'd like to kindly clarify that we have mentioned rationalization is mainly used for NLP (the right part of L93–94). Although our experiments are conducted on text and graph data, the theoretical analysis in Sec4 is based on abstract variables and is not restricted to a specific scenario. Could you kindly clarify your concern in more detail regarding why you believe the analysis may not generalize to other scenarios? This would help us address your concern more effectively.\n\nWe believe such correlations may also arise in the image domain. For example, consider a cat-vs-dog classification dataset in which every original image contains a green pixel. If the generator selects a subset of pixels as the rationale, it may (during this selection process) consistently include the green pixel as part of the rationale for all images labeled 1, while excluding it from images labeled 0. This kind of sampling bias would introduce a spurious correlation.\n\nHowever, our focus in this paper is solely on explaining why such situations can arise (i.e., identifying a potential risk or vulnerability). Assessing how likely this is to occur in real-world settings is somewhat beyond the scope of this paper. We appreciate your suggestion and will clarify and narrow down the scope in our revision.\n\n\n**Weakness2**. Complexity and running time.\n\n**A2**. Thanks for the valuable suggestion. Below is a comparison between the complexity of vanilla RNP and RNP+A2I (referred to as A2I for short). In general, A2I introduces an additional attacker module, and as a result, A2I has approximately 1.5 times the number of parameters of vanilla RNP. However, we would like to note that introducing extra modules is common practice in the field of rationalization. For example, our A2I model has a comparable number of parameters to Inter-RAT, a previously proposed variant of RNP.\n\nBelow are the practical time and memory consumption of RNP and A2I on the Beer-Appearance dataset (we add the results of Inter-RAT as a reference).\n\n\n| Beer-Appearance | batch size  |lr| epochs  |memory (MB)| RTX3090 hours  |\n|:---:|:---:|:--------:|:-------:|:---:|:---:|\n|Inter-RAT|  256 |  0.001   |20| 3660 |2.34 |\n|RNP | 128 |  0.0001  |   200   | 2184|0.37|\n| A2I| 128 |  0.0001  |   200   |2784|0.61|\n| A2I | 128 |  0.0001  |   100   | 2784 |0.31|\n|A2I | 256 | - |    -    |  3664  |-|\n\nPer epoch, RNP consumes only about 60% of the training time compared to RNP+A2I. However, A2I converges significantly faster than vanilla RNP and typically requires fewer training epochs (about 50% on most datasets). Since memory usage is influenced by batch size, we further assign A2I a batch size of 256 (the same as Inter-RAT). Under this setting, the memory usage is 3664 MB, which is comparable to that of Inter-RAT.\n\nRegarding the second question, \"How does the proposed method scale to large datasets?\", we are not entirely sure about the specific concern. Are you referring to datasets with more samples, or datasets with longer text inputs? If you mean the former, we believe the table in the Appendix addresses this concern: the largest dataset we use (Hotel-Cleanliness) contains 150k text samples. If you mean the latter, we follow one of our baseline Inter-RAT to test our A2I on the Movie dataset, which has much longer texts (the average length is about 775, and we set the max length to be 1024). The results are as follows (the results of other baselines are copied from Inter-RAT):\n\n| Movie| S | P| R | F1|\n|---|---|---|---|---|\n| RNP| 20.0 | 35.6 | 21.1 | 24.1|\n| A2R | 20.0 | 48.7 | 31.9 | 34.9|\n| INVRAT| 20.0 | 33.9 | 24.3 | 28.3|\n| Inter-RAT| 20.0 |35.7| 35.8 | 35.7|\n| A2I| 20.6 | 45.0 |31.9| **37.1** |\n\nOur A2I still significantly outperforms RNP in terms of F1 score on the long-text dataset.\n\n**Weakness3**. It does not thoroughly explore conditions under which the adversarial intervention might fail or succeed universally.\n\n**A3**. We are sorry, but we are not very sure about your specific concern. Could you please specify it with more details? \n\nWe would like to clarify that our analysis already covers both sides of cooperative rationalization and how our attack succeeds in each case. Specifically, in the first part of Sec 4.3 (L241–316), we discuss how our attacker can correct the predictor and generator when they select the wrong rationales. Then, from L318 to the end of Sec4.3, we discuss how, when the predictor and generator select the correct rationales, our attacker does not introduce any negative impact. We think these two cases together cover both sides of the issue.\n\n**Typos**. Thank you very much for the careful reading, it refers to Fig.3. We will fix it in our revision._", "claims_and_evidence": "The paper mentioned \"such a cooperative game could unintentionally introduce a sampling bias during rationale extraction\", which can be a very interesting finding. However, the claim seems to broad and the authors did not present comprehensive insights into how this can generalize to a wider range of different scenarios. I would suggest to narrow down the scope of the claim of this paper to the focused areas, including text-based and graph-based ones.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "They appear to be generally correct and the intuition makes sense to me as well.", "experimental_designs_or_analyses": "They appear to be carefully designed and I do not have major complaints about the experimental designs and analyses.", "relation_to_broader_scientific_literature": "This paper is properly positioned as a part of the broader scientific literature with clarifications about the relationship between itself and other works.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths :\n\n1 - This paper highlights a previously underexplored source of spurious correlations introduced by cooperative rationalization, even in clean datasets.\n\n2 - It also proposes a practical adversarial inspection and instructional method to mitigate identified biases effectively.\n\n3 - The paper demonstrates the effectiveness of the proposed approach across diverse datasets and multiple model architectures (GRUs, BERT, GCN).\n\nWeaknesses:\n\n1 - This paper involves broad claims that may not hold in the domain not discussed in this paper.\n\n2 - I did not see much analysis about complexity and running time. How does the proposed method scale to large datasets? Extra adversarial inspections and instructions could significantly increase training time and computational resources.\n\n3 - Although the paper provides theoretical explanations, it does not thoroughly explore conditions under which the adversarial intervention might fail or succeed universally.", "other_comments_or_suggestions": "Typos exist, e.g., Figure ?? in Appendix B.3.", "questions_for_authors": "See weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "seOpVMCPDb", "reviewer": "Reviewer_cGvE", "summary": "This paper proposes TeaRGIB to enhance the robustness of dynamic graph representation learning methods by using the Information Bottleneck (IB) principle to reduce the redundancy and noise in dynamic graphs. TeaRGIB first employs von Neumann entropy (VNGE) to model the evolution of dynamic graphs. And it decomposes the dynamic graph learning problem into learning compact structure and representation given the previous temporal graphs. Experimental results show that the proposed method enjoys good link prediction accuracy and robust to several types of noise.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. But it seems that you may have mistakenly pasted review comments intended for another paper here. So, we will wait for your correction.", "claims_and_evidence": "The reviewer is confused about the motivation of the structural evolution loss. The author needs to elaborate on why von Neumann entropy (VNGE) could capture the structural evolution. And what is the motivation of aligning von Neumann entropy (VNGE) with the mutual information? Is there any theoretical justification for this alignment?", "methods_and_evaluation_criteria": "The decomposition of the dynamic graph information bottleneck objective in Section 4.1 follows GIB [1] and utilizes the local dependency assumption, which is similar to Dynamic GIB [2]. The author needs to discuss the difference between Dynamic GIB and the proposed method to highlight the novelty.\n\n[1]. Graph Information Bottleneck. NeurIPS 2020.\n[2]. Dynamic Graph Information Bottleneck. WWW2024.", "theoretical_claims": "The reviewer checks most of the proofs, and they seem to be good. But further theoretical justification for structural evolution in Section 4.2 is necessary.", "experimental_designs_or_analyses": "The proposed TeaRGIB only adopts the Graph Attention Network as the backbone network. The generalization ability of TeaRGIB to other (dynamic) graph neural network architectures is questionable.", "relation_to_broader_scientific_literature": "The key contribution is to extend the Graph Information Bottleneck principle to the dynamic graph learning domain. The proposed method could enhance the robustness of dynamic graph learning models. However, the proposed method is similar to Dynamic GIB as pointed out in the Methods And Evaluation Criteria Part. The author is encouraged to further discuss the difference between the two work, or the contribution would be limited.", "essential_references_not_discussed": "[1]. Dynamic Graph Information Bottleneck. WWW 2024.\n[2]. TempME: Towards the Explainability of Temporal Graph Neural Networks via Motif Discovery. NeurIPS 2024.", "other_strengths_and_weaknesses": "Please see above.", "other_comments_or_suggestions": "The author is encouraged to address the concerns in the above parts.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: All reviewers agree that this is a valuable contribution to understanding and eliminating spurious correlations in cooperative rationalization.
Decision: Accept (poster)