Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making
Authors: Yanrui Du
Sendong Zhao
Haochun Wang
Yuhan Chen
rui bai
Zewen Qiang
Muzhen Cai
Bing Qin
Venue: EMNLP 2023 Findings (2023)
Keywords: rationale
reliable link
two-stage framework
TL;DR: Our proposed SADM framework establishes a more reliable link between the generated rationale and model decision while improving task performance and rationale quality.

Abstract: Explaining black-box model behavior with natural language has achieved impressive results in various NLP tasks. Recent research has explored the utilization of subsequences from the input text as a rationale, providing users with evidence to support the model decision. Although existing frameworks excel in generating high-quality rationales while achieving high task performance, they neglect to account for the unreliable link between the generated rationale and model decision. In simpler terms, a model may make correct decisions while attributing wrong rationales, or make poor decisions while attributing correct rationales. To mitigate this issue, we propose a unified two-stage framework known as Self-Attribution and Decision-Making (SADM). Through extensive experiments on five reasoning datasets from the ERASER benchmark, we demonstrate that our framework not only establishes a more reliable link between the generated rationale and model decision but also achieves competitive results in task performance and the quality of rationale. Furthermore, we explore the potential of our framework in semi-supervised scenarios.

Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making
Yanrui Du, Sendong Zhao; Haochun Wang, Yuhan Chen, Rui Bai
Zewen Qiang, Muzhen Cai, Bing Qin
Harbin Institute of Technology, Harbin, China { yrdu, sdzhao, hewang, yhchen, rbai, zwqiang, mzcai,qinb) @ir.hit.edu.cn
Abstract
Cluim: The Nice Gays directed by Stephen Spielherg.
Passage: The Nive Guys is 2016 American nea-noir action comedy film directed by Shane Black and wriften by Black and Authony Eagaragi. .. The Nice Gays premlered May 2016 the 2016 released Warner Bros
Explaining black-box model behavior with nat- ural language has achieved impressive results various NLP tasks. Recent research has ex- plored the utilization of subsequences from the input text as a rationale, providing users with evidence support the model decision. Al- though existing frameworks excel in generating high-quality rationales while achieving high task performance, they neglect account for the unreliable link between the generated ra- tionale and model decision. In simpler terms, model may make correct decisions while at- tributing wrong rationales, or make poor deci- sions while attributing correct rationales. To mitigate this issue, propose unified two- stage framework known as Self-Attribution and Maisinn (SADM). Through extensive experiments on five reasoning datasets from the ERASER benchmark, we demonstrate that our framework not only establishes link between the generated rationale and model decision but also achieves competitive results in task performance and the quality of rationale. Furthermore, we explore the potential of our framework semi- sup-ssupe scenarios.
Film Festival
Gold Answer: REFUTE.
Model Output: Answer: SUPPORT. Rationale: The
Guys 2016 American aco-noir action comedy film directed by Shane Black and written by Black and Anthony Bagarozzi
Clain: The Others (2001 film) won awards.
Passage: The Others -LRB- Los Otros -RRB- 2001 Spanish-American supernatural gothic horror film with elements of psychological horror. The Others was nominated for Saturn Awards including Best Director and Writing for Amenābar and Best Performance Younger Actor for Alakina Mann...
Gold Answer: SUPPORT.
Model Output Answer: SUPPORT. Rationale: The Others -IRBB- Otros -RRB-
Spanish- American supernatural gothic horror film with elements of psychological horror.
reliable
Figure I: With the claim and passage input, the FID- Ex framework generates the model decision, followed by the rationale output. It observed that the gener- ated rationale does not convincingly justify the model decision. underlined part of the passage represents the manually annotated rationale.
1 Introduction
Large-scale pre-trained models (Lewis et al., 2019; Touvron et al., 2023) have achieved state-of-the- art results various tasks (Wang et al., 2023; Du et al., 2023a; Chen et al., 2023), but their decision- making process is opaque. Recent work (Geirhos al.. 2020; Lai al., 2021; Wang al., 2021; Du et al., 2022; Liu al., 2023; Du et al., 2023b) have revealed that models often rely on superficial clues for predictions, which can make their deci- sions unconvincing. Therefore, it is valuable to motivate models to provide trustworthy rationales to back up their decisions, which facilitates their implementation in real-world applications.
Recent studies (Ismail al., 2021; Shen et al., 2022) have concentrated on the ERASER (DeY- oung al., 2019) benchmark, which encourages models to obtain the extracted subsequences from the input text as a rationale to support their decision. The WT5 (Narang et al., 2020) framework and its variant FID-Ex (Lakhotia et al., 2020) have shown superiority on this benchmark, which generates the rationale and classification decision in a parallel way. However, such parallel frameworks raise a serious problem: is the link between the rationale and classification decision generated by models re- liable? In the upper example Fig. 1, we observe that the model attributes the correct rationale, that
Corresponding author

---

"The Nice Guys is a 2016 American neo-noir action comedy film directed by Shane Black", but still mistakenly supports the claim that "The Nice Guys was directed by Stephen Spielberg". In the lower example of Fig. 1, it is evident that the model at- tributes the rationale that "The Others (Los Otros) is 2001 Spanish-American supernatural gothic horror film with elements of psychological horror", which is entirely unrelated to the claim. However, despite the wrong rationale, the model still cor- rectly supports the claim that "The Others (2001 film) won awards". These instances highlight a significant challenge in developing a model with explanations in natural language form, that the gen- erated rationale does not genuinely support and convincingly justify the model decision.
2
Background
2.1 Task Form
We work with supervised data containing quadru- ples (q, (r),,r Here, represents a question or claim, p represents a passage that can answer or judge q, corresponds to the rationale, typically a subsequence extracted from the passage p, and y represents the classification target. Our objective is to train a model where the input is denoted as = (q. (). The desired outcome from the model is twofold: classification result and a rationalc to explain its decision-making behavior, that is, (r, v) - [g,p).
2.2 Related Work
Rationale. The rationale is defined as the con- densed and logically coherent subsequences from the input text, yet still adequate for the model to make correct decisions (Lei et al., 2016; Linardatos et al., 2020; Burkart and Huber, 2021). Previ- ous works (McDonnell et al., 2016, 2017; Arous et al., 2021) have shown that rationales bring many benefits, including more reliable judgments, greater transparency for evaluating human raters, and added value from the rationales themselves.
To mitigate the above issue, we introduce a uni- fied two-stage framework called Self- Attribtttittiinn and Decision-Making (SADM). Our SADM frame- work adopts distinct architectures for training and inference processes. For the training process, we train the model by jointly optimizing both the self- attribution and decision- decision-making objectives. For the inference process, we adopt a two-stage for- mat, which is inspired by the two-stage inference theory of human (Evans, 1984). The model is first prompted to extract the rationale from the given input (known the self-attribution stage), and then, the model is prompted to utilize the extracted rationale to make informed decisions (known as the decision- makiong stage). Moreover, our SADM framework incorporates the Fusion-In- Decoder (FID) (Izacard and Grave, 2020) archi- tecture to address the challenges posed by lengthy texts, and the Sentence Mark (SM) (Lakhotia et al., 2020) strategy to mitigate the issue of random and irrelevant rationale generation during the self- attribution stage. To further enhance the model's comprehension at the decision-making stage, we also introduce a Reasoning Augment Learning (RAL) strategy.
Methods.
Existing methods are mainly divided into two categories: pipeline-basee based frameworks and parallel frameworks.
For pipeline-based frameworks, one way is post-hoc explanation. After models make deci- sions, humans attempt to analyze why models give a specific decision. Common methods include at- tention mechanism (Tenney et al., 2019) (assign soft weights to tokens by self-attention matrix), LIME (Ribeiro et al., 2016) (approximate model be- havior locally by repeatedly perturbing inputs), and gradient (Sundararajan et al., 2017; Smilkov et al., 2017) (gradient of each token vector to represent their attribution), etc. However, recent work (Feng et al., 2018; Serrano and Smith, 2019; Jain and Wal- lace, 2019; Pruthi et al., 2019; Brunner et al., 2019; Zhou et al., 2022) have pointed out that the above methods often exhibit counterintuitive behaviors and lack credibility. The other way is a pre-hoe ex- planation. Models encouraged to first generate the rationale and then make decisions based on it. The BERT2BERT framework utilizes two indepen- dent models to extract rationales and make deci- sions, with the reparameterization method to jointly optimize two models during training. Based on the
In our experiments, we introduce the RSQ met- ric to quantitatively assess the reliable link be- tween generated rationales and model decisions. Experimental results consistently show significant improvements in the RSQ metric for our SADM framework. For task performance and the quality of rationale, our SADM framework also outperforms strong baselines overall. Moreover, we conduct ablation experiments to analyze the contribution of each component within our framework.

---

rationale
T Tnswer
Football is a sport that Terry Crews played professionally.
Extraet Rationale from the Passage for jadying Claim
Refer to the following
information judge Claim.
Passage
Self-attribution
rationale
50: Terry Alas Crews -LRB- born July 30, and former American football player. St:..
artist,
Claim
LM Model
Rationale:
54:
played defensive
snd and linebacker in National Football League -LRB- NEL RKB- for the Los Angeles Ntms.
Passage
Shared Parameiers
Indexing
Decision-making
Rationale
Terry -LRB- July
Claim
LM Model
Answer: Trree.
American actor, artist,
and former American football player. In football, Crews
defensive linebacker National Foothall League -LRB- NFL RRB- for the Los Angeles Rams.
Rationale
Passage
Optional
Figure 2: Illustration of SADM framework: During the inference process, two-stage format is adopted. Firstly, the model is prompted to generate the rationale, and subsequently, the model is prompted to make a decision based on the generated rationale.
BERT2BERT framework, the IB framework pro- poses information bottleneck method instead of the reparameterization method to jointly optimize the models. Moreover, the QUASER framework incorporates sentence selector and a multi-task training objective to improve the model's s task per- formance, which aims at the to-nncee---seuunce (seq2seq) models.
ently, our study concentrates on extracted rationale scenarios. In comparison to parallel frameworks, our proposed pipeline SADM framework demon- strates more pronounced advantages.
SADM
In this section, we first introduce our overall SADM framework (Sec. 3.1), and then we describe how our SADM framework works in detail from two aspects: Training Process (Sec. 3.2) and Inference Process (Sec. 3.3).
For parallel frameworks, WT5 and FID-Ex are representative, where the latter is variant of the former. For the WT5 framework, task-specific phrases such as "explain fact verification task" are prepended to the input text. The model then gen- erates a classification decision, followed by the ex- tracted rationale. Notably, the position of the clas- sification decision and extracted rationale can be interchanged, referred to as WT5-INVERSE (WT5- INV). Empirical evidence suggests that WT5 gen- erally outperforms WT5-INV in terms of perfor- mance. Furthermore, FID-Ex retains the same mode as WT5 but introduces the fusion-in-decoder architecture to address input length limitations. Ad- ditionally, employs sentence mark strategy to prevent the generation of random rationales.
3.1 Overall Framework
The theory of human inference (Evans, 1984) sug- gests that there are two processes involved in hu- inference: the heuristic process and the ana- lytic process. During the heuristic process, indi- viduals gather task-relevant information, while the analytic process involves manipulating and process- ing the gathered information to make judgments. Drawing inspiration from this cognitive thinking, we propose our SADM framework, as depicted in Fig. 2. Firstly, we employ the Trationale tem- plate to prompt the model to generate task-related rationale, the process called self-attrib seffaatribution. Sub- sequently, we employ the other Tanswer template to prompt the model to make decision based on the generated rationale, the process called decision- making. As for the choice of prompt templates,
Furthermore, prior research (Wiegreffe al., 2020) has measured the association between free- text rationales and model decisions. Their findings indicate that the parallel frameworks offer substan- tial advantages over the pipeline framework. Differ-

---

T Pationale
Input
Target
tive decisioon we provide Tonswers q, and human- annotated rationale r as input, with the golden target as the learning objective. We adopt a joint training strategy that involves adding the losses Faationate and deccision of two objectives. Moreover, to calculate the losses Lrationate and Liccciioo we employ the teacher-foreing strategy. Given an input sequence Ii,.... Ig and target sequence y1,-- Y/u, we maximize the probability ply/ ...,,,,,. zt. 91, 24 yi-1) for each y in the target sequence y1, Va to obtain the loss.
B
O rationale
annwer
Clnim
O
O decision
Passage
5
Rationale
RAL Strategy
Answer
2
Figure 3: The format of training samples with and with- out the RAL strategy.
Sentence Mark (SM). Recent research (Tam et al., 2022) has highlighted the creative nature of generative models. However, in our specific task, there is concern that the generative model may produce random and irrelevant natural lan- guage rationale. To mitigate this issue, we adopt the sentence mark strategy (Lakhotia et al., 2020), which involves adding an index number be- fore each sentence in the passage p. For instance, a passage consists of n sentences (s1,..., Af- ter adding the index, the passage takes the form of (S1 $1, ........ Sx Sx), where uppercase charac- ters are the sentence indexes. If applying the SM strategy, when optimizing the objective Orationale during the training process, need take the sentence indexes of human-annotated rationales the learning objective instead of the rationale in natural language form.
we utilize natural language oriented toward human understanding. Consider the FEVER dataset as an example. For the Tratioaate template, we design it as follows: "Extract the rationale from the pas- sage to assess the claim". Similarly, the Tanswer template formulated as: "Refer to the following information to judge the claim". Notably, we use discrete prompt templates, which do not introduce additional parameters or training costs.
Moreover, similar FID-Ex and QUASER frameworks, we implement the FID architecture based on the seq2seq models to effectively han- dle lengthy text inputs. The FID architecture can be described as follows: Firstly, the lengthy pas- sage is divided into multiple segments denoted as {seg1; sC92, .... seg.). Next, q is combined with each segment and encoded separately in the coder module generate multiple vector repre- sentations {e1, E2, .... en). Finally, all the vector representations are concatenated as e1 @ c2... @ en and forwarded to the decoder module for further processing.
Reasoning Augment Learning (RAL). Cogni- tive science (Payne et al.. 1988) shows that humans have the ability to make reasonable decisions re- gardless of whether they possess fine-grained in- formation (such as human-annotated rationale) coarse-grained information (such as the whole pas- sage). Therefore, we imitate human beings, in- tending to equip the model with the ability to per- ccive information at different levels of granularity, thereby enhancing its reasoning ability. To accom- plish this, we leverage the wealth of information available in supervised data. As shown in Fig. 3, for the objective Odecisions add two for- mats of training samples. We respectively provide the model with Tanswer, q. and p as input, as well as T Tanswers 4, r, and p as input. The golden tar- get is regarded as the learning objective and the calculated loss is added to Ledccssion
3.2 Training Process
For the training process, we introduce our training objective, the Sentence Mark (SM) strategy, and the Reasoning Augment Learning (RAL) strategy.
Training Objective. Our SADM framework is designed to achieve two training objectives:
Objective Orationale: Training the model to gen- crate a relkted rationale at the prompt of Trationate template.
Objective :decision: Training the model to make decision based on the generated rationale at the prompt of Tanewer template.
As shown in Fig. 3, for the objective Orationale- we provide the model with Trationates q. and p as input, while using the human-annotated ra- tionale r as the learning objective. For objec-
3.3 Inference Process
The inference process initiates an auto regrerregeesive process. During the inference process, we em-

---

ploy two-stage process shown in Fig. 2: self-
rede: Samples where both the generated rationale
attribution and decision-making.
and model decision are correct.
rudyy Samples where both the generated ratio-
Self-attribution. When the model is prompted with the rationate template, generates the ratio- nale based on the claim ! and passage If with the SM strategy, the model first generates sentence indexes, which are then used to locate the corre- sponding rationale from the passage p. And if with- out the SM strategy, the model directly generates the rationale in natural language form.
nale and model decision are wrong.
redw: Samples where the generated rationale is
correct, but the model decision is wrong.
Tudde Samples where the generated rationale is
wrong, but the model decision is correct.
The RSQ metric is calculated follows.
Num(rede + Fan
RSQ Num(rade + Tudl rodw + Twdl))
Decision-making. When the model is prompted with the T Tanswer template, it makes a decision based the generated rationale. Moreover, we provide the model with the option to make deci- sion by considering combination of the generated rationale and passage.
(1)
where Num represents the number of samples. As for how to assess whether the generated rationale or model decision is correct, for the model decision, we determine whether the predicted target aligns with the golden target. For the generated rationale, assess its correctness using the recall (described in detail in App. A). If the recall rate exceeds a certain threshold, mainly set 0.5 our work, we consider the generated rationale to be correct. Based on the RSQ metric, we also propose RSQ- W and RSQ-C metrics to guide more detailed analysis. The RSQ-W measures the proportion of wrong decisions made by the model when the model attributes the correct rationales. The RSQ-W metric is follows:
4 Evaluation Metrics
Consistent with prior work (Paranjape et al., 2020; Ghoshal et al., 2022), we assess task performance using accuracy and evaluate the quality of gener- ated rationales using the Intersection-Over-nii F1 score (IOU F1) and Token F1 (TF1). We also follow the prior research (Wiegreffe al., 2020) take Rationale Accuracy (R-Ace) to evaluate the quality of generated rationales. Additionally, we introduce novel metric, the Reasoning Success Quotient (RSQ), to gauge the extent of the reliable link between the generated rationale and model decision.
RSQ-W = Num(redw)
(2)
Num(rede + rodww
IOU F1 and TF1 metries. The IOU F1 and TF1 metries are used evaluate the quality of rationale at the sentence level and the token level respec- tively. Whether at the sentence level or the token level, the precision and recall rates are calculated for each test sample. Based on them, the Fl value can be calculated. To obtain the IOU F1 or TF1 metries, the corresponding Fl values of all test sam- ples are averaged. The detailed calculation process is described in App. A.
The RSQ-C measures the proportion of correct decisions made by the model when the model at- tributes the wrong rationales. The RSQ-C metric is as follows:
Num(rude)
RSQ-C Num(r + d.)
(3)
Experiment
5.1 Datasets
The statistics of datasets used in our experiments are shown in Tab. 2. The FEVER (Thorne et al., 2018) dataset aims to judge whether the given passage supports or refutes the claim. The Mul- tiC (Khashabi et al., 2018) dataset aims assign True or False to the question concatenating with the answer choice based on the given passage. The BoolQ (Clark al., 2019) dataset aims answer the question with True or False labels based on the given passage. The Evidence Inference (Evi
R-Ace metric. We first train model f with ques- tions and annotated rationales and then evaluate the performance (accuracy) of model ff on generated rationales to reflect the quality of the generated rationales.
RSQ metric. We propose the RSQ metric mea- sure the reliable link between the generated ratio- nale and model decision. Specifically, we catego- rize the test samples into four classes:

---

Perf.
IOU FI
TFIt
R-Acct
RSQT
RSQ-W,
RSQ-CU
FEVER
BERT2BERT
85.0
81.7
WTS
91.9
76.6
85.3
74.9
6.7
86.9
WTS-INV
91.4
76.5
84.9
75.4
6.8
85.1
FID-Ex(C=I)
92.7
85.4
86.4
92.1
82.2
5.5
82.7
SADM(C=1)
93.1
85.9
86.9
92.2
83.5
4,9
81.6
MultiRC
BERT2BERT
63.3
41.6
WTS
78.0
68.0
76.6
69.6
20.7
72.3
WTS-INV
77.2
66.6
75.5
68.4
21.7
72.5
FID-Ex(C=l)
79.1
72.0
77.4
77.6
72.1
19.8
72.6
SADM(C=I)
80.1
72.9
78.1
78.8
75.2
18.4
69.3
BoolQ
BERT2BERT
62.3
31.5
WTS
71.8
44.1
63,2
54.0
23.6
67.4
WTS-INV
69.7
42.6
61.6
52.8
26.3
66.1
FID-Ex(C=11)
73.9
52.3
64.2
72.8
61.9
20.1
65.5
FID-Ex(C=10)
72.4
52.2
64.3
73.2
61.5
22.8
64.9
BADM(C=1)
74.3
52,9
64.5
72.9
63.1
17.1
62.9
SADM(C=10)
72.5
51.9
64.3
73.6
61.4
22.6
63.9
Evi Inf
BERT2BERT
70.8
53.9
WTS
66.8
21.1
55.2
46.6
17.9
62.8
WTS-INV
62.9
22.4
55.7
52.1
15,9
57.9
FID-Ex(C=1)
63.5
32.6
51.3
66.3
54.0
23.1
57.7
FID-Ex(C=10)
75.4
51.4
66.4
75,2
66.1
12.9
60.5
SADM(C=1)
68.3
30.9
49.1
65.4
57.5
8.4
57.4
SADM(C=10)
75.6
52.2
67.9
76.0
68.0
11.4
58.7
Moy Rev
BERT2BERT
86.0
15.7
WTS
90.9
30.2
50,9
21.1
8.7
90.9
WTS-INV
FID-Ex(C=I)
90.5
57.1
68.2
87.9
68.8
7.8
86.4
FID-Ex(C=6)
96.0
57.8
67.3
95.5
41.7
2.5
95.8
SADM(C=)
94.9
63.9
73,2
89.9
90.5
4.3
84.6
SADM(C=6)
96.5
62.7
71.4
96.5
55.8
1.8
94.5
Table 1: Experimental results(%) in full-supervised scenarios. Perf. represents task performance and C represents the count of segments set in FID architecture.
Inf) (Lehman al., 2019) dataset concatenates the (intervention, outcome, comparator) triplet into the question, and aims judge whether the interven- tion significantly increases, decrcases, or has no effect on the outcome based on the given passage. The Movie Reviews (Mov Rev) (Zaidan and Eis- ner, 2008) dataset aims to analyze the sentiment of the given passage with positive or negative la- bels, where the question is uniformly set to "What is the sentiment of this review?". Overall, all five datasets belong to the reasoning lasks. Morcover, the ERASER benchmark provides the annotated ra- tionale at the phrase level for the Mov Rev dataset, and the sentence level for the others. Following the prior work, we convert the phrase level to the sentence level annotations.
Train Dev / Test
# Toks
Sents
FEVER MultiRC
97,957 6,12/ 61,111
288
11
24,029 3,214/ 4,848
300
14
BooiQ
6,363 1,491 2,807
3,391
165
Evi Inf
7,958 9722 9959
4,658
153
Mov Rev
1,600 200 200
774
37
Table Statistics of datasets in our experiments.
5.2 Training Details
Consistent with previous work (Paranjape al., 2020; Ghoshal al., 2022), we select T5-base our main model and use the integrated interface TSForConditionalCGeerrtion from huggingface to load the model. We run all experiments on single NVIDIA 80g-a100 GPU machine. We set the learn- ing rate to le-4, the batch size to 16, and the total
httg::/htugginnaaccccooo//tttttt

---

training steps 15,000 steps, where evaluate the IOU F1 metric on the validation set every 500 steps to choose the best checkpoint. For datasets with lengthy input, we apply the FID architecture. For the BoolQ, Mov Rev, and Evi Inf datasets, we use maximum of 512 subword tokens per segment input, where we use 10 segments for the BoolQ and Evi Inf datasets and 6 for the Mov Rev dataset. For the FEVER and MultiRC datasets, only use one segment and set the maximum input subword lengths to 512 and 1024 respectively.
Perf.
IOU
TF1
RSQ
FEVER
IB
88.8
66.6
FID FID-Ex(C=I)
91.5
83.9
85.3
82.1
SADM
92.1
84,8
86.2
82.8
MultiCC
IB
66.4
54.4
FID-Ex(C=I)
78.4
71.5
76.8
72.3
SADM(C=1)
79.9
72.6
77.6
74.4
BoolQ
IB
63.4
32.3
FID-Ex(C=1)
70.7
46.4
60.6
55.8
FID-Ex(C=10)
65.7
49.5
60.9
53.1
SADM(C=1)
73.9
50.3
61.5
61.7
5.3 Baselines
SADM(C=10)
73.4
47.5
61.3
57.8
our study, we compare our SADM framework against several baselines, including BERT2BERT, IB, WT5, WT5-INV, and FID-Ex frameworks. The experimental results for BERT2BERT and IB are reported from the original work, where BERT2BERT is applied in full-supervised scenar- ios and IB is applied in supervipe scenarios. For WT5, WT5-INV, and FID-Ex frameworks, re-implemented them based on the details provided important to
Evi Inf
IB
46.7
13.3
FID-ExC==I
55.1
26.2
47.8
57.6
FID-Ex(C=10)
64.1
43.1
61.3
57.7
SADM(C=1)
59.2
23.9
41.2
59.7
SADM(C=10)
73.6
45.6
62.9
65.8
Mov Rev
85.4
43.4
FID=Ex(C=I)
86.4
59.4
70.4
73.4
FID-Fx(C=6)
94.4
55.6
66.5
46.7
SADM(C=1)
87.9
56.9
68.9
76.3
SADM(C=6)
91.1
54.9
67.3
50.2
the original work. However, note that there is limited availability open-source code for these baselines, which presents challenge in aligning the TFI metric. To ensure fairness, we do not report the TFI metric mentioned in the prior work. On the other hand, for task performance and IOU Fl metrics, successfully aligned them.
Table 3: Experimental results(%) in semi-supervised scenarios. Perf. represents task performance and represents the count of segments in FID architecture.
vides improvements for the Evi Inf dataset, whereas does not exhibit substantial gains for the BoolQ and Evi Inf datasets. We attribute this observation to the fact that, in the Evi Inf dataset, rationale tends to appear in the middle toward the end of the passage. Hence, addressing the limitation of input length with the FID becomes imperative.
5.4 Experiment Results
Full-supervised scenario. We select the WT5, WT5-INV,FID-Ex and BERT2BERT frameworks as baselines. Since the WT5-INV framework can not obtain a stable performance on the Mov Rev dataset, we do not report its results. As shown in Tab. 1, experimental results demonstrate the promising potential of the SADM framework. For both task performance (Perf.) and the quality of rationale (IOU F1, TF1, and R-Ace), our frame- work demonstrates varying degrees of improve- ment across five datasets. Notably, our framework has exhibited more significant improvements the RSQ metric, which indicates a more reliable link between the generated rationale and model deci- sion. Specifically, we observe 1.3 points improve- ment the FEVER dataset, 3.1 points improve- the MultiRC dataset, 1.2 points improve- ment on the BoolQ dataset, 1.9 points improvement on the Evi Inf dataset, and 21.7 points improvement on the Mov Rev dataset. Furthermore, experimen- tal results show that the FID architecture only pro-
Semi-supervised scenario. Considering the ex- pensive cost of rationale annotation, semi- supervised scenarios are more likely to be applied in the real world. We select the IB framework, variant of the BERT2BERT framework, and the FID-Ex framework, which demonstrates good per- formance the full-supervised scenario, as base- lines. Following previous settings, we utilize only 25% of the training data with annotated rationales. As shown in Tab. 3, on the Mov Rev dataset, our SADM framework achieves lower performance than the FID-Ex framework in task performance and the quality of rationale but still outperforms in RSQ metric. On the other four datasets, we observe an average improvement of 3.7 points task performance, 1.3 points in IOU F1, 0.9 points in TF1, and 4.2 points the RSQ metric. Over-

---

Perf.
IOU F1
RSQ
RCP
Threshold
0.6
0.8
FEVER
FEVER
w/o SM
92.4
76.3
85.1
75.9
FID-Ex(C=1)
75.4
75.2
75.2
75.1
75.1
w/o RAL
92.0
85.6
86.6
82.5
93.7
SADM(C=1)
77.2
76.8
76.7
76.7
76.7
SADM
93.1
85.9
86.9
83.5
94.5
MulüRCC
passage
93.3
76.3
FID-Ex(C=1)
59.9
53.8
52.9
52.9
52,9
MultiRC
SADM(C=1)
63.9
56,9
55.8
55.8
55.8
w/o SM
77.9
67.1
75.3
70.6
BoolQ
w/o RAL
79.5
72.8
77.8
74.1
79.9
FID-Ex(C=1)
59.7
57.1
55.6
52.6
52.4
SADM
80.1
72.9
78.1
75.2
81.5
FID-Ex(C=6)
59.3
57.7
55.8
55.8
52.7
passage
80.9
74.8
SADM(C=1)
61.1
58.8
57.4
54.9
54.8
BoolQ
SADM(C=6)
59.2
56.9
54.8
52.4
52.2
w/o SM
72.8
44.6
63.8
57.4
Evi Inf
w/o RAL
72.9
52.1
64.0
62.4
80.3
SADM
74.3
52.9
64.5
63.1
80.7
FID-Fx(C=1)
46.7
46.6
46.6
46.6
46.6
FID-Ex(C=6)
54.5
52.5
52.5
52.5
52.5
passage
75.2
61.3
SADM(C=1)
47.2
47.1
47.1
47.1
47.1
Evi Inf
SADM(C=6)
56.9
55.1
54.9
54.9
54.9
w/o SM
67.4
21.2
55.2
47.6
Mov Rev
w/o RAL
63.8
30.9
49.1
56.9
88.4
FID-Ex(C=1)
56.8
41.7
27.1
13.1
SADM
68.3
31.2
49.9
$7.5
91.4
FID-Ex(C=6)
41.7
15.6
W passage
66.9
57.2
SADM(C=I)
84.9
72.9
50.3
23.6
14.6
Mov
SADM(C==)
55.8
21.1
8.5
w/o
89.5
30.5
49.3
23.1
w/o RAL
89.4
56.7
68.6
66.8
99.4
Table 5: Experimental results(%) with different thresh- olds chosen the RSQ metric.
SADM
94.9
63.9
73.2
90.5
99.9
W passage
95.9
89.4
Table 4: Experimental results(%) of our ablation study.
portion of correct decisions that can be made when the model is provided with the annotated rationale the decision-making stage. Our experimental sults show that when the RAL strategy is removed, the RCP metric decreases by an average of 1.3 points across the five datasets. Such phenomenon underscores the critical significance the RAL strategy in enhancing the model's s reasoning ability at the decision-making stage.
all, our framework demonstrates more significant advantages in the supe-supe scenario.
6 Analysis
In our analysis, conduct ablation experiments (in Sec. 6.1) to evaluate the effectiveness of each strategy in our SADM framework. We also con- sider different choices of threshold in the RSQ met- ric (in Sec. 6.2) to provide robust results. Further- more, we provide quantitative analysis to evaluate the lack of a reliable link between the rationale and model decision in the competitive FID-Ex frame- work, which will be shown in detail in App. B.
Additionally, introduced in Sec. 3.1, the pas- sage is optional input at the decision-makin stage. For our SADM framework, the model can make decision based on combination of gener- ated rationale and passage. As shown Tab. 4, ex- perimental results show that considering both gen- crated rationale and passage at the decision-makinn stage can improve task performance, but slightly damage the reliable link between the generated ra- tionale and model decision. Such phenomenon is reasonable. When the model provided with more contextual information, it will make more correct decisions, but at the same time, the information it focuses on will be more scattered.
6.1 Ablation Study
We evaluate the performance SADM without SM strategy and SADM without the RAL strat- egy. As shown in Tab. 4, experimental results show that the performance of the SADM frame- work decreases to certain extent when either the SM strategy RAL strategy is removed, which indicates that both the SM strategy and RAL strat- egy play positive role. Notably, we specifically verify the effect of the RAL strategy on model rea- soning ability. We propose the Rationale-Centir Precision (RCP) metric, which focuses on the pro-
6.2 Choice of Threshold
To ensure standardized evaluation of the gener- ated rationale, we have selected threshold of 0.5 for the recall rate in the RSQ metric. However, we are concerned that the choice the threshold will

---

bring bias in our experimental results. Therefore. we have also conducted experiments with alter- native threshold values of 0.6, 0.7, 0.8, 0.9, and 1.0, which allows us to present a robust evaluation. As shown in Tab. 5, compared to strong baselines, results consistently showcase the significant advan- tage of our SADM framework across the entire range of threshold values in the RSQ metric.
2021. Marta: Leveraging human rationales for ex- plainable text classification. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 5868-5876.
Gino Brunner, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, and Roger Wat- tenhofer. 2019. On identifiahility in transformers. arXiv preprint arXi::998.0022)
Nadia Burkart and Marco F Huber. 2021. survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Rexearch, 70:245- 317.
7 Conclusion
Our proposed SADM framework establishes more reliable link between the generated rationale and model decision while improving task perfor- mance and rationale quality. Furthermore, we ob- serve significant advantages in semi-supervised sce- narios. In the future, we will explore how to opti- mize framework to attain greater performance improvements.
Yuhan Chen, Nuwa Xi. Yanrui Du, Haochun Wang. Chen Jianyu, Sendong Zhao, and Bing Qin. 2023. From artificially real to real: Leverag- ing pseudo data from large language models for low-resource molecule discovery. arXiv preprint arXiv:2309.05208.
Christopher Clark, Kenton Lee, Ming-Wa Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint 1XXv:1905.11044
8 Limitation
Despite the numerous advantages exhibited by our SADM framework, we believe that it still has the following limitations:
Jay Young Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong. Richard Socher, and Byron C Wallace. 2019. Eraser: A benchmark to evaluate rationalized nip models. arXiv preprint 1rXiv:1911.03422
Our SADM framework achieves good perfor- mance in full-supervieed and semi-supervised scenarios, but in the future, we will put more ef- fort into thinking about how apply our SADM framework in the unsupervised scenario. In our work, we only design natural language oriented toward human understanding as prompt templates. Is this necessarily the best for the model? We will further explore their influence.
Yanrui Du Jing Yan, Yan Chen, Jing Liu, Sendong Zhao, Hua Wu, Haifeng Wang, and Bing Qin. 2022. Less learn shorteut: Analyzing and mitigating learning of spurious feature-label correlation. arXiv preprint 2rx.v:2205.12593
Yanrui Du Sendong Zhao, Yuhan Chen, Rai Bai, Jing Liu, Hua Wu, Haifeng Wang, and Bing Qin. 2023a. The calla dataset: Probing Ilms' interactive knowl- edge acquisition from chinese medical literature. arXiv preprint arXiv:2309.04 309.0499.....
9 Ethics Statement
We conduct all experiments on publicly available datasets with authorization from the respective maintainers. All data we use do not involve per- sonal privacy, ethical issues, or sensitive topics.
Yanrui Du, Sendong Zhao, Yuhan Chen, Rai Bai, Jing Liu, Hua Wu, Haifeng Wang, and Bing Qin. 2023b. Gls-ese: A simple but effective strategy to mitigate chinese stm models' over-reliance on superficial clue. arXiv preprint S3Vv:2339004166
Acknowledgements
Jonathan St BT Evans. 1984. Heuristic and analytic pro- cesses in reasoning. British Journal of Psychology. 75(4):451 468..
This work was supported by the National Key R&D Program of China [2021ZDO] 13302]; ; the National Natural Science Foundation of China [62206079]; and the Heilongjiang Provincial Natural Science Foundation of China (YQ2022F006).
Shi Feng. Eric Wallace, Alvin Grissom II, Mohit Iyyer. Pedro Rodriguez, and Jordan Boyd-Graber. 2018 Pathologies of neural models make interpretations difficult. arXiv preprint arXivv1804.0778..
Robert Geirhos. förn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020.
References
Ines Arous, Ljiljana Dolamic, Jie Yang, Akansha Bhard- waj, Giuseppe Cuccu, and Philippe Cauré-Mauroux.
Shorteut learning deep neural networks. Nature Machine Intelligence, 2(1 1:665-677.

---

Asish Ghoshal, Srinivasan Iyer, Bhargavi Paranjape. Kushal Lakhotia, Scott Wen-tau Yih. and Yashar Mehdad. 2022. Quaser: Question answering with scalable extractive rationalization. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1208-1218.
Tyler McDonnell, Mücahid Kutlu, Tamer Elsayed, and Matthew Lease. 2017. The many benefits of amno- tator rationales for relevance judgments. In IJCAI, pages 4909-4913.
Tyler McDonnell, Matthew Lease, Mucahid Kutlu, and Tamer Elsayed. 2016. Why that relevant? col- lecting annotator rationales for relevance judgments. In Proceedings of the AAAI Conference Human Computation and Crowdsourcing, volume 4, pages 139-148.
Aya Abdelsalam Ismail, Hector Corrada Bravo, and Soheil Feizi. 2021. Improving deep learning inter- pretability by saliency guided training. Advances in Neural Information Processing Systems, 34:26726- 26739.
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. Wt5?! training text-to-text models to explain their predictions. arXiv preprint Sttiv:2200.145444
Gautier Izacard and Edouard Grave. 2020. Leverag- ing passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282.
Bhargavi Paranjape, Mandar Joshi, John Thickstun. Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020.
Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation. arXiv preprint arXiv: 1902.
An information bottleneck approach for controlling conciseness rationale extraction. arXiv preprint arXiv:2005.00652
Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface: A challenge set for reading com- prehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Compulational Linguisties: Human Language Technologies, Volume (Long Pa- pers), pages 262.
John Payne, James Bettman, and Eric J Johnson. 1988. Adaptive strategy selection in decision mak- ing. Journal of experimental psychology: Learning, Memory, and Cognition, 14(3):534.
Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, and Zachary C Lipton. 2019. Learning to deceive with attention- based explanations. arXiv preprint arXiv: 1909.0791.
Yuxuan Lai, Chen Zhang, Yansong Feng, Quzhe Huang, and Dongyan Zhao. 2021. Why machine read- ing comprehension models learn shortcuts? arXiv preprint arXiv:2106.0102
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference knowledge discovery and data mining, pages 1135- 1144.
Kushal Lakhotia, Bhargavi Paranjape, Asish Ghoshal, Wen-tau Yih, Yashar Mehdad, and Srinivasan Iyer. 2020. Fid-ex: Improving sequence-to-sqquenc mod- els for extractive rationale generation. arXiv preprint arXiv:2012.15482
Sofia Serrano and Noah A Smith. 2019. Is attention interpretable? arXiv preprint arXiv:1906.0373:
Eric Lehman. Jay De Young, Regina Barzilay, and By- ron C Wallace. 2019. Inferring which medical treat- ments work from reports of clinical trials. arXiv preprint arXiv: 190.00166.
Hua Shen, Tongshuang Wu, Wenbo Guo, and Ting- Hao' Kenneth Huang. 2022. Are shortest ratio- nales the best explanations for human understanding? arXiv preprint 0rXiv:2003.0876
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. arXiv preprint 1606
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. Smoothgrad: removing noise by adding noise. arXiv preprint 1.Xi::7706.0825.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Chazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: De- noising sequence-to cecnnncettt-sqquee to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:/910. 13461.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. Interna- rional conference on machine learning, pages 3319- 3328. PMLR.
Pantelis Linardatos, Vasilis Papastefanopoulls, and Sotiris Kotsiantis. 2020. Explainable ai: review of machine learning interpretability methods. Entropy, 23(1):18.
Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. 2022. Eval- uating the factual consistency large language models through summarization. arXiv preprint 10X0422211.08412.
Yanchen Liu, Jing Yan, Yan Chen, Jing Liu, and Hua Wu. 2023. Smoa: Sparse mixture of adapters to mitigate multiple dataset biases. arXiv preprint arXiv:2302. 14413.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950.

---

James Thome. Andreas Vlachos. Christos Christodoulopoulos, and Arpit Mittal. 2018. Fever: large-scale dataset for fact extraction and verification. arXiv preprint arXiv: 003505555.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint 17Xiv:23022139771.
Haochun Wang. Sendong Zhao, Zewen Qiang, Zijian Li. Nuwa Xi. Yanrui Du, MuZhen Cai, Haoqiang Guo, Yuhan Chen, Haoming Xu, et al. 2023. Knowledge- tuning large language models with structured medical knowledge bases reliable response generation chinese. arXiv preprint arXiv:2309.04175
Tianlu Wang, Diyi Yang, and Xuezhi Wang. 2021. Iden- tifying and mitigating spurious correlations for im- proving robustness nip models. arXiv preprint 10.07:2100.07366.
Sarah Wiegreffe, Ana Marasovié, and Noah A Smith. 2020. Measuring association between labels and free- text rationales. arXiv preprint arXiv:2010.1270 127622........
Omar Zaidan and Jason Eisner. 2008. Modeling an- notators: A generative approach to learning from annotator rationales. In Proceedings of the 2008 ference on Empirical methods in natural language processing, pages 31-40.
Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, and Julie Shah. 2022. Do feature attribution methods rectly attribute features? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36. pages 9623-9633.

---

A IOU F1 and TF1
Annotated
Generated
Matched
Rationale
Rationale
U
IOU F1. IOU FI is used to assess the quality of rationale the sentence level. As shown in the left of Fig. 4, it is assumed that the annotated rationale consists of four sentences (highlighted in green) and the generated rationale consists of three sentences (highlighted in yellow), of which two sentences match the sentences from annotated rationales (connected part). In this sample. the pre- cision ratio is calculated as 2/3 = 0.67, which represents the ratio of matched sentences to the total number of sentences from the generated ratio- nale. The recall ratio is calculated 2/1 0.5,
$
5
5g
Figure 4: Illustration of IOU Fl metric.
Nestor Carbonell played Godzilla
famons his roles Richard Alpert in ABC drama series Lost_... He also known for his regular roles Luis Rivera the sitcom Suddenly Susan, and Baimanuel on the live-action sitcom The Tick...
Passage: He perkaps
which represents the ratio of matched sentences to the total number of sentences from the annotated rationale. By considering both precision and recall, the F1 value can be calculated for each test sam- ple. Finally, the FI values of all test samples are averaged to obtain the Macro IOU F1 score.
Gold Answer: REFUTE.
Origin
Model Ourpur: Answer: REFUTE. Rationale: He
perhups famous his roles Richard Alpert drama series Lost.
Mask
Additionally, shown in the right of Fig. 4, it illustrates the approach for determining whether two sentences match. A ratio computed by di- viding the length of the intersection between two sentences by the length of their union. If this ratio surpasses specific threshold (typically set as 0.5 in prior work), the two sentences are considered to be match. In our study, utilize the longest common substring determine the length of the sentence intersection, and the matching score is subsequently calculated.
Model Output: Answer: <PAD> Rationale: He is perhaps must fumous for roles as Richurd Alpert ABC druma series Lost.
Figure 5: Illustration of the method for analyzing the FID-Ex framework.
technique to the model decision and then initialize the model with the prompt "Answer :< pad > Explanation :" to encourage the generation of rationale. We believe that if there is reliable link between the generated rationale and the model de- cision, the IOU FI and TF1 metrics will change significantly after the masking of the model deci- sion. However, interestingly, as presented Tab. 6, despite masking the model decision, we observe minimal changes in both the IOU Fl and TFI met- rics. This somewhat suggests that, for the FID-Ex framework, the rationale may be generated indepen- dently, with reliable link to the model decision.
TF1. TFI is used to assess the quality of rationale the token level. In each test sample, we define the set tokens from the annotated rationale as Q1, and the set of tokens from the generated rationale as Q2. The intersection of these two sets is denoted as Q. The precision rate is computed by dividing the length of set Q by the length of set Q2, while the recall rate is calculated by dividing the length of set Q by the length of set Q1. By considering precision and recall, the Fl value can be calculated. Finally, the Fl values of all test samples are averaged to obtain the Macro TFI score.
IOU F1 Mask
TFI
FID-Ex
Origin
Origin
Mask
FEVER
85.4
85.2
86.4
86.3
MultiRC
72.1
72.2
77.4
77.3
B Quantitative Analysis
BoolQ
52.3
49.1
64.2
59.8
Evi Inf
32.6
32.3
51.3
$1.2
Our competitive baseline FID-Ex framework in- volves generating the classification decision fol- lowed by rationale in parallel way. To quantita- tively assess the reliable link between the generated rationale and model decision, we conduct exper- iments illustrated in Fig. 5. We apply a masking
Mov Rev
57.1
57.3
68.2
68.1
Table 6: Quantitative analysis of FID-Ex framework.


Reviews:
[{"review_id": "50l0ltryJy", "reviewer": "Reviewer_AqvU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer AqvU\nThanks for taking the time to review our work. Thank you for recognizing our efforts in addressing a focused and meaningful problem, and appreciating the effectiveness of our proposed method and the high quality of our paper. We have summarized your main questions as follows and will address them one by one.\n\n### Q1: Since previous works have studied the process of pipeline framework (first I->R and then IR->O), this work is less exciting.\n\nDespite extensive discussions surrounding the pipeline framework, recent work [2] claims that a significant challenge still confronts its implementation.\n\n- Under the same scale of parameters, a considerable performance gap can be observed when compared to the end-to-end frameworks (such as I-OR or I-RO frameworks). This phenomenon is particularly evident as shown in the performance metrics outlined in Table 2 of our paper. The pipeline BERT2BERT framework's performance notably lags behind that of the end-to-end WT5, WT5-INV, and FID-Ex frameworks.\n\nExcitingly, our proposed pipeline framework (SADM) outperforms the end-to-end frameworks without adding any additional parameters. Furthermore, compared to different frameworks, our proposed framework can establish a more reliable link between the generated rationales and decisions. We believe that all of the above are crucial for constructing an explanation model.\n\n\n### Q2: Different variants should be compared to more comprehensively evaluate the performance, e.g., I->RO, I-O, and then IO->R, etc.\n\nIn our work, we have conducted a comprehensive and detailed comparison of different variants. However, expression problems in writing may have caused some confusion for you and made you misunderstand our work. To shed light on the selection process for different variants, we provide the following elucidation.\n\nOur work focuses on explanation models capable of making decisions while providing rationales to support their decisions. For the I-O framework, it lacks the capacity to provide rationales to support their decisions. Therefore, we have not taken the I-O framework into consideration.\n\nFor explanation models, there are the following frameworks:\n- I-OR\n- I-RO\n- I-O, O-R\n- I-R, IR-O\n\nWe choose the top-performing and most representative method from each framework, establishing a robust baseline for our study. Within our baselines, WT5 and FID-Ex belong to the I-OR framework, WT5-INV belongs to the I-RO framework, and BERT2BERT and IB belong to the I-R, IR-O framework.\n\nAs for the I-O, O-R framework, it has rarely been considered in recent work due to its often counterintuitive behavior (as introduced in Lines 145-150). Besides, the work[1] has explicitly highlighted the superior performance of the I-R, R-O framework (the BERT2BERT method) in comparison to the I-O, O-R framework (including methods like LIME and Attention Score). Meanwhile as illustrated in Table 2 and Table 3, our proposed framework significantly outperforms the BERT2BERT framework. Consequently, any concerns surrounding the I-O, O-R framework are redundant.\n\n\n\n\n### Q3: Explore the potential of LLMs.\n\nIn the current era, it is crucial to explore the potential of Large Language Models (LLMs). But in our scenario, we have the following concerns:\n\n- Since LLMs are completely black-box (like chatgpt and llama), we are not sure whether they have seen the test data during the training phase. Therefore, it is difficult for us to provide a fair comparison.\n\n- Evaluating LLMs within a standardized framework is intricate. LLMs often generate some free-text rationales regardless of our control. Therefore, IOU and TF1 metrics are unsuitable.\n\n- In our comparison experiments, the parameter size used by all methods is the same, that is, 220M. If compared to LLMs, the benefit of model parameter size is hard to ablate.\n\nBased on the above concerns, we do not compare LLMs in our main experiments. But we agree with your valuable suggestion that we should explore the potential of LLMs. We have the following plans in the camera-ready version:\n- Clarify our concerns about LLMs.\n- Following parameter-efficient fine-tuning technology (such as LoRA), we apply our SADM framework to the base model Llama to explore the potential of LLMs.\n\nOverall, thanks again for taking the time to review our work. I hope our response can clear up your confusion and improve your perception of our work, which is of the utmost importance to us. In the camera-ready version, we will make changes to some of the details you suggested.\n\n[1]Jay DeYoung et al., ERASER: A Benchmark to Evaluate Rationalized NLP Models.\n\n[2]Wiegreffe et al., Measuring Association Between Labels and Free-Text Rationales.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This work studies an important problem, which is meaningful.\n\n2. The proposed method is shown to be effective on various datasets.\n\n3. The paper is well-written and easy to follow.", "reasons_to_reject": "1. The idea of the proposed method is not new, which makes this work less exciting. Previous works have studied the process of I->R and then IR->O where I, R and O means the input, rationale and output. Hence, the contribution of this work seems to be more on the engineering side.\n\n2. Different variants should be compared to more comprehensively evaluate the performance, e.g., I->RO, I-O and then IO->R, etc. These variants can benefit the evaluation and help to understand the contribution of different components in the method. Also, I suggest the authors explore the potential of LLMs on this problem, since some results show that LLMs seem to be pretty good at generating consistent answers and rationales.", "questions_to_authors": "", "paper_topic_and_main_contributions": "Topic:\n\nThis work studies an important problem, which is the consistency between the model predictions and the corresponding generated rationales. This work proposes a two-stage framework, which is to prompt the model to generate the rationale first, and then generate the answer based on the rationale. Experiments on different datasets demonstrate the effectiveness of the proposed method.\n\nContribution:\n\nThis work proposes a framework that can generate consistent answers and rationales.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "1. Hui Liu, et al. Towards Explainable NLP: A Generative Explanation Framework for Text Classification. ACL 2019.", "Typos_Grammar_Style_and_Presentation_Improvements": "1. Line 552: Fig. -> Table"}, "extra_text": {}}, {"review_id": "nr5OwcJcTJ", "reviewer": "Reviewer_WBPQ", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer WBPQ\nThanks for taking the time to review our work. Thank you for acknowledging the effectiveness of our proposed SADM framework. We have summarized your main questions as follows and will address them one by one.\n\n### Q1: What are the differences compared to the previous work[1]?\n\nWe summarize the differences from the work [1] in the following four aspects:\n\n- **Different scenarios:** The work[1] focuses on the free-text rationale scenario, whereas our work focuses on the extracted rationale scenario.\n- **Different motivations:** The work[1] aims to conduct an analysis to determine whether the pipeline framework (I-R, R-O) or the end-to-end (I-RO) framework is better suited for the free-text rationale scenario. Our work aims to enhance the reliable link between generated rationales and decisions in the extracted rationale scenario. To achieve this, we introduce a novel SADM framework. \n- **Different conclusions:** Due to different scenarios, we come to different conclusions.  The work[1] concludes that pipeline frameworks are not suitable for the free-text rationale scenario, while our work concludes that pipeline frameworks are well suited for the extracted rationale scenario.\n- **Different analysis perspectives:** In the work[1], the analysis of the correlation between generated rationales and decisions focused solely on the end-to-end framework, without comparing it to the pipeline framework. In contrast, our work comprehensively compares the performance of the pipeline framework and the end-to-end framework, including the performance of decisions, the quality of generated rationales, and the correlation between generated rationales and decisions. \n\nTo further resolve your confusion, we provide some additional clarification as follows:\n- **Free-text rationale scenario vs extracted rationale scenario.** For the free-text rationale scenario, the goal is to have the model make the correct choice to question based on the candidate choices and generate coherent paragraphs of free text as rationales to support its decision. For the extracted rationale scenario, the goal is to have the model provide the answer to the question based on the given passage while extracting rationales from the passage to support its decision. \n\n- **Why different conclusions are drawn in different scenarios?** The work[1] has revealed that in the free-text rationale scenario, pipeline frameworks face the following challenges: 1. Cascading errors due to poor quality rationale generated during the I-R phase. 2. The insufficient generated rationales result in poor performance. 3. Double the number of parameters to reach comparable performance to an end-to-end (I→OR) framework, but still often perform worse. However, these challenges appear to be non-existent in the extracted rationale scenario. Concerning challenges 1 and 2, as illustrated in Table 2 of our paper, our SADM framework ensures the quality of generated rationales during the I-R phase. We attribute this capability to the feasibility of employing information extraction (IE) tasks on the t5-base model. However, for the free-text rationale scenario, generating free-text still presents challenges. Besides, for the performance of models, our proposed pipeline framework has surpassed the end-to-end framework. As for challenge 3, this is addressed in our proposed framework. As depicted in Lines 247-252 and shown in Figure 2, our SADM framework shares parameters between the two stages during the training phase and employs a joint training strategy. For the inference stage, we use the shared model to perform two stages separately. Experiments show that our framework can achieve better performance without adding additional parameters.\n\n- **The analysis for the pipeline framework was not considered in the work[1].** Although the pipeline framework exhibited poorer performance in the free-text rationale scenario, this does not necessarily imply that the pipeline framework would perform a worse correlation between generated rationales and decisions. For instance, due to wrong rationales, the model provides incorrect decisions, also indicating a strong correlation. \n\n\n\n### Q2: Missing the evaluation[1].\n\nThe work [1], to address the challenge of the free-text rationale scenario, introduces novel evaluation metrics designed to assess the quality of generated rationales and quantify the correlation between generated rationales and decisions. Conversely, for the extracted rationale scenario, there is already a well-established evaluation system. We adhere to the evaluation metrics IOU and TF1 presented by the ERASER Benchmark [3] to evaluate the quality of generated rationales, consistent with prior research [4] [5] [6]. Building upon the IOU metric, we further introduce a metric tailored to enhance the measurement of the correlation between generated rationales and decisions within the extracted rationale scenario.\n\n- **The quality of generated rationale:** Due to the inadequacy of IOU and TF1 in assessing the quality of free-text rationales, the work[1] proposed a novel evaluation metric. This evaluation metric involves first training a model M with questions and annotated rationales, and then evaluating the performance of model M on generated free-text rationales to reflect the quality of the generated rationales. Despite this evaluation metric not having been previously considered in the extracted rationale scenario, in order to enhance the robustness of our work, we incorporate this evaluation metric and conduct an additional experiment as follows. \n\n\n|   Framework    | FEVER | MultiRC | BoolQ | Evi Inf | Mov Rev |\n|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|\n| Gold    | 94.08    | 80.41    | 78.56    | 90.09    | 100    |\n| FID-Ex(C=1)    | 92.13    | 77.62    | 72.78    | 66.32    | 87.94    |\n|   FID-Ex(C=10)    | -    | -    | 73.24    | 75.23    | 95.48    |\n| SADM(C=1)    | **92.21**    | **78.75**    | 72.92    | 65.38    | 89.95    |\n| SADM(C=10)    | -    | -    | **73.64**    | **76.02**    | **96.48**    |\n\n\n\nWe conduct a comparison between our SADM framework and the state-of-the-art FID-Ex framework. In this evaluation, a larger experimental value corresponds to a higher quality of generated rationales. The pinnacle of performance is represented by the ''Gold'' standard. Notably, the experimental results unequivocally demonstrate that our SADM framework continues to surpass the strong baselines, underscoring its exceptional performance and superiority.\n\n\n- **The correlation between generated rationales and decisions:**  In the free-text rationale scenario, assessing the validity of generated rationales is challenging due to the inherent ambiguity of standard rationales. This makes it difficult to directly measure the correlation between the generated rationale (R) and the output (O). Therefore, in the work[1], for the free-text rationale scenario, a novel evaluation approach is proposed. This approach involves perturbing the input (I), observing changes in both the model's performance (O), and the quality of the generated rationales (R), and assessing if they exhibit similar trends. While this evaluation approach indirectly reflects the correlation between R and O, there remains a concern about coincidental alignment in their change trends. In contrast, in the extracted rationale scenario, where standard rationales are well-defined, the use of statistical metrics proposed by the ERASER benchmark enables the assessment of the validity of each rationale. This facilitates a straightforward measurement of the correlation between R and O, as discussed in Lines 332-354. Therefore, in our work, we propose a new metric RSQ for evaluating the correlation between generated rationales and decisions for the extracted rationale scenario, building on the work of ERASER benchmark [3].\n\n\n\n\n\n### Q3: Missing a baseline [2].\nThe work[3] has indicated that the performance of the pipeline framework introduced in work [2] is notably inferior to that of our selected baseline, the BERT2BERT pipeline framework. It is important to note that our work not only delves into the pipeline framework but also encompasses the comprehensive exploration of end-to-end frameworks. Consequently, we choose the top-performing and most representative methods from each framework as our strong baselines. Therefore, for the pipeline framework, the BERT2BERT framework is chosen by us.\n\n\n\n### Q4: Further clarification of method.\n\nThank you for your valuable suggestions, we will provide further background information on the SM technique in the camera-ready version. Besides, we need to clarify that the RAL technique is first proposed in our work, and we will further clarify the motivation of the RAL technique and the overall goal of the SADM framework.\n\n### Q5: Lack of citations.\n\nThank you for promptly pointing out the issues in our writing. The citations for WT5 and FID-Ex have been marked in Lines 46-47, and we will cite the references for BERT2BERT[3] and IB[4] in the camera-ready version.\n\n### Q6: Lack of Human evaluation.\n\nFor our extracted rationale scenario, automated metrics are a good measure of the quality of the generated rationale. Recent research[3] [4] [5] [6] over the past three years has uniformly employed IOU and TF1 metrics to assess the quality of generated rationales. Therefore, we keep consistent with them for a fair comparison. Besides, in previous work[3], IOU and TFI metrics have been also shown to align with human evaluations. We believe that human evaluation is more necessary and indispensable for the free-text rationale scenario.\n\nOverall, thanks again for taking the time to review our work. I hope our response can clear up your confusion and improve your perception of our work, which is of the utmost importance to us. In the camera-ready version, we will make changes to some of the details you suggested, such as adding missing references, clarifying the motivation of the method, etc. We will do everything to make our work more convincing.\n\n\n[1]Wiegreffe et al., Measuring Association Between Labels and Free-Text Rationales.\n\n[2]Jain et al., Learning to Faithfully Rationalize by Construction.\n\n[3]Jay DeYoung et al., ERASER: A Benchmark to Evaluate Rationalized NLP Models.\n\n[4]Bhargavi Paranjape et al., An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction.\n\n[5]Asish Ghoshal Srinivasan Iyer et al., QUASER: Question Answering with Scalable Extractive Rationalization.\n\n[6]Kushal Lakhotia et al., FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed method seems effective as compared to baseline methods.", "reasons_to_reject": "- The distinction between this work and previous pipeline frameworks [1, 2] remains unclear, despite the application of new techniques such as SM and RAL. Additionally, the omission of important baseline methods in this paper further complicates the evaluation. Without a clear comparison and distinction, the novelty of this work remains ambiguous. To strengthen the paper's contribution, it is crucial to explicitly highlight the unique aspects that set it apart from existing approaches and conduct a comprehensive comparison with relevant baseline methods. This will provide a clearer understanding of the original contributions and potential advancements made by this work.\n- The method section (Sec. 3) could be improved to enhance its clarity. It would greatly benefit from including background information on previous techniques (e.g., SM, RAL) to provide readers with a better context for understanding the proposed method. Additionally, explicitly formulating the objectives (Sec. 3.2) would make the proposed method more accessible and understandable.\n- Lack of citations in Section 2.2 (e.g., BERT2BERT, IB, WT5 and FID-Ex).\n- It would be better to include human evaluations on the generated rationales.\n\n[1] Wiegreffe et al., Measuring Association Between Labels and Free-Text Rationales\n\n[2] Jain et al., Learning to Faithfully Rationalize by Construction", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on the research problem of loosely linked model-generated rationales to its predictions, such as correct predictions associated with wrong rationales or wrong predictions associated with correct rationales. To address this problem, a two-stage framework called Self-Attribution and Decision-Making (SADM) is proposed, where the model is trained jointly with both the self-attribution and decision-making objectives during training, while first prompted to extract the rationale from the given input and then prompted to utilize the rationale to make a decision during inference. Besides, the sentence mark (SM) strategy is applied to prevent the generative model from generating random and irrelevant rationale. And reasoning augment learning (RAL) is used to enhance the model’s reasoning ability. The proposed method is compared with several baselines (BERT2BERT, IB, WT5, WT5-INV, and FID-Ex) on five datasets in the ERASER benchmark. Experiments demonstrate the effectiveness of the proposed method.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "[1] Wiegreffe et al., Measuring Association Between Labels and Free-Text Rationales\n\n[2] Jain et al., Learning to Faithfully Rationalize by Construction"}, "extra_text": {}}, {"review_id": "AXdKeWNgnj", "reviewer": "Reviewer_TZrn", "summary": "", "questions": "1. What is the proportion of the unreliable link problem? And how many problems does SADM address? ", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer TZrn\nThanks for taking the time to review our work. Thank you for recognizing our efforts in addressing a focused and meaningful problem, and appreciating the effectiveness of our proposed method and the adequacy of our experiments. We have summarized your main questions as follows and will address them one by one.\n\n### Q1: What is the proportion of the unreliable link problem? And how many problems does SADM address?\n\nIn our work, we introduce the RSQ metric to measure the correlation between generated rationales and decisions. However, for two cases: a. the model might make the correct decision but provide incorrect rationales; b. make incorrect decisions while giving the correct rationales. The RSQ metric fails to indicate whether our proposed framework can improve the above cases separately and to what extent. Therefore, to make our paper more complete, we provide another qualitative analysis as follows.\n\nAs introduced in Lines 336-343, we divide the test samples into four classes:\n- $r_c d_c$: Samples where both the generated rationale and model decision are correct.\n- $r_w d_w$: Samples where both the generated rationale and model decision are wrong.\n- $r_c d_w$: Samples where the generated rationale is correct, but the model decision is wrong.\n- $r_w d_c$: Samples where the generated rationale is wrong, but the model decision is correct.\n\nFor the case that the model may make wrong decisions while attributing correct rationales, we propose the RSQ-W metric. The purpose is to observe the proportion of wrong decisions made by the model when the model attributes the correct rationales. The RSQ-W metric is as follows:\n$$\nRSQ-W=\\frac{Num(r_c d_w)}{Num(r_c d_c + r_c d_w)} \n$$\n\nFor the case that the model may make correct decisions while attributing wrong rationales, we propose the RSQ-C metric. The purpose is to observe the proportion of correct decisions made by the model when the model attributes the wrong rationales. The RSQ-C metric is as follows:\n$$\nRSQ-C=\\frac{Num(r_w d_c)}{Num(r_w d_w + r_w d_c)} \n$$\n\nThe experimental results are as follows.\n\n\n|       | RSQ-W | RSQ-C |\n|-----------------|-----------------|-----------------|\n|  FEVER  |     |      |\n|  WT5  |  6.67%   | 86.94%    |\n| WT5-INV| 6.76%   | 85.06%    |\n|FID-Ex(C=1)| 5.47% | 82.72%   |\n|SADM(C=1)|  **4.97%**  | **81.57%**   |\n|  MultiRC  |     |      |\n|  WT5  |  20.68%   | 72.34%    |\n| WT5-INV| 21.71%   | 72.52%    |\n|FID-Ex(C=1)| 19.81% | 72.54%   |\n|SADM(C=1)|  **18.36%**  | **69.34%**   |\n|  BoolQ  |     |      |\n|  WT5  |  23.56%   | 67.41%    |\n| WT5-INV| 26.26%   | 66.10%    |\n|FID-Ex(C=1)| 20.05% | 65.53%   |\n|FID-Ex(C=10)|  22.80%  | 64.96%   |\n|SADM(C=1)| **17.01%** | **62.95%**   |\n|SADM(C=10)|  22.61%  | 63.89%  |\n|  Evi Inf  |     |      |\n|  WT5  |  17.91%   | 62.80%    |\n| WT5-INV| 15.96%   | 57.97%    |\n|FID-Ex(C=1)| 23.08% | 57.73%   |\n|FID-Ex(C=10)|  12.87%  | 60.52%   |\n|SADM(C=1)| **8.42%** | **57.45%**   |\n|SADM(C=10)|  11.44%  | 58.75%  |\n|  Mov Rev  |     |      |\n|  WT5  |  8.71%   |    90.95% |\n| WT5-INV| -   | -    |\n|FID-Ex(C=1)| 7.86% |  86.44%  |\n|FID-Ex(C=10)| 2.50%   | 95.80%  |\n|SADM(C=1)|   4.30%   |   **84.62%** |\n|SADM(C=10)|  **1.85%**  |  94.51% |\n\nIn this evaluation, a smaller experimental value corresponds to a better performance of the model. Experimental results show that our proposed SADM framework achieves stable improvements for both cases. I hope the quantitative analysis we provide will address your concerns.\n\nOverall, thanks again for taking the time to review our work. I hope our response can clear up your confusion and improve your perception of our work, which is of the utmost importance to us. In the camera-ready version, we will add the above additional analysis to our paper to make our work more complete.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper finds an important problem: the link between the rationale and classification decisions generated by models is not reliable. \n2. To solve this problem, this paper proposes SADM,  which could jointly optimize both the self-attribution and decision-making objectives. SADM is prompted to extract rationales and employs a reasoning augment learning strategy to decision-making. Extensive experimental results show the effectiveness of the proposed method. Besides, SADM also incorporates Fusion-In-Decoder to address the challenges of lengthy texts and Sentence Mark strategy to mitigate the issue of irrelevant rationale generation.\n3. Extensive experiments show the effectiveness of SADM. The experiments include full-supervised scenarios and semi-supervised scenarios, and SADM both achieve the best performance compared to the baselines.", "reasons_to_reject": "1. The proposed problem includes two aspects: a. model may make correct decisions while attributing wrong rationales; b. make poor decisions while attributing correct rationales. Actually, the first problem seems to be normal. Considering the faithfulness of rationale extraction,  the model may just make correct decisions while attributing wrong rationales. This is a long-term research topic in recent interpretability community. Therefore, only the second problem need to solve.\n2. This paper only qualitatively analyze the problem (Examples in Figure 1). However, this is not enough. To make this paper more qualitative, this paper should statistically analyze the proportion of the unreliable link problem. And this paper should show whether the proposed method could solve them. This would make this paper more complete.", "questions_to_authors": "1. What is the proportion of the unreliable link problem? And how many problems does SADM address? ", "paper_topic_and_main_contributions": "This paper finds that the problem that the rationale does not genuinely support and convincingly justify the model decision. In specific, the model may make correct decisions while attributing wrong rationales or make poor decisions while attributing correct rationales. To solve such problems, this paper proposes self-attribution and decision-making (SADM), which jointly optimize both the self-attribution and decision-making objectives. SADM is prompted to extract rationales and employs a reasoning augment learning strategy to decision-making. Extensive experimental results show the effectiveness of the proposed method.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for self-rationalization. The proposed method is an instance of I->R, IR->O, (given input, generate rationale. Given input + rationale, generate output), which is a good design that 1) improves the accuracy of O because it is conditioned on R, 2) increases the consistency between R and O. 

While this setup is not new, e.g.[1], the details of how the I->R model and the IR->O model are implemented led to improved results in the 5 datasets of the ERASER benchmark.

During the rebuttal, the authors added new results to quantify the consistency between R and O using their method vs. previous work, and showed greater consistency. This result is an afterthought, so the authors should make sure to add it to the paper.


[1]Wiegreffe et al., Measuring Association Between Labels and Free-Text Rationales.
Decision: Accept-Findings