ZARA: Improving Few-Shot Self-Rationalization for Small Language Models
Authors: Wei-Lin Chen
An-Zi Yen
Cheng-Kuang Wu
Hen-Hsen Huang
Hsin-Hsi Chen
Venue: EMNLP 2023 Findings (2023)
Keywords: free-text explanation
rationale
self-rationalization
TL;DR: This work improves few-shot self-rationalization for small LMs via self-training with a novel zero-shot rationale-answer augmentation strategy.

Abstract: Language models (LMs) that jointly generate end-task answers as well as free-text rationales are known as self-rationalization models.
Recent works demonstrate great performance gain for self-rationalization by few-shot prompting LMs with rationale-augmented exemplars.
However, the ability to benefit from explanations only emerges with large-scale LMs, which have poor accessibility.
In this work, we explore the less-studied setting of leveraging explanations for small LMs to improve few-shot self-rationalization.
We first revisit the relationship between rationales and answers.
Inspired by the implicit mental process of how human beings assess explanations, we present a novel approach, Zero-shot Augmentation of Rationale-Answer pairs (ZARA), to automatically construct pseudo-parallel data for self-training by reducing the problem of plausibility judgement to natural language inference.
Experimental results show ZARA achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric.
In addition, we conduct human and quantitative evaluation validating ZARA's ability to automatically identify plausible and accurate rationale-answer pairs.

ZARA: Improving Few-Shot Seff-aationllzation for
Small Language Models
Wei-Lin Chen'
An-Zi Yen? Cheng-Kuang Wu' Hen-Hsen Huang" Hsin-Hsi Chen'
National Taiwan University, Taiwan
National Yang Ming Chiao Tung University, Taiwan
Academia Sinica, Taiwan w chenenlg. csie. ntu. edu. tw hhchen@ntu. edu. tw
Abstract
Question:
Which sentence more nonsensical? (A) I drove my to the gas station.
Language models (LMs) that jointly generate end-task answers well as free-text rationales are known as self-rationalization models. Re- cent works demonstrate great performance gain for rationaatinii by few-shot prompt- ing LMs with rationale-augmented exemplars. However, the ability benefit from expla- nations only emerges with large-scale LMs. which have poor accessibility. In this work, we explore the less-studied setting of leveraging explanations for small LMs to improve few- shot self reffraationllization. We first revisit the relationship between rationales and answers. Inspired by the implicit mental process of how human beings assess explanations, we present a novel approach, Zero-shot A Augmentation of Ratior pairs (ZARR),, to automat- ically construct pseudo- paraloelaaallelll data for self- training by reducing the problem of plausibility judgement to natural language inference. perimental results show ZARA achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric. In addition, we conduct human and quantita- tive evaluation validating ZARA's ability to automatically identify plausible and accurate aationale-answer pairs.
(B) I drove my computer to gas station.
Rationale:
You can drice compater.
Answer: (B)
Implicit Plausibility
Plausible?
Score
x
Approsimate
Plausibility Threshold
Explicit Plausibility
Score
Figure I: The role of our plausibility agent. As the main component of ZARA, the agent, i.c, the approximator, imitates how humans assess the plausibility of explana- tions, in explicir fashion.
al., 2020), free-text explanation is more expres- sive, inherently apt for human comprehension and brings richer information in addition to input con- text (Camburu et al., 2018: Wiegreffe er al., 2021). Yet, the construction of NLE datasets is expensive and challenging due to quality control issues such as inconsistency and under-specifiiattion (Wiegr- effe and Marasovic, 2021). The development of in- terpretable NLP systems which can provide NLEs in few-shot necessitated.
1 Introduction
Driven by the concerns of whether the decisions made by the artificial intelligence models are trust- worthy, providing free-text, natural language expla- nations (NLEs) has drawn substantial attention in the research community (Camburu et al., 2018; Li et al., 2018; Rajani et al., 2019; Aggarwal et al., 2021; Chen al., 2022). Comparing with popular explanation techniques within the input scope, e.g., attributing feature importance scores to tokens (Li et al., 2::6; Godin et al., 2018) or extracting frag- ments of text highlights (Lei et al., 2016; Jain
Recent works (Wei et al., 2022; Wang et al., 2022b; Lampinen et al., 2022) achieve few-shot self self-rationalization, i.e., jointly generating free- text explanations and end-task labels. by extending the usage of NLEs compose chain-of-thought (CoT) input-rationall-outpi demonstrations for prompt- learning. Comparing with standard
We use the term "frec-text explanation" and "natural lan- guage ex explanation" interchangeably. and the term "expinan- tion" can also refer to the rationale generated by the model.
httrs:://githuu.oo///tttapaapp wot.:///nmnnnlaaa//.aa...

---

prompting (i.e.. without rationales), prompting with rationalc-augmented exemplars triggers LM's complex reasoning ability, significantly boosting the end-task performance. However, the main draw- back is that only excessively large LMs (generally 100B-plus) demonstrate this ability to leverage ex- planations, which sharply emerges when scaling model size sufficiently (Wei et al., 2022; Lampinen al., 2022).
posed standardized few-shot self-rationalizattio benchmark- 3,4% 5.1% and 3.0% 5.8% for task accuracy and the associated explanation metric, respectively. Additionally, we validate the approximator's ' ability with both human and quan- titative evaluations. The results suggest our ap- proximator can effectively select plausible expla- nations that lead to higher accuracy for end-task predictions. In summary, our main contributions are three-fold:
In this work, we explore the less-studied setting of improving few-shot self-rationalization only re- lying on affordable, small LMs (200M~2.7B), We adopt self- (Scudder, 1965), simple yet effective methodology that is not practical for large LMs in most real-world scenarios. We first investi- gate the relationship between the generated expla- nations and end-task predictions, and find plausible explanations are usually paired with correct label predictions. Namely, plausibility is strong indi- cator for answer correctness. Motivated by this finding, we propose Zero-shot Augmentation of Rational-Answer pairs (ZARA) for self-training.
We show how to leverage explanations for small LMs by an in-depth analysis of the rela- tionship between rationales and task labels.
2. We propose ZARA, a novel approach for small LMs to improve self-raatinnlizzaiinn with self-training.
3. Our NLI-based approximator sheds light on the potential of automatic evaluation for expla- nation plausibility and post-hoe verification for label accuracy.
Specifically, we reduce the problem of ussessing rationale plausibility to the task of natural language inference (NL.), and propose a zero-shot plausi- bility approximalor towards automatic assessment of the generated rationales, without requiring any ground-truth labels or golden explanations. The approximator can be viewed as an agent for plau- sibility judgement. As illustrated in Figure 1, to determine the plausibility of the rationale, humans implicitly ask themselves whether they can draw conclusions to the predicted answer by understand- ing the task, the input question, and the supported rationale with their logic and reasoning. To ap- proximate such process explicitly, the approximator leverages the ability of textual entailment yield a probability score indicating the explanation plau- sibility. Connecting to the trffntrainng paradigm, we first train a self-rationaliizaiion model by few- shot prompt-based learning with natural language prompts, and leverage the approximator to collect pseudo-parallel data, i.e, unlabeled inputs paired with high cogh-confident rationale-an -nswer pairs, for ere- ating an augmented training which is then used learn an improved self-rattonalizztioon model.
2 Background and Motivation
Given a trained self-rationalization model fo(-) and an input sequence x, we denote prediction fo(x) (f, a). where f is the generated free-text rationale and is the predicted answer, typically a classification label. Note that f and à are parsed from the output sequence of fo().. Evaluation of a self ralionaatoniliioation model requires assessing both à for the end-task performance and f for the quality of the explanation. With the lack of an ideal and unified automatic metric, the current gold standard for determining the quality of is conduct hu- man evaluation to check its plausibility (Marasović et al., 2020; Kayser et al., 2021; Wiegreffe et al., 2022; Marasovic et al., 2022). An ideal is con- sidered to be plausible if it is able to justify à, that is, providing a logical and reasonable explanation supporting the model's prediction.
However, if A is deemed plausible by humans, does not mean à is correct. As the example in Table 1, commonsense would know "bed" is likely the answer, yet the generated explanation for the corresponding prediction "couch" still plausi- ble. Plausibility illustrates the degree of convince- ment towards the model's prediction, regardless of whether the model is actually making an accurate prediction or not (Jacovi and Goldberg, 2021).*
With various small-size LMs, experiments show our approach notably improves the FEB bench- mark? (Marasovic et al., 2022) recently pro-
htttps///gi wtt.coccmaaaaa.aa aomaalleeai/eeb (1 (Licensed under the Apache License 2.0.)
FFor confounder-free setting, prior works (Kayser et al.,

---

The Percentage of Correct/Incorrect Answer conditioned on Plausibility
ECQA
Not plausible
78.9
aa
36.4
A16
67.02.4
Plausible 96.8
20
40
60
80
100
0
20
60
80
100
20
60
80
100
20
40
80
100
Conrectio]
IncomeetiSi
Figure 2: The percentage of correct/incorrec answers when explanations are plausible and not plausible. The y-axis denotes the percentage of explanations deemed plausible/not plausible by humans.
Instamce
as the dataset for experiments throughout this work. FEB consists of four sub-tasks from ex- isting English-language explainable datasets with free-text explanations: (1) Nonsensical sentence selection (COMVE: Wang et al., 2019). Given sentences, select the sentence that is less likely to make sense. (2) Offensiveness classification (SBIC; Sap et al., 2020). Classify a given post as offensive or not. (3) Natural language inference (E-SNLI; Camburu et al., 2018). Classify the re- lationship between two sequences as entailment, neutral, contradiction. (4) Multiple-choice com- monsense QA (ECQA; Aggarwal al., 202 I). Given question, select the correct answer from five choices.
Question: Danny is human. He needed to catch flight, hut stayed long object made for hamans relox Where might he have been?
Choices: buthroom & couch särport d. homes bed
Prodiction()): couch
Label(a); bed
Explanation: Couch an objocct made for humans relax Danny might have couch.
Table 1:
A plausible but inaccurate prediction of an instance from ECQA dataset output by our model.
Naturally, generating plausible explanations that can justify the wrong answers should be much harder comparing to justifying the correct answers. Since such i usually demonstrates a slight pivot from commonsense yet still introduces a sound rea- son to support the inaccurate à. We hypothesize this plis-blasiible explanation towards inaccurate end- task prediction is not the circumstance in most cases of (ā, f). In other words, if f is considered be plausible, it is likely that is correct pre- diction. Hence, the first research question arises: RQI: "To what extent do plausible explanations imply correct label predictions?" And we could verify RQI, the follow-ur up question would be RQ2: "Is it possible to automatically identify plausible i and utilize (f, à) for further model improvement?"
The goal for each sub-task is the same, namely, to predict label for the underling classification task and generate a free-text explanation support- ing the model's decision. Each sub-task has 60 episodes, and each episode is train-test split with 48 training examples and 350 evaluation examples. This design of no extra validation data encompasses the FLEX principles (Bragg et al., 2021) for per- forming robust few-shot NLP evaluation to avoid per-episode paper-parameter tuning, which could inflate the evaluation results considerably as shown in previous work (Gao et al., 2021). ).. Hence, single set of parammte is used across all episodes.
In the following of our work, we answer RQ/ by inspecting the interrelationship between the plau- sibility of f and the correctness of à (Section 4), where we show evidence supporting the linkage to RQ2. Ergo, we propose ZARA coupled with self- training to accomplish RQ2 (Section 5), improving few-shot self-rationaliizaioo models.
Correlation between Plausibility and Correctness
described Section 2, following we attempt to answer RQI by measuring the correlation be- tween the plausibility of é and the correctness of à. We conduct human studies on results from a self-rationalizztion model (without self-training) using the FEB dataset. We adopt prompt-base based fine- tuning with natural language prompt on sequence- to-sequence language model perform few-shot self-rationalization.
3 Datasets and Tasks
We adopt FEB (Marasovic al., 2022), newly proposed few-shot self-rationalizzatoon benchmark,
2021: Marasovic et al., 2022) only evaluates with à = a. i.c. explanation for the correctly predicted answer. This may overestimate the quality of explanations (Wiegreffe et al., 2022).
For each episode of the sub-task, we train a self-

---

Mapping and Example
Example
Example
choicel: drove my to the gas station.
Post: just when i thought women couldn't it get any stupider.
choice2: i drove my computer to the gas station.
Answer: choice2
Answer: offensive
Rationale: you drive computer.
Rationale: this post implies that women stupid.
Mapping
Mapping
COM VE
Premise: [Rationale]
SBIC
Premise: The post: [Post]
Hypothesis: [Answer's sentence]
Hypothesis: The post is [Answer] because
NLJ Class: Contradiction
[Rationale]
NLI Class: Entailment
Mapped example
Mapped example
Premise: you can't 't drive a computer.
Premise: The post just when i thought women couldn't 't get any stupider. The post is offensive because this post implics that women are stupid.
Hypothesis: drove my computer to the gas station.
Hypothesis:
Example
Example
Premise: woman black mesh skirt plays
Question: what is place that has a bench nestled in trees?
acoustic guitar.
Hypothesis: a woman is wearing black.
Choices:(a) state park (b) bus stop (c) bus depot (d)
Answer: Entailment
statue (e) train station
Rationale: The woman wearing black mesh, she wearing black.
Answer: (a)
Rationale: state park protected public garden.
public gardens generally have benches for people
to sit and relax. gardens are places with lots of
E-SNLI
ECQA
trees and plants.
Mapping
Mapping
Premise: [Premise] [Rationale].
Premise: Because [Rationale]
Hypothesis: [Hypothesis]
Hypothesis: The answer
question
NLI Class: [Answer]
"[question] is [Answer's choice].
NLI Class: Entailment
Mapped example
Mapped example
Premise: A woman in black mesh skirt plays acoustic guitar. The woman wearing black mesh, she is wearing black.
Premise: Because state park is [...] gardens places with lots of trees and plants.
Hypothesis: The answer of the question "what is place that has bench nestled in trees?"
Hypothesis: woman wearing black.
park.
Table The mapping design for the four sub-tasks with picked examples, See Section for description about each sub-task.
rationalization model with the training set and gen- erate rationale-answer pairs on the test set. We then gather all predictions from the 60 episodes and randomly select 350 examples for human stud- ies. We present the description of the task, the input instance and the iitionale-aswwer pair (f, a) for the annotators, and ask them to judge the plausi- bility of f, i.e., whether it can justify Following prior works (Marasović et al., 2020; Marasovic et al., 2022), the annotator determines the plausi- bility by assigning labels from ("no", "weak no", "weak yes", "yes"). We then map labels to plausi- bility scores {1, 2, 3, 4) and instances with average scores above 2.5 are deemed plausible. We provide inter-annotator agreement details in Appendix C.
are judged as plausible, they are much more likely paired with correctly predicted answers in constrast to implausible ones. This verifies our hypothesis (discussed in Section 2) and shows plausibility to be strong signal for correct label predictions. Our results also align with the prior work (Wiegreffe et al., 2021), where they find relfiaationllizationn models demonstrate high label-rationale associa- tion against robustness testing. In conclusion, iden- tifying (f: à) pairs that have plausible f spurs great potential for boosting model performance, and con- nects us to RQ2.
Zero-Shot Augmentation of Rationale-Answer Pairs
The results are shown in Figure 2. We can ob- serve that for all sub-tasks, when the explanations
As shown in Section 4, plausible explanations im- ply that the corresponding task predictions are more

---

Augment
Question
Rationale
Answer
Plausible?
Finctune
Rationale
Question
Language
Generate
NLI Mapping
Plausibility Approximator
Model
Answer
Figure 3: Overview the self-trrining paradigm for ZARA. A language model is tne-tuned to generate predictions for unlabeled instances, which are then mapped NLI formats. The approximator then identifies high-confident (likely plausibile) ones as augmentation for learning new model.
likely be correct. Following we present ZARA the approach towards automatically judging the plausibility of generated explanations, and lever- ages the high confident aationale-answer pairs to boost model performance via self-training.
justifies by stating "you can't drive computer.", which explains why the answer is nonsensical by providing information refuting the answer sentence, resulting in a contradiction relationship between the two. Hence, the approximator can estimate the degree of plausibility by referring to the score of the contradiction class.
5.1 Reduce plausibility judgement to NLI
Given rationalc-answer pair (f, à) output by a self-rationalizzation model, human evaluates whether is plausible by understanding the input context and the task objective, and applying reason- ing ability determine if f justifies à. Specifically, humans implicitly form propositions from the input context and rationale by understanding the prob- lem (the task). Then do inference, i.e., apply logic and reasoning to draw conclusions, in their mind to decide if the propositions support the predicted answer. This mental process of assessing plausibil- ity resembles determining the relationship between a premise and hypothesis. Driven by this for- mulation, we reduce the problem of judging the plausibility of explanations to the task of natural language inference (NLI), and construct a zero- shot approximator, which leverages existing NLI models to automatically approximate the human judgement of plausibility.
The approximator. For developing the approxima- tor, we ensemble three state-of-the-an pre-trained NLI models by averaging their output scores for the decision of NLI class. Specifically, we adopt RoBERTa (Liu et al., 2019), DeBERTa (He et al., 2020), and BART (Lewis et al., 2020), trained on the MultiNLI corpus (Williams et al., 2018 8), one of the largest available NLI dataset. The approxi- mator is zero-shot, i.e., all three models used off-the-shelf (See Appendix A for details) without any fine-tuning on our dataset, accommodating the few-shot, data scarcity setting.
5.2 Self-training
In the self-training paradigm, a trained model augments its own training set by constructing pseudo-paran data with predictions on unlabeled instances, where the most confident predictions are collected as new training examples and used to re-train an improved model. For applying self-training, most works focus classification tasks (Miyato al., 2018; Xie al., 2020; Gera et al., 2022) with common strategies based on operations of confidence scores such as probabil- ity values to select new examples. E.g., finding predictions that are far from the decision bound- ary (Slonim et al., 2011).
NLI Mapping. The formulation NLI requires the mapping of (Ir, a) + (p, h), where v, p, and h are the input instance, premise, and hypothesis, respectively. We manually create the mappings for each FEB sub-task as shown Table 2. Construct- ing such mappings can be easily achieved with minimal effort compared with human evaluation on all f. Consider the COMVE example in Table 2, the goal is to select the nonsensical sentence from two sentences. As we see "i drove my computer the gas station." is nonsensical, and the rationale
However, the adoption of self-training for self- rationalization differs from typical classification tasks in two aspects: (1) Compared with fixed clas- sification labels, the target space of neural sequence generation is much more complex. (2) The selec-
*One can simply design the mapping by observing the training data.

---

tion requires considering both the task label â and the rationale with their relationship. By a proxy model, i.e, the approximator, we could reduce the target dimensions to fixed class labels to address the former. For the latter, we could resolve it by only considering the plausibility of since plausi- ble f likely implies correct à as shown in Section 4, Following we introduce our self-training paradigm of Z-RA-a train-judge-train procedure. See Fig- ure 3 for illustration.
mixture of QA datasets, our self-rationalization model for all experiments. The model performs few-shot learning via fine-tuning with natural language prompts. We experiment with three model sizes: UNIFIEDQA-base (200M), UNI- Plrge (770M), and UNIMEDQA-3B (2.7B). The results presented in Section 4 are conducted with UNITEDQA--B. More details of the experimental setups and configurations are provided in Appendix A.
Given an episode E consisting a training split Draan and a test split Diest, where an example in E is an imputrrationale-nnswe tuple (zr, r, a). We first train LM Mo on Draain for self reff-rationalization by prompt-based fine-tuning with natural language prompts. The trained model is denoted as M1.
Method Model
Acc.
A
BERTse.
A
FEB
Base
67.3p.7
61.0g.s
6.6
ZARA
Base
74.1
67.6).s
COMVE
FEB
Large
81.30.4
5.9
73.90.4
5.8
Next, we perform inference with M1 on unla- beled instances Dunlabled. where unllabeed is a non-overlapping set randomly sampled from other episodes with size (Dunlabled= = [Prest/- For each prediction, the input and the generated rationale- answer pair (f, â) are mapped to the II.I format, i.e., (z, f. a) + (p,h), ht), and passed to the zero-shot plausibility approximator." The approximator au- tomatically judges the plausibility of f, where the most confident predictions are selected by plau- sibility threshold a, i.e., a probability score (See Appendix for details). This process does not require any ground truth label or golden rationale.
ZARA
Large
87.2
79.70.2
FEB
3B
89.0p.4
81.0p.s
4.7
4.9
ZARA
3B
93.7
85.9)
GPT-3
INSTRUCTGPT
74.0 50.0
67.61.3
Random
FEB
Base
67.5p.4
65.3n.4 4 5.0
7.ARA
Base
72.%
70.3)
FEB
Large
71.10.4
68.50 4 3.5
3.6
SBIC
ZARA
Large
74.7
72.0
FEB
3B
71.70.5
68.9g.5
4.4
4.4
ZARA
3B
76.1
73.3/
GPT-3
INSTRUCTGPT
74.22.4 50.0
71.51.4
Random
The collected high-confident (ar, P. â) predic- tions become instances to augment Dirain- Also, we ensure the added instances are balanced for classification tasks by downsampling majority classes. We then re-train Mo on the augmented training split to obtain our final relfirationnlizaaiion model My, and evaluate on Drest-
FEB
Base
75.0p.3
67.50
3.0
3.0
ZARA
Base
78.0%
70.5
FEB
Large
84.80.3
76.6u.3
ENSL
ZARA
Large
86.0
1.2
77.7..2
1.1
FEB
3B
87.4p.2
79.10.2
2.1
2.2
ZARA
3B
89.5
81.3
GPT-3
INSTRUCTGPT
65.40.5 33.3
59.8u.b
Random
6 Experiments
FEB
Base
41.4o.a
36.7a.a
-1.9
-2.0 34.70.2
ZARA
Base
39.5o.2
In this section, we discuss the experimental setup and present the results of our proposed method, ZARA, for improving few-shot self-rationalizzation via self-training. We also perform human and quan- titative evaluations validate the automatic plausi-
FEB
Large
57.20.4
$1.00.3
0.4
0.0
ECQA
ZARA
Large
57.60.2
51.00.2
FEB
65.9u.4
$9.0u.a
4.1
3.2
ZARA
3B
70.0
62.2].2
GPT-3
INSTRUCTGPT
60.6,
54.41.3
bility assessment for our approximator.
Random
20.0
6.1 Model
Table 3: Experiments comparing FEB (Marasovic et al.. 2022) and ZARA, where the results with GPT-3 (175B) and random bascline. i.c., number of classes also pre- sented. We assess the statistical significance of ZARA's performance gain over the FEB baseline by adopting one-sided McNemar's 's test (McNemar, 1947), where denotes p < 0.01.
For comparison purposes, we follow FEB and use UNIFIeDQA (Khashabi et al., 2020), a T5 (Raffel et al., 2020) variant trained on a multi-task
"Depending on the mapping design, some sub-tasks do not require input content z to form the premise and hypothesis.

---

ComvE
SBIC
E-SNLI
ECQA
3.5
Pleadbiity
2.5
134
204
3/4
4/4
124
24
304
414
1/4
2/4
324
464
114
214
3/4
4/4
Figure 4: Human judgement plausibility scores under different pseudo-plausibility percentile bins. 3. and indicate 1~25thh, 25,550th, 50~75th, and 75~1000th percentile group, respectively, Ranked by the approximator's output probability, probabiiity i.e. pseudo-plausibilit scores.
6.2 Main results
plausibility. To recap, the approximator's output probability of the corresponding NLI class (based on the mapping design in Table represents an estimation of plausibility degree, i.e., a pseudo- plausibility score. We use the same batch of anno- tated data from Section 4. That is, 350 randomly se- lected examples generated by the stage-one model with human judgement of plausibility value {1, 2. 3, 4) mapped from ("no", "weak no", "weak yes", "yes") and averaged across annotators.
The evaluation metrics of FEB are accuracy and BERTscore (Zhang et al., 2019) for end-task labels and explanations, respectively. For each sub-task, train 60 models (one per episode) and report the mean and standard eтror of accuracy/BERT score in Table 3. also provide statistics on number of instances added for augmentation Appendix D. To the best of our knowledge, we present the first results on the newly introduced FEB benchmark (besides their original approach in the paper).
The results are presented in Figure 4. We group the instances into four bins, each containing 25% of data according the percentile ranking of their plauudddplluibblityyy score. In general, the median performance of human plausibility judgement in- creases with higher percentile groups, especially for the COMVE and SBIC sub-tasks. Interestingly, due to the nature of NLI model of the approxima- tor, its output (i.e., pseudo-plausibiiity scores) may be effected by spurious surface features learned only for NLI tasks (transferred from the MultiNLI dataset), giving rise to the larger interquartile range of the top percentile group in E-SNLI. Overall, the results show our approximator is capable of reflect- ing human plausibility judgement.
We experiment with three model sizes: base. large and 3B. In ZARA, both training stages adopt models the same size; the original FEB base- line only involves training one model (one stage). As observed in Table 3, our method substantially outperforms the FEB baseline for all datasets. In general, COMVE, SBIC and E-SNLI demonstrate relatively consistent improvements across model size. The only anomoly is for ECQA. We hypoth- esize the pader-parameterized models (base and large) suffer forgetting from continuous learning with the augmented data (Kirkpatrick et al., 2017), since ECQA may require commonsense knowledge which is not presented in the FEB training data but is encoded in models' parameters originally. How- ever, for the 3B model-which is still significantly smaller than most large-scale LMs great perfor- mance gain with ZARA is exhibited.
Correctness evaluation As stated in Section 4, plausible rationales likely indicate correct answer predictions. We further evaluate our approximator regarding this property by checking the end-task answer accuracy of the data subset selected for augmentation from stage-one model's prediction pool. We consider three selection strategies: (1) ZARA, i.e., our proposed method, which selects con- fident (high-scoring) predictions; (2) Random, the data subset is selected randomly from prediction pool; (3) Lowest, in contrast to ZARA, we select a subset from the data with lowest-ranking pseudo- plausibility scores.
6.3 Approximator evaluation
Plausibility evaluation We conduct human eval- uation to validate our approximator. Specifically, the human evaluation can be considered as a meta- evaluation for evaluating the approximator's 's ability to evaluate explanations, i.e., its ability to assess
"BERTscore is one of the most correlated automatic NLG metries with human judgement of plausibility for free-text explanation, as shown by Kayser et al. (2021).
For each episode, the number of augmented

---

ComVE
SBIC
E-SNLI
ECQA
90
Accuracy (N)
80
80
65
60
Large
Base
Base
ZARA
Random
Lawest
Figure The average accuracy (per episode) of the selected subset from stage-one model's generated results by different selection strategies: (1) ZARA, (2) Random, and (3) Lowest.
instances for (2) and (3) are determined by (I), i.e., we randomly select n instances or select n bottom--anking instances, where n is the number of instances for augmentation using ZARA. The re- sults are shown in F'igure 5. We can observe ZARA consistently outperforms Random and Lowest with substantial margins under different model sizes across all four datasets, and Lowest demonstrates the poorest accuracy. This suggest our approxima- tor able to verify label predictions post-hoc, i.c., the high/low pseudo-plausibility score suggests the prediction is accurate/inaccurate.. In conclusion, the overall evaluation results suggest our approxi- mator effectively extract rationale-answe pairs which are more plausible and accurate.
larity to ours is STaR (Zelikman al., 2022). Give an initial training set consisting of a large amount of labels and few seed rationales, STaR iteratively tinettunes a GPT-J model to build an augmented training set to bootstrap itself. The fundamental difference between ZARA and STaR is that STaR needs ground-truth labels to select and generate rationales for augmentation, whereas, ZARA aug- ments rationale- pairs zero-shot, without any requirements of ground labels or golden rationales. Another related work by Ye and Durrett (2022) leverages NLES to boost end-task predic- tions post-hoc, via training calibrator. In com- parison, directly improve relionattnaliioooonn and approximator does not require any further training, Moreover, all LMs used by Ye and Durrett (2022) are 175B.
7 Related Work
7.1
Few-shot self-rationalization
7.2 Leveraging NLI for downstream tasks
To provide NLEs under low supervision, Maraso- vie et al. (2022) propose the FEB benchmark and establish the first results by exploring natu- ral language prompt-based fine-tuning. Wiegreffe et al. (2022) focus on improving NLEEs with an overgenerate-and-Fe
The framework of NLJ has been expanded to bene- fit many NLP tasks. Welleck et al. (2019) develop a dataset to improve dialogue models by framing the dialogue consistency problem as NLI. Hon- ovich et al. (2021); Dziri et al. (2022) use NLI to design automatic metrics evaluating factuality of knowledge -rougggg-grondeddd dialogue systems. Falke et (2019); Kryscinski et al. (2020); Laban et al. (2022) use NLI models detect factual errors abstractive summarization tasks. For question an- swering, Chen et al. (2021) propose a framework to verify QA systems' predictions with NLI by training models to generate memise-hypothesii pairs from QA instances. Driven by human rea- soning, Yin et al. (2019) approach text classifica- tion in zero-shot by formulating it as entailment problem-givee -iven the input text (premise), humans mentally construct hypotheses "the text is about [label choice]" to determine the answer and adopt out-of-the--bo NLI models for predictions.
pipeline: prompting GPT-3 with gold labels to generate explanation candidates which are then filtered by model trained with hu- man annotations. Recent works (Wei et al., 2022; Wang et al., 2022b; Huang et al., 2022) leverage rationale-aus -ughnale--ugmmmeedd augmented ohain-of-thought (CoT) inputs to prompt frozen large-scale LMs in few-shot. Con- current works (Wang et al., 2022a; Ho et al., 2022; Hsieh et al., 2023) propose pipeline frameworks to distill knowledge by prompting a large "teacher" LM to generate diverse reasoning rationales which are then used to fine-tuning small "student" LM. In comparison, ZARA directly optimizes small LMs on downstream tasks, without access any large LMs.
previous work that shares a conceptual simi-

---

8 Conclusion
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natu- ral language inference with natural language expla- nations. Advances in Neural Information Processing Systems, 31.
In this work, we first show evidences that plausible explanations imply correct label predictions, and leverage novel NLI approximator to automati- cally identify plausible rationales paired with cor- rect answers from unlabeled results.s By collecting such rationale-answer pairs with self-training, we effectively improve the performance of few-shot self- relf-aationallzationn for small LMs. Moreover, we demonstrate the potential for automatic evaluation of free-text explanations. In light of this, we be- lieve developing a supervised approximator with a unified II.I mapping schema across tasks to be a promising avenue for future works.
Jifan Chen, Eunsol Choi, and Greg Durrett. 2021. Can NLI models verify QA systems' predictions? In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 3841-3854, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
Wei-Lin Chen, An-Zi Yen, Hen-Hsen Huang, and Hsin- Hsi Chen. 2022. Learning to generate explanation from e-hospital services for medical suggestion. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2946-2951, Gyeongju. Republic of Korea. International Com- mittee on Computational Linguistics.
Limitations
The success of the approximator relies on the qual- ity of the NLI mapping. Though we showcase great improvement across four different tasks, if the com- plexity of task makes the mapping construction non-trivial, the created mapping might not be able accurately reflect human plausibility judgement of the generated rationales, and the benefit of self- training could not be guaranteed. Namely, the ap- proximator may identify noisy instances that would instcad hurt model performance.
Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter. 2022, Evaluating attribution in dialogue sys- tems: The BEGIN benchmark. Transactions of the Association for Computational Linguistics, 10:1066- 1083.
Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Rank- ing generated summaries by correctness: An interest- ing but challenging application for natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 2214-2220, Florence, Italy. Association for Computational Linguistics.
Acknowledgements
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 38 16-3830, Online. Association for Computa- tional Linguistics.
We thank the reviewers for their insightful com- ments. This research was partially supported by National Science and Technology Council, Taiwan, under grants MOST 1002221--E-002-123 120---2-- and NSTC 11 111-2634-F-002-0233 and Ministry of Education (MOE) in Taiwan, under grants NTU- 112L900901.
Ariel Gera, Alon Halfon, Eyal Shnarch, Yotam Perlitz. Liat Ein-Dor, and Noam Slonim. 2022. Zero-shot text classification with self-training. arXiv preprint arXiv:2210.1554.
References
Shourya Aggarwal, Divyanshu Mandowara, Vishwa- jeet Agrawal, Dinesh Khandelwal, Parag Singla, and Dinesh Garg. 2021. Explanations for Common- senseQA: New Dataset and Models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Conference Natural Language Processing (Volume 1: Long Papers), pages 3050-3065, Online. Association for Computational Linguistics.
Fréderic Godin, Kris Demuynck, Joni Dambre, Wes- ley De Neve, and Thomas Demeester. 2018. Ex- plaining character-aware neural networks for word- level prediction: Do they discover linguistic rules? In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 3775-2284, Brussels, Belgium. Association for Com- putational Linguistics.
Jonathan Bragg, Arman Cohan, Kyle Lo, and Iz Beltagy. 2021. Flex: Unifying evaluation for few-shot nlp. Advances in Neural Information Processing Systems, 34: 150000
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoddng-eehanced bert with disentangled attention. In International Conference on Learning Representations.

---

Namgyu Ho, Laura Schmid, and Sc- Young Yun. 2022. Large language models are reasoning teachers. arXiv preprint arXiv:2212.1000 10071.
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 9332-9346, Online. Association for Computa- tional Linguistics.
Or Honovich, Leshem Choshen, Roce Aharoni, Ellaa Neeman, Idan Szpektor, and Omri Abend. 2021. q²: Evaluating factual consistency in knowledge- grounded dialogues via question generation and ques- tion answering. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 7856-7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-visiting NLI- based models for inconsistency detection in summa- rization. Transactions of the Association for Compu- tational Linguistics, 10:163-177.
Andrew K Lampinen, Ishita Dasgupta, Stephanie CY Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L McClelland, Jane X Wang, and Felix Hill. 2022. Can language models learn from explanations in context? arXiv preprint arXiv:2204.02322
Cheng-Yo Yu Hsich, Chun-Liang Li, Chih-Kuan Ych, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. arXiv preprint wrxiv:2305.02301.
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016 Rationalizing neural predictions. Proceedings of the 2016 Conference on Empirical Methods in Nat- ural Language Processing, pages 107-117, Austin, Texas, Association for Computational Linguisties.
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-improve. arXiv preprint arXiv:2210.1 1670.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising seuueecceto-sequenee pre-tree pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics. pages 7871-7880, Online. Association for Computa- tional I Linguistics.
Alon Jacovi and Yoav Goldherg. 2021. Aligning faithful interpretations with their social attribution. Transac- tions of the Association for Computational Linguis- tics, 9:294-310.
Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, and By- ron C. Wallace. 2020. Learning to faithfully ratio- nalize by construction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pages 4459-4473, Online. Association for Computational Linguistics.
Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016. Visualizing and understanding neural models in NLP. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 681 -691, San Diego, Califoria. Association for Computational Linguistics.
Maxime Kayser, Oana-Maria Camburu. Leonard Salewski, Cornelius Emde, Virginie Do, Zeynep Akata, and Thomas Lukasiewicz. 2021. e-vil: A dataset and benchmark for natural language explana- tions in vision-language tasks. In Proceedings of the IEFF/CVF International Conference Computer Vision, pages 1254..
Qing Li, Qingyi Tao, Shafiq Joty, Jianfei Cai, and Jiebo Luo. 2018. Vqa-e: Explaining, elaborating, and en- hancing your answers for visual questions. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 552-567.
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Han- naneh Hajishirzi. 2020. UNIFIEDQA: Crossing for- mat boundaries with a single QA system. In Find- ings of the Association for Computational Linguistics: EMNLP 2020, pages 1896-1907, Online. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11 1907.11692
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins. Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Grabska-Barwe Barwinska, et al. 2017. Over- coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):35113352.
Ana Marasovic, Iz Beltagy, Doug Downey, and Matthew Peters. 2022. Few-shot selffrratonallization with nat- ural language prompts. In Findings of the Associa- tion for Computational Linguistics: NAACL 2022, pages 410 424, Seattle, United States. Association for Computational Linguistics.
Ana Marasović, Chandra Bhagavatula, Jae sung Park, Ronan Le Bras, Noah A. Smith, and Yejin Choi. 2020. Natural language rationales with full-stack visual reasoning: From pixels to semantic frames to
Wojeiech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In

---

commonsense graphs. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 28 10-2829, Online. Association for Computational Linguisties.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou, 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.
Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153-155)
Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguisties. pages 3731-374) Florence, Italy. Association for Computational Linguistics.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. 2018. Virtual adversarial training: a regularization method for supervised and semi- supervised learning. IEEE Iransactions on pattern analysis and machine intelligence, 199991997-
Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022. Reframing human-Al collaboration for generating free-text ex- planations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 632-658, Seattle, United States. Association for Computational Linguisties.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Much. Learn. Res., 21(140):1667
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain your- self! leveraging language models for commonsense reasoning. Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 4932-4942, Florence. Italy, Association for Computational Linguistics.
Sarah Wiegreffe and Ana Marasovic. 2021, Teach me to explain: A review of datasets for explainable natural language processing. In Thirty-fitth Conference on Neural Information Processing Systems Datasets and Benchmarks Truck (Round 1).
Sarah Wiegreffe, Ana Marasović, and Noah A. Smith. 2021. Mcasuring association between labels and free-text rationales. Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing, pages 10266-10284, Online and Punta Cana, Dominican Republic. Association for Compu- tational Linguistics.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Juraf- sky, Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power im- plications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490, Online. Association for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guisties: Human Language Technologies, Volume (Long Papers), pages 11 12-1122, New Orleans, Louisiana. Association for Computational Linguis- tics.
Henry Scudder. 1965. Probability of error of some adaptive pattern-recognitionn machines. IEEE Trans- actions on Information Theory, 11333363--71.
Noam Slonim, Elad Yom-Tov, and Koby Crammer. 2011. Active online classification via information maximization. In Tienty- -econd International Joint Conference on Artificial Intelligence.
Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong. and Quoe Le. 2020. Unsupervised data augmenta- tion for consistency training. Advances in Newral Information Processing Systems, 33:6256-6266
Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. 2019. Does it make sense? and why? a pilot study for sense making and explana- tion. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4020-4026, Florence, Italy, Association for Compu- tational Linguistics.
Xi Ye and Greg Durrett. 2022. The unreliability of explanations in few-shot prompting for textual rea- soning. Advances in neural information processing systems.
Peifeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen, and Xiang Ren. 2022a. Pinto: Faithful language reasoning using preepttgenerated rationales. arXiv preprint 1..002221..01662.
Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. Bench- marking zero-shot text classification: Datascts, eval- uation and entailment approach. In Proceedings of the 2019 Conference on Empirical Methods Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3914-3923, Hong Kong. China. Association for Computational Linguistics.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quae Le,, Ed Chi, and Denny Zhou. 2022b. Self-consistency improves chain of thought reasoning in language models. arXiv preprini 220iv 11171.

---

Eric Zelikman, Yuhuai Wu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reason- ing. arXiv preprint ar ar03v:2203. 14465.
Avg. # Instances Above Different Thresholds
SBIC
E SKLI
BCOA
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In International Confer- ence on Learning Representations.
Avg. Instances
A Experimental setups and configurations
85
We prepare the training data and format as natural language prompts using the scripts provided by the FEB paper's 's repository. We train our base and large model (UNIFIEDAA-baas and UNIFIEDQA-large) by NVIDIA GeForce RTX 3090, and 3B model (UNIFIEDQA-3B) by NVIDIA RTX A6000. For parameer (HP) settings, we follow the origi- nal setup in the FEB paper for stage-one training. and for stage-two, we set maximum epochs to 48 and keep other HPs the same as stage-one. We do not perform additional HP search.
Plausibiity Threshold
Figure The average number of training set instances (per episode) with probability score above different thresholds.
2022) and could require more fine- fine-grained analysis in the future (Marasovic et al., 2022).
D Data augmentation details
Table 4 reports the average number of additional test set instances added per episode for stage-two training. For COMVE, SBIC, and E-SNLI, about one-third of the test data are selected, with only minor differences against model sizes. On the other hand, ECQA shows notable increment on the 3B model, yet significant lower addition number in general comparing to the other three sub-tasks. which may attribute to the nature of difficulty for commonsense question answering.
For the approximator, We use facebook/bart- large-mali, miccrosofiiddeeet--llrgg--- roberta-large-mali from the Hugging Face Hub.
and
B Plausibility threshold
To estimate a threshold that is sufficient but not overly strict, we compute the average number of training set instances (which are required to be plau- sible) per episode with probability scores above dif- ferent threshold values, as shown in Figure 6. The dotted line represents the segment with the smallest slope, indicating increasing the threshold results in the largest data lost. The starting, i.e., smaller, v-value of the dotted line chosen our plausibil- ity threshold. Thus, 0.9 for COMVE, E-SNLI and ECQA, and 0.8 for SBIC.
Model
ComVE
SBIC
E-SNLI
ECQA
Base
109.0
113.6
123.8
13.2
Large
132.3
109.0
128.6
13.4
3B
191.5
107.6
98.3
48.4
Table 4: The comparison for average number of in- stances added (per episode) between model size.
C Human annotation details
We invite three annotators conduct human eval- uation and compute inter-anmotator agreements by Randolph's on 100 overlapping annotation ex- amples. We record k of 0.60, 0.56, 0.36, and 0.49 for COMVE, SBIC, E-SNLI, and ECQA, respec- tively. The low (0.36) to moderate (0.60, 0.56, 0.49) agreements align with prior works' observations on evaluating plausibility of free-text explanation, reflecting the task subjectivity (Wiegreffe et al.,
"The annotators include two graduate students and one Ph.D. student. As our tasks do not require specific domain expertise, the payment is determined by the minimum wage.


Reviews:
[{"review_id": "izJEp4mqsh", "reviewer": "Reviewer_wxRY", "summary": "", "questions": "- I would like to see more insights on different off-the-shelf NLI models on the the ZARA's eventual performance. Also, it would be interesting to see how the performance compares if based on a more recent NLI model such as WaNLI (\"WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation\", https://wanli.allenai.org/) that is presumably more robust for out-of-domain test cases.\n\n- Why the model size is repeated twice for ZARA in Table 3? Were there different combinations of sizes not included in the paper?\n\n- I found the NLI conversion for ECQA a bit odd due to the insertion of the word \"Because\" for premise, which makes the sentence incomplete.\n\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive feedback! Regarding the “Questions For The Authors”:\n\n> I would like to see more insights on different off-the-shelf NLI models on the ZARA's eventual performance. Also, it would be interesting to see how the performance compares if based on a more recent NLI model such as WaNLI (\"WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation\", wanli.allenai.org/) that is presumably more robust for out-of-domain test cases.\n\n- Thank you for the great suggestions, we agree these additional insights would be very interesting and will try to incorporate a discussion on different off-the-shelf NLI models trained with NLI dataset with different characteristics, including the one mentioned (WaNLI).\n\n> Why the model size is repeated twice for ZARA in Table 3? Were there different combinations of sizes not included in the paper?\n\n- In Table 3, we repeat the model size twice to indicate the models used in the two stages (initial stage and self-training stage) of our proposed ZARA, i.e., initial model size - self-training model size. Considering the models used in the two stages are identical, we will simplify the notation in the revised version to reduce confusion.\n\n> I found the NLI conversion for ECQA a bit odd due to the insertion of the word \"Because\" for premise, which makes the sentence incomplete.\n\n- Thank you for pointing this out! We did not notice this at the time. We agree that the incomplete sentence is a bit odd and could be refined.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper presents a clever use of off-the-shelf NLP models to determine the plausibility of the model predictions combined with the free text rationalization. The proposed approach demonstrates an improvement over previous approaches.\n\n- The paper is well-written and the experiments and analyses are thoughtful.", "reasons_to_reject": "- No significant reason to reject. The paper presents a solid work that is worthy of acceptance.", "questions_to_authors": "- I would like to see more insights on different off-the-shelf NLI models on the the ZARA's eventual performance. Also, it would be interesting to see how the performance compares if based on a more recent NLI model such as WaNLI (\"WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation\", https://wanli.allenai.org/) that is presumably more robust for out-of-domain test cases.\n\n- Why the model size is repeated twice for ZARA in Table 3? Were there different combinations of sizes not included in the paper?\n\n- I found the NLI conversion for ECQA a bit odd due to the insertion of the word \"Because\" for premise, which makes the sentence incomplete.\n\n", "paper_topic_and_main_contributions": "This paper presents an approach to few-shot self-rationalization via self-training by demonstrating a clever use of off-the-shelf NLI models for filtering potentially incorrect model predictions and the corresponding rationalization.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IULnKBYiZb", "reviewer": "Reviewer_JA6T", "summary": "", "questions": "More critical questions in the Weaknesses section. More questions - \n1. L212 Which version of the self-rationalizing model do you use in order to validate your hypothesis in Section 4? (some more details about the exact data split used, model size, number of shots, etc, would be good to add to the paper)\n2. Section 5.2: What is the difference between ZARA and STaR (Zelikman et. al) in the self-training approach? How many rationales are actually plausible in STaR vs. the improvement yielded by ZARA?\n3. L359 - M_0 is retrained on the augmented training split, why not M_1? It is also confusing if ZARA is continually trained approach? Are there more sequential versions of M?\n4. Are improvements in Table 3 significant? (would love to see some significance tests here, specifically for base/large models as they are understudied in self-rationalizing literature)\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive comments! To alleviate your concerns raises in “Reasons To Reject”:\n\n> The core idea of the paper lies in L164-168. The paper shows in Section 4 that measuring correlation between plausibility of a rationale correlates with the correctness of the answer. It is unclear if the paper tries to distinguish between actually plausible rationales vs. rationales that probably leak labels directly? This comes about as a question specifically because the agreements between annotators is low (which I recognize is similar to prior work). But has the paper looked into why plausibility annotations have low agreement?\n\n- We did not conduct fine-grained analysis to distinguish “actually plausible rationales” and “rationales that probably leak labels”, however, our core idea holds since the rationales are not priorly given, where rationales and answers are both generated by the model.\n\n- To the best of our understanding, “rationales that probably leak labels” would not affect the agreements between annotators since if the model indeed generates rationales that leak labels which are contradicting to the predicted answer labels, the annotators should all be able to score these rationales as low plausibility.\n\n- Regarding the reasons why plausibility annotations have low agreement, as stated in Lines 878-880, prior works attribute this to the subjectivity nature of the task and requires further analysis in the future, which we agree. We did not conduct such an analysis as our goal is to show plausible rationales generated by the model have a high probability to be paired with correct labels in the predictions.\nWe believe one explanation for low agreement is that different annotators require different levels of *convincement* to agree with the reasons provided by rationales.\n\n> The standard argument about correlation does not imply causation also holds here. How can one conclusively say that the performance improvements are being shown because of plausible rationales being added, rather than simply adding more rationales? Clearly, ZARA requires adding more rationales to its training pipeline (almost double than what FEB uses in certain tasks). How does a similar few-shot experiment setting with FEB (with compare number of samples) hold?\n\n- The goal of this work—self-rationalization—is to generate both rationales and answer labels jointly, thus requiring rationale-label pairs as the training data. We believe there is very little reason to assume that simply adding more data, i.e., rationale-label pairs with potentially wrong labels, could lead to performance improvements, especially in our few-shot, prompt-based fine-tuning setting.\nAlso, we validate ZARA’s ability to select rationale-label pairs which have correctly predicted labels as shown in Figure 5 and discussed in Section 6.3.\nAnd the few-shot setting holds in the sense that the number of used instances with gold rationale-label pairs remains the same as in FEB.\n\n> The way the data splits are created is extremely confusing (L336 onwards). How are D_unlabeled for a given episode taken? If they are randomly sampled from other episodes, doesn't it mean that there can be leaks in the test set on average?\n\n- As described in Lines 343-345, D_unlabeled is non-overlapped with D_test, and is randomly selected from another episode. For better clarity, we will revise our wordings in Line 343-345 and explicitly state that D_unlabeled does not contain instances from D_test. Thank you for pointing this out!\n\n> Human evaluations are done to validate the 0-shot plausibility approximation. In Figure 4, these judgements are compared. How does the plausibility of M_0 hold with M_1 and M_2?\n\n- In our self-training procedure (as elaborate in Section 5.2), the model M_0 refers to the original model before any fine-tuning. After the initial fine-tuning, M_0 becomes M_1, and we adopt M_1 to perform inference on D_unlabeled. Lastly, we again fine-tune M_0 but on the augmented dataset with additional instances selected by ZARA on M_1, and the resulting trained model is M_2. Here the human evaluation present in Figure 4 is to validate the ability of ZARA to select plausible rationales, so the evaluation is performed on rationales generated by M_1.\n\nWe address the following two comments jointly as they are relevant and dependent.\n\n> How plausible are FEB's rationales? (It is unclear if there is an actual increase in plausibility of rationales since no baseline plausibility comparisons are made?)\n\n> Human evaluations are conducted for the plausibility approximator but not for the rationales generated by the final model (M_2)? How does the quality/relevance/association of rationales improve? (given that the eventual purpose of self-rationalization models are to also provide explanations for its predictions).\n\n- For a unified comparison of previous work (i.e., FEB) and future work, we assessed the plausibility of rationales by BERTscore and show ZARA outperforms FEB (Table 3). As described in footnote 6, prior studies have shown BERTscore to be one of the most correlated automatic metrics with human judgement of plausibility for rationales (which is also the reason why BERTscore is chosen as FEB’s evaluation metric in the first place). However, we agree an additional human evaluation of the plausibility on M_2's generated rationales would be beneficial, and we plan to incorporate it in our camera-ready version.\n\nRegarding the “Questions For The Authors:”\n\n> Which version of the self-rationalizing model do you use in order to validate your hypothesis in Section 4? (some more details about the exact data split used, model size, number of shots, etc, would be good to add to the paper)\n\n- We validate our hypothesis in Section 4 using model M_1’s generated results.\nFor model size, we use the UnifiedQA-3B model (described in Lines 379-380).\nDetails including data split used and number of shots are described in Lines 197-199, which is the original FEB benchmark setup.\n\n> What is the difference between ZARA and STaR (Zelikman et. al) in the self-training approach?\n\n- The fundamental difference between ZARA and STaR is that STaR needs ground-truth labels to select and generate rationales for augmentation, whereas, ZARA augments rationale-label pairs without any requirements of ground-truth labels or golden rationales.\n\n> How many rationales are actually plausible in STaR vs. the improvement yielded by ZARA?\n\n- The comparison is not suitable as the goal and setting between STaR and ZARA are different. The objective of STaR is to improve the end-task accuracy given a training set consisting of a large amount of labels and a few seed rationales, where these ground-truth labels are required when training STaR to generate rationales and select augmentation data for self-training. However, the goal of ZARA is to jointly improve rationale quality and end-task accuracy in a few-shot manner, that is, the training data only includes a few rationale-answer pairs. In sum, the STaR framework can not be adopted when only a few rationale-answer pairs are allowed as training data (i.e., in the setting of ZARA).\n\n> M_0 is retrained on the augmented training split, why not M_1? It is also confusing if ZARA is continually trained approach? Are there more sequential versions of M?\n\n- We retrain the initial model M_0 following prior self-training approaches, e.g., STaR (Zelikman et. al). In our setting, ZARA is not (although it can be) a continually trained approach. We achieve great empirical improvements (Table 3) with only one self-training iteration as described in Section 5.2.\n\n> Are improvements in Table 3 significant? (would love to see some significance tests here, specifically for base/large models as they are understudied in self-rationalizing literature)\n\n- Thanks for bringing this up, following we report results of significance tests by adopting one-sided McNemar’s test (McNemar, 1947) to assess the statistical significance of ZARA’s performance gain over the FEB baseline, and “v” denotes p value < 0.01. We will add the details into our final version. As observed, base and large have significant results in 3 out of the 4 subtasks, and 3B is significant in all 4 subtasks.\n- ComVE\n\n    |       | Accuracy | BERTscore |\n    | ----- |:--------:|:---------:|\n    | Base  |    v     |     v     |\n    | Large |    v     |     v     |\n    | 3B    |    v     |     v     |\n\n- SBIC\n\n    |       | Accuracy | BERTscore |\n    | ----- |:--------:|:---------:|\n    | Base  |    v     |     v     |\n    | Large |    v     |     v     |\n    | 3B    |    v     |     v     |\n\n- E-SNLI\n\n    |       | Accuracy | BERTscore |\n    | ----- |:--------:|:---------:|\n    | Base  |    v     |     v     |\n    | Large |    v     |     v     |\n    | 3B    |    v     |     v     |\n\n- ECQA\n    |       | Accuracy | BERTscore |\n    | ----- |:--------:|:---------:|\n    | Base  |         |          |\n    | Large |         |          |\n    | 3B    |    v     |     v     |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper proposes an excellent yet simple approach to improve accuracy of small self-rationalizing models. The trick to add a rationale filter on the basis of their plausibility is a good design idea that can be easily applied for improved performance. Furthermore, it has been shown in prior work that rationalization emerges as model scales (Wei et. al 2022), so the paper's approach for improving rationalization for smaller models is extremely relevant for the community!\n\n\n", "reasons_to_reject": "RR1 Motivation: The core idea of the paper lies in L164-168. The paper shows in Section 4 that measuring correlation between plausibility of a rationale correlates with the correctness of the answer. It is unclear if the paper tries to distinguish between *actually plausible* rationales vs. rationales that probably leak labels directly? This comes about as a question specifically because the agreements between annotators is low (which I recognize is similar to prior work). But has the paper looked into *why* plausibility annotations have low agreement?\n\nRR2 Experiments - \n1. The standard argument about correlation does not imply causation also holds here. How can one conclusively say that the performance improvements are being shown because of *plausible* rationales being added, rather than simply adding *more* rationales? Clearly, ZARA requires adding more rationales to its training pipeline (almost double than what FEB uses in certain tasks). How does a similar few-shot experiment setting with FEB (with compare number of samples) hold?\n2. The way the data splits are created is extremely confusing (L336 onwards). How are D_unlabeled for a given episode taken? If they are randomly sampled from other episodes, doesn't it mean that there can be leaks in the test set on average?\n3. Human evaluations are done to validate the 0-shot plausibility approximation. In Figure 4, these judgements are compared. How does the plausibility of M_0 hold with M_1 and M_2? How plausible are FEB's rationales? (It is unclear if there is an actual increase in plausibility of rationales since no baseline plausibility comparisons are made?)\n\nRR3 Human evaluations - Human evaluations are conducted for the plausibility approximator but not for the rationales generated by the final model (M_2)? How does the quality/relevance/association of rationales improve? (given that the eventual purpose of self-rationalization models are to also provide explanations for its predictions).", "questions_to_authors": "More critical questions in the Weaknesses section. More questions - \n1. L212 Which version of the self-rationalizing model do you use in order to validate your hypothesis in Section 4? (some more details about the exact data split used, model size, number of shots, etc, would be good to add to the paper)\n2. Section 5.2: What is the difference between ZARA and STaR (Zelikman et. al) in the self-training approach? How many rationales are actually plausible in STaR vs. the improvement yielded by ZARA?\n3. L359 - M_0 is retrained on the augmented training split, why not M_1? It is also confusing if ZARA is continually trained approach? Are there more sequential versions of M?\n4. Are improvements in Table 3 significant? (would love to see some significance tests here, specifically for base/large models as they are understudied in self-rationalizing literature)\n", "paper_topic_and_main_contributions": "The paper proposes an approach called ZARA, that is designed to improve self-rationalization capability for smaller models. A filter that separates plausible rationales from generations are added back to the training pool of the model, leading to improvements in the model performance as compared to prior baselines like FEB. This design is motivated by empirical proof that plausibility correlates with correctness of answers.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BGbuspguBJ", "reviewer": "Reviewer_x5eQ", "summary": "", "questions": "Would it not make more sense to use an ensemble of NLI models each trained on different NLI datasets, such as ANLI?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive feedback! Regarding the points mentioned in “Reasons To Reject”:\n\n> One concern that I have is that the framework is not completely “automatic”. Instead, mappings must be developed for each subtask that needs to be improved upon.\n\n- We acknowledge that the mapping design is not fully “automatic”. Though, as discussed in Lines 281-284, constructing such mappings only requires minimal efforts via observing a few instances as opposed to human evaluation of the plausibility of all generated instances. Building an automated framework with zero human-in-the-loop is left as our future work. Thanks for your comment!\n\n> What is an episode? It says that you train 60 models, one for each episode but I am unclear of the difference between episodes since they all belong to the same subtask in the dataset. Due to the number of models trained, it seems to be rather inefficient in the few-shot setting.\n\n- An episode is a train-test split with 48 training examples and 350 evaluation examples, where different episodes are constructed with examples randomly sampled from the full dataset of the subtasks. The goal of such a design is for building a reliable few-shot evaluation setup recommended by “Flex: Unifying evaluation for few-shot nlp (Bragg et al., 2021)” (described in Section 2 of the FEB paper), and in practice, we only need to train one model.\n\nRegarding the “Questions For The Authors”:\n\n> Would it not make more sense to use an ensemble of NLI models each trained on different NLI datasets, such as ANLI?\n\n- Thank you for the suggestion and yes we agree! Ensembling different NLI models trained on different NLI datasets is a promising and insightful setup which we will investigate in the future.\n\nWe will fix all typos in the final version, thank you!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper proposes a simple yet effective methodology to find additional instances for self-training language models for improved rationalization.\n\nRather than focus on improving large language models, the authors focus on improving smaller and more accessible models with their framework.\n", "reasons_to_reject": "One concern that I have is that the framework is not completely “automatic”. Instead, mappings must be developed for each subtask that needs to be improved upon.\n\nWhat is an episode? It says that you train 60 models, one for each episode but I am unclear of the difference between episodes since they all belong to the same subtask in the dataset. Due to the number of models trained, it seems to be rather inefficient in the few-shot setting.\n", "questions_to_authors": "Would it not make more sense to use an ensemble of NLI models each trained on different NLI datasets, such as ANLI?", "paper_topic_and_main_contributions": "In this paper, the authors propose a framework, ZARA, to improve self-rationalization in small language models. The framework consists of an approximator, which converts samples into NLI-type examples and utilizes an ensemble of NLI models to make predictions on unlabeled instances. The samples that are given highly confident scores are used to retrain the language model and improve self-rationalization. The results demonstrate that this framework is able to improve upon existing results in the FEB dataset. In addition, the approximator is able to select unlabeled samples with plausible rationales as produced by the base model, leading to higher accuracy.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 209: “As described in Section 2, following we attempt”\n\nSection 5.1 has some grammatical errors such as line 259: “a human” and line 463: “humans”\n\nLine 438: “affected”\n\nLine 524: “evidence”\n\nLine 528"}}]

Author rebuttal:

Meta-review: This paper proposes a method for self-training of small language models to improve their rationalization capabilities.  They propose to use NLI to evaluate explanation plausibility. Figure 2 establishes that plausible explanations are correlated with correct predictions; if plausible explanations *cause* correct predictions, then training on them to produce more plausible explanations should led to better self-rationalization models. The paper uses this observation to filter model predictions for use in the self-training stage.  Experiments on FEB with a UnifiedQA model show stronger accuracy across the board with the proposed technique.

Reviewers liked the use of off-the-shelf models to improve rationalization. They praised the overall clarity of the paper.

I agree with these strengths and think this is a solid piece of research. However, the differences from STaR don't seem that substantial: the main difference is that ground truth labels are not needed and a different filtering criterion (plausibility) is used. I do believe the authors (in the response to JA6T) that it's hard to properly compare the techniques, so it's more of a conceptual similarity.  However, I would say that this paper should be cited more prominently in the present paper (right now it appears to be "nocited" and only appears in the citations).

JA6T also raises a valid point about causation being derived from data that shows correlation. However, the ultimate test of this is in the final end-to-end results, so I don't see this as a critical soundness weakness. It should be clarified in the paper, though.
Decision: Accept-Findings