SlowGuess/NAACL_2022_OCR huggingface dataset:
Interpretable machine learning: Fundamental principles and 10 grand challenges. arXiv: 2103.11251.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490, Online. Association for Computational Linguistics.
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online. Association for Computational Linguistics.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/mesh-transformer-jax.
Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. 2019. Does it make sense? and why? a pilot study for sense making and explanation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4020-4026, Florence, Italy. Association for Computational Linguistics.
Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv:2201.11903.
Sarah Wiegrefe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022. Reframing human-ai collaboration for generating free-text explanations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics.
Sarah Wiegrefe and Ana Marasovic. 2021. Teach me to explain: A review of datasets for explainable nlp. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS).
Sarah Wiegrefe, Ana Marasovic, and Noah A. Smith. 2021. Measuring association between labels and free-text rationales. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10266-10284, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguistics.
Jialin Wu and Raymond Mooney. 2019. Faithful multimodal explanation for visual question answering. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 103-112, Florence, Italy. Association for Computational Linguistics.
Fan Yang, Mengnan Du, and Xia Hu. 2019. Evaluating explanation without ground truth in interpretable machine learning. 1907.06831.
Yordan Yordanov, Vid Kocijan, Thomas Lukasiewicz, and Oana-Maria Camburu. 2021. Few-shot out-of-domain transfer learning of natural language explanations. In Proceedings of the Deep Generative Models and Downstream Applications Workshop at NeurIPS 2021.
Eric Zelikman, Yuhuai Wu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. arXiv:2203.14465.
Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. 2018. Record: Bridging the gap between human and machine commonsense reading comprehension. arXiv:1810.12885.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. *Bertscore: Evaluating text generation with bert*. In *The International Conference on Learning Representations (ICLR)*.
A.1 Input Formats in Related Work
$\mathbf{SQUAD}_{\mathrm{T5}}$  (T5's prompt for SQUAD)
- question: [question] context: [paragraph]
UNIFIEDQA's prompts (basis for QASIMPLE)
- Multiple-choice QA: [question] \n (A) [choice1] (B) [choice2]...
- Extractive QA: [question] \n [paragraph]
UNIFEW
- Single text classification: Topic? \\\\n(A) [class1] (B) [class2] (C) [class3] \\\\n[document]
- Sentence-pair classification: [sentence1] Is [sentence1]? \\\\n (A) Yes (B) No (C) Maybe
- Relation classification: [mention1] to [mention1]?  $\backslash \backslash n$  (A) [class1] (B) [class2] (C) [class3]
- Entity recognition: What is the type of the entity between the # marks? \n [text... #mention1# ...text... #mention2# ...text]
A.2 Prompts Used in §4
We use  $\approx$ T5 for E-SNLI, and QAsIMPLE for other datasets:
- E-SNLI explain nli hypothesis: [hypothesis] premise: [premise]
- ECQA: explain [question] \n (A) [choice1] (B) [choice2] (C) [choice3] (D) [choice4] (E) [choice5]
COMVE: explain what is more nonsensical? \n choice1: [sentence1] choice2: [sentence2]
- SBIC: explain what is this? \n post: [post]
Prompts for GPT-3 are given in Figure 1. The output always takes the form of "[answer/label] because [explanation]".
SBIC:
Is the following post offensive? and why?
Post: [post]
Answer: [Yes | No]
Reason: [explanation]
E-SNLI:
Answer the question and provide a reason why the answer is correct. [premise]
Question: Is [hypothesis]?
Answer: [Yes | No | Maybe]
ECQA:
Answer the question from the provided choices, and provide a reason why the answer is correct.
Question: [question]
Choices: [choices]
Answer: [one of the choices]
COMVE:
Which of the two choices makes more sense? and why?
Choice1: [choice1]
Choice2: [choice2]
Answer: [Choice1 | Choice2]
Figure 1: GPT-3 prompt templates for all datasets.
FEB Task	Similar T5 Pretraining Tasks
E-SNLI	MNLI(Williams et al., 2018)	Classify the entailment relation between two sequences
ECQA	RECORD(Zhang et al., 2018)	Answer a cloze-style query about a passage given entities in it
COMVE	COPA(Roemmle et al., 2011)	Select one of two sequences as the cause/effect of a premise
SBIC	COLA(Warstadt et al., 2019)	Classify a sentence as acceptable or not
Table 6: The first column shows tasks that we have included in FEB. Tasks on the right are included in T5's pretraining and they are similar to FEB's tasks. We explore self-rationalization prompts for FEB's tasks based on the tasks on the right, and compare them to prompts designed as span infilling and QA (§3).
GPUs	8 NVIDIA A100s 48 GB on Google Cloud
Implementation	https://github.com/allenai/feb
Hyperparameter	Assignment
max step number	300
batch size	4 (1 for T5/UNIFIEDQA-3B)
grad. accumulation steps	1 (4 for T5/UNIFIEDQA-3B)
learning rate	3e-5
learning rate scheduler	linear
warmup steps	0
decoding	greedy
Table 7: Hyperparameters used in our experiments.
Accuracy	BERTscore
Cos	INFILLING (b)	34.30.4	29.60.3
INFILLING (n)	40.10.4	34.70.3
≈T5	51.70.4	44.60.4
SQUADt5	51.10.3	44.10.3
QASIMPLE	60.00.3	48.60.3
Table 8: A comparison of all prompt types introduced in §3 on CoS-E. We do not support using CoS-E in the future given the reported issues with it (Narang et al., 2020; Wegreffe and Marasovic, 2021), especially since ECQA is introduced.
Size	Accuracy	BERTscore
COS-	BASE	58.30.3	50.40.2
LARGE	69.40.3	60.10.3
3B	75.40.3	65.30.3
GPT-3	68.41.3	59.51.2
Table 9: The effect of scaling the UNIFIEDQA model size on self-rationalization of CoS-E. We do not support using CoS-E in the future given the reported issues with it (Narang et al., 2020; Wegreffe and Marasovic, 2021), especially since ECQA is introduced.
Prompt	Accuracy	BERTscore
E-SNLI	Is...?	38.70.4	34.70.4
+ tags	48.20.6	43.20.6
What is...?	60.70.8	54.70.8
+ tags	77.90.3	70.10.3
ECQA	SQUADT5	36.50.3	32.40.3
RANDOM BASELINE	20.0	-
COMVE	Is...?	50.40.2	45.50.1
+ tags	50.20.1	45.30.1
What is...?	50.50.2	45.70.2
+ tags	54.50.5	49.20.4
SBIC	Is...?	63.40.6	61.10.6
+ tags	63.80.5	61.70.5
What is...?	66.70.5	64.30.5
+ tags	67.00.5	64.60.6
Table 10: A comparison between  $\mathrm{SQUAD}_{\mathrm{T5}}$  prompts with "Is...?" and "What is...?" questions. See §3.1 for more info. We also inspect the effects of adding answer choices and tags to the input. Tags are a single word descriptions of the input elements; e.g., E-SNLI's tags are "premise:" / "hypothesis:" before premise / hypothesis.
Sentence1: The stove was cleaned with a cleaner. Sentence2: The stove was cleaned with a mop. Nonsensical Sentence: Sentence2 Explanation: A mop is too large to clean the stove.
Prompt: INFILLING  $\times$  BASIC
Input: explain sensemaking choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop. <extra_id_0> because <extra_id_1>
Output: <extra_id_0> choice2 <extra_id_1> A mop is too large to clean the stove. <extra_id_2>
Prompt: INFILLING  $\times$  NATURAL SOUNDING
Input: explain sensemaking choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop. It is <extra_id_0> that choice2 is less common because <extra_id_1>
Output: <extra_id_0> True <extra_id_1>A mop is too large to clean the stove. <extra_id_2>
Prompt:  $\approx \mathrm{T}5\times$  COPA
Input: explain sensemaking choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop. Less common is choice2
Output: True because a mop is too large to clean the stove.
Prompt:  $\mathrm{SQUAD}_{\mathrm{T5}} \times \mathrm{YES} / \mathrm{NO} + \mathrm{TAGS}$
Input: explain sensemaking question: Is choice2 more nonsensical? context: choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop.
Output: Yes because a mop is too large to clean the stove.
Prompt:  $\mathrm{SQUAD}_{\mathrm{T5}} \times$  WHAT IS...? + TAGS
Input: explain sensemaking question: What is more nonsensical? context: choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop.
Output: choice2 because a mop is too large to clean the stove.
Prompt:  $\mathrm{QA}_{\mathrm{SIMPLE}} \times \mathrm{YES} / \mathrm{NO}$
Input: explain is choice2 more nonsensical? \\\\n The stove was cleaned with a cleaner. The stove was cleaned with a mop.
Output: yes because a mop is too large to clean the stove.
Prompt:  $\mathrm{QA}_{\mathrm{SIMPLE}} \times \mathrm{YES} / \mathrm{NO} + \mathrm{TAGS}$
Input: explain is choice2 more nonsensical? \n choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop.
Prompt:  $\mathrm{QA}_{\mathrm{SIMPLE}} \times \mathrm{YES} / \mathrm{NO} + \mathrm{TAGS} + \mathrm{CHOICES}$
Input: explain is choice2 more nonsensical? \n (A) yes (B) no \n choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop.
Prompt: QASIMPLE  $\times$  WHAT IS...?
Input: explain what is more nonsensical? \n The stove was cleaned with a cleaner. The stove was cleaned with a mop.
Prompt:  $\mathrm{QA}_{\mathrm{SIMPLE}} \times \mathrm{WHAT~IS...? + TAGS}$
Input: explain what is more nonsensical? \n choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop.
Prompt:  $\mathrm{QA}_{\mathrm{SIMPLE}} \times \mathrm{WHAT~IS...?} + \mathrm{TAGS} + \mathrm{CHOICES}$
Input: explain what is more nonsensical? \n (A) choice1 (B) choice2 \n choice1: The stove was cleaned with a cleaner. choice2: The stove was cleaned with a mop.
Table 11: COMVE self-rationalization prompts that we design and test. INFILLING marks span-filling prompts;  $\approx$ T5 prompts made by following the most similar T5 pretraining task (Table 1);  $SQUAD_{T5}$  prompts designed following SQUAD's formatting in T5 pretraining; and QASIMPLE prompts made following UNIFIEDQA. This table shows variations of these prompt types. We refer to spans "choice1:"/“choice2:" as TAGS, and to "(A) yes (B) no"/"(A) choice1 (B) choice2" as CHOICES. YES/No and WHAT is...? refer to a question type. Following Hendrycks et al. (2021), we add  $
$  to the end of our QASIMPLE prompts. More info in §3.