Original: When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data
Revised: When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data

Original: Anonymous ACL submission

Original: Abstract
Revised: Abstract

Original: Many methods now exist for conditioningmodels on task instructions and user-providedexplanations for individual data points.
Revised: Many methods now exist for conditioning models on task instructions and user-provided explanations for individual data points.

Original: Thesemethods show great promise for improvingtask performance of language models beyondwhat can be achieved by learning from individual ( x, y ) pairs.
Revised: These methods show great promise for improving task performance of language models beyond what can be achieved by learning from individual ( x, y ) pairs.

Original: In this paper, we (1) provide a formal framework for characterizingapproaches to learning from explanation data,and (2) we propose a synthetic task for studying how models learn from explanation data.
Revised: In this paper, we (1) provide a formal framework for characterizing approaches to learning from explanation data, and (2) we propose a synthetic task for studying how models learn from explanation data.

Original: In the ﬁrst direction, we give graphical models for the available modeling approaches, inwhich explanation data can be used as modelinputs , as targets , or as a prior .
Revised: In the ﬁrst direction, we give graphical models for the available modeling approaches, in which explanation data can be used as model inputs , as targets , or as a prior .

Original: In the seconddirection, we introduce a carefully designedsynthetic task with several properties makingit useful for studying a model’s ability to learnfrom explanation data.
Revised: In the second direction, we introduce a carefully designed synthetic task with several properties making it useful for studying a model’s ability to learn from explanation data.

Original: Each data point in thisbinary classiﬁcation task is accompanied by astring that is essentially an answer to the whyquestion: “why does data point x have labely ?"
Revised: Each data point in this binary classiﬁcation task is accompanied by a string that is essentially an answer to the why question: “why does data point x have label y ?"

Original: (Miller, 2019).
Revised: (Miller, 2019).

Original: We aim to encourage research into this area by identifying key considerations for the modeling problem and providing an empirical test bed for theories of howmodels can best learn from explanation data.
Revised: We aim to encourage research into this area by identifying key considerations for the modeling problem and providing an empirical test bed for theories of how models can best learn from explanation data.

Original:  Introduction
Revised:  Introduction

Original: A long line of past work has sought to use freetext explanations, rationales and other similar datato improve machine learning models.
Revised: A long line of past work has sought to use freetext explanations, rationales, and other similar data to improve machine learning models.

Original: Proposedmethods use explanations to constrain or regularizethe learned model (Zaidan et al., 2007;Small et al.
Revised: Proposed methods use explanations to constrain or regularize the learned model (Zaidan et al., 2007;

Original:  Ba et al., 2015; Zhang et al., 2016; Srivastavaet al., 2017; Liang et al., 2020), to automaticallylabel data for data augmentation (Hancock et al. Wang et al., 2019a; Awasthi et al., 2020), asadditional supervision (Narang et al., 2020; Haseet al., 2020; Pruthi et al., 2021) or intermediate
Revised: Small et al., 2011; Ba et al., 2015; Zhang et al., 2016; Srivastava et al., 2017; Liang et al., 2020), to automatically label data for data augmentation (Hancock et al., 2018; Wang et al., 2019a; Awasthi et al., 2020), as additional supervision (Narang et al., 2020; Hase .com/peterbhase/ExplanationRoles .

Revised: An extended technical report on this topic is available at: https://arxiv.org/abs/2102.02201 .

Original: Illustrative Example # Illustrative Example #
Revised: Illustrative Example # Illustrative Example #

Original: When asked for travel times, give them in terms of travel by car.
Revised: When asked for travel times, give them in terms of travel by car.

Original: How many hours does it take to travel from Addis Ababa to Dessie?
Revised: How many hours does it take to travel from Addis Ababa to Dessie?

Original: About 8 hours.
Revised: About 8 hours.

Original: Addis Ababa and Dessie are 400km apart by road, and assuming you could average 50kph in a car, the travel time would be about 8 hours.
Revised: Addis Ababa and Dessie are 400km apart by road, and assuming you could average 50kph in a car, the travel time would be about 8 hours.

Original: What are the names of people in the text?
Revised: What are the names of people in the text?

Original: She was in particular interested in Babbage's work on the Analytical Engine.
Revised: She was in particular interested in Babbage's work on the Analytical Engine.

Original: Lovelace first met him in June 1833, through their mutual friend, and her private tutor, Mary Somerville.
Revised: Lovelace first met him in June 1833, through their mutual friend, and her private tutor, Mary Somerville.

Original: Babbage, Lovelace, Mary Somerville.
Revised: Babbage, Lovelace, Mary Somerville.

Original: Names will refer to people, who can work on things, meet others, and be tutors.
Revised: Names will refer to people, who can work on things, meet others, and be tutors.

Original: Not all capitalized things are names.
Revised: Not all capitalized things are names.

Original: Engines are not people, and here June is a date.
Revised: [Figure] are not people, and here June is a date.

Original: structured variables (Camburu et al., 2018; Rajani [Figure] et al., 2019; Wiegreffe et al.,2020), and simply as
Revised: et al., 2020; Pruthi et al., 2021) or intermediate structured variables (Camburu et al.,

Original: [Figure]
Revised: Rajani et al.,

Original:  model inputs (Rupprecht et al.,2018; Co-Reyes [Figure]
Revised: Wiegreffe et al., 2020) and simply as model inputs (Rupprecht et al.,

Original:  et al., 2019; Zhou et al., 2020).[Figure]
Revised: Co-Reyes et al., 2019; Zhou et al., 2020).

Original: However, there are many tasks in NLP where

Original: [Figure]

Original: improvements in performance prove elusive even

Original: [Figure]

Original:  when using thousands of explanations as additional[Figure] data (Narang et al., 2020; Hase et al., 2020).A
Revised: However, there are many tasks in NLP where improvements in performance prove elusive even when using thousands of explanations as additional data (Narang et al., 2020; Hase et al., 2020).

Original: [Figure]

Original: few observations could explain this situation: (1)

Original: [Figure] the modeling space has not been fully explored[Figure] yet for these tasks, but improvements are possible; (2) pretrained language models already store the knowledge that the explanations would have provided, so they do not need them; (3) the language models do not need any information that is not already learnable from the task’s input-output pairs.
Revised: A few observations could explain this situation: (1) the modeling space has not been fully explored for these tasks, but improvements are possible; (2) pretrained language models already store the knowledge that the explanations would have provided, so they do not need them; (3) the language models do not need any information that is not already learnable from the task’s input-output pairs.

Original: We do not yet know which explanation is best, and therefore it would be helpful to more deeply understand the motivations behind existing modeling approaches, as well as to carefully consider why we think we can improve modeling performance using explanation data in the ﬁrst place.
Revised: We do not yet know which explanation is best, and therefore it would be helpful to more deeply understand the motivations behind existing modeling approaches.

Original: In this paper, we (1) present a formal framework for characterizing approaches to learning from explanation data, and (2) we propose a synthetic task for studying how models learn from natural language data.
Revised: In this paper, we (1) present a formal framework for characterizing approaches to learning from explanation data, and (2) we propose a synthetic task for studying how models learn from natural language data.

Original: Speciﬁcally, we ﬁrst present graphicalmodels for various approaches where explanationdata is used either as model inputs , targets , or priors , and we characterize existing methods according to these graphical models.
Revised: Speciﬁcally, we ﬁrst present graphical models for various approaches where explanation data is used either as model inputs , targets , or priors , and we characterize existing methods according to these graphical models.

Original: Then, based on pastresults, we suggest which models might be mostappropriate for explanation data.
Revised: Then, based on past results, we suggest which models might be most appropriate for explanation data.

Original: Next, we presenta synthetic task which shares important propertieswith NLP tasks involving explanation data.
Revised: Next, we present a synthetic task which shares important properties with NLP tasks involving explanation data.

Original: Constructing this task helps us carefully specify themanner in which we expect explanations to be useful to models.
Revised: Constructing this task helps us carefully specify the manner in which we expect explanations to be useful to models.

Original: We provide simple experimentalveriﬁcation that the task is solvable by existing
Revised: We provide simple experimental veriﬁcation that the task is solvable by existing

Original: Transformer models when using explanations asadditional data but very difﬁcult to solve withoutthem.
Revised: Transformer models when using explanations as additional data but very difﬁcult to solve without them.

Original: Our aim is to outline promising approachesin the area and contribute a concrete test bed toassist others in developing new models for learningfrom natural language explanations.
Revised: Our aim is to outline promising approaches in the area and contribute a concrete test bed to assist others in developing new models for learning from natural language explanations.

Original:  Formalizing the Roles of Explanations
Revised:  Formalizing the Roles of Explanations

Original: In what follows, we discuss our framework formodeling with explanations and relevant work(Sec. 2.1), as well as promising approaches forlearning from explanations (Sec. 2.2).
Revised: In what follows, we discuss our framework for modeling with explanations and relevant work (Sec. 2.1), as well as promising approaches for learning from explanations (Sec. 2.2).

Original: What Is an Explanation?
Revised: What Is an Explanation?

Original: We use the term “explanation” to refer to the data one might collect ifasking a person to answer the question, “Why doesdata point x have label y ?”
Revised: We use the term “explanation” to refer to the data one might collect if asking a person to answer the question, “Why does data point x have label y ?”

Original: This is a formulation ofthe explanation as an answer to a why-question ofthe kind discussed in Miller (2019).
Revised: This is a formulation of the explanation as an answer to a why-question of the kind discussed in Miller (2019).

Original: Rather than tryto give a formal deﬁnition of the kind of data generated from this question, we proceed with someillustrative examples, shown in Fig.
Revised: Rather than try to give a formal deﬁnition of the kind of data generated from this question, we proceed with some illustrative examples, shown in Fig.

Original: 2. Formal Framework and Relevant Work
Revised: 2. Formal Framework and Relevant Work

Original: In this section, we lay out our theory of how explanations may be used in modeling a task, in astandard supervised learning setup for obtaining a
Revised: In this section, we lay out our theory of how explanations may be used in modeling a task, in a standard supervised learning setup for obtaining a

Original: MAP estimate of model parameters:
Revised: MAP estimate of model parameters:

Original: [Equation]

Original: [Figure]

Original: [Equation]
Revised: [Equation]

Original: where Y is a set of labels for inputs X .
Revised: where Y is a set of labels for inputs X .

Original: We referto the role of Y in this probabilistic model as thetarget , X as an input , and p ( θ ) as a prior .
Revised: We refer to the role of Y in this probabilistic model as the target , X as an input , and p ( θ ) as a prior .

Original: Belowwe describe existing approaches to adding explanations into this framework.
Revised: Below we describe existing approaches to adding explanations into this framework.

Original: An overview of thecorresponding graphical models is shown in Fig.[Figure] do
Revised: An overview of the corresponding graphical models is shown in Fig.

Original: Using Explanations as Targets.
Revised: Using Explanations as Targets.

Original: Explanations are often used as additional supervision (shown  as Multi-Task in Fig. 2).
Revised: Explanations are often used as additional supervision (shown [Figure] as Multi-Task in Fig. 2).

Original: For instance, Pruthi et al.
Revised: For instance, Pruthi et al.

Original: (2021) consider using attention weight explanations (from a model) as targets in a multitask framework, and they observe accuracy improvements in what is essentially model distillation.
Revised: (2021) consider using attention weight explanations (from a model) as targets in a multitask framework, and they observe accuracy improvements in what is essentially model distillation.

Original: Meanwhile, natural language explanations appear as targets in a multi-task framework, using datasets with explanations for each data point (Camburu et al., 2018;
Revised: Meanwhile, natural language explanations appear as targets in a multi-task framework, using datasets with explanations for each data point (Camburu et al., 2018;

Original: Narang et al., 2020; Hase et al., 2020;
Revised: Narang et al., 2020; Hase et al., 2020;

Original: Wiegreffe et al., 2020).
Revised: Wiegreffe et al., 2020).

Original: None of these works ﬁnd improvements in task performance from incorporating explanations.
Revised: None of these works ﬁnd improvements in task performance from incorporating explanations.

Original: It is perhaps even concerning that a model could learn to generate coherent “explanations” without the learning of this ability inﬂuencing the models that are found for the task.
Revised: It is perhaps even concerning that a model could learn to generate coherent “explanations” without the learning of this ability inﬂuencing the models that are found for the task.

Original: Using Explanations as Inputs.
Revised: Using Explanations as Inputs.

Original: Additional inputs may be valuable for solving some tasks.
Revised: Additional inputs may be valuable for solving some tasks.

Original: One family of approaches uses explanations as model inputs for each data point (Per Data Point Input in
Revised: One family of approaches uses explanations as model inputs for each data point (Per Data Point Input in

Original: Fig. 2).
Revised: Fig. 2).

Original: Talmor et al.
Revised: Talmor et al.

Original: (2020) systematically study
Revised: (2020) systematically study

Original: RoBERTa’s ability to combine pieces of knowledge for a task by including relevant factoids in the text input.
Revised: RoBERTa’s ability to combine pieces of knowledge for a task by including relevant factoids in the text input.

Original: Co-Reyes et al.
Revised: Co-Reyes et al.

Original: (2019) provide online natural language feedback to RL agents, and Rupprecht et al.
Revised: (2019) provide online natural language feedback to RL agents, and Rupprecht et al.

Original: (2018) take a similar approach to interactiveimage segmentation with language feedback.
Revised: (2018) take a similar approach to interactive image segmentation with language feedback.

Original: More commonly, approaches do not use humanexplanations at test time.
Revised: More commonly, approaches do not use human explanations at test time.

Original: In ExpBERT (Murtyet al., 2020), a model conditions on vector representations of an input x and a single “global”set of explanations in order to make each prediction (Global Set in Fig. 2).
Revised: In ExpBERT (Murty et al., 2020), a model conditions on vector representations of an input x and a single “global” set of explanations in order to make each prediction (Global Set in Fig. 2).

Original: This approach maynot scale well to large numbers of explanations,however.
Revised: This approach may not scale well to large numbers of explanations, however.

Original: Zhou et al.
Revised: Zhou et al.

Original: (2020) treat explanations aslatent variables, and at inference time they retrieveexplanations from the training data (Retrieval in Fig. 2).
Revised: (2020) treat explanations as latent variables, and at inference time they retrieve explanations from the training data (Retrieval in Fig. 2).

Original: A number of works condition on explanations generated at test time using generative modelslearned with human explanations as supervision,which are represented asSingle Structured Variable and Per-Label Structured Variable in Fig.
Revised: A number of works condition on explanations generated at test time using generative models learned with human explanations as supervision, which are represented as Structured Variable and Per-Label Structured Variable in Fig.

Original: 2(Camburu et al., 2018; Rajani et al., 2019; Kumarand Talukdar, 2020;
Revised: 2 (Camburu et al., 2018; Rajani et al., 2019; Kumar and Talukdar, 2020;

Original: Hase et al., 2020; Wiegreffeet al., 2020; Zhao and Vydiswaran, 2021).
Revised: Hase et al., 2020; Wiegreffe et al., 2020; Zhao and Vydiswaran, 2021).

Original: Whilesuch structured variables could be useful in principle, these methods have not produced sustainedimprovements in model accuracy.
Revised: While such structured variables could be useful in principle, these methods have not produced sustained improvements in model accuracy.

Original: Lastly, large language models have recentlyopened the door for using explanations in few-shotin-context learning (Brown et al., 2020).
Revised: Lastly, large language models have recently opened the door for using explanations in few-shot in-context learning (Brown et al., 2020).

Original: We represent this approach as Few-shot In-context Learningin Fig.
Revised: We represent this approach as Few-shot In-context Learning in Fig.

Original: We do not draw the dependencies between distinct data points in the context that wouldbe implied by the attention graph of Transformers, but instead represent the dependence of eachdata point on the unknown task τ , which modelsevidently do inference over at test time.
Revised: We do not draw the dependencies between distinct data points in the context that would be implied by the attention graph of Transformers, but instead represent the dependence of each data point on the unknown task τ , which models evidently do inference over at test time.

Revised: Initial work in this direction suggests that models of a sufﬁciently large size (280B parameters) can learn from explanations provided in a few-shot in-context learning setting (Lampinen et al., 2022).

Original: Using Explanations as Priors.
Revised: Using Explanations as Priors.

Original: We group togetherapproaches to deﬁning a distribution over modelparameters, including those conditioning on data,p ( θ | data ) .
Revised: We group together approaches to deﬁning a distribution over model parameters, including those conditioning on data, p ( θ | data ) .

Original: This is a prior over model weights notin the sense that the distribution is independent ofdata (which it is not), but rather that the posteriorparameters are conditioned on the prior.
Revised: This is a prior over model weights not in the sense that the distribution is independent of data (which it is not), but rather that the posterior parameters are conditioned on the prior.

Original: Explanations have been used to constrain the learnedmodel (Srivastava et al., 2017, 2018) or to placepriors over how features are weighted or extracted(Zaidan et al., 2007;
Revised: Explanations have been used to constrain the learned model (Srivastava et al., 2017, 2018) or to place priors over how features are weighted or extracted (Zaidan et al., 2007;

Original: Small et al., 2011; Zhang et al.
Revised: Small et al., 2011; Zhang et al., 2016;

Original: ,2016; Ross et al., 2017; Bao et al., 2018; Selvarajuet al., 2019; Liang et al., 2020; Stammer et al. ,2020; Pruthi et al., 2021; Stacey et al., 2022).
Revised: Ross et al., 2017; Bao et al., 2018; Selvaraju et al., 2019; Liang et al., 2020; Stammer et al., 2020; Pruthi et al., 2021; Stacey et al., 2022).

Original: Otherworks map directly from text to model parameters(Ba et al., 2015; Andreas et al., 2018).
Revised: Other works map directly from text to model parameters (Ba et al., 2015; Andreas et al., 2018).

Original: These methods are all effectively described by Regularizer or
Revised: These methods are all effectively described by Regularizer or

Original: Hypernetwork in Fig.
Revised: Hypernetwork in Fig.

Original: Lastly, a few approaches[Figure] learn to use explanations for automatically labeling data for data augmentation purposes (Hancock et al., 2018; Wang et al., 2019b; Awasthi et al., 2020), which is effectively ﬁtting to data from a prior distribution given by the labeling mechanism (Data Augmentation in Fig. 2).
Revised: Lastly, a few approaches learn to use explanations for automatically labeling data for data augmentation purposes (Hancock et al., 2018; Wang et al., 2019b; Awasthi et al., 2020), which is effectively ﬁtting to data from a prior distribution given by the labeling mechanism (Data Augmentation in Fig. 2).

Original: 2. Promising Models
Revised: 2. Promising Models

Original: Based on our review of existing approaches, we make a few key observations that we believe will assist in the design of future techniques:
Revised: Based on our review of existing approaches, we make a few key observations that we believe will assist in the design of future techniques:

Original: Using free-text explanations as structured vari- ables and as targets do not appear to be promis- ing approaches at the moment (Hase et al., 2020;{+
+}
Revised: Using free-text explanations as structured variables and as targets do not appear to be promising approaches at the moment (Hase et al., {+
+}

Original:  Narang et al., 2020).
Revised: ; Narang et al., 2020).

Original: Free-text explanations may be useful as priors in computer vision (Liang et al., 2020), but we know of no successful use case for tasks besides Stacey et al.
Revised: Free-text explanations may be useful as priors in computer vision (Liang et al., 2020), but we know of no successful use case for tasks besides Stacey et al.

Original: (2022), which effectively reduces free-text explanations to a bag of words.
Revised: (2022), which effectively reduces free-text explanations to a bag of words.

Original: The only cases we know of where free-text ex-

Original: planations improve model performance on NLP tasks is when they are used as model inputs via the Global Set model (Murty et al., 2020)or a Retrieval model (Zhou et al., 2020).
Revised: The only cases we know of where free-text explanations improve model performance on NLP tasks is when they are used as model inputs via the Global Set model, (Murty et al., 2020) a Retrieval model (Zhou et al., 2020), and an InContext Learning model using 280B parameters (Lampinen et al., 2022).

Original: The upshot of these results is that the most promising

Original: approaches for

Original: learning

Original: from

Original:  explanation data are likely those treating explanations as inputs (in a manner that does not require new explanations at test time).
Revised: The upshot of these results is that the most promising approaches for learning from explanation data are likely those treating explanations as inputs (in a manner that does not require new explanations at test time).

Original: However

Original: , we recommend that other graphical models not be ruled out completely, in 
Revised: However, we recommend that other graphical models not be ruled out completely, in case there are promising methods in those families that have yet to be explored.

Original: case

Original: there are promising

Original: methods

Original: in those

Original: families that have yet

Original: to be

Original: explored.

Original:  Synthetic Task
Revised:  Synthetic Task

Original: Following recent work using synthetic data to investigate sequence modeling questions (Liu et al., 2021; Lovering et al., 2021), we design a syntheticdataset so that we can carefully control severalimportant data properties.
Revised: Following recent work using synthetic data to investigate sequence modeling questions (Liu et al., 2021; Lovering et al., 2021), we design a synthetic dataset so that we can carefully control several important data properties.

Original: In Fig.
Revised: In Fig.

Original: 3, we show anexample data point and description of how it getsits label.
Revised: 3, we show an example data point and description of how it gets its label.

Original: The premise of our task is to classifysequences by counting different integers in them.
Revised: The premise of our task is to classify sequences by counting different integers in them.

Original: Core Idea Behind Data.
Revised: Core Idea Behind Data.

Original: We wish to design atask where, for a data point ( x, y ) , an explanatione communicates information about why input xreceives label y .
Revised: We wish to design a task where, for a data point ( x, y ) , an explanation [Figure] Indicator A feature indicating what information from the explanation is relevant for the

Revised: [Figure]

Revised: 's

Original: The premise of the task is thata binary label for a sequence of integers x is determined by whether there are more of an integera in the sequence than  there are of an integer b .
Revised: label An explanation that says why the input received its label, when understood properly Description: The sequence has label because there are more s than s. The index maps to , and indicator says to count rather than . If there were more s than s, the label would be .

Revised: There is a one-to-one map between index values and tuples.

Revised: Count whether there are more of integer a than integer b e communicates information about why input x receives label y .

Revised: The premise of the task is that a binary label for a sequence of integers x is determined by whether there are more of an integer a in the sequence than there are of an integer b .

Original: We refer to integers ( a, b ) that need to be countedas the label reason .
Revised: We refer to integers ( a, b ) that need to be counted as the label reason .

Original: This label reason forms thebasis of the explanation for each data point, andit is always exactly speciﬁed by the ﬁrst two integers in x , which we term the index and indicator .
Revised: This label reason forms the basis of the explanation for each data point, and it is always exactly speciﬁed by the ﬁrst two integers in x , which we term the index and indicator .

Original: For every data point x , there is an explanatione = ( index , m, n, r, d ) where the label reason isgiven by either ( m, n ) or ( r, d ) .Whether the label reason is the ( m, n ) integer pair or the ( r, d )pair is dictated by the indicator . As representedin Fig.
Revised: For every data point x , there is an explanation e = ( index , m, n, r, d ) where the label reason is given by either ( m, n ) or ( r, d ) .

Original: 3, ( a, b ) = ( m, n ) if theindicator is 1 and( a, b ) = ( r, d ) if the indicator is 2.
Revised: Whether the label reason is the ( m, n ) integer pair or the ( r, d ) pair is dictated by the indicator .

Original: As representedin Fig.
Revised: As represented in Fig.

Original: 3, ( a, b ) =
Revised: 3, ( a, b ) =

Original: ( m, n ) if the indicator is 1 and( a, b ) =
Revised: ( m, n ) if the indicator is 1 and ( a, b ) =

Original: ( r, d ) if the indicator is 2.
Revised: ( r, d ) if the indicator is 2.

Original: We call the datae an explanation because it is a direct encoding ofa natural language explanation for the data ( x, y ) .
Revised: We call the data e an explanation because it is a direct encoding of a natural language explanation for the data ( x, y ) .

Original: For the data point in Fig.
Revised: For the data point in Fig.

Original: 3, this natural languageexplanation is “input x receives label 1 because itcontains more 80’s than 40’s, and we do not needto count 17’s or 27’s for this sequence."
Revised: 3, this natural language explanation is “input x receives label 1 because it contains more 80’s than 40’s, and we do not need to count 17’s or 27’s for this sequence."

Original: Proposed Dataset.
Revised: Proposed Dataset.

Original: We describe the proposeddataset using some default data parameters for preliminary experiments, but any speciﬁc numbers appearing below are easily adjusted.
Revised: We describe the proposed dataset using some default data parameters for preliminary experiments, but any speciﬁc numbers appearing below are easily adjusted.

Original: See Supplement
Revised: See Supplement

Original: D for the full generative process.
Revised: D for the full generative process.

Original: Train set: 5000 sequences of 20 integers (in- cluding index and indicator ).
Revised: Train set: 5000 sequences of 20 integers (including index and indicator ), each accompanied by an explanation.

Original: There are  unique values of index in the dataset drawnfrom unif (1 , 10000) , so there are 10 points foreach index , whose values of m, n, r, and d aredrawn from unif (1 , 100) while requiring that m (cid:54) =
Revised: There are 500 unique values of index in the dataset drawn from unif (1 , 10000) , so there are 10 points for each index , whose values of m, n, r, and d are drawn from unif (1 , 100) while requiring that m (cid:54) =

Original: n (cid:54) =
Revised: n (cid:54) =

Original: r (cid:54) =
Revised: r (cid:54) =

Original: d .
Revised: d .

Original: The corresponding 10 values of indicator are split between 1 and 2 .
Revised: The corresponding 10 values of indicator are split between 1 and 2 .

Original: Half of the

Original:  points have label y =1 , i e., either # m> # n or # r> # d , depending on which feature is causal.
Revised: Half of the points have label y =1 , i.e. either # m> # n or # r> # d , depending on which feature is causal.

Original: In each x i , after m, n, r, and d have been ran- domly placed into the sequence, unﬁlled slots are ﬁlled with samples from unif (1 , 100) .
Revised: In each x i , after m, n, r, and d have been randomly placed into the sequence, unﬁlled slots are ﬁlled with samples from unif (1 , 100) .

Original: Dev set: 10 , 000 points, none appearing in Train, with the same 500 index values, and twice the number of points per index as Train.
Revised: Dev set: 10 , 000 points, none appearing in Train, with the same 500 index values, and twice the number of points per index as Train.

Original: Test set: 50 , 000 points of similar construction to the Dev set, but with ﬁve times the points per index as Train.
Revised: Test set: 50 , 000 points of similar construction to the Dev set, but with ﬁve times the points per index as Train.

Original: Analogous Properties to Human-Curated Data
Revised: Analogous Properties to Human-Curated Data.

Original: We claim that aspects of our synthetic task are analogous to properties that natural language data might take on, which we represent in Fig.
Revised: We claim that aspects of our synthetic task are analogous to properties that natural language data might take on, which we represent in Fig.

Original: First, e is an explanation in the sense that, when understood properly, it is a plausible answer to the question: “why does point x have label y ?”
Revised: First, e is an explanation in the sense that, when understood properly, it is a plausible answer to the question: “why does point x have label y ?”

Original: The explanation describes the feature that causes the label, i.e. the integers that should be counted.
Revised: The explanation describes the feature that causes the label, i.e. the integers that should be counted.

Original: We suggest that the index in a sequence is analogous to the topic of some text or the things it refers to: it is an easily computable feature that connects the input to the appropriate explanation.
Revised: We suggest that the index in a sequence is analogous to the topic of some text or the things it refers to: it is an easily computable feature that connects the input to the appropriate explanation.

Original: Meanwhile, the indicator is a feature that tells how information from an explanation is relevant to deciding the label.
Revised: Meanwhile, the indicator is a feature that tells how information from an explanation is relevant to deciding the label.

Original: Similarly, an explanation might only be understood in the context of the input it explains.
Revised: Similarly, an explanation might only be understood in the context of the input it explains.

Original:  Initial Modeling Experiment.
Revised:  Initial Experiments

Revised: We include experiments below that (1) show explanation data is helpful for solving our task and (2) demonstrate why the task is hard without explanation data.

Original: We train a retrieval-based model similar to Zhou et al.
Revised: We make use of a retrieval-based model similar to Zhou et al.

Original: (2020) on our dataset (details in Appendix A and B), which learns to retrieve explanations from the training dataset to help with prediction at test time.
Revised: (2020), which learns to retrieve explanations from the training dataset to help with prediction at test time (details in Appendix B and C).

Revised: This model is composed of a RoBERTabase classiﬁer (Liu et al., 2019) and a SentenceRoBERTa model used for retrieval (Reimers and

Revised: Gurevych, 2019).

Revised: The baseline in our experiments is the RoBERTa classifer on its own.

Revised: 4. Explanation Retrieval Enables a Model to Solve Our Task

Revised: Design.

Original: Our preliminary results show that this modelachieves 98%+ accuracy on our task, while a comparable baseline without access to explanations achieves only 62% accuracy (Appendix Fig. 5).
Revised: Using our default dataset containing one explanation per training point, we measure model accuracy with retrieval in a 3 × 2 design. There are three conditions for the retrieval model: (1) ﬁxed , where the Sentence-RoBERTa retriever is ﬁxed and only the classiﬁer is trained, (2) learned , where both classiﬁer and retriever are trained end- [Figure] [Figure]

Revised: ᴇᴀɴ

Revised: [Figure]

Revised: ᴇ

Revised: [Figure]

Revised: FixedLearnedOptimal

Revised: Is Explanation Retrieval Helpful? of tasks increases (equivalent to the number of points per task decreasing), reaching accuracies as low as 62.2% at num-tasks = 500 .

Revised: Meanwhile, we observe that providing the index does slightly ease the task inference, but the models can by no means memorize the map from index to the task information.

Revised: Regarding model capacity, we ﬁnd that using RoBERTa-large increases model accuracy when the number of num-tasks is relatively low (less than 250), but after this point RoBERTa-base performs better (see Fig.in Appendix B).

Revised: Lastly, we see that increasing the training set size can greatly improve model performance even with num-tasks = 500 , reaching 87 .

Revised: 11% with 50 , 000 training points (trend shown in Fig.

Revised: 14 in Appendix B).

Revised: However, we will see in the next section that, in terms of improving model accuracy, even this 10 x increase in training size is less effective than using retrieved explanations withtraining points.

Revised: [ added transition off sample size point – PH ] [ roberta-large results in appendix.

Revised: better at low- regime,

Revised: 6.2. RQ2: Can retrieval of past explanations enable a model to solve our task?

Revised: Design.

Revised: Using the full-info explanations and data with num-tasks = 500 , we measure model accuracy with retrieval in a 3 ⇥ 2 design.

Revised: There are three conditions for the retrieval model: (1) ﬁxed , where the Sentence-RoBERTa retriever is ﬁxed and only the classiﬁer is trained, (2) learned , where both classiﬁer and retriever are trained end-to-end, and (3) optimal where the optimal retrieval model is used and the classiﬁer is trained.

Revised: Note that we know the optimal retrieval model assigns the highest probabilities to explanations with index e matching the query point’s index x , so by using a retriever p ( e i | x i ) =

Revised: exp ( [ index e = index x ]) and a context size lower than n task , we can ensure the retrieved explanations are always relevant.

Revised: There are two conditions for the conditioning mechanism used: (1) T EXT C AT with C

Revised: = k =6 ,

Revised: (2) H-M EAN with C =4 and k =4 , which approximately matches the computational cost of the T EXT C AT condition

Revised: Results.

Revised: Shown in Fig.

Revised: 6, the results show that retrieval with Sentence-BERT improves model accuracy by around 29 percentage points over a no-retrieval baseline.

Revised: Each conditioning mechanism sees roughly the same improvement.

Revised: Additionally, we can learn a retrieval model that does nearly as well as the optimal retrieval model, improving over the ﬁxed condition by another 7 points.

Revised: [ should we add some more reasons + conclusions/takeaways of these numerical results?

Revised: [ added a couple takeaway sentences – PH

Revised: – MB

Revised: Thus, retrieval of explanations allows the model to perform much better than a no-retrieval baseline .

Revised: We see a large improvement in performance from retrieval even when the baseline could learn to infer the task information directly from the index value in each input.

Revised: In fact, explanation retrieval outperforms a no-retrieval baseline with as many as 50 , 000 training data points (a 10 x increase), which obtains 87 .

Revised: 11% accuracy.

Revised: 6.3. RQ3: Can models aggregate information across explanations for better prediction?

Revised: Design.

Revised: We run the same experiment design as for

Revised: RQ2, using evidential and recomposable explanations (see

Revised: Sec. 3.3).

Revised: With evidential explanations, we shift each integer in the explanation (excluding the index ) independently by zero-mean, discrete noise ✏ ⇠ unif (  2 , 2) .

Revised: We use the 2-piece condition for recomposable explanations, meaning two explanations combine to give the full task information.

Revised: As in RQ1, we show results here for values of C = k

Revised: =

Revised: T EXT C AT

Revised: C k =

Original: Thus our explanations appear to contain highly useful information for solving the task, and our dataset should provide a useful testing ground for future models.
Revised: Results. We display the results in , we observe that for evidential explanations, learned retrieval is close to-end, and (3) optimal where the optimal retrieval model is used and the classiﬁer is trained. We know the optimal retrieval model retrieves explanations with an index matching the query point’s index .

Revised: The two conditioning mechanisms, H-M EAN and

Revised: T EXT C AT , differ in how they combine information across multiple retrieved explanations to produce a ﬁnal prediction (see Appendix B.1).

Revised: Results.

Revised: The results in Fig.

Revised: 4 show that explanation retrieval can reach accuracies above 98%, improving accuracy by around 37 points over a no-explanation baseline.

Revised: We also ﬁnd that the learned retrieval model does as well as the optimal retrieval model, improving over the ﬁxed condition by about 7 points.

Revised: Thus, access to explanations allows the model to perform much better than a no-explanation baseline.

Revised: In fact, the explanation retrieval model outperforms a no-explanation baseline with as many as 50 , 000 training data points (a 10 x increase), which obtains 87.11% accuracy.

Revised: 4. Why Is The Task Hard Without Explanations?

Revised: Design.

Revised: We measure test accuracy as a function of how many unique explanations (and therefore label reasons ) there are in the data.

Revised: While keeping the train set size ﬁxed at 5000 points, we vary how many points share the same explanation ( index , m, n, r, d ) .

Revised: By default there arepoints per index , and with 5000 points this means that there are 500 unique explanations in the data.

Revised: We use many as 2500 points per index , meaning using two unique explanations.

Revised: The experiment conditions also vary in how task information is available in the input: (1) for With Explanation, each 20-integer sequence x i has its explanation appended to it; (2) for No Explanation, only x i is given, which requires the model to learn the map

Revised: [Figure]

Revised: index → ( m, n, r, d ) ; (3) for No Index, the index is omitted from the input, so the model must infer the label reason from the sequence’s contents alone.

Revised: Results.

Revised: The results are shown in Fig.

Revised: We see that, when the number of unique explanations (and therefore possible label reasons ) is small, the No

Revised: Explanation model can achieve an accuracy as high as if it had been directly given the label reason, i.e. as high as the With Explanation condition.

Revised: Yet,

Revised: No Explanation model accuracy falls off quickly with the number of unique explanations, reaching accuracies as low as 62.2% with 500 explanations.

Revised: Evidently, with this many unique explanations, it is too difﬁcult to learn the map between the index and the latent label reason .

Revised: Without the index in the input (No Index condition), it is even harder to infer the label reason.

Revised: While accuracy does rise signiﬁcantly with the size of the training data (see

Revised: Fig. 4), even using 10x as much train data does not close the gap with the explanation retrieval model.

Original:  Discussion & Conclusion
Revised:  Discussion & Conclusion

Original: We  show that ourdataset contains useful explanations  for solving the synthetic task, but we emphasize that if a model already “knew" the information in the explanations, it might not need them.
Revised: We present a synthetic dataset with key similarities to natural language explanation data, and we show that our explanations are highly useful for model learning. However, we emphasize that if a model already “knew" the information in some explanations, it might not need them.

Original: Of course, this may  occur with  large pretrained models.
Revised: This may plausibly occur with sufﬁciently large pretrained models that store a great deal of factual knowledge (Petroni et al., 2019).

Original: Similarly, needed information might be learnable from ( X, Y ) data alone.
Revised: Similarly, the necessary information might be learnable from ( X, Y ) data alone.

Original: Future work into models we outline in this paper (Fig. 2) will beneﬁt from testing their methods on controlled synthetic taskslike ours as a test of their ability to learn from explanation data.
Revised: Future work on modeling approaches we outline in this paper (Fig. 2) will beneﬁt from testing their methods on controlled synthetic tasks as a test of their ability to learn from explanation data.

Original: Then, further analysis will be helpful for showing to what extent explanations contain novel information that is not learned elsewhere in pretraining or ﬁnetuning. 
Revised: Then, further analysis will be helpful for understanding how explanations contain novel information that is not learned elsewhere in pretraining or ﬁnetuning. Acknowledgements

Revised: We thank Miles Turpin and Ethan Perez for helpful discussion of the topics represented here, as well as Xiang Zhou, Prateek Yadav, and our anonymous reviewers for helpful feedback on the work.

Revised: This work was supported by NSF-CAREER

Revised: Award 1846185, DARPA Machine-Commonsense (MCS)

Revised: Grant N66001-19-2-4031, a Google PhD

Original:  Ethical Considerations
Revised: Fellowship, Microsoft Investigator Fellowship, and Google and AWS cloud compute awards. The views contained in this article are those of the authors and not of the funding agency. Ethical Considerations

Original: There are several positive broader impacts fromdesigning methods for learning from human explanations.
Revised: There are several positive broader impacts from designing methods for learning from human explanations.

Original: Foremost among them is the promise ofbetter aligning learned models with human priorson what kinds of behaviors are good, which couldbe especially helpful when these priors are hardto robustly encode in supervised learning objectives or unlikely to be learned from the availabledata.
Revised: Foremost among them is the promise of better aligning learned models with human priors on what kinds of behaviors are good, which could be especially helpful when these priors are hard to robustly encode in supervised learning objectives or unlikely to be learned from the available data.

Original: Explanations can also greatly improve modelsample efﬁciency, which is broadly beneﬁcial fordifﬁcult, time-consuming, or human-in-the-looptasks where acquiring a large amount of data isexpensive and slow.
Revised: Explanations can also greatly improve model sample efﬁciency, which is broadly beneﬁcial for difﬁcult, time-consuming, or human-in-the-loop tasks where acquiring a large amount of data is expensive and slow.

Original: There are still some possible downsides andlimitations to this methodology, mainly involvingoverconﬁdence in what explanations can provide.
Revised: There are still some possible risks to this methodology, mainly involving overconﬁdence in what explanations can provide.

Original: For instance, just because explanations improvea model’s performance does not mean the modelwill behave exactly as a human would.
Revised: For instance, just because explanations improve a model’s performance does not mean the model will behave exactly as a human would.

Original: We risk anthropomorphizing machine learning models whenwe suppose their learned interpretations of explanations matches our own.
Revised: We risk anthropomorphizing machine learning models when we suppose their learned interpretations of explanations matches our own.

Original: Meanwhile, the method wepropose is limited primarily by the suitability of theavailable explanation data. As we observe in ourexperiments, explanations can be extremely helpful, but the explanations collected for past datasetsdo not automatically satisfy all of the propertiesneeded for them to be useful. We hope that futurework can build on our results in order to better understand how to collect explanations for data whichare useful for modeling. adjusted
Revised: A Additional Experiments

Original: models generate non-trivial explanations of their behavior in natural language? In Findings of EMNLP . Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017.
Revised: We give additional experimental results with our synthetic dataset in an extended technical report on this topic, available here: https://arxiv.

Original: Billion-scale similarity search with gpus.

Original: IEEE

Original: Transactions on Big Data .

Original: Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick

Original: Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and

Original: Wen-tau Yih. 2020.
Revised: org/abs/2102.02201 .

Original: Dense passage retrieval for open-domain question answering EMNLP , pages 6769– Computational Linguistics.
Revised: Additional experiments are conducted to answer a research questions including:

Original: Sawan Kumar and Partha Talukdar

Original:  explanations In ACL 2020 . Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau
Revised: Can explanations help models learn to use strong (causal, generalizable) features rather than weak ones?

Original: Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela intensive NLP NeurIPS .
Revised: What is the best way to compute explanation representations for prediction?

Original: IJCNLP

Original: Hong Kong, China

Original: Association for Computational

Original: Linguistics. Andrew Slavin Ross, Michael C. Hughes, Finale Doshi
Revised: Can models aggregate information across several retrieved explanations?

Original: Christian Rupprecht, Iro Laina

Original: Nassir Navab

Original: Gre

Original: gory D

Original: Harger

Original: Federico Tombari with

Original: Conference Ramprasaath Ramasamy Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry P. Heck, Dhruv Batra, and Devi Parikh.
Revised: What makes an explanation relevant across data points?

Original: Taking a HINT: leveraging explanations to make vision andlanguage models more grounded. In ICCV , pages2591–2600. IEEE.
Revised: What enables a retrieval model to ﬁnd relevant explanations for a new data point?

Original: Kevin Small, Byron C Wallace, Carla E Brodley, and Thomas A Trikalinos. 2011. The constrained weightspace
Revised: How does the co-dependence between classiﬁer and retrieval model inﬂuence the viability of joint training?

Original: Shashank Srivastava, I. Labutov, and T. Mitchell.

Original: Shashank Srivastava, Igor Labutov, and Tom Mitchell

Original: menting language inference with relevant explanation.

Original: In AAAI .

Original: Wangchunshu Zhou, Jinyi Hu, Hanlin Zhang, Xiaodan Liang, Maosong Sun, Chenyan Xiong, and Jian

Original: Tang. Towards interpretable natural language understanding with explanations as latent variables. In NeurIPS . [Figure]
Revised: Does retrieval of explanations improve model performance on existing natural language datasets?

Original: A Our Model for Initial Experiments
Revised: B Our Model for Initial Experiments

Original: Here, we introduce our chosen model for incorporating explanation data, which makes use of explanations as model inputs after they are retrievedfrom the training data (the “Retrieval” graphicalmodel in Fig. 2).
Revised: Here, we introduce our chosen model for incorporating explanation data, which makes use of explanations as model inputs after they are retrieved from the training data (the “Retrieval” graphical model in Fig. 2).

Original: Our approach is similar to Lewiset al.
Revised: Our approach is similar to Lewis et al.

Original: (2020), who marginalize over latent documents retrieved from Wikipedia for question answering, question generation, and fact veriﬁcation.
Revised: (2020), who marginalize over latent documents retrieved from Wikipedia for question answering, question generation, and fact veriﬁcation.

Original: The marginal distribution is given as:
Revised: The marginal distribution is given as:

Original: [Equation]
Revised: [Equation]

Original: where top- k gets the top k texts as ranked by the retrieval model, p η ( e | x ) .
Revised: where top- k gets the top k texts as ranked by the retrieval model, p η ( e | x ) .

Original: Note that we never retrievea data point’s own explanation when predicting itslabel.
Revised: Note that we never retrieve a data point’s own explanation when predicting its label.

Original: We do so because explanations can leak thelabel (Hase et al., 2020) and this approach matchesthe test-time distribution, where we assume explanations are not collected for new data points (seediscussion in Sec. 2).
Revised: We do so because explanations can leak the label (Hase et al., 2020) and this approach matches the test-time distribution, where we assume explanations are not collected for new data points (see discussion in Sec. 2).

Original: Zhou et al.
Revised: Zhou et al.

Original: (2020) also propose to use explanations as latent variables and retrieve explanationsat inference time, but they do not learn the retrievalmodel, marginalize over the latents during inference, or prohibit data point’s own explanationsfrom being retrieved.
Revised: (2020) also propose to use explanations as latent variables and retrieve explanations at inference time, but they do not learn the retrieval model, marginalize over the latents during inference, or prohibit data point’s own explanations from being retrieved.

Original: In our experiments, we compare with their original approach and a versionwhere we marginalize over the latents and learn theretrieval model.
Revised: In our experiments, we compare with their original approach and a version where we marginalize over the latents and learn the retrieval model.

Original: The form of p η ( e | x ) follows Lewis et al.
Revised: The form of p η ( e | x ) follows Lewis et al.

Original: (2020)and Karpukhin et al.
Revised: and Karpukhin et al.

Original: Given a query x ,unnormalized probabilities are computed as:
Revised: Given a query x , unnormalized probabilities are computed as:

Original: [Equation]
Revised: [Equation]

Original: where f η embeds each sequence into a vector.
Revised: where f η embeds each sequence into a vector.

Original: To compute top- k ( p η ( ·| x )) , we search throughthe training explanations using FAISS (Johnsonet al., 2017).
Revised: To compute top- k ( p η ( ·| x )) , we search through the training explanations using FAISS (Johnson et al., 2017).

Original: We discuss methods for computingp θ ( y | x, e ) and f η ( e | x ) in Sec.
Revised: We discuss methods for computing p θ ( y | x, e ) and f η ( e | x ) in Sec.

Original: A.1.
Revised: B.1.

Original: Because it maybe helpful to reason over multiple explanations at once, we extend this model to allow for explanations to be composed into a single “document.”
Revised: Because it may be helpful to reason over multiple explanations at once, we extend this model to allow for explanations to be composed into a single “document.”

Original: Assuming explanations to be conditionally independent given x , we can compute the probability of a set of explanations E = { e c } Cc =1 as
Revised: Assuming explanations to be conditionally independent given x , we can compute the probability of a set of explanations E = { e c } Cc =1 as

Original: [Equation]
Revised: [Equation]

Original: where (1) a context size C will control the size of the explanation set, (2) a value of k implies that the top Ck will be retrieved, and (3) we sort these Ck explanations into sets in order of their probability
Revised: where (1) a context size C will control the size of the explanation set, (2) a value of k implies that the top Ck will be retrieved, and (3) we sort these Ck explanations into sets in order of their probability

Original: p η ( e | x ) .
Revised: p η ( e | x ) .

Revised: [Figure]

Original: We represent the overall approach in Fig.
Revised: We represent the overall approach in Fig.

Original: 4 for one method of computing p θ ( y | x, E ) (described fully in Sec. A.1), where explanations are concatenated with the query sequence.
Revised: 6 for one method of computing p θ ( y | x, E ) (described fully in Sec. B.1), where explanations are concatenated with the query sequence.

Original: Flowing from left to right, Fig.
Revised: Flowing from left to right, Fig.

Original: 4 shows how explanations are retrieved from the training data conditioned on a query sequence x , then allocated into k classiﬁer inputs with C explanations each.
Revised: 6 shows how explanations are retrieved from the training data conditioned on a query sequence x , then allocated into k classiﬁer inputs with C explanations each.

Original: The k classiﬁer predictions are aggregated by marginalizing over the latent variable, Z = E .
Revised: The k classiﬁer predictions are aggregated by marginalizing over the latent variable, Z = E .

Original: Modeling Assumptions.
Revised: Modeling Assumptions.

Original: In using retrieval, we make a few assumptions.
Revised: In using retrieval, we make a few assumptions.

Original: First, since the number of forward passes per data point scales with k , we require a relatively small value of k , i.e. k ≤ 10 , for reasonable computational efﬁciency in SGDbased training.
Revised: First, since the number of forward passes per data point scales with k , we require a relatively small value of k , i.e. k ≤ 10 , for reasonable computational efﬁciency in SGDbased training.

Original: Hence, we must assume that this summation is sufﬁciently similar to the full summation over latent variables.
Revised: Hence, we must assume that this summation is sufﬁciently similar to the full summation over latent variables.

Original: This assumption is more likely to hold when (1) a small number of documents account for most of the probability mass in p η ( e | x ) , and (2) a pretrained model p η ( e | x ) yields a decent initial rank-ordering, such that some of the best documents are in the top- k .
Revised: This assumption is more likely to hold when (1) a small number of documents account for most of the probability mass in p η ( e | x ) , and (2) a pretrained model p η ( e | x ) yields a decent initial rank-ordering, such that some of the best documents are in the top- k .

Original: The exact value of k we use depends on the experiment.
Revised: The exact value of k we use depends on the experiment.

Original: A second, more basic assumption is that explanations will be useful in predicting other data points’ labels.
Revised: A second, more basic assumption is that explanations will be useful in predicting other data points’ labels.

Original: Such an assumption is needed since we never condition on a data point’s own explanation.
Revised: Such an assumption is needed since we never condition on a data point’s own explanation.

Original: Lastly, duringretrieval we assume that explanations are independent given x , i.e. p ( E | x ) =(cid:81)
Revised: Lastly, during retrieval we assume that explanations are independent given x , i.e. p ( E | x ) =

Original:  e ∈ E p ( e | x ) .
Revised: (cid:81) e ∈ E p ( e | x ) .

Original: Thiscould be a poor assumption when, for instance,explanations each contribute one of a number ofneeded facts, in which case it would be helpfulto retrieve additional explanations conditioned onwhat has already been retrieved.
Revised: This could be a poor assumption when, for instance, explanations each contribute one of a number of needed facts, in which case it would be helpful to retrieve additional explanations conditioned on what has already been retrieved.

Original: A. Conditioning Mechanisms
Revised: B. Conditioning Mechanisms

Original: In this section we describe the methods used tocompute p θ ( y | x, E ) and p η ( e | x ) (see Sec.
Revised: In this section we describe the methods used to compute p θ ( y | x, E ) and p η ( e | x ) (see Sec.

Original: A forthe overall model description).
Revised: B for the overall model description).

Original: For the classiﬁerp θ ( y | x, E ) , we use two methods, T EXT C AT and
Revised: For the classiﬁer p θ ( y | x, E ) , we use two methods, T EXT C AT and

Original: H-M EAN , which are described below.
Revised: H-M EAN , which are described below.

Original: Then wedescribe the retrieval model, which is based on
Revised: Then we describe the retrieval model, which is based on

Original: Sentence-BERT (Reimers and Gurevych, 2019).
Revised: Sentence-BERT (Reimers and Gurevych, 2019).

Original: T EXT C AT .
Revised: T EXT C AT .

Original: Represented in Figure 4, this methodtakes a straightforward approach to conditioningon a set of explanations: concatenating C explanations and the input x to form a longer sequence oftext.
Revised: Represented in Figure 6, this method takes a straightforward approach to conditioning on a set of explanations: concatenating C explanations and the input x to form a longer sequence of text.

Original: Each of the original sequences is separatedby a special token, e.g. [SEP] for BERT.
Revised: Each of the original sequences is separated by a special token, e.g. [SEP] for BERT.

Original: In ourexperiments, we pass this longer sequence into a
Revised: In our experiments, we pass this longer sequence into a

Original: RoBERTa-base model.
Revised: RoBERTa-base model.

Original: After pooling the outputtoken representations, we pass the resulting vector to a 1-layer MLP for classiﬁcation.
Revised: After pooling the output token representations, we pass the resulting vector to a 1-layer MLP for classiﬁcation.

Original: We usemean pooling for our synthetic task and NLI; forrelation extraction tasks, we concatenate the representations corresponding to the initial tokens in thesubject and object words, since this is an especiallyeffective pooling technique (Baldini Soares et al.,2019).
Revised: We use mean pooling for our synthetic task and NLI; for relation extraction tasks, we concatenate the representations corresponding to the initial tokens in the subject and object words, since this is an especially effective pooling technique (Baldini Soares et al., 2019).

Original: This approach allows the model to reason overall of the explanations and the input together.
Revised: This approach allows the model to reason over all of the explanations and the input together.

Original: Whilethe method may be limited by the fact that somemodels can face difﬁculties in processing longpieces of text (Beltagy et al., 2020), this issue ispartly mitigated by marginalizing over k sets of explanations.
Revised: While the method may be limited by the fact that some models can face difﬁculties in processing long pieces of text (Beltagy et al., 2020), this issue is partly mitigated by marginalizing over k sets of explanations.

Original: As a result of the marginalization, theﬁnal prediction can be conditioned on a far highernumber ( Ck ) of individual explanations than couldﬁt in the context alone.
Revised: As a result of the marginalization, the ﬁnal prediction can be conditioned on a far higher number ( Ck ) of individual explanations than could ﬁt in the context alone.

Original: H-M EAN .
Revised: [Equation]

Original: By H-M EAN , we refer to the kind ofunweighted hidden representation averaging usedin Co-Reyes et al.
Revised: By H-M EAN , we refer to the kind of unweighted hidden representation averaging used in Co-Reyes et al.

Original: and
Revised: and

Original: Zhou et al.
Revised: Zhou et al.

Original: H-M EAN works by ﬁrst obtaining representationsof the input x and a single explanation e at a time,then passing the unweighted average of these representations to an MLP.
Revised: H-M EAN works by ﬁrst obtaining representations of the input x and a single explanation e at a time, then passing the unweighted average of these representations to an MLP.

Original: For a fair comparison with
Revised: For a fair comparison with

Original: T EXT C AT , we use the same token pooling and a1-layer MLP.
Revised: T EXT C AT , we use the same token pooling and a 1-layer MLP.

Original: So with C explanations to condition on, x (cid:48)
Revised: So with C explanations to condition on, x (cid:48)

Original: =
Revised: =

Original: concatenate ( x, e ) , and vector representations from RoBERTa ( x (cid:48) ) , H-M EAN obtains a single representation as
Revised: concatenate ( x, e ) , and vector representations from RoBERTa ( x (cid:48) ) , H-M EAN obtains a sin- gle representation as

Original: [Equation]
Revised: [Equation]

Original: which is then passed to the MLP for classiﬁcation.
Revised: which is then passed to the MLP for classiﬁcation.

Original: H-M EAN does not face the same sequence length limitations as T EXT C AT , but by separately processing of each explanations H-M EAN may fail to integrate information across explanations.
Revised: H-M EAN does not face the same sequence length limitations as T EXT C AT , but by separately processing of each explanations H-M EAN may fail to integrate information across explanations.

Original: This method also becomes expensive when we marginalize over
Revised: This method also becomes expensive when we marginalize over

Original: E (which is what allows retrieval to be learned), as it requires Ck forward passes for a single prediction.
Revised: E (which is what allows retrieval to be learned), as it requires Ck forward passes for a single prediction.

Original: A. Retrieval
Revised: B. Retrieval

Original: We use a similar approach to retrieval as in Lewis et al.
Revised: We use a similar approach to retrieval as in Lewis et al.

Original: (2020), namely using vector representations of sequences from a pretrained transformer to compute
Revised: (2020), namely using vector representations of sequences from a pretrained transformer to compute

Original: [Equation]
Revised: [Equation]

Original: which is followed by computing top-
Revised: which is followed by computing top-

Original: Ck ( p η ( ·| x ) .
Revised: Ck ( p [Equation] [Equation] x ) .

Original: We use an approximate but sub-linear time search method (FAISS) to ﬁnd the top- Ck points (Johnson et al., 2017).
Revised: We use an approximate but sub-linear time search method (FAISS) to ﬁnd the top- Ck points (Johnson et al., 2017).

Original: In our experiments we ﬁnd that it is necessary to use Sentence-BERT (Reimers and
Revised: In our experiments we ﬁnd that it is necessary to use Sentence-BERT (Reimers and

Original: Gurevych, 2019) as our pretrained f η , rather than simply a pretrained RoBERTa model.
Revised: Gurevych, 2019) as our pretrained f η , rather than simply a pretrained RoBERTa model.

Original: SentenceBERT is a network trained to produce semantic representations of sentences that can be compared under cosine similarity.
Revised: SentenceBERT is a network trained to produce semantic representations of sentences that can be compared under cosine similarity.

Original: In our experiments, we use the Sentence-RoBERTa-base model trained on a combination of several NLI and semantic textual similarity tasks, with mean pooling of token representations.
Revised: In our experiments, we use the Sentence-RoBERTa-base model trained on a combination of several NLI and semantic textual similarity tasks, with mean pooling of token representations.

Original: We normalize the representations we obtain from this model, so that our inner product is equivalent to a cosine similarity.
Revised: We normalize the representations we obtain from this model, so that our inner product is equivalent to a cosine similarity.

Original: Note that during training, we never condition on a data point’s own explanation when predicting its label.
Revised: Note that during training, we never condition on a data point’s own explanation when predicting its label.

Original: This is an important constraint for matching the train and test-time distributions.
Revised: This is an important constraint for matching the train and test-time distributions.

Original: At test time, we assume we have access only to past (training) explanations, since they can be expensive to collect and conditioning on explanations at test time can lead to label leakage, meaning what is essentially the beneﬁt of human labeling could be mistaken as improvements in model performance.[Figure]
Revised: At test time, we assume we have access only to past (training) explanations, since they can be expensive to collect and conditioning on explanations at test time can lead to label leakage, meaning what is essentially the beneﬁt of human labeling could be mistaken as improvements in model performance.

Original: Is Explanation Retrieval Helpful? of tasks increases (equivalent to the number of points per task decreasing), reaching accuracies as low as 62.2% at num-tasks = 500 .

Original: Meanwhile, we observe that providing the index does slightly ease the task inference, but the models can by no means memorize the map from index to the task information.

Original: Regarding model capacity, we ﬁnd that using RoBERTa-large increases model accuracy when the number of num-tasks is relatively low (less than 250), but after this point RoBERTa-base performs better (see Fig.in Appendix B).

Original: Lastly, we see that increasing the training set size can greatly improve model performance even with num-tasks = 500 , reaching 87 .

Original: 11% with 50 , 000 training points (trend shown in Fig.

Original: 14 in Appendix B).

Original: However, we will see in the next section that, in terms of improving model accuracy, even this 10 x increase in training size is less effective than using retrieved explanations withtraining points.

Original: [ added transition off sample size point – PH ] [ roberta-large results in appendix.

Original: better at low-task regime, worse in high-task regime – PH ]

Original: 6.2. RQ2: Can retrieval of past explanations enable a model to solve our task? Design.

Original: Using the full-info explanations and data with num-tasks = 500 , we measure model accuracy with retrieval in a 3 ⇥ 2 design.

Original: There are three conditions for the retrieval model: (1) ﬁxed , where the Sentence-RoBERTa retriever is ﬁxed and only the classiﬁer is trained, (2) learned , where both classiﬁer and retriever are trained end-to-end, and (3) optimal where the optimal retrieval model is used and the classiﬁer is trained.

Original: Note that we know the optimal retrieval model assigns the highest probabilities to explanations with index e matching the query point’s index x , so by using a re- triever p ( e i | x i ) =

Original: exp ( [ index e = index x ]) and a context size lower than n task , we can ensure the retrieved explana- tions are always relevant.

Original: There are two conditions for the conditioning mechanism used: (1) T EXT C AT with C

Original: = k =6 , and (2) H-M EAN with C =4 and k =4 , which approximately matches the computational cost of the T EXT C AT condition.

Original: Results.

Original: Shown in Fig.

Original: 6, the results show that retrieval with Sentence-BERT improves model accuracy by around 29 percentage points over a no-retrieval baseline.

Original: Each conditioning mechanism sees roughly the same improvement.

Original: Additionally, we can learn a retrieval model that does nearly as well as the optimal retrieval model, improving over the ﬁxed condition by another 7 points.

Original: [ should we add some more reasons + conclusions/takeaways of these numerical results?

Original: [ added a couple takeaway sentences – PH ] – MB ]

Original: Thus, retrieval of explanations allows the model to perform much better than a no-retrieval baseline .

Original: We see a large improvement in performance from retrieval even when the baseline could learn to infer the task information directly from the index value in each input.

Original: In fact, explanation retrieval outperforms a no-retrieval baseline with as many as 50 , 000 training data points (a 10 x increase), which obtains 87 .

Original: 11% accuracy.

Original: 6.3. RQ3: Can models aggregate information across explanations for better prediction? Design.

Original: We run the same experiment design as for

Original: RQ2, using evidential and recomposable explanations (see

Original: Sec. 3.3).

Original: With evidential explanations, we shift each integer in the explanation (excluding the index ) independently by zero-mean, discrete noise ✏ ⇠ unif (  2 , 2) .

Original: We use the 2-piece condition for recomposable explanations, meaning two explanations combine to give the full task information.

Original: As in RQ1, we show results here for values of C = k =6 for T EXT C AT and C =

Original: k

Original: =4 for H-M EAN .

Original: Results.

Original: We display the results in Fig.

Original: First, we observe that for evidential explanations, learned retrieval is close

Original: B Training Details 
Revised: C Training Details C. Runtimes.

Original: B. Runtimes.

Original: Regarding training times, we run most experimentson a single NVIDIA RTX 2080 GPU, with runtimes as follows: 4.0 hours for 40 epochs of the noretrieval RoBERTa-base using the synthetic dataset;5.7 hours for 40 epochs of RoBERTa-large in thesame setting; 8.6 hours for 20 epochs of learnedretrieval with RoBERTa-base models on syntheticdata.
Revised: Regarding training times, we run most experiments on a single NVIDIA RTX 2080 GPU, with runtimes as follows: 4.0 hours for 40 epochs of the noretrieval RoBERTa-base using the synthetic dataset; 5.7 hours for 40 epochs of RoBERTa-large in the same setting; 8.6 hours for 20 epochs of learned retrieval with RoBERTa-base models on synthetic data.

Original: B. Training Hyperparameters and Analysis
Revised: C. Training Hyperparameters and Analysis

Original: For optimization, we use AdamW with a learningrate of 1e − 5 and gradient norm clipping at norm1.
Revised: For optimization, we use AdamW with a learning rate of 1e − 5 and gradient norm clipping at norm 1.

Original: For the LR, we use a linear warmup and decayschedule peaking at 10% of the training steps forexperiments with synthetic data and at 1% for experiments with existing datasets (given the largertraining set sizes).
Revised: For the LR, we use a linear warmup and decay schedule peaking at 10% of the training steps for experiments with synthetic data and at 1% for experiments with existing datasets (given the larger training set sizes).

Original: The batch size is set to 10 acrossall experiments.
Revised: The batch size is set to 10 across all experiments.

Original: We decide how often to rebuild the representations of training explanations while learning theretrieval model by tuning across frequency valuesin the range {10%, 20%, 33%, 50%, 100%} (i.e. torebuild at this percentage of every epoch), as wellas never rebuilding.
Revised: We decide how often to rebuild the representations of training explanations while learning the retrieval model by tuning across frequency values in the range {10%, 20%, 33%, 50%, 100%} (i.e. to rebuild at this percentage of every epoch), as well as never rebuilding.

Original: In our synthetic setting, theonly noticeable drop in performance comes fromnever rebuilding.
Revised: In our synthetic setting, the only noticeable drop in performance comes from never rebuilding.

Original: As long as representations arere-encoded at least as often as every epoch, wenotice no difference in ﬁnal test accuracy, thoughin early experiments we observed that rebuildingmore often improved training stability.
Revised: As long as representations are re-encoded at least as often as every epoch, we notice no difference in ﬁnal test accuracy, though in early experiments we observed that rebuilding more often improved training stability.

Original: To err onthe safe side of training stability, we re-encode therepresentations every 20% of each epoch in all experiments except e-SNLI with full data, where were-encode every 30% of each epoch.
Revised: To err on the safe side of training stability, we re-encode the representations every 20% of each epoch in all experiments except e-SNLI with full data, where we re-encode every 30% of each epoch.

Original: Additionally, we use the stop-gradient function when computing the gradient of p η ( e | x ) as follows:
Revised: Additionally, we use the stop-gradient function when computing the gradient of p η ( e | x ) as follows:

Original: [Equation]
Revised: [Equation]

Original: meaning that we do not differentiate through the explanation embeddings, but only through the query data point embeddings.
Revised: meaning that we do not differentiate through the explanation embeddings, but only through the query data point embeddings.

Original: In early experiments, we found that this decision contributed to training stability, while improving computational efﬁciency, and we conﬁrm that we observe no differences in model accuracy as a result.
Revised: In early experiments, we found that this decision contributed to training stability, while improving computational efﬁciency, and we conﬁrm that we observe no differences in model accuracy as a result.

Original: C Initial Experiment

Original: Below, we give the experimental design and results for our initial experiment with our synthetic dataset.

Original: Design.

Original: We measure model accuracy with retrieval in a 3 × 2 design.

Original: There are three conditions for the retrieval model: (1) ﬁxed , where the Sentence-RoBERTa retriever is ﬁxed and only the classiﬁer is trained, (2) learned , where both classiﬁer and retriever are trained end-to-end, and (3) optimal where the optimal retrieval model is used and the classiﬁer is trained.

Original: Note that we know the optimal retrieval model assigns the highest probabilities to explanations with index e matching the query point’s index x , so by using a retriever p ( e i | x i ) =

Original: exp ( 1 [ index e = index x ]) and a context size lower than n task , we can ensure the retrieved explanations are always relevant.

Original: There are two conditions for the conditioning mechanism used: (1) T EXT C AT with C = k =6 , and (2) H-M EAN with C =4 and k =4 , which approximately matches the computational cost of the T EXT C AT condition.
Revised: C. Experiment Conﬁdence Intervals

Original: We compute conﬁdence intervals for our synthetic data tasks to represent seed variance around some mean seed performance.
Revised: We compute conﬁdence intervals for our synthetic data tasks to represent seed variance around some mean seed performance.

Original: We represent seedvariance in ﬁgures rather than sample variance because the sample variance is fairly low with 50 , 000test points and could be driven arbitrarily low withmore generated test points.
Revised: We represent seed variance in ﬁgures rather than sample variance because the sample variance is fairly low with 50 , 000 test points and could be driven arbitrarily low with more generated test points.

Original: For instance, the 95%conﬁdence interval for a model accuracy of 90%would be ± 0 .
Revised: For instance, the 95% conﬁdence interval for a model accuracy of 90% would be ± 0 .

Original: To calculate seed variance, werun 10 random seeds for our baseline condition(no-retrieval) with the default synthetic task setup.Results.
Revised: To calculate seed variance, we run 10 random seeds for our baseline condition (no-retrieval) with the default synthetic task setup.

Original: Shown in Fig.

Original: 5, the results show thatretrieval with Sentence-BERT can reach accuraciesabove 98%, improving model accuracy by around37 percentage points over a no-retrieval baseline.

Original: Each conditioning mechanism sees roughly thesame improvement.

Original: Additionally, we ﬁnd that thelearned retrieval model does as well as the optimalretrieval model, improving over the ﬁxed conditionby about 7 points.

Original: Thus, retrieval of explanationsallows the model to perform much better thana no-retrieval baseline .

Original: We see a large improvement in performance from retrieval even when thebaseline could learn to infer the task information directly from the index value in each input.

Original: In fact, explanation retrieval outperforms a no-retrieval baseline with as many as 50 , 000 training data points (a10 x increase), which obtains 87.11% accuracy.

Original: D Synthetic Task Generative Process
Revised: D Synthetic Task Generative Process

Original: The required parameters to the data generation include: (1) a training sample size sample-size and (2)num-tasks , the number of unique integer pairs to becounted, or, equivalently, the number of points perindex , n task .
Revised: The required parameters to the data generation include: (1) a training sample size sample-size and (2) num-tasks , the number of unique integer pairs to be counted, or, equivalently, the number of points per index , n task .

Original: In all experiments, we use a maximuminteger value of 100 to appear in the sequences,and a maximum index value of 10 , 000 .
Revised: In all experiments, we use a maximum integer value of 100 to appear in the sequences, and a maximum index value of 10 , 000 .

Original: We givethe general generative process below.
Revised: We give the general generative process below.

Original: Note thatthe dev and test sets are constructed with the extraconstraint that sequences must not appear in thetraining data.
Revised: Note that the dev and test sets are constructed with the extra constraint that sequences must not appear in the training data.

Original: Further note that this is the genericversion of generative process, and in some experiments the process is altered.
Revised: Further note that this is the generic version of generative process, and in some experiments the process is altered.

Original: For example, in RQ3,indicator is always 1 and the construction of themap from index values to ( m, n ) tuples occurs in aspecial way described in the experimental designfor RQ3.
Revised: For example, in RQ3, indicator is always 1 and the construction of the map from index values to ( m, n ) tuples occurs in a special way described in the experimental design for RQ3.

Original: Sample { index t } num-tasks τ = from the uniform dis- tribution over integers {1,...,10000} without replacement.
Revised: Sample { index t } num-tasks τ = from the uniform distribution over integers {1,...,10000} without replacement.

Original: Sample { ( m, n, r, d ) t } num - tasks τ = from the uniform distribution over integers, unif ([1 , 100] 4 ) ,without replacement and requiring that m (cid:54) =
Revised: Sample { ( m, n, r, d ) t } num - tasks τ = from the uniform distribution over integers, unif ([1 , 100] 4 ) , without replacement and requiring that m (cid:54) =

Original: n (cid:54) =
Revised: n (cid:54) =

Original: r (cid:54) =
Revised: r (cid:54) =

Original: d . 
Revised: d . 3.

Original: Deﬁne the set { ( index , m, n, r, d ) index ) } for in- dex and ( m, n, r, d ) drawn from their respective sets, without replacement, in an arbitrary order.
Revised: Deﬁne the set { ( index , m, n, r, d ) index ) } for index and ( m, n, r, d ) drawn from their respective sets, without replacement, in an arbitrary order.

Original: Compute the number of points per index , n task = sample-size // num-tasks .
Revised: Compute the number of points per index , n task = sample-size // num-tasks .

Original: For each index ∈ { index t } num-tasks τ =
Revised: For each index ∈ { index t } num-tasks τ =

Original: (a) Sample a vector of length n task , balanced between 1 s and 2 s, that gives the values of { indicator p } Pp =1 for the P points with that index .
Revised: (a) Sample a vector of length n task , balanced between 1 s and 2 s, that gives the values of { indicator p } Pp =1 for the P points with that index .

Original: (b) Sample a vector of length n task , balanced between 0s and 1s, representing whether the features 1 [ # m> # n ] and 1 [ # r> # d ] should correlate (1 implies 
Revised: (b) Sample a vector of length n task , balanced between 0s and 1s, representing whether the features 1 [ # m> # n ] and 1 [ # r> # d ] should correlate (1 implies they are equal, and 0 unequal).

Original: they are equal,

Original: and 0 unequal

Original: This balance changes when the strong-weak correlation is intended to change.
Revised: This balance changes when the strong-weak correlation is intended to change.

Original: (c) Sample a vector of length n task , balanced between 0 s and 1 s, representing whether ( m, n ) or ( r, d ) should be the more numer- ous integers in the sequence (so that there 
Revised: (c) Sample a vector of length n task , balanced between 0 s and 1 s, representing whether ( m, n ) or ( r, d ) should be the more numerous integers in the sequence (so that there is no bias, even randomly, between features by size).

Original: is no bias, even randomly, between features

Original: by size).

Original: (d) For i ∈ 1 : n task :

Original: i. Place the index in the ﬁrst element of an empty array, and the indicator in the second.
Revised: i. Place the index in the ﬁrst element of an empty array, and the indicator in the second.

Original: ii.
Revised: ii.

Original: Based on the i th elements of the three vectors described above, allocate sam- ples of the integers in ( m, n, r, d ) index 
Revised: Based on the i th elements of the three vectors described above, allocate samples of the integers in ( m, n, r, d ) index into the remaining 18 slots.

Original: into the remaining 18 slots.

Original: iii.
Revised: iii.

Original: If there are any remaining slots af- ter these integers are randomly allo- cated, ﬁll them with i.i
Revised: If there are any remaining slots after these integers are randomly allocated, ﬁll them with i.i.d.

Original: d. samples from unif (1 , 100) .
Revised: samples from unif (1 , 100) .