SlowGuess/ACL_2020_OCR huggingface dataset:
Sawan Kumar
sawankumar@iisc.ac.in
Partha Talukdar
ppt@iisc.ac.in
The recent growth in the popularity and success of deep learning models on NLP classification tasks has accompanied the need for generating some form of natural language explanation of the predicted labels. Such generated natural language (NL) explanations are expected to be faithful, i.e., they should correlate well with the model's internal decision making. In this work, we focus on the task of natural language inference (NLI) and address the following question: can we build NL systems which produce labels with high accuracy, while also generating faithful explanations of its decisions? We propose Natural-language Inference over Label-specific Explanations (NILE), a novel NLI method which utilizes auto-generated label-specific NL explanations to produce labels along with its faithful explanation. We demonstrate NILE's effectiveness over previously reported methods through automated and human evaluation of the produced labels and explanations. Our evaluation of NILE also supports the claim that accurate systems capable of providing testable explanations of their decisions can be designed. We discuss the faithfulness of NILE's explanations in terms of sensitivity of the decisions to the corresponding explanations. We argue that explicit evaluation of faithfulness, in addition to label and explanation accuracy, is an important step in evaluating model's explanations. Further, we demonstrate that task-specific probes are necessary to establish such sensitivity.
Deep learning methods have been employed to improve performance on several benchmark classification tasks in NLP (Wang et al., 2018, 2019). Typically, these models aim at improving label accuracy, while it is often desirable to also produce explanations for these decisions (Lipton, 2016;
Chakraborty et al., 2017). In this work, we focus on producing natural language explanations for Natural Language Inference (NLI), without sacrificing much on label accuracy.
There has been growing interest in producing natural language explanations for deep learning systems (Huk Park et al., 2018; Kim et al., 2018; Ling et al., 2017), including NLI (Camburu et al., 2018). In general, the explanations from these methods can typically be categorized as post-hoc explanations (Lipton, 2016). Camburu et al. (2018) propose an NLI system which first produces an explanation and then processes the explanation to produce the final label. We argue that these explanations also resemble post-hoc explanations (Section 4.2). Further, existing methods don't provide a natural way to test the faithfulness of the generated explanations, i.e., how well do the provided explanations correlate with the model's decision making.
We therefore propose Natural-language Inference over Label-specific Explanations (NILE)<sup>1</sup>, which we train and evaluate on English language examples. Through NILE, we aim to answer the following question:
Can we build NLI systems which produce faithful natural language explanations of predicted labels, while maintaining high accuracy?
Briefly, in NILE, we first generate natural language explanations for each possible decision, and subsequently process these explanations to produce the final decision. We argue that such a system provides a natural way of explaining its decisions. The key advantage is the testability of these explanations, in themselves, as well as in terms of the sensitivity of the system's prediction
<sup>1</sup>NILE source code available at https://github.com/SawanKumar28/nile
8730
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8730-8742 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
Figure 1: Overview of NILE: A Premise and Hypothesis pair is input to label-specific Candidate Explanation Generators  $G$  which generate natural language explanations supporting the corresponding label. The generated explanations are then fed to the Explanation Processor  $S$ , which generates label scores using the evidence present in these explanations (see Figure 3 for the architectures used in this work). In addition to the explanations, NILE also utilizes the premise and hypothesis pair (See Section 4.4.2 for a discussion on the challenges in building such a system). Please see Section 4 for details.
to these explanations.
We choose NLI due to its importance as an NLP task, and the availability of e-SNLI, a large dataset annotated both with entailment relation labels and natural language human explanations of those labels (Camburu et al., 2018; Bowman et al., 2015).
In summary, we make the following contributions in this work.
1. We propose NILE, an NLI system which generates and processes label-specific explanations to infer the task label, naturally providing explanations for its decisions.
2. We demonstrate the effectiveness of NILE compared to existing systems, in terms of label and explanation accuracy.
3. Through NILE, we provide a framework for generating falsifiable explanations. We propose ways to evaluate and improve the faithfulness of the system's predictions to the generated explanations. We claim that task-specific probes of sensitivity are crucial for such evaluation.
We have released the source code of NILE to aid reproducibility of the results.
Explainability of a model's predictions has been studied from different perspectives, including feature importance based explanations (Ribeiro et al., 2016; Lundberg and Lee, 2017; Chen et al., 2018), or post-hoc natural language explanations (Huk Park et al., 2018; Kim et al., 2018; Ling et al.,
2017). Hendricks et al. (2018) produce counterfactual natural language explanations for image classification given an image and a counter-class label. Camburu et al. (2018) propose a model for NLI to first generate a free-form natural language explanation and then infer the label from the explanation. However, as noted by Oana-Maria et al. (2019a), the system tends to generate inconsistent explanations. We reason that requiring a model to generate an explanation of the correct output requires it to first infer the output, and the system thus resembles post-hoc explanation generation methods.
Given the diversity of desiderata and techniques for interpretability, the need for understanding interpretation methods and evaluating them has grown. Difficulty in building interpretation models and the lack of robustness of the same are some of the major issues in existing deep neural networks systems (Feng et al., 2018; Ghorbani et al., 2019; Oana-Maria et al., 2019b). Given these observations, measuring faithfulness, i.e., how well do the provided explanations correlate with the model's decision making, is crucial. DeYoung et al. (2019) propose metrics to evaluate such faithfulness of rationales (supporting evidence) for NLP tasks.
Through NILE, we propose a framework for generating faithful natural language explanations by requiring the model to condition on generated natural language explanations. The idea of using natural language strings as a latent space has been explored to capture compositional task structure (Andreas et al., 2018). Wu et al. (2019) explore improving
8731
visual question answering by learning to generate question-relevant captions. Rajani et al. (2019) aim to improve commonsense question answering by first generating commonsense explanations for multiple-choice questions, where the question and the choices are provided as the prompt. Similar to (Camburu et al., 2018), they learn by trying to generate human-provided explanations and subsequently conditioning on the generated explanation. In NILE, we instead aim to produce an explanation for each possible label and subsequently condition on the generated label-specific explanations to produce the final decision.
In this section, we discuss the datasets (Section 3.1) and pre-trained models (Section 3.2) used to build NILE.
SNLI: The Stanford NLI dataset (Bowman et al., 2015) contains samples of premise and hypothesis pairs with human annotations, using Amazon Mechanical Turk. The premises were obtained from pre-existing crowdsourced corpus of image captions. The hypotheses were obtained by presenting workers with a premise and asking for a hypothesis for each label (entailment, neutral and contradiction), resulting in a balanced set of  $\sim 570\mathrm{K}$  pairs.
e-SNLI: Camburu et al. (2018) extend the SNLI dataset with natural language explanations of the ground truth labels. The explanations were crowdsourced using Amazon Mechanical Turk. Annotators were first asked to highlight words in the premise and hypothesis pairs which could explain the labels. Next, they were asked to write a natural language explanation using the highlighted words.
Similar to Camburu et al. (2018), for all our experiments, we filter out non-informative examples where the explanations contain the entire text of the premise or hypothesis. In particular, we drop any training example where the uncased premise or hypothesis text appears entirely in the uncased explanation. This leads to a training data size of  $\sim 532\mathrm{K}$  examples.
3.2 Pretrained Language Models
Transformer architectures (Vaswani et al., 2017) pre-trained on large corpora with self-supervision have shown significant improvements on various NLP benchmarks (Devlin et al., 2019; Radford
et al., 2019; Yang et al., 2019; Liu et al., 2019; Lan et al., 2019). Improvements have been demonstrated for text classification as well as text generation tasks (Lewis et al., 2019; Raffel et al., 2019). In this work, we leverage the implementation of transformer architectures and pre-trained models provided by Wolf et al. (2019).
GPT-2: We use the GPT-2 architecture (Radford et al., 2019), which is trained using a causal language modeling loss (CLM), and includes a left-to-right decoder suitable for text generation. In particular, we use the gpt2-medium model. This model has 24 layers, 16 attention heads and a hidden size of 1024 ( $\sim$ 345M parameters). For text generation, the model can be finetuned using CLM on desired text sequences.
RoBERTa: For classification modules, we leverage RoBERTa (Liu et al., 2019), which is trained using a masked language modeling loss (MLM). In particular, we use the roberta-base model. This model has 12 layers, 12 attention heads and a hidden size of 768 ( $\sim$ 125M parameters). For downstream classifications tasks, a classification layer is added over the hidden-state of the first token in the last layer.
4 Natural-language Inference over Label-specific Explanations (NILE)
The overall architecture employed in NILE is shown in Figure 1. We introduce the notation used in this paper in Section 4.1. We then discuss the motivation for the major design choices in Section 4.2.
NILE performs the following steps to produce labels and explanations:
1. Candidate Explanation Generators: Label-specific Candidate Explanation Generators first generate explanations supporting the respective labels (Section 4.3).
2. Explanation Processor: The Explanation Processor takes the explanations and also the premise and hypothesis pairs as input to produce the task label (Section 4.4). We also build NILE-PH, where the Explanation Processor has access only to the generated explanations (Section 4.4.1).
We note that NILE-PH more naturally fits the desiderata described in Section 1, while we design and evaluate NILE for the more general case
8732
where the Explanation Processor also accesses the premise and hypothesis pair.
In Section 4.5, we describe comparable baseline architectures.
4.1 Notation
We denote each data point by  $(p,h)$ , where  $p$  is the premise and  $h$  the hypothesis sentence.  $G$  denotes a model trained to generate natural language explanations. Specifically,  $G_{x}$  denotes a model which generates natural language explanations  $t_x$  of type  $x$ , where  $x \in \{\text{entail, contradict, neutral}\}$ . We denote the human-provided gold explanation for the correct predictions as  $t_g$ .  $S$  denotes a module which predicts label scores. The true label for an example is denoted by  $y$ , while a model prediction is denoted by  $y'$ , and label scores by  $l_x$ .
Figure 2: Existing alternative architectures.: A. Posthoc generation: Given an input instance, first the label is predicted and then an explanation generated conditioned on the label and the input text. B. ExplainThen-Predict (Camburu et al., 2018): Given the input instance, first the desired explanation is generated, and then the label is predicted using only the generated explanation. We argue that neither architecture provides a natural way to test the sensitivity of the model's predictions to the generated explanation. Please see Section 4.2 for details.
4.2 Why do it this way?
In this section, we describe the motivation for adopting a two-step pipelined approach.
Label-specific explanations: Consider two alternative existing architectures in Figure 2. In Figure 2A, a model  $S_{\mathrm{pre}}$  is trained directly on the example sentences  $(p \& h)$  to produce a label  $(y')$ , which together with the example sentences are used to produce an explanation  $t_g'$  using  $G_{\mathrm{post}}$ . It can be argued that while the target explanations may regularize the system, there is no reason for  $t_g'$  to be aligned with the reason why the model chose a particular label.
Figure 2B corresponds to a model which has also been trained on e-SNLI (Camburu et al., 2018).
$G_{\mathrm{pre}}$  is first trained to produce natural language explanations  $t_g^\prime$  using human-provided explanations  $(t_{g})$  as targets, using only the example sentences as inputs. A model  $S_{\mathrm{post}}$  then chooses the label corresponding to the generated explanation  $t_g^\prime$ . While at first, it appears that this system may provide faithful explanations of its decisions, i.e., the generated explanations are the reason for the label prediction, we argue that it may not be so.
In Figure 2B,  $G_{\mathrm{pre}}$  is required to generate the explanation of the correct label for an example. It must first infer that label and then produce the corresponding explanation. Further analysis of the free-form human-provided explanations has revealed clear differences in the form of explanations, through alignment to label-specific templates (Camburu et al., 2018; Oana-Maria et al., 2019a). The Explanation Processor  $S_{\mathrm{post}}$  then only needs to infer the form of  $t_g'$ .  $G_{\mathrm{pre}}$  then resembles post-hoc generation methods, with the label (as the form of  $t_g'$ ) and explanation  $t_g'$  being produced jointly. The claim is supported by inconsistencies found in the generated explanations (Oana-Maria et al., 2019a).
Neither architecture allows a natural way to test the sensitivity of the model's predictions to its explanations. In NILE, we first allow explanations for each label, and then require the Explanation Processor to select the correct explanation. This allows us to naturally test whether the model's predictions are indeed due to the selected explanation. This can be done, for example, by perturbing the input to the Explanation Processor.
A pipelined approach: We use a pipelined approach in NILE (Figure 1). The Candidate Explanation Generators are first trained using human-provided explanations. The Explanation Processor takes as input the generated label-specific explanations. This prevents the system from producing degenerate explanations to aid task performance. It also allows perturbing the generated explanations to probe the system in a more natural way compared to an unintelligible intermediate state of a learnt model. We believe that systems can be designed to work in this setting without compromising task performance.
4.3 Candidate Explanation Generators
We train label-specific explanation generators,  $G_{x}$ ,  $x \in \{\text{entail, contradict, neutral}\}$ , using human-provided explanations of examples with the corresponding label. For example, to train  $G_{\text{entail}}$ , we
8733
Figure 3: Explanation Processor architectures. A. Independent (Ind) collects evidence for a label symmetrically from the corresponding explanation. B. Aggregate (Agg) allows handling missing explanations by looking for contradictory evidence. C. Append (Apn) allows arbitrary evidence collection for each label. Please see Section 4.4.1 for details. Premise and hypothesis sentences are processed by additionally providing them to each block  $F_{z}$  where  $z \in \{\mathrm{Ind}, \mathrm{Agg}, \mathrm{Apn}\}$ . Please see Section 4.4.2 for details.
collect all triplets  $(p, h, t_g)$  annotated as entailment. We create text sequences of the form: "Premise:  $p$  Hypothesis:  $h$  [EXP]  $t_g$  [EOS]" to fine-tune a pretrained language model, where [EXP] and [EOS] are special tokens added to the vocabulary. During fine-tuning, the language modeling loss function is used only over the explanation tokens.
Next, we create prompts of the form "Premise:  $p$  Hypothesis:  $h$  [EXP]" and require each trained language model to independently complete the sequence. In this way we obtain label specific explanations  $t_x$ ,  $t_x = G_x(p, h)$ , for  $x \in \{\text{entail, contradict, neutral}\}$ .
4.4 Explanation Processor
The Explanation Processor in NILE takes as input the generated label-specific explanations, as well as the premise and hypothesis pair to generate label scores  $l_{x}$ ,  $x \in \{\text{entail, contradict, neutral}\}$ . During training, these scores are passed through a softmax layer and a cross-entropy loss is used to generate the training signal. During testing, the label with the maximum score is selected.
We leverage a pre-trained roberta-base model for all our experiments, and fine-tune it as specified in the following subsections. In each case, any intermediate scores are generated through transformations of the first token ([CLS]) embedding from the last layer. We define:
$$
F _ {\mathrm {m o d e l}} (\mathrm {i n p}) = \tanh (W. \mathrm {C L S} _ {\mathrm {e m b e d}} (\mathrm {i n p}))
$$
where inp is a pair of sequences in NILE, a single sequence in NILE-PH, and  $W$  are the learnable parameters for the model.
For simplicity, and to elucidate the desired behavior, we first describe how explanations are processed in NILE-PH (Section 4.4.1). We then dis
cuss the construction of NILE, a potential issue, and a fix for the same (Section 4.4.2).
4.4.1 Processing Explanations
In this section, we describe how explanations are processed in NILE-PH, which is generalized in NILE (Section 4.4.2). We experiment with three architectures, described below (also see Figure 3).
A. Independent: In the Independent model, explanations are fed to  $F_{\mathrm{Ind}}$ , which generates a score for each explanations independently:
$$
l _ {x} = W _ {\text {I n d}} F _ {\text {I n d}} \left(t _ {x}\right) \tag {1}
$$
where  $x \in \{\text{entail, contradict, neutral}\}$ . We expect this score to represent the truthfulness of the input explanation.
B. Aggregate: The Independent model would need all three explanations to be available to reliably produce label scores. We believe a system should be able to handle one or more missing or ambiguous explanations. For example, the entailment explanation: "tentail: A dog is a cat" would provide evidence for contradiction. To capture this notion, we require the Explanation Processor to produce two intermediate scores  $V_{1}$  and  $V_{2}$ , where we expect  $V_{1}$  to collect evidence supporting an input claim and  $V_{2}$  to collect evidence against an input claim:
$$
V _ {i} (x) = W _ {\mathrm {A g g}, i} F _ {\mathrm {A g g}} \left(t _ {x}\right), \text {w h e r e} i \in \{1, 2 \} \tag {2}
$$
The intermediate score are then aggregated into the final label scores:
$$
l _ {\text {e n t a i l}} = \operatorname {C m b} \left(V _ {1} \left(t _ {\text {e n t a i l}}\right), V _ {2} \left(t _ {\text {c o n t r a d i c t}}\right)\right)
$$
$$
l _ {\text {c o n t r a d i c t}} = \operatorname {C m b} \left(V _ {1} \left(t _ {\text {c o n t r a d i c t}}\right), V _ {2} \left(t _ {\text {e n t a i l}}\right)\right) \tag {3}
$$
$$
l _ {\text {n e u t r a l}} = V _ {1} \left(t _ {\text {n e u t r a l}}\right)
$$
where Cmb is the LogSumExp function. The reason for this choice of aggregation is that while evidence against entailment might point to contradiction and vice versa, evidence against neutral doesn't necessarily provide any information about entailment or contradiction relations.
C. Append: Finally, to allow the model to reason arbitrarily between the three generated explanations, we created a single sequence,  $\text{concat}_{\text{ecn}}$ : "entailment:  $t_{\text{entail}}$  contradiction:  $t_{\text{contradict}}$  neutral:  $t_{\text{neutral}}$ ", and generate the scores as follows:
$$
l _ {x} = W _ {\mathrm {A p n}, x} F _ {\mathrm {A p n}} \left(\operatorname {c o n c a t} _ {e c n}\right) \tag {4}
$$
where  $x\in \{$  entail, contradict, neutral  $\}$
4.4.2 Processing Premise and Hypothesis
In NILE, to process premise  $p$  and hypothesis  $h$ , we first concatenate  $p$  and  $h$  into  $\mathrm{concat}_{ph}$ : "Premise:  $p$  Hypothesis:  $h$ ". The label scores are then obtained as in Section 4.4.1, by modifying Equation 1, 2 and 4 as follows: replace  $F_{z}(x)$  by  $F_{z}(\mathrm{concat}_{ph}, x)$ , where  $z \in \{\mathrm{Ind}, \mathrm{Agg}, \mathrm{Apn}\}$ . We note that appending the example sentences to the generated explanations (as in Append) would result in having no control over whether the explanations are used for the final prediction. The case for Independent and Aggregate is not immediately clear. We now discuss a potential issue with these architectures when processing premise and hypothesis text, and suggest a fix for the same.
The issue: We expect NILE to answer the question: Is  $(\mathrm{concat}_{ph}, t_x)$ , where  $x \in \{\text{entail, contradict, neutral}\}$ , a valid instance-explanation pair? The Independent and Aggregate architectures for NILE have been designed such that the model can't ignore the label-specific explanations. For example, the Independent model will produce identical scores for each output label, if it chooses to completely ignore the input explanations. However, the model is still free to learn a different kind of bias which is an outcome of the fact that natural language explanations convey ideas through both content and form. If the form for explanations of different labels is discriminative, an unconstrained learning algorithm could learn to infer first the type of explanation and use it to infer the task. For example, given the input  $(\mathrm{concat}_{ph}, t_x)$ , where  $x \in \{\text{entail, contradict, neutral}\}$ , if a model could learn whether  $t_x$  is an entailment explanation, it then only has to output whether  $\mathrm{concat}_{ph}$  corresponds
to an entailment relation. Essentially, high label accuracy can be achieved by inferring first what task to do using only the form of  $t_x$ .
The fix: To prevent NILE from exploiting the form of an explanation as described above, we create additional training examples, where we require NILE to score valid instance-explanation pairs higher. In particular, we sample negative explanations for an instance, of the same form as the correct label. For example, an instance labeled as entailment would have an additional training signal: Score (concatph,  $t_\mathrm{entail}$ ) higher than (concatph,  $t_\mathrm{entail}^\prime$ ) and (concatph,  $t_\mathrm{entail}^{\prime \prime}$ ), where  $t_\mathrm{entail}^\prime$  and  $t_\mathrm{entail}^{\prime \prime}$  are randomly sampled entailment form explanations.
We note that the fix leaves room for other kinds of biases to be learnt. However, the key advantage with NILE is that it is easy to design probes to test for such biases and subsequently fix them (see Section 5.3).
4.5 Baselines
We now describe baselines which use the same underlying blocks as NILE, for generating explanations and classification.
NILE:post-hoc: To understand the drop in performance which could be associated with constraining models as we have done, we train a model with full access to input examples (See Figure 2A).
$$
l _ {x} = W _ {x} F _ {\mathrm {p r e}} (p, h)
$$