SlowGuess/ACL_2020_OCR huggingface dataset:
Further, we provide a strong baseline for post-hoc generators using this model, where using the model's predictions, we simply pick the corresponding label-specific generated explanation.
$$
t _ {g} ^ {\prime} = G _ {\mathrm {p o s t}} (l _ {x}) = t _ {x}
$$
We note that the model's predictions have no sensitivity to the generated explanations in NILE: posthoc.
ExplainThenPredictAttention (ETPA): Following (Camburu et al., 2018), (see Figure 2B), we train a pipelined system, where we first learn to generate the gold explanation  $t_{g}^{\prime}$ , followed by a classification of  $t_{g}^{\prime}$  to predict the label:
$$
t _ {g} ^ {\prime} = G _ {\text {p r e}} \left(\text {c o n c a t} _ {e c n}\right)
$$
$$
l _ {x} = W _ {x} F _ {\text {p o s t}} \left(t _ {g} ^ {\prime}\right)
$$
8735
Model	SNLI Dev	SNLI Test	Explanation evaluation on first 100 SNLI Test Samples
Label Accuracy	Label Accuracy	A: Correct Labels	Averaged over annotators	Annotators in-agreement
B: Correct Expl.	B/A	C: Correct Expl.	C/A
SemBERT# (Zhang et al., 2019)	92.2	91.9	-	-	-	-	-
ETPA (Camburu et al., 2018)	Reported	81.71	-	-	64.27	-	-
Reproduced	86.98	86.22	77	71.2	92.47	59	76.62
NILE:post-hoc	91.86	91.49	90	81.4	90.44	68	75.56
NILE-PH	Independent	84.69	84.13	78	72.0	92.31	61	78.21
Aggregate	85.71	85.29	80	73.4	91.75	62	77.50
Append	88.49	88.11	85	78.0	91.76	66	77.65
NILE-NS	Independent	91.56	90.91	88	80.8	91.82	69	78.41
Aggregate	91.55	91.08	89	80.6	90.56	68	76.40
Append	91.74	91.12	89	80.4	90.34	67	75.28
NILE	Independent	91.29	90.73	91	82.4	90.55	69	75.82
Aggregate	91.19	90.91	90	81.4	90.44	68	75.56
Table 1: Comparison of label and explanation accuracy on the in-domain SNLI evaluation sets. Models are selected using the Dev set label accuracy over 5 runs with different seeds of random initialization. Mean (and standard deviation) over the 5 runs are reported in the Appendix. # indicates the best reported result at https://nlp.stanford.edu/projects/snli/ at the time of writing. Note that SemBERT does not provide natural language explanations and is reported here only for reference. Bold numbers indicate highest among methods that produce explanations. Explanations are evaluated on the first 100 SNLI Test examples. We present reported numbers of ETPA (Camburu et al., 2018) as well as the results with our reproduction of ETPA. ETPA (reproduced) is directly comparable with NILE (Section 4.5). NILE-PH competes with or outperforms ETPA baselines on label accuracy, while NILE-NS and NILE provide significant gains in label accuracy. NILE and NILE-NS are competitive with the best reported results in terms of label accuracies. We report the number of correct explanations, averaged across annotators (B) as well as when all annotators agree on correctness (C). All NILE variants are able to provide more correct explanations than the ETPA baseline. We also report the percentage of correct explanations in the subset of correct label predictions (B/A, C/A). On this metric, NILE variants are comparable with the ETPA baseline. However, the real value of NILE lies in being able to probe the faithfulness of its decisions (Section 5.3). Further, NILE explanations generalize significantly better on out-of-domain examples (See Table 2). Please see Section 5.1 for details.
Q1 How does NILE compare with the baselines and other existing approaches in terms of final task performance, and explanation accuracy, on in-domain evaluation sets (train and test on SNLI)? (Section 5.1)
Q2 How well does NILE transfer to out-of-domain examples (train on SNLI, and test on MNLI)? (Section 5.2)
Q3 How faithful are the model's predictions to the generated explanations? (Section 5.3)
We provide training details in Appendix A, and examples of generated label-specific explanations in Appendix B.
5.1 In-domain Results
We report the label accuracies of the baselines and proposed architectures on the SNLI Dev and Test set in Table 1. We also report explanation accuracies, obtained through human evaluation of the generated explanations in the first 100 test examples. Binary scores on correctness were sought from five annotators (non-experts in NLP) on the generated explanations. For both label and explanation accuracies, we report using a model selected using the SNLI Dev set label accuracy across 5 runs with 5 different seeds of random initialization. Please see the Appendix for more details on the 5 runs. First, through NILE:post-hoc, we provide a strong baseline for obtaining high label and explanation accuracy. Our aim in this work is to learn explanations that serve as the reason for the model's
8736
Model	MNLI Dev	MNLI Dev-mm	Explanation evaluation on first 100 MNLI Dev Samples
Label Accuracy	Label Accuracy	A: Correct Labels	Averaged over annotators	Annotators in-agreement
B: Correct Expl.	B/A	C: Correct Expl.	C/A
ETPA (Camburu et al., 2018)	Reproduced	56.11	56.42	48	22.67	47.22	14	29.17
NILE:post-hoc	79.29	79.29	69	47.67	69.08	35	50.72
NILE-PH	Independent	54.95	55.35	46	34.33	74.64	28	60.87
Aggregate	56.45	56.66	49	34.67	70.75	26	53.06
Append	61.33	61.98	58	43.33	74.71	34	58.62
NILE-NS	Independent	74.84	75.20	68	49.67	73.04	37	54.41
Aggregate	75.73	76.22	69	49.33	71.50	37	53.62
Append	77.07	77.22	72	52.33	72.69	38	52.78
NILE	Independent	72.91	73.04	64	45.67	71.35	33	51.56
Aggregate	72.94	73.01	63	45.67	72.49	34	53.97
Table 2: Testing the generalization capability of NILE on the out-of-domain MNLI Dev sets. Training and model selection is done on the SNLI dataset (Section 5.1), and evaluation on the out-of-domain MNLI Dev (matched) and MNLI Dev-mm (mismatched) sets. Label accuracies are reported for both MNLI Dev (matched) and MNLI Dev-mm (mismatched) sets, while explanations are evaluated on the first 100 MNLI Dev set examples. We report the number of correct explanations, averaged across annotators (B) as well as when all annotators agree on correctness (C). All NILE variants provide more correct explanations than the ETPA baseline (B, C). Further, the percentage of correct explanations in the subset of correct label predictions (B/A, C/A) is significantly better for all NILE variants. The results demonstrate that NILE provides a more generalizable framework for producing natural language explanations. Please see Section 5.2 for details.
predictions. Nevertheless, we are able to compete or outperform this baseline, in terms of explanation accuracy, while incurring a only a small drop in label accuracy. All variants of NILE, including NILE-PH and NILE-NS (which is not trained using negative samples of explanations as described in Section 4.4.2), produce more correct explanations than the ETPA baseline. NILE-PH:Append, NILE and NILE-NS provide gains over label accuracies compared to the ETPA baseline. Additionally, NILE and its variants provide natural ways to probe the sensitivity of the system's predictions to the explanations, as demonstrated in the subsequent sections. Finally, the explanations generated by all NILE variants generalize significantly better on out-of-distribution examples when compared to the ETPA baseline (See Section 5.2).
5.2 Transfer to Out-of-domain NLI
To test the generalization capability of NILE, we do training and model selection on the SNLI dataset (Section 5.1), and evaluate on the out-of-domain MNLI (Williams et al., 2018) development sets. Transfer without fine-tuning to out-of-domain NLI has been a challenging task with transfer learning
for generating explanations in MNLI being particularly challenging (Camburu et al., 2018). We report label accuracies on the Dev (matched) and Dev-mm (mismatched) sets, and explanation evaluation on the first 100 Dev samples in Table 2. Explanation evaluation was done by three annotators (who also annotated the SNLI explanations). While the label accuracies follow a similar pattern as the in-domain SNLI Test set, all variants of NILE provide gains in the quality of generated explanations. All variants of NILE produce more correct explanations (B, C) as well as a higher percentage of correct generated explanations among correct predictions (B/A, C/A). This demonstrates that NILE, through intermediate label-specific natural language explanations, provides a more general way for building systems which can produce natural language explanations for their decisions.
5.3 Evaluating Faithfulness using Sensitivity Analysis
NILE and its variants allow a natural way to probe the sensitivity of their predictions to the generated explanations, which is by perturbing the explanations themselves. In this way, NILE resembles
8737
Model	I+ Exp	I only	Exp only
NILE-NS	Independent	91.6	33.8	69.4
Aggregate	91.6	33.8	74.5
Append	91.7	91.2	72.9
NILE	Independent	91.3	33.8	46.1
Aggregate	91.2	33.8	40.7
Table 3: Estimating the sensitivity of the system's predictions to input explanations through erasure. During testing, we erase either the instance or the explanations from the input to NILE-NS and NILE. The results seem to indicate that NILE-NS's predictions are more faithful, in the sense of having a higher sufficiency. However, as demonstrated subsequently, the sensitivity of NILE-NS's prediction to the input explanations is not as desired. Please see Section 5.3 for details.
Model	Dev Set	Shuffled Dev Set
NILE-NS	Independent	91.6	88.1
Aggregate	91.6	89.6
Append	91.7	88.5
NILE	Independent	91.3	35.3
Aggregate	91.2	31.6
Table 4: Probing the sensitivity of the system's predictions by shuffling instance-explanation pairs. Each instance is attached to a randomly selected explanation of the same form as the original pair. The results demonstrate a much weaker link between NILE-NS's predictions and associated explanations. On the other hand, NILE behaves more expectedly. Note that the baselines don't allow a similar mechanism to test their faithfulness, and such testability is a key advantage of NILE. Please see Section 5.3 for details.
explanation systems which provide input text fragments as reasons for their decisions. DeYoung et al. (2019) propose metrics to evaluate the faithfulness of such explanations. Following their work, we first attempt to measure the explanations generated by the methods proposed in this paper for comprehensiveness (what happens when we remove the explanation from the input) and sufficiency (what happens if we keep only the explanations). In Table 3, we show these measures for NILE and NILE-NS. The results seem to indicate that explanations for both NILE and NILE-NS are comprehensive, while having higher sufficiency in the case of NILE-NS. We first note that the comprehensiveness of these systems is ensured by design, and the input is indistinguishable without an explanation. Second, we argue that sufficiency may indicate correlations which don't necessarily exist
in the system otherwise. We study the sensitivity of the explanations through a probe motivated by an understanding of the task and the training examples (see Section 4.4.2). We perturb the instance-explanation inputs such that for each test instance, the explanation is replaced by a randomly selected explanation of the same label. The results (Table 4) indicate that NILE-NS is more robust to random perturbations of input explanations, and presumably uses the form of the explanation to infer the task (see Section 4.4.2 for a discussion). It is true that NILE behaves expectedly as we have specifically designed NILE to prevent the associated bias, and that this could potentially lead the system to learn other such biases. However, a key advantage of the proposed architecture is the ability to identify and fix for such biases. We leave it as an interesting and challenging future work to find and fix more such biases.
In this paper we propose NILE, a system for Natural Language Inference (NLI) capable of generating labels along with natural language explanations for the predicted labels. Through extensive experiments, we demonstrate the effectiveness of this approach, in terms of both label and explanation accuracy. NILE supports the hypothesis that accurate systems can produce testable natural language explanations of their decisions. In the paper, we also argue the importance of explicit evaluation of faithfulness of the generated explanations, i.e., how correlated are the explanations to the model's decision making. We evaluate faithfulness of NILE's explanations using sensitivity analysis. Finally, we demonstrate that task-specific probes are necessary to measure such sensitivity.
We thank the anonymous reviewers for their constructive comments. This work is supported by the Ministry of Human Resource Development (Government of India). We would also like to thank HuggingFace for providing a state-of-the-art Transformers library for natural language understanding. Finally, we want to thank the annotators who annotated generated explanations for correctness.
Jacob Andreas, Dan Klein, and Sergey Levine. 2018. Learning with latent language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2166-2179, New Orleans, Louisiana. Association for Computational Linguistics.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Computational Linguistics.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural Language Inference with Natural Language Explanations. In Advances in Neural Information Processing Systems, pages 9539-9549.
Supriyo Chakraborty, Richard Tomsett, Ramya Raghavendra, Daniel Harborne, Moustafa Alzantot, Federico Cerutti, Mani Srivastava, Alun Preece, Simon Julier, Raghuveer M Rao, et al. 2017. Interpretability of Deep Learning Models: A Survey of Results. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), pages 1-6. IEEE.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. 2018. Learning to Explain: An Information-Theoretic Perspective on Model Interpretation. In International Conference on Machine Learning, pages 882-891.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2019. ERASER: A Benchmark to Evaluate Rationalized NLP Models. arXiv preprint arXiv:1911.03429.
Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. 2018. Pathologies of Neural Models Make Interpretations Difficult. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3719–3728, Brussels, Belgium. Association for Computational Linguistics.
Amirata Ghorbani, Abubakar Abid, and James Zou. 2019. Interpretation of Neural Networks is Fragile. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3681-3688.
Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, and Zeynep Akata. 2018. Generating counterfactual explanations with natural language. In ICML Workshop on Human Interpretability in Machine Learning, pages 95-98.
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. 2018. Multimodal Explanations: Justifying Decisions and Pointing to the Evidence. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8779-8788.
Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. 2018. Textual Explanations for Self-Driving Vehicles. In Proceedings of the European conference on computer vision (ECCV), pages 563-578.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. BART: Denoising Sequence-to-Sequence Pretraining for Natural Language Generation, Translation, and Comprehension. arXiv preprint arXiv:1910.13461.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunson. 2017. Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158-167, Vancouver, Canada. Association for Computational Linguistics.
Zachary C Lipton. 2016. The Myth of Model Interpretability. arXiv preprint arXiv:1606.03490.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692.
Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems, pages 4765-4774.
Camburu Oana-Maria, Shillingford Brendan, Minervini Pasquale, Lukasiewicz Thomas, and Blunsom Phil. 2019a. Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations. arXiv preprint arXiv:1910.03065.
8739
Camburu Oana-Maria, Giunchiglia Eleonora, Foerster Jakob, Lukasiewicz Thomas, and Blunsom Phil. 2019b. Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods. arXiv preprint arXiv:1910.02065.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog, 1(8).
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. arXiv preprint arXiv:1910.10683.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144. ACM.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in neural information processing systems, pages 5998-6008.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems. In Advances in Neural Information Processing Systems, pages 3261-3275.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium. Association for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace's Transformers: State-of-the-art Natural Language Processing. ArXiv, abs/1910.03771.
Jialin Wu, Zeyuan Hu, and Raymond Mooney. 2019. Generating question relevant captions to aid visual question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3585-3594, Florence, Italy. Association for Computational Linguistics.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Advances in neural information processing systems, pages 5754-5764.
Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, and Xiang Zhou. 2019. Semantics-aware BERT for Language Understanding. arXiv preprint arXiv:1909.02209.
Model	SNLI Dev
Label Accuracy
Mean	Stddev
ETPA	Reproduced	86.96	0.02
NILE:post-hoc	91.77	0.06
NILE-PH	Independent	84.53	0.18
Aggregate	85.47	0.26
Append	88.30	0.12
NILE-NS	Independent	90.17	2.76
Aggregate	91.44	0.06
Append	91.57	0.14
NILE	Independent	91.09	0.19
Aggregate	90.94	0.22
Table 5: Mean and Standard Deviation for label accuracies on SNLI Dev set are reported. NILENS:Independent system has a high standard deviation and relatively lower mean accuracy. This is due to a bad random initialization with seed 219. When seed 219 results are excluded, the mean and standard deviation are 91.41 and 0.20 respectively.
For fine-tuning gpt2-medium language models for explanation generation as well as roberta-base models, we leverage code and pre-trained models from the "transformers" library available at https://github.com/huggingface. In each case we train on the train split for three epochs. Apart from batch size, sequence length and seed for random initialization, we keep the other hyperparameters fixed throughout the experiments. We don't do any fine-tuning on seeds of random initialization.
8740
For roberta-base models, we report results through model selection on models trained using 5 seeds of random initialization - 42, 219, 291, 67 and 741. Model selection is done using label accuracies on SNLI Dev set. In Table 5, we report the mean and standard deviation for the label accuracies across 5 runs.
We ran our experiments on GeForce GTX 1080 Ti GPUs. We adjust the batch size to be the largest multiple of 16 to fit on the GPU memory ( $\sim$ 12GB). We now list all the hyper-parameters used.
GPT2: The hyper-parameters used for finetuning gpt2-medium include a maximum sequence length of 128, batch size of 2, learning rate of 5e-5, Adam epsilon of 1e-8, max gradient norm of 1.0 and a seed of 42. For generating text, we used greedy decoding.
Model	Batch size	Max seq length
ETPA	Reproduced	32	100
NILE:post-hoc	32	100
NILE-PH	Independent	32	50
Aggregate	32	50
Append	32	100
NILE-NS	Independent	16	100
Aggregate	16	100
Append	16	200
NILE	Independent	16	100
Aggregate	16	100
Table 6: Hyper-parameters (batch size and maximum sequence length) used for fine-tuning roberta-base
RoBERTa: The fixed parameters for fine-tuning robera-base included a learning rate of 2e-5, Adam epsilon of 1e-8 and max gradient norm 1.0. Other parameters are captured in Table 6.
B Generated Explanations
In this section, we list the label-specific explanations generated for five Dev set examples, along with the premise, hypothesis and gold labels, for both SNLI (Section B.1) and MNLI (Section B.2) datasets.
B.1 SNLI
PairID: 4705552913.jpg#2r1n
gold label: neutral
Premise: Two women are embracing while
holding to go packages.
Hypothesis: The sisters are hugging goodbye while holding to go packages after just eating lunch.
entailment: Sisters are women.
contradiction: The women cannot be embracing if they are hugging goodbye.
neutral: Two women embracing does not imply they are sisters hugging goodbye.
PairID: 2407214681.jpg#0r1e
gold label: entailment