SlowGuess/NAACL_2022_OCR huggingface dataset:
Task	Method	Leaves	Steps	Intermediates	Overall
F1	All-Cor.	F1	All-Cor.	F1	All-Cor.	All-Cor.
Gold	EntailmentWriter	98.7	86.2	50.5	37.7	67.6	50.3	34.4
IRGR	97.9	89.4	50.2	36.8	62.1	45.6	32.3
Gold+Dist.	EntailmentWriter	84.3	38.5	35.7	23.5	62.6	50.9	22.4
IRGR	69.9	23.8	30.5	22.3	47.7	56.5	22.0
Table 3: Entailment tree scores for baseline methods and IRGR, along four different dimensions (test set). The "Gold" and "Gold+Dist." tasks do not require retrieval and evaluates solely on the model's entailment tree generation capabilities.
Therefore, IRGR-retriever results should be considered an upper bound since the generator might not produce the desirable intermediate steps used for queries.
Table 1 shows the R@25 and All-Correct metrics results for different methods. Our premise retrieval module performs consistently better than baselines. For instance, the "IRGR-retriever (cond.)" outperforms the retriever from EntailmentWriter by  $14.2\%$  on R@25 and  $28.8\%$  on All-Correct metric. Note that "IRGR-retriever" may retrieve a variable number of premises (greater than 25), so we are not reporting R@25 for this method.
4.4.2 Entailment Tree Generation Results
We compare our method against EntailmentWriter baseline model on entailment tree generation. As shown in Table 2, our method outperforms the EntailmentWriter in all metrics. The overall tree structure better matches the golden tree, where the score for Overall All-Correct metric has an impressive increase of over  $300.0\%$ . Note that EntailmentWriter uses the T5-11B model, which has around 10 times more parameters than our model.
We also show the ablation results of combining different retrieval modules with our proposed generation module on Table 2. The "w/o iter." method does not iteratively retrieve premises, relying on one-shot retrieval at the beginning of the generation. As for the "w/o iter. & cond." method, the model does not use the conditioned retrieval, only relying on the trained dense retrieval with the hypothesis  $h$  as the query instead.
The work of Dalvi et al. (2021) defines two other simplified entailment tree generation tasks for further ablation studies. We report the results for what they define as "Task-1" and "Task-2", which are generation tasks where the golden premises are given as input, disregarding the retrieval component. Results in Table 2 report what they define
as "Task-3". For clarity, we rename "Task-1" and "Task-2" to "Gold" and "Gold+Dist.", respectively, and show the results in Table 3. In the "Gold" task, each context uses the golden leaves as input, while the "Gold+Dist." task uses the golden leaves plus some distractors (up to 25 distractors). When comparing models with the same number of parameters (we use their reported T5-large results), the generation results without retrieval are roughly the same as the EntailmentWriter method. This experiment shows that the iterative generation can create accurate explanations compared to a single pass generation when using golden retrieved premises.
4.5 Results Breakdown
We investigate how well the system performs relative to the number of steps in the gold tree. Figure 5 contains two graphs with results breakdown. The graph on the top shows the all-correct metric values for all three tasks (golden, golden + distractors, and retrieval). The bottom graph shows all F1 metrics (leaves, steps, and intermediates), but only for the "retrieval" task.
The results demonstrate that generating entailment trees becomes increasingly difficult as the size of the tree increases. The IRGR model cannot perfectly predict trees with more than four steps for any of the three different tasks. For the "retrieval" task (without the golden leaf sentences provided as input), the IRGR model cannot successfully generate trees with three or more steps. This could be explained by the fact that the all-correct metric is very strict, and missing or misplacing a single leaf sentence can result in an incorrect tree.
This downwards trend is also present in the "Break Down by Metrics" graph. Most noticeably, the "Intermediates (F1)" metric is especially challenging, having values close to zero for entailment trees with more than five steps. This metric is one
Figure 5: Result breakdown for number of steps in explanation (entailment steps).
of the main bottlenecks that lowers the value of the "Overall All-Correct" metric.
To understand the strengths and weaknesses of our model, we conduct further analysis of the output of the IRGR. When analyzing errors in the generation of entailment trees, we use the results on the development set for the task with distractors. We manually annotate 50 predicted trees that contain some error compared to the golden tree. We categorize the different types of errors, identifying both individual generated steps errors and entailment tree errors.
4.6.1 Retrieval Error Analysis
We use ENTAILMENT BANK's development set to automatically compute metrics that will give us some insights into the type of errors made by the IRGR-retriever module. We use "IRGR-retriever (cond.)" to fetch a set of 25 premises for each data point, where we identify the set of true positives (correctly retrieved premises) and the set of false negatives (missing premises).
To understand if the false negatives are more challenging to retrieve than the true positives, we compute the number of overlapping uni-grams and bi-grams between premises and hypotheses in these
two sets. We notice that true positives contain  $28.5\%$  more uni-gram overlap and  $68.6\%$  more bi-gram overlap to the hypothesis compared to the false negatives. These results suggest that premises lexically dissimilar to the hypothesis are, in theory, more challenging to retrieve.
We also investigate how the depth (number of edges in a path from the tree root) of a leaf node in the gold tree correlates to the errors of the IRGR-retriever module. We compute the average depth of true positive nodes as 2.3, while for false-negative nodes, the average depth is 3.0. These results strengthen the idea that leaf nodes deeper in the tree tend to be harder to retrieve, as depicted in Figure 4.
4.6.2 Entailment Step Error Analysis
The first error case is called invalid entailment steps (56% of errors), meaning that the conclusion of a step did not follow from the premises. For instance, in "kilogram is used to measure heavy objects"  $\wedge$  "an automobile is usually a heavy object"  $\Rightarrow$  "kilogram can be used to measure the mass of an automobile", the model assumes that "measure" is the same as "measure of mass", even though that is not explicitly stated.
The second error case accounts for misevaluation and irrelevance (27% of errors). It happens when the step is correct but does not match the golden tree, or when the step is correct but is not relevant or well placed in the final entailment tree. In the third error case, labeled repetition (17% of errors), the conclusion directly copied the premises, not creating a new sentence for the intermediate step.
4.6.3 Entailment Tree Error Analysis
When analyzing errors between the entire generated and golden trees, we noticed that incorrect or missing leaves (52% of errors) is the most common type of problem. For instance, when explaining the hypothesis "light year can be used to measure the distance between the stars in milky way" the premises "the milky way is a kind of galaxy" and "a galaxy is made of stars" are missing from the generated tree, making it impossible to explain the second part of the hypothesis.
The remaining errors are categorized as invalid or skipped steps (32% of errors), where the model commonly concludes an invalid conclusion from premises. This error often overlaps with missing leaves due to the fact that the model uses
fewer premises when it skips important intermediate steps; Imperfect evaluation (12% of errors), where the tree produced is valid, but does not match the golden tree; Disconnected or degenerate trees (4% of errors), where the generated output does not form a tree, or follows the desired output format.
As deep learning models become more ubiquitous in the natural language field, it is desirable that users can understand the model's answer by inspecting the reasoning chain from simple premises to the answer hypothesis. To generate rich, systematic explanations, we proposed a method that can iteratively generate and retrieve premises to produce entailment trees. We show how our approach has advantages over previous baselines, where the retrieved premises and generated explanations are more accurate.
In future work, we plan to improve the generation module by leveraging the structure of the entailment tree instead of relying purely on the encoder-decoder models. This idea could potentially fix the issues with "invalid entailment steps" and "repetition", which account for  $73\%$  of entailment step errors. We also plan to understand how explanations can be generated in the case of a false hypothesis, where we would expect the model to build a conclusion explaining why a statement is incorrect. It could help users verify false claims and understand the meaning behind their incorrectness.
We are thankful to Felicity M. Lu-Hill for proofreading this paper. The research leading to this paper was supported in part by the Machine Learning, Reasoning, and Intelligence Program of the Office of Naval Research.
James Allen. 1988. Natural language understanding. Benjamin-Cummings Publishing Co., Inc.
Tiberiu Boros, Stefan Daniel Dumitrescu, and Sonia Pipa. 2017. Fast and accurate decision trees for natural language processing tasks. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP 2017, pages 103-110, Varna, Bulgaria. INCOMA Ltd.
Kaj Bostrom, Xinyu Zhao, Swarat Chaudhuri, and Greg Durrett. 2021. Flexible generation of natural language deductions. In Proceedings of the 2021 Confer-
ence on Empirical Methods in Natural Language Processing, pages 6266-6278, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 9539-9549. Curran Associates, Inc.
Ruben Cartuyels, Graham Spinks, and Marie-Francine Moens. 2020. Autoregressive reasoning over chains of facts with transformers. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6916-6930, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. Explaining answers with entailment trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7358-7370, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A survey of the state of explainable AI for natural language processing. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 447-459, Suzhou, China. Association for Computational Linguistics.
Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443-4458, Online. Association for Computational Linguistics.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrievalaugmented language model pre-training. arXiv preprint arXiv:2002.08909.
Shuguang Han, Xuanhui Wang, Mike Bendersky, and Marc Najork. 2020. Learning-to-rank with BERT in tf-ranking. CoRR, abs/2004.08476.
Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543-3556, Stroudsburg, PA, USA. Association for Computational Linguistics.
Harsh Jhamtani and Peter Clark. 2020. Learning to explain: Datasets and models for identifying valid reasoning chains in multihop question-answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online. Association for Computational Linguistics.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems, volume 33, pages 9459-9474. Curran Associates, Inc.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Ro{bert}a: A robustly optimized {bert} pretraining approach.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019a. Explain yourself! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019b. Explain yourself! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. SentenceBERT: Sentence embeddings using Siamese BERTnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982-3992, Hong Kong, China. Association for Computational Linguistics.
Danilo Ribeiro, Thomas Hinrichs, Maxwell Crouse, Kenneth Forbus, Maria Chang, and Michael Witbrock. 2019. Predicting state changes in procedural text using analogical question answering. In 7th Annual Conference on Advances in Cognitive Systems.
Danilo Neves Ribeiro and Kenneth Forbus. 2021. Combining analogy with language models for knowledge extraction. In 3rd Conference on Automated Knowledge Base Construction.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881-7892, Online. Association for Computational Linguistics.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and TieYan Liu. 2020. Mpnet: Masked and permuted pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 33, pages 16857-16867. Curran Associates, Inc.
Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. 2021. ProofWriter: Generating implications, proofs, and abductive statements over natural language. In *Findings of the Association for Computational Linguistics: ACL-IJCNLP* 2021, pages 3621-3634, Online. Association for Computational Linguistics.
Marco Valentino, Mokanarangan Thayaparan, Deborah Ferreira, and André Freitas. 2021. Hybrid autoregressive inference for scalable multi-hop explanation regeneration. In 36th AAAI Conference on Artificial Intelligence.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, and Peter Jansen. 2020. WorldTree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 5456-5473, Marseille, France. European Language Resources Association.
Wenhan Xiong, Xiang Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Scott Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oguz. 2021. Answering complex open-domain questions with multi-hop dense retrieval. In International Conference on Learning Representations.
Chen Zhao, Chenyan Xiong, Jordan Boyd-Graber, and Hal Daumé III. 2021. Multi-step reasoning over unstructured text with beam dense retrieval. In *North American Association for Computational Linguistics*.
The IRGR-generator used the T5-large model from HuggingFace library. The best models were chosen according to the best "Overall All-Correct" metric on the validation set. During training, we used the following hyper parameters: learning rate:  $3 \cdot 10^{-5}$ , epochs: 15, training batch size: 4, validation batch size: 4, max number of input tokens: 512, max number of output tokens: 256, warm-up steps: 0, weight decay: 0.
The IRGR-retriever module uses the version all-mpnet-base-v2 from the SentenceTransformers library. During training, we used the following hyper parameters: learning rate:  $5 \cdot 10^{-5}$ , epochs: 10, training batch size: 32, validation batch size: 32, loss function: cosine similarity loss, warm-up steps: 0, weight decay: 0.
A.2 Entailment Tree Evaluation Metrics Details
The alignment algorithm between the nodes from gold and predicted entailment trees can be roughly described as follows:
1. For each intermediate node  $e_i \in \mathcal{E}$  and  $e_j \in \mathcal{E}^*$ , compute the set of leaf sentences in which the nodes are ancestors.
2. Align each intermediate node  $e_i$  to the first golden node  $e_j$  for which the Jaccard similarity of their respective set of associated leaf sentences is maximum. If any node  $e_i$  is associated with no gold nodes  $e_j$  (Jaccard similarity is zero), then  $e_i$  is aligned to a blank node (no conclusion).
Given the list of aligned nodes, the metrics "Intermediates (F1, All-Correct)" compute the similarity between two generated intermediate steps using BLEURT (Sellam et al., 2020), a learned evaluation metric based on BERT. We use the BLEURT-Large-512 model to compute textual similarity scores. The prediction between intermediate nodes is considered correct if  $BLEURT(e_i,e_j) > 0.28$
(this threshold was originally picked by Dalvi et al. (2021) using a subset of 300 manually labeled pairs).
A.3 Generator Input and Output Examples
The generation is done in multiple steps. Below are the input and output examples for a tree with two entailment steps  $(\mathrm{T} = 1$  and  $\mathrm{T} = 2)$ . In the following example, only the golden premises are used, which is how the generator is trained. At test time this format is extended to use the retrieved premises instead.
INPUT T=1: “hypothesis: notebook paper can be recycled many times; sent1: recyclable means a material can be recycled / reused many times sent2: paper is recyclable sent3: notebook paper is a kind of paper;”
OUTPUT T=1: "sent2 & sent3 -> int1: notebook paper is recyclable;"
INPUT T=2: “hypothesis: notebook paper can be recycled many times; sent1: recyclable means a material can be recycled / reused many times; sent2 & sent3 -> int1: notebook paper is recyclable;”
OUTPUT T=2: "int1 & sent1 -> hypothesis;
Note that the input for  $\mathbf{T} = 2$  removed the premises used in the previous entailment step, i.e. "sent2" and "sent3", and added the generated entailment step from  $\mathbf{T} = 1$  to the end of the input.
1Model available in https://huggingface.co/t5-large 2Model available in https://huggingface.co/sentence- transformers/all-mpnet-base-v2