SlowGuess/NAACL_2022_OCR huggingface dataset:
Danilo Ribeiro $^{1,2,*}$  Shen Wang $^{2}$ , Xiaofei Ma $^{2}$ , Rui Dong $^{2}$ , Xiaokai Wei $^{2}$ , Henry Zhu $^{2}$ , Xinchi Chen $^{2}$ , Zhiheng Huang $^{2}$ , Peng Xu $^{2}$ , Andrew Arnold $^{2}$ , Dan Roth $^{2}$ ,
<sup>1</sup>Northwestern University, <sup>2</sup>AWS AI Labs
{dnribeiro}  $@$  u.northwestern.edu,
{shenwa, xiaofeim, ruidong, xiaokaiw, henghui} @amazon.com,
{xcc, zhiheng, pengx, anarnld, drot}  $@$  amazon.com
Large language models have achieved high performance on various question answering (QA) benchmarks, but the explainability of their output remains elusive. Structured explanations, called entailment trees, were recently suggested as a way to explain and inspect a QA system's answer. In order to better generate such entailment trees, we propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR). Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. The IRGR model iteratively searches for suitable premises, constructing a single entailment step at a time. Contrary to previous approaches, our method combines generation steps and retrieval of premises, allowing the model to leverage intermediate conclusions, and mitigating the input size limit of baseline encoder-decoder models. We conduct experiments using the EN-TAILMENT-BANK dataset, where we outperform existing benchmarks on premise retrieval and entailment tree generation, with around  $300\%$  gain in overall correctness.
Large neural network models have successfully been applied to different natural language tasks, achieving state-of-the-art results in many natural language benchmarks. Despite this success, these results came with the expense of AI systems becoming less interpretable (Jain and Wallace, 2019; Rajani et al., 2019a).
With the desire to make the output of such models less opaque, we propose a question answering (QA) system that is able to explain their decisions not only by retrieving supporting textual evidence (rationales), but by showing how the answer to a question can be systematically proven from simpler textual premises (natural language reasoning).
Task: explain a hypothesis from premises
Figure 1: Task has as input a hypothesis H (e.g. an answer to a question) and a corpus of premises C (simple textual evidences), the goal is to generate an entailment tree that explains the hypothesis H by using premises from C.
These explanations are represented using entailment trees, as depicted in Figure 1. First introduced by Dalvi et al. (2021), entailment trees represent a chain of reasoning that shows how a hypothesis (or an answer to a question) can be explained from simpler textual evidence. In comparison, other explanation approaches such as retrieval of passages (rationales) (DeYoung et al., 2020) or multi-hop reasoning (chaining) (Jhamtani and Clark, 2020) are less expressive than entailment trees, which are comprised of multi-premise textual entailment steps.
In order to generate such entailment trees, previous works (Tafjord et al., 2021; Dalvi et al., 2021; Bostrom et al., 2021) have used encoder-decoder models that takes as input a small set of retrieved premises and output a linearized representation of the entailment tree. Such models are limited by (1) the language model's fixed input size, and they may construct incorrect proofs when the retrieval module cannot fetch all relevant premises at once
*Work done during an internship at the AWS AI. Code and model checkpoints are publicly available at https://github.com.amazon-research/irgr.
Findings of the Association for Computational Linguistics: NAACL 2022, pages 465 - 475
Figure 2: Comparison among different natural language explanation approaches. The images show (A) plain textual explanations (B) retrieval of evidence passages (C) multi-hop explanations (D) entailment trees. The approach in (D) allows for more detailed inspection of the reasoning behind an explanation. Nodes in gray are retrieved from a corpus, nodes in blue are generated, and the red node is the hypothesis or answer that is being explained.
(2) such approaches do not leverage the partially generated entailment trees. In contrast, we propose Iterative Retrieval-Generation Reasoner (IRGR), a novel architecture that iteratively searches for suitable premises, constructing a single entailment step at a time. At every generation step, the model searches for a distinct set of premises that will support the generation of a single step, therefore mitigating the language model's input size limit and improving generation correctness.
Our contributions are two-fold. First, we design a retrieval method that is able to better identify premises needed to generate a chain of reasoning, which explains a given hypothesis. Our retrieval method outperforms previous baselines by  $48.3\%$ , while allowing for a dynamic set of premises to be retrieved. Secondly, we propose an iterative retrieval-generation architecture that constructs partial proofs and augments the retrieval probes using intermediate generation results. We show that integrating the retrieval module with iterative generation can significantly improve explanations. Our proposed approach achieves new state-of-the-art result on entailment tree generation with over  $306\%$  better results on the All-Correct metric (strict comparison with golden data), while using a model with one order of magnitude fewer parameters.
Traditionally, natural language processing (NLP) frameworks were based on white-box methods such as rule-based systems (Allen, 1988; Ribeiro et al., 2019; Ribeiro and Forbus, 2021) and decision trees (Boros et al., 2017), which were inherently inspectable (Danilevsky et al., 2020). More recently, large deep learning language models (black-box methods) have gained popularity (Song et al., 2020; Raffel et al., 2020), but their improvements in result quality came with a cost: the system's outputs
lack explainability and inspectability.
There have been many attempts to mitigate this issue, including input perturbation (Ribeiro et al., 2018) and premises selection (DeYoung et al., 2020). One promising explanation approach is to combine the model's output with a human-interpretable explanation. For instance, Camburu et al. (2018) introduced the concept of natural language explanation in their e-SNLI dataset while Rajani et al. (2019b) expanded this idea to commonsense explanations. Jhamtani and Clark (2020) further explored the notion of explanation in multihop QA, where explanations contain a chain of reasoning, instead of simple textual explanations. Different from these explanation approaches, our work generates explanations in the form of entailment trees, introduced by Dalvi et al. (2021), which are composed of multi-premise textual entailment steps. Entailment trees are more detailed explanations, making it easier to inspect the reasoning behind the model's answer. Figure 2 shows a diagram comparing different natural language explanation methods according to their structure and use of textual evidence.
The first approach used to generate entailment trees was based on the EntailmentWriter model by Tafjord et al. (2021). However, their approach is limited by the input size of the encoder-decoder language models, where a fixed set of supporting facts is used to generate an explanation. Instead, our model iteratively fetches a set of premises using dense retrieval conditioned on previous entailment steps, allowing for more precise explanations.
Our work is also related to some recent approaches that combine retrieval and neural networks for QA tasks (Karpukhin et al., 2020; Guu et al., 2020). The work of Lewis et al. (2020) combined dense retrieval with encoder-decoder models, where a different set of passages were retrieved
Figure 3: IRGR is composed of two modules, IRGR-retriever and IRGR-generator. The IRGR-retriever iteratively fetches a set of premises from a corpus  $C$  in order to generate an entailment tree (structured explanation for a given hypothesis). The IRGR-generator computes a single entailment step at a time, and the intermediate generated steps are stored and used for subsequent retrieval and generation.
for each generated character. Conditioning the retrieval of a passage on previously retrieved passages has been explored in the context of multi-hop QA (Zhao et al., 2021; Xiong et al., 2021), and multi-hop explanations (Valentino et al., 2021; Cartuyvels et al., 2020). However, these approaches either are not used to generate explanations or do not use inferred intermediate reasoning steps to retrieve premises.
The problem input consists of a corpus of premises  $C$  (simple textual statements) and a hypothesis  $h$ . The objective is to generate an entailment tree  $T$  that explains the hypothesis  $h$  by using a subset of the premises in  $C$  as building blocks. Entailment trees are represented as a tuple  $T = (h, \mathcal{L}, \mathcal{E}, S)$ , where leaf nodes  $l_i \in \mathcal{L}$  are retrieved from the corpus (i.e.  $\mathcal{L} \subseteq C$ ), internal tree nodes  $e_i \in \mathcal{E}$  are intermediate conclusions (new sentences not present in corpus  $C$ , note that intermediate conclusions are generated by the model), and  $s_i \in S$  is a list of entailment steps that can explain the hypothesis  $h$ , which is always the tree root and the final conclusion. An illustration of the problem and expected entailment tree can be found in Figure 1.
Each entailment step  $s_i$  represents one inference step from a conjunction of premises to a conclusion. For instance, “ $l_1 \wedge l_2 \Rightarrow e_1$ ” or “ $l_1 \wedge l_2 \wedge e_1 \Rightarrow h$ ” could be valid entailment steps in  $S$ . Note that the root of  $T$  is always the node representing  $h$ .
Our approach, which we call Iterative Retrieval-Generation Reasoner (IRGR), consists of two modules, the IRGR-retriever and the IRGR-generator. The initial input to the model is the hypothesis  $h$  and the corpus of premises  $C$ . The generation process is performed through multiple iterations. At each iteration step  $t \geq 1$  the IRGR-retriever selects a subset of premises from the corpus  $\mathcal{L}_t \subseteq C$ . The IRGR-generator outputs one entailment step  $s_t$  per iteration until the entailment tree  $T$  is fully generated. Given  $S_{1:t-1} = (s_1, \ldots, s_{t-1})$  as the list of the entailment steps generated up to the previous iterations  $t-1$ , the generator takes as input  $\mathcal{L}_t$  and  $S_{1:t-1}$  and produces the next entailment step  $s_t$ . The generation stops when the entailment step's conclusion is the hypothesis  $h$ , i.e., the proof is finished. Formally, the  $t$ -th iteration of the generation process is defined as:
$$
\mathcal {L} _ {t} = \operatorname {I R G R - r e t r i e v e r} (h, s _ {t - 1}) \tag {1}
$$
$$
s _ {t} = \operatorname {I R G R - g e n e r a t o r} \left(h, \mathcal {L} _ {t}, \mathcal {S} _ {1: t - 1}\right) \tag {2}
$$
The IRGR-retriever searches over the premises in corpus  $C$  using dense passage retrieval (Karpukhin et al., 2020). Meanwhile, the IRGR-generator was implemented using T5, the Text-to-Text Transformer (Raffel et al., 2020), while any other sequence-to-sequence language model could also be used. An overview of the model can be seen in Figure 3.
3.2.1 IRGR-retriever
The IRGR-retriever module proposed in this work aims to retrieve premises from the corpus  $C$ . In existing baseline models the retrieval is done in one single step, fetching a fixed set of premises before generation (Tafjord et al., 2021). However, the generation of entailment trees requires a different set of leaves for each entailment step. To address this issue, our IRGR-retriever fetches  $k_{t}$  premises from  $C$  to produce  $\mathcal{L}_t$  at iteration step  $t$ . Note that the size of  $C$  can be very large ( $k_{t} < |C|$ ). The value  $k_{t}$  is chosen such that the size of  $\mathcal{L}_t$  is small enough to fit in the context of a language model while still being large enough to fetch as many premises as possible (in our experiments, the value  $k_{t}$  is always below 25). We define the retrieval probability of a premise  $c \in C$  at a certain iteration step  $t$  as:
$$
P (c \mid h, s _ {t - 1}) = \frac {\exp \left(\langle \mathbf {c} , \mathbf {q} _ {\mathbf {t}} \rangle\right)}{\sum_ {c ^ {\prime} \in C} \exp \left(\langle \mathbf {c} ^ {\prime} , \mathbf {q} _ {\mathbf {t}} \rangle\right)} \tag {3}
$$
Where  $\phi$  is the sentence encoder function used to encode both premises and hypothesis, transforming the input text into a dense vector representation in  $\mathbb{R}^M$ . The values  $\mathbf{c} = \phi(c)$ ,  $\mathbf{c}' = \phi(c')$  and  $\mathbf{q_t} = \phi(h, s_{t-1})$  are dense  $M$ -dimensional vectors. The operator  $\langle . \rangle$  represents the inner product between two vectors.
The encoder follows the Siamese Network architecture from Reimers and Gurevych (2019). We select a set of  $N$  positive and negative examples in the form of query-value pairs  $\{(q_j, c_j)\}_{j=1}^N$  for training. Queries  $q_j$  encode both the hypothesis  $h$  and previous entailment step  $s_{t-1}$  by concatenating their textual values. The positive examples are taken from the golden entailment trees, where  $c_j \in \mathcal{L}$ . For negative examples, we pair a query  $q_j$  with either random premises from  $C$  or premises retrieved using the not fine-tuned version of the encoder (hard negatives).
We define  $\hat{y}_j$  as the label given to the training example  $(q_j, c_j)$ . For positive examples, the label  $\hat{y}_j$  depends on how close the leaf node  $l_i \in \mathcal{L}$  is to the intermediate step  $s_{t-1}$  in the golden tree:
$$
\hat {y} _ {j} = \left\{ \begin{array}{l l} 0, & \text {i f n e g a t i v e} \\ \lambda , & \text {i f p o s i t i v e a n d} l _ {i} \notin \operatorname {a n t} \left(s _ {t - 1}\right) \\ 1, & \text {i f p o s i t i v e a n d} l _ {i} \in \operatorname {a n t} \left(s _ {t - 1}\right) \end{array} \right. \tag {4}
$$
Where  $\operatorname{ant}(s_{t-1})$  denotes the set of antecedents in some entailment step  $s_{t-1}$ , and  $l_i \in \operatorname{ant}(s_{t-1})$  means that the leaf node  $l_i$  is used in the entailment
Figure 4: Entailment tree example showing how some retrieval examples are challenging. Leaf sentences are not always directly related to hypothesis.
Algorithm 1: Conditional Retrieval
Data: hypothesis  $h$ , corpus  $C$ , number of retrieved premises  $k_{0}$
Result: retrieved premises  $\mathcal{L}_0$
Parameter: conditioning factor  $\omega$
$$
Q \leftarrow \{h \}; / \star \text {s e t o f q u e r i e s} \star /
$$
$$
\mathcal {L} _ {0} \leftarrow \{\};
$$
for  $i\gets 0$  to  $k_{0}$  do
$$
C ^ {\prime} \leftarrow \{c \in C: c \notin \mathcal {L} _ {0} \};
$$
$$
i \geq \omega \text {t h e n}
$$
$$
\begin{array}{l} l _ {i} \leftarrow \arg \max  _ {(c \in C ^ {\prime})} P (c \mid Q); \\ Q \leftarrow Q \cup \{l _ {i} \}; \end{array}
$$
$$
\left| \begin{array}{l} l _ {i} \leftarrow \arg \max  _ {(c \in C ^ {\prime})} P (c \mid h); \end{array} \right.
$$
$$
\mathcal {L} _ {0} \leftarrow \mathcal {L} _ {0} \cup \{l _ {i} \};
$$
step  $s_{t-1}$ . The value  $\lambda \in [0:1]$  is used to give lower priority to leaf nodes not relevant to the current entailment step ( $\lambda = 0.75$  gave the best results in our experiments). Finally, we fine-tune the encoder  $\phi$  by minimizing the following loss function  $L_{\phi}$ , where  $N$  is the number of training examples:
$$
L _ {\phi} = \frac {1}{N} \sum_ {j = 1} ^ {N} \left(\hat {y} _ {j} - \frac {\langle \phi \left(q _ {j}\right) , \phi \left(c _ {j}\right) \rangle}{\| \phi \left(q _ {j}\right) \| \| \phi \left(c _ {j}\right) \|}\right) \tag {5}
$$
One significant challenge is that for the first generation step, when  $t = 1$ , the list of previously generated entailment steps  $S_0$  is empty. The retrieval only depends on  $h$ , meaning  $\mathcal{L}_1 = \mathrm{IRGR - retriever}(h)$ . It is more difficult to retrieve premises for leaf nodes when the entailment tree's
depth is large since the leaf nodes have low syntactic and semantic similarity with the hypothesis  $h$ . For instance, the example in Figure 4 shows how leaf node "a human is a kind of animal" (depth 3) is needed to build the entailment tree, but is syntactically distinct from hypothesis "an astronaut requires the oxygen in a space suit backpack to breath".
To mitigate this problem, we perform a conditional retrieval on the first step, where the retrieval module uses partial results as part of the query, as depicted in Algorithm 1. This algorithm assumes that leaf nodes (premises) further from the root node (hypothesis) are more similar to each other than to the root node itself. The parameter  $\omega$  (value  $\omega = 15$  yields the best results on development set) is used to split the search, such that part of the retrieved premises only depend on the hypothesis  $h$ . In contrast, the other parts of the retrieved premises depend on the hypothesis and previously retrieved premises stored in the set  $Q$ .
3.2.2 IRGR-generator
The IRGR-generator consists of a sequence-to-sequence model that outputs one single entailment step given a context. One key aspect of this module is encoding the input and output as plain text.
Encoding Entailment Trees: Entailment trees are linearized from leaves to root. Each leaf node  $l_i \in \mathcal{L}$ , intermediate node  $e_i \in \mathcal{E}$  and root node  $h$  are encoded with the symbols "sent", "int" and "hypothesis", respectively. The entailment steps represent conjunctions with "&" and entailment with the symbol " $->$ ". For instance, the entailment tree depicted in Figure 1 can be represented as:
Note that the text of intermediate nodes have to be explicitly represented, since they are not part of the corpus  $C$ . Ultimately, they have to be generated by the model. The input to the model encodes the hypothesis  $h$  and retrieved premises  $l_i^t \in \mathcal{L}_t$ , which are straightforwardly encoded as follows:
Method	R@25	All-Correct
Okapi BM25	45.01	22.35
EntailmentWriter	59.76	34.70
IRGR-retriever (sing.)	64.41	40.29
IRGR-retriever (cond.)	68.28	44.70
IRGR-retriever*	-	51.47
Table 1: Retrieval results. The methods with * retrieves more than 25 premises from corpus.
When a leaf sentence  $l_{t}^{i}$  is used in the entailment step, it is removed from the context for the following step, and the premise sent identifier is not used to encode new retrieved premises. A detailed example of input and output for the IRGR-generator module is shown in Appendix A.3.
We evaluate our architecture on the ENTAILMENT-BANK dataset (Dalvi et al., 2021), which is comprised of 1,840 questions (each associated with a hypothesis  $h_i$  and entailment tree  $T_i$ ) with 5,881 total entailment steps. On average, each entailment tree has 7.6 nodes (including leaf, intermediate, and root) and around 3.2 entailment steps. The corpus of premises  $C$  has around 11K entries and is derived from the WorldTree V2 (Xie et al., 2020) in addition to a few premises created by the EntailmentBank annotators.
4.2.1 Retrieval
We evaluate our IRGR-retriever module using two different sets of metrics. The first one is "Recall at  $\mathbf{k}$ " (R@k), a standard evaluation metric for information retrieval. The second metric "All-Correct" is more strict, and the results are only considered correct if all the premises from the golden tree are retrieved. Formally, given the retrieved premises  $\mathcal{L}$  and the set of gold premises  $\mathcal{L}^*$ , the metrics R@k is given by  $|\mathcal{L} \cap \mathcal{L}^*| / |\mathcal{L}^*|$ , and the metric All-Correct is 1 if  $|\{x \in \mathcal{L}^* : x \notin \mathcal{L}\}| = 0$  or 0 otherwise. For our experiments, we consider  $k = 25$  since that's roughly the maximum number of sentences that can fit in the T5 language model's 512 tokens context.
Method	Leaves	Steps	Intermediates	Overall
F1	All-Cor.	F1	All-Cor.	F1	All-Cor.	All-Cor.
EntailmentWriter	39.7	3.8	7.8	2.9	36.4	13.2	2.9
IRGR	45.6	12.1	16.3	11.8	38.8	36.5	11.8
- w/o iter.	46.6	10.0	11.3	8.2	36.3	35.6	8.2
- w/o iter. & cond.	36.1	3.8	6.0	3.2	27.6	14.7	3.2
Table 2: Entailment tree scores for baseline methods and IRGR, along four different dimensions (test set). F1 scores measure predicted/gold overlap, while All-Correct scores are 1 when all the predictions for a tree are correct, 0 otherwise.
4.2.2 Entailment Tree Generation
We adopt the evaluation metrics defined by Dalvi et al. (2021), which compares the generated entailment tree  $T = (h, \mathcal{L}, \mathcal{E}, \mathcal{S})$  with the golden entailment tree  $T^{*} = (h, \mathcal{L}^{*}, \mathcal{E}^{*}, \mathcal{S}^{*})$ . The metrics evaluate the correctness along four dimensions: (1) leaf nodes, (2) entailment steps, (3) generated intermediate nodes, (4) and overall correctness. The first step is to align the nodes from  $T$  with the nodes from  $T^{*}$  by Jaccard similarity (alignment algorithm and further details of metrics described in Appendix A.2). This method tries to ignore variations between predicted and gold trees that do not change the semantics of the output. The four metric dimensions are described below as follows. For each metric with F1 value, there is also a strict "All-Correct" metric that is equal to 1 when  $F1 = 1$  and 0 otherwise.
Leaf (F1, All-Correct): Tests if the predicted and golden leaf nodes match. This metric compares the sets  $\mathcal{L}$  and  $\mathcal{L}^*$  using F1 score.
Steps (F1, All-Correct): Tests if the predicted entailment steps follow the correct structure. Given that  $s_i \in S$  matches  $s_j \in S^*$  according to the alignment algorithm, tests if the premises of  $s_i$  are equal to those of  $s_j$ , and computes the F1 score according to the set of all matched steps.
Intermediates (F1, All-Correct): Tests if the sentences of the generated intermediate nodes are correct. Given that intermediate nodes  $e_i \in \mathcal{E}$  and  $e_j \in \mathcal{E}^*$  were matched by the alignment algorithm, the F1 score is computed by comparing the textual similarity between the set of the aligned and correct pairs  $e_i$  and  $e_j$ .
Overall (All-Correct): Tests all previous metrics together. The All-Correct value is only 1 if the All-Correct values for leaves, steps, and intermediates
are 1. Note that this is a strict metric, and any semantic difference between  $T$  and  $T^{*}$  will cause the score to be zero.
All experiments were conducted using a machine with 4 Tesla V100 GPUs with 16GB of memory. Our code is based on HuggingFace's Transformers (Wolf et al., 2020) implementation of the t5-large model (Raffel et al., 2020). The retrieval module uses the Sentence Transformers (Reimers and Gurevych, 2019) sentence embeddings by fine-tuning the all-mpnet-base-v2 encoder. Please refer to Appendix A.1 for further details on hyper-parameters and training settings.
4.4.1 Retrieval Results
We compare our retrieval module against two baselines: Okapi BM25 and the retrieval module of EntailmentWriter, which constitutes of a classifier that retrieves relevant sentences using RoBERTA (Liu et al., 2020) and performs re-ranking with Tensorflow-Ranking-BERT (Han et al., 2020).
For comparison, we break down the results of our approach (the IRGR-retriever module) into three variations. The IRGR-retriever (sing.) method retrieves premises from the corpus using a single query element, namely the hypothesis  $h$ . The IRGR-retriever (cond.) method performs conditioned retrieval as described by Algorithm 1. This retrieval method is not iterative and fetches a fixed set of premises per example. Finally, IRGR-retriever tries to emulate the retrieval when combined with the generation module. It not only performs conditional retrieval, but also fetches a different set of premises for each iteration depending on the generated intermediate nodes. In this retrieval experiment, the IRGR-retriever uses the intermediate nodes from the golden entailment trees.