SlowGuess/NAACL_2021_OCR huggingface dataset:
Zhengzhong Liang<sup>1</sup>, Steven Bethard<sup>2</sup> and Mihai Surdeanu<sup>1</sup>
<sup>1</sup> Computer Science Department, The University of Arizona, Tucson, AZ
$^{2}$ School of Information, The University of Arizona, Tucson, AZ
{zhengzhongliang, bethard, msurdeanu}@email.arizona.edu
Many state-of-the-art (SOTA) language models have achieved high accuracy on several multi-hop reasoning problems. However, these approaches tend to not be interpretable because they do not make the intermediate reasoning steps explicit. Moreover, models trained on simpler tasks tend to fail when directly tested on more complex problems. We propose the Explainable multi-hop Verbal Reasoner (EVR) to solve these limitations by (a) decomposing multi-hop reasoning problems into several simple ones, and (b) using natural language to guide the intermediate reasoning hops. We implement EVR by extending the classic reasoning paradigm General Problem Solver (GPS) with a SOTA generative language model to generate subgoals and perform inference in natural language at each reasoning step. Evaluation of EVR on Clark et al. (2020)'s synthetic question answering (QA) dataset shows that EVR achieves SOTA performance while being able to generate all reasoning steps in natural language. Furthermore, EVR generalizes better than other strong methods when trained on simpler tasks or less training data (up to  $35.7\%$  and  $7.7\%$  absolute improvement respectively).<sup>1</sup>
Large pretrained language models such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) have been successfully used in multi-hop reasoning problems (Banerjee et al., 2020; Asai et al., 2019; Yadav et al., 2019). Usually, these pretrained language models solve multi-hop reasoning problems in a discriminative end-to-end manner: these models take the question and all the relevant evidence as the input, and produce the final answer to the question. This raises two problems. First, this direction lacks interpretability, i.e., it is hard to
(Input Facts:) Alan is blue. Alan is rough. Alan is young. Bob is big. Bob is round. Charlie is big. Charlie is blue. Charlie is green. Dave is green. Dave is rough.
(Input Rules:) Big people are rough. If someone is young and round then they are kind. If someone is round and big then they are blue. All rough people are green.
Q1: Bob is green. True/false? [Answer: T]
Q2: Bob is kind. True/false? [Answer: F]
Figure 1: An example taken from (Clark et al., 2020). The context includes two types of statements: facts and rules. Multiple facts and rules are usually needed to answer the questions. For example, to prove "Bob is green," the model needs to construct the reasoning chain "All rough people are green  $\leftarrow$  Big people are rough  $\leftarrow$  Bob is big".
know which individual reasoning steps are taken in each iteration and why. Second, the trained models usually suffer from the compositionality generalization problem, meaning that they tend to fail when the number of reasoning steps are much larger in the evaluation set than in the training set (Hupkes et al., 2020; Hahn, 2020; Clark et al., 2020).
Newell (1994) categorized cognitive processes based on their time scales: unconscious activities take around  $50~\mathrm{ms}$ , whereas conscious actions can vary from  $100~\mathrm{ms}$  to hours. Importantly, Newell (1994) argued that conscious actions are sequences of simple conscious/unconscious actions. Extrapolating from cognitive science to natural language processing (NLP), in this paper we ask the question: can we design an interpretable multi-hop reasoning system that sequentially applies neural networks trained on simpler tasks? Further, motivated by the finding from cognitive science that people might use internal monologues to guide their reasoning, we want to explore whether it is possible to use natural language to guide this sequential process.
In this paper, we propose a solution for these important questions. We provide a neural implementation for a classic planning/reasoning paradigm that is designed to mimic the human reasoning process:
The code is available at: https://github.com/clulab/releases/tree/master/naacl2021-evr
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1225-1250 June 6-11, 2021. ©2021 Association for Computational Linguistics
the General Problem Solver (GPS) (Newell et al., 1959). We augment GPS with a SOTA sequence-to-sequence (Seq2Seq) model and apply this model recursively to achieve high interpretability and better generalization of compositionality for a synthetic QA task (Clark et al., 2020).
The contributions of our paper are the following:
(1) We extend (Clark et al., 2020)'s dataset with natural language intermediate goals/statements necessary to answer each question.
(2) We propose a neural GPS to address this QA task while generating all intermediate goals/statements in natural language.
(3) Evaluation on the above task shows that our proposed method achieves SOTA performance. Importantly, our method generalizes better when trained only on simpler tasks (26.5% to 35.7% absolute improvement), and less training data (7.7% absolute improvement) compared with other strong reasoning methods.
2 Task Description and Baseline Models
We build our approach from the multi-hop reasoning problem proposed in (Clark et al., 2020), which we summarize first. Figure 1 shows an example from this dataset. Each reasoning problem consists of the context  $C$ , the question  $Q$  and the answer  $A = \{True, False\}$ .  $C$  includes facts  $F$  and rules  $R$ . To answer a question, multiple statements in  $C$  need to be combined.
2.1 Proof Strategy
The proofs of the questions are provided by the creators of the dataset, and each question is proved by one of three available strategies: "proof", "inv-proof" and "fail-to-prove". "Proof" directly proves a statement is true using the facts and the rules; "inv-proof" proves a statement is false using the facts and rules; and, lastly, "fail-to-prove" means the statement could not be explicitly proved to be true or false given the rules and facts. In the latter case, a positive statement is considered to be False, and a negative statement is considered to be True.
For example, assuming we are given the facts and rules in Figure 1, the reasoning chain provided to prove "Bob is green" is using the "proof" strategy. Conversely, "Alan is not green" is false by "inv-proof", because "Alan is green" can be proved by "All rough people are green  $\leftarrow$  Alan is rough". Finally, "Alan is nice" is false and "Alan is not nice" is true due to "fail-to-prove".
2.2 Dataset Details
The dataset is synthesized using hand-crafted rules and formal language, then translated to natural language. Some language variation is inserted (e.g., in Figure 1 the rules are expressed differently). Depending on the number of rules and facts needed, there are 5 partitions in the dataset: DU0, DU1, DU2, DU3 and DU5, where "DU" stands for "Depth Upto". "DU0" means the reasoning depth of the questions is 0, i.e., the questions can be answered by just looking at the facts without applying any rules. DU5 means the questions may require applying the rules for upto 5 times (but DU5 also has questions that require applying the rules for 0 to 4 times).
Additionally, a "birds-electricity" dataset is also provided to test the model's generalization ability. The  $F$ ,  $R$ ,  $Q$  are generated by similar templates of DU0 to DU5, but with different entities/predicates/attributes that do not appear in the DU0 to DU5 partitions.
Summing up all partitions, the dataset has approximately 500K questions, and the train/dev/test ratio is 70/10/20. More details can be found in Clark et al., 2020).
2.3 Baseline Models
We will compare our approach against two strong baselines. The first baseline is a RoBERTa classifier from the original paper of Clark et al. (2020). In this approach, the questions are solved in a text classification manner. That is, for each question, the model takes  $C$  and  $Q$  as the input, and calculates the probability of  $A$  being true or false. We abbreviate this baseline as "RT" in our paper.
The second baseline is PROVER (Saha et al., 2020), which handles the reasoning problem as a graph problem. This approach takes the input  $C$  and  $Q$  to produce both the final answer  $\{True, False\}$ , and a graph that indicates the reasoning path. We abbreviate this baseline as "PR".
2.4 Shortcomings and Opportunities
We see two shortcomings for these research directions, which motivate our proposed idea.
Shortcoming #1: Limited interpretability RT models this reasoning problems as a text classification task over a "bag of evidences". That is, RT takes all context and produces an answer in a single forward inference process. Although its predictions can often achieve high accuracy, there are several
Figure 2: (A) GPS's working cycle. (B) A toy example showing the goal and operators (action 1 and 2). This problem can be solved in two cycles: in the first cycle, the proposer reads the goal state  $\text{On}(C, A)$  and search for available actions. In this case action 2 is proposed because its effect matches the goal state. Next, the goal is updated as  $\text{Clear}(A)$  (i.e., preconditions of action 2). In the second cycle, the proposer reads the new goal  $\text{Clear}(A)$  and action 1 is proposed. By applying action 1 the goal is further updated as  $\text{On}(B, A)$ . Since  $\text{On}(B, A)$  is the initial state, the proposer then terminates the reasoning process.
problems with this and similar directions, which limit their interpretability. First, it is unclear which part of the context was used by the answering engine. Second, although some methods such as PR improve over RT by producing an explanation consisting of multiple supporting facts at the end, it is still hard to explain the underlying reasoning process in human understandable terms. Finally, it has been shown theoretically and empirically that neural networks suffer from the compositionality generalization problem (Hupkes et al., 2020; Hahn, 2020). That is, neural networks have limited ability to learn recursive patterns, and they fail to generalize to recursive patterns that are much deeper than the ones seen in training.
Shortcoming #2: Differences from the human cognitive process People do not usually solve very complex problems at once. Instead, people constantly generate subgoals and solve complex problems incrementally (Newell, 1966). Second, verbal strategies are sometimes used to guide one's reasoning (Bacon et al., 2003). This is different from the approaches taken by both RT and PR.
A desired explainable multi-hop verbal reasoner: Motivated by these shortcomings, we propose several desired characteristics for an ideal problem solver. First, the method should be able to decompose complex problems into simple ones that are easy to answer. This should not only increase the interpretability of the reasoning process, but also help reduce the compositionality generalization problem, because the unseen distributions
(the complex problem) can be reduced to a series of seen distributions (simple problems). Second, each reasoning step should be guided by natural language, so that each step is easily explainable to the human end user.
3 Neural General Problem Solver for Multi-hop Verbal Reasoning
In this section, we first review a classic planning/reasoning paradigm that is designed to mimic the human reasoning process, the General Problem Solver (GPS) (Newell et al., 1959), then propose our neural implementation of it.
3.1 The General Problem Solver
GPS works in cycles (Figure 2 (A)): in each cycle, the operator proposer  $P$  reads the current goal  $G$  to propose an operator  $O = P(G)$ . Then the proposed operator is used by the executor  $E$  to update the goal  $G = E(G, O)$ . The cycle stops when the goal is satisfied, or no new operators are proposed. Figure 2 (B) shows a toy example from the block world, where the agent starts from the goal state and searches for a sequence of actions to reach the initial state.
3.2 The Neural General Problem Solver
Although GPS has been widely used to mimic the human reasoning process, it has shortcomings. First, the representations of the goals are usually in a formal language, which has limited expressiveness and readability compared to natural language. Second, the proposer uses human-crafted rules to match and propose operators, which may not be flexible enough to handle situations that diverge from the training examples.
Due to these drawbacks, we propose to add neural components to GPS (a working cycle is shown in Figure 3). More specifically, the neural GPS has the following characteristics:
Goal (Extended to Working Memory): First, the goal is represented in natural language instead of formal language to enable better readability and expressiveness. Second, the goal is extended to a working memory buffer, which contains not only the goal but also other information that might be useful to the reasoning process.
Operator Proposer and Executor: The operator proposer is no longer using explicit rules. Instead, we use a Seq2Seq neural network to directly map
Figure 3: (A) A working cycle for the proposed neural GPS. The goal in GPS is extended to a working memory buffer (WM), and multiple operators might be generated in a cycle. (B) The composition of the working memory and possible values stored in each field.
Figure 4: A walk through example of EVR solving the problem mentioned in Figure 1. The format in the example is the same as the actual output of our method. The red solid lines indicate the path that proves the goal statement. An example of the actual output of our model is shown in Appendix A.5.
the text in the working memory to a sequence of operators. The operators are later used by the executor, which also has neural components, to update the working memory buffer.
3.3 Adapting the Neural GPS to Multi-hop Verbal Reasoning
In this section we explain in detail the working flow of the neural GPS (referred as Explainable Verbal Reasoner or EVR later) and the design of each of its components for multi-hop verbal reasoning.
A Walk-Through Example: Figure 4 shows an example of how our method solves the problem in Figure 1. Every two consecutive blocks form a working cycle (e.g., patterns 1&2 or 3&4). In each cycle, the odd pattern is the operator proposing stage, and the even pattern is the executing stage.
For example, pattern 1 shows an operator proposing stage in a cycle. There are two episodic
Buffers available for the operator proposer, with the first buffer storing some general knowledge about the problem, and the second describing the goal. The operator proposer, a Seq2Seq neural model, first concatenates the two episodic buffers, then proposes "GENERATE_SUBGOALS" as the operator. At the executing stage, the executor (another Seq2Seq neural model), takes the two episodic buffers in the working memory and the "GENERATE_SUBGOALS" operator to produce the subgoals: judge whether the facts can prove the goal or the rules can prove the goal. Finally, the newly generated subgoals replace the old goal in the episodic buffer (i.e., the goals in pattern 3 and 7 are different from pattern 1, because the goal in pattern 1 is replaced) and one working cycle is finished. At pattern 10, the EVR discovers that the new goal is to prove "bob is rough", so another recursive search process starts (largely repeating
the process of pattern 1 to 10).
Working Memory: Working memory is a global memory space with several storage fields, where each field is indexed by a textual key (Figure 3 (B)). In this verbal reasoning problem, three types of information can be stored in the working memory: episodic buffer (indexed by the key "EPISODIC BUFFER"), fact buffer (indexed by FACT BUFFER_  $[i]$ , since there are probably more than one fact buffers), and rule buffer (indexed by RULE BUFFER_  $[i]$ ).
The episodic buffer stores three types of information: (a) general statements about the reasoning task that are useful throughout the reasoning process (e.g., row 1 in Figure 3 (B)); (b) goal (row 2, Figure 3 (B)). Similar to GPS, a description of goal should be included in the working memory. We use natural language to describe the goals, and goals are updated periodically; (c) inferred knowledge during the reasoning process (row 3, Figure 3 (B)).
Note that the working memory is not equivalent to the input text in the patterns in Figure 4. The input text in Figure 4 is obtained from the working memory, but some information in the working memory might not be needed by many patterns. For example, fact buffers and rule buffers are only the input for pattern 6 and 10, whereas other patterns do not use them.
Ideally, at each cycle, both the proposer and the executor need to determine what information to use from the working memory, and the executor also needs to determine what information to modify in the working memory. This is a hard problem: assume there are  $n$  pieces of information in the working memory, there will be  $2^n$  ways to read from/modify the memory. Therefore we make the following simplifications for this verbal reasoning problem. First, the size of episodic buffer is fixed to 2, and the first episodic buffer slot (i.e., "there are  $X$  fact buffers and  $Y$  rule buffers") can not be modified; the second episodic buffer slot is constantly modified as the new subgoals are generated. Second, the fact buffers and the rule buffers could not be modified. Third, the proposer and the executor only use the two episodic buffers as the input by default. The fact buffer and rule buffer can be used as input, but only when explicit commands are generated (e.g., pattern 5 and 6 in Figure 4).
Operator Proposer: We use a SOTA Seq2Seq language model, Google T5 (Raffel et al., 2020), as the operator proposer. The proposer concatenates
the two episodic buffer slots as a single piece of text, and uses this text as the input to produce a sequence of operators.
Executor: The executor has three functions: (1) parse the operators, (2) call the correct neural module given the operator, to get the answer/get the subgoal, and (3) update the working memory. Since the operators are fairly simple, we just use several if-else conditions to determine what actions need to be taken. Some examples are given in Table 1. As discussed above, the changing of working memory is restricted to changing the second episodic buffer slot. The major part of the executor is the neural module, which is responsible for generating subgoals (pattern 2,4,8 in Figure 4), answering questions (pattern 6 in Figure 4) and deriving statements (pattern 10 in Figure 4). Again we use Google T5 as this neural module to read from the working memory and produce textual output. More details can be found in Appendix A.3.
Operators: We design a simple domain specific language (DSL) as the operators. The meaning of some DSL commands are shown in Table 1.
3.4 Training Data Generation
Finally, to train the working flow we proposed in Section 3.3, 12 patterns of training data need to be generated (only 10 are shown in Figure 4). The generation strategies for some critical patterns are shown in Table 2. In summary, we write rules to generate the input text and the output text for each pattern. Around 1M training samples are generated for the 12 patterns in total. $^2$
We implemented three variants of EVR to learn these 12 tasks:
EVR1: This is the EVR baseline. For this baseline we use three distinct T5 models: one to learn pattern 6 data, one to learn pattern 10 data, and one to learn the rest of the patterns. The fact buffer size is set to 5 and the rule buffer size is set to 3 (5 facts per fact buffer and 3 rules per rule buffer).
EVR2: The fact buffer size and rule buffer size are the same as the EVR1. However, we use a single T5 model to learn all patterns of data. This is to test whether multi-task learning helps or harms the performance of EVR.
EVR3: We use three T5 models like EVR1, but the fact buffer size is set to 20 and the rule buffer
2The data generation code can be found at: https://github.com/clulab/releases/tree/master/naacl2021-evr
Operator	Example	Description
AND/OR	Pattern 2, 4, 8 in Figure 4	Conjunction/disjunction operator to connect two branches.
GENERATE_SUBGOALS	Pattern 2, 4, 8 in Figure 4	When the neural model takes this operator it generates subgoals
NAF/CWA	Pattern 6 in Figure 4	NAF is the abbreviation for “Negation as Failure”. It is generated when a negative statement (e.g., “bob is not nice”) is not contradicted by any facts. “CWA” is the abbreviation for “Close World Assumption”. It is output when a positive statement (e.g., “Bob is nice”) is not confirmed by any facts.
GET(FACT_BUFFERED[i]) THEN	Pattern 5, 6 in Figure 4	“GET(FACT_BUFFERED[i])” is to get the text of the facts in FACT_bufferED[i] (e.g., in pattern 6 Figure 4, FACT_bufferED_1 contains the text from fact 1 to fact 5). “THEN” connects two commands that need to be executed sequentially. Finally, the neural module in the executor takes the episodic buffer and fact buffer as the input to produce the output.
RUN(EPISODIC_BUFFER, FACT_BUFFERED[i])
Table 2: The strategy to generate the training data for some of the patterns. Each pattern has an input and an output to be generated. [statement] is the query to prove, e.g., "bob is green". Patterns 7, 8, 9, 11 and 12 are not included because they are generated in the similar manner as the patterns shown here. Please see Appendix A.2 for the generation strategies of all patterns.
size to 10. Therefore for all problems there will be only one fact buffer and one rule buffer. Since there are more facts and rules in the buffer, the input text to the T5 will be longer. And since there are more rules in the rule buffer, the number of matched rules could be more, so the target text could be potentially longer. We conjecture the longer input and output will make the model harder to train.
4.1 Training and Evaluation
We use T5 small for all experiments, with the learning rate set to 1e-4. In each epoch, the models are trained on 24,000 training examples, and evaluated on the first 2,000 dev samples. Edit distance between the generated text and target text is used to
D	0	1	2	3	4	5	all
Cnt	6299	4434	2915	2396	2134	2003	20192
DU1
RT	100.0	99.0	36.8	23.1	11.4	12.3	63.5
PR	—	—	—	—	—	—	73.7
EVR1	99.7	98.8	97.8	95.5	91.7	90.7	97.0
EVR2	99.5	98.3	96.8	93.1	89.3	88.1	95.9
EVR3	99.8	99.5	99.0	98.9	98.2	97.9	99.2
DU5
RT	100.0	98.4	98.4	98.8	99.2	99.8	99.2
PR	100.0	99.0	98.8	99.1	98.8	99.3	99.3
EVR1	99.5	98.2	96.5	92.8	88.3	86.2	95.5
evaluate the models performance on the dev set. The training is stopped when the edit distance on the dev set starts to increase.
In addition to the accuracy of the model's prediction of the final answer (T/F), we also report the quality of the generated proofs. We extracted the critical facts/rules from EVR's reasoning process and reconstructed the reasoning chain in the same format as the provided proofs in the dataset. The proof is considered correct as long as the generated reasoning chain matches one of the provided reasoning chains. We use depth- $\{0,1,2,3,4,5\}$  data (i.e., all depths) from DU5 to test all methods.
4.2 EVR Compared with Other Baselines
Table 3 lists the performance of the three EVR variants for QA. We compare EVR against RT and PR trained on the same DU1 data (top part of the table), and all the data (DU5) (bottom part). EVR outperforms the other two baseline methods trained on DU1 on nearly all splits. The best performing EVR is EVR3, which successfully maintains a 97.9 accuracy on depth-5 testing data, and a 99.2 accuracy on all testing splits. EVR3 trained on DU1 approaches the performance of the other methods trained on DU5. This indicates that when the training data are abundant, longer input or output does not harm the performance of our method.