SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite, Deepak Ramachandran
{mehrankazemi, yquan, bhatid, njkim, xxujasmine,
vimbrasaite, ramachandrand}@google.com
Automated reasoning with unstructured natural text is a key requirement for many potential applications of NLP and for developing robust AI systems. Recently, Language Models (LMs) have demonstrated complex reasoning capacities even without any finetuning. However, existing evaluation for automated reasoning assumes access to a consistent and coherent set of information over which models reason. When reasoning in the real-world, the available information is frequently inconsistent or contradictory, and therefore models need to be equipped with a strategy to resolve such conflicts when they arise. One widely-applicable way of resolving conflicts is to impose preferences over information sources (e.g., based on source credibility or information recency) and adopt the source with higher preference. In this paper, we formulate the problem of reasoning with contradictory information guided by preferences over sources as the classical problem of defeasible reasoning, and develop a dataset called BoardgameQA for measuring the reasoning capacity of LMs in this setting. BoardgameQA also incorporates reasoning with implicit background knowledge, to better reflect reasoning problems in downstream applications. We benchmark various LMs on BoardgameQA and the results reveal a significant gap in the reasoning capacity of state-of-the-art LMs on this problem, showing that reasoning with conflicting information does not surface out-of-the-box in LMs. While performance can be improved with finetuning, it nevertheless remains poor.
A fundamental goal of AI since its early days has been automatically applying logical or deductive reasoning to draw new conclusions from existing knowledge [29, 20]. Since a large amount of knowledge is available in the form of natural language, tremendous effort has been put into developing models that can understand and reason over natural language [23, 42, 54, 32, 12, 57] (see [35] for a survey). Recent years have seen substantial improvements in this direction thanks to advancements in pretrained language models (LMs) [8, 9] that can handle unstructured data more flexibly, combined with advanced prompting techniques [52, 31], and modular reasoning approaches [23, 12].
Existing work in automated reasoning in natural language usually assumes that the provided knowledge is consistent and reliable. But in many applications, the collection of information one has to reason with is inconsistent and contradictory. This is the case, for instance, when reasoning is performed with information found in different online sources or social media (e.g., retrieval-augmented LMs [17, 3]). When input sources are contradictory, one can consider various strategies to resolve the contradictions. One simple and practical formulation, which we adopt in this work, is to resolve the conflicts based on preferences over the information sources: when a con
flict arises, the information from the source with a higher preference should be used to solve the reasoning problem. Depending on the application, preferences can be assigned based on different criteria, e.g., based on the credibility of websites or social media users, or based on the recency of the information with newer information being preferred over older information. Exceptions to generics can also be expressed as preferences; for example, generic knowledge such as "birds fly" (see also [6]) should be overridden by exceptions such as "penguins are birds but do not fly" (see also [1]) when reasoning about penguins. Figure 1 demonstrates an example of a reasoning problem with conflicting information, where the conflict is resolved based on recency.
Reasoning with conflicting information guided by preferences can be formulated as a form of the classical defeasible reasoning problem [33, 19, 28]. In this work, we study the reasoning ability of LMs in this setting. Toward this goal, we create a synthetic dataset where each example contains a defeasible theory (a set of input facts, possibly contradictory rules, and preferences over the rules), and a question about that theory. Answering the questions in the dataset requires multi-hop reasoning and conflict resolution over the input theory. The
Facts: Fiona wants to travel from country X to country Y and stay for 5 days. Does Fiona need to book a Covid19 test appointment?
Info. Source 1 (date: 2022-04-20)
Info. Source 2 (date: 2022-04-27)
All travelers entering Y from X need to show negative COVID19 test results.
Travelers visiting Y for less than a month do not require covid19 tests anymore.
Reasoning: Fiona is traveling from X to Y and wants to stay for less than a month so both sources apply. The conclusions from the two sources conflict, but source 2 may be preferred since it is more recent. So Fiona doesn't need aovid test, and so does not need to book an appointment.
Figure 1: A reasoning problem with contradictory information (conflict resolved based on recency).
difficulty level (e.g., the depth, amount and type of conflicts, etc.) of the examples in the dataset can be controlled automatically, enabling targeted comparisons of various aspects of reasoning.
We also note that while a large number of logical reasoning benchmarks provide all the knowledge needed to answer questions [48, 41, 42, 18], such benchmarks do not reflect common real-world scenarios where implicit background knowledge plays an important role in reasoning. Moreover, models that translate the textual examples into logical form and then leverage off-the-shelf solvers may excel on these datasets, which does not reflect the true performance of such models in real-world applications. For these reasons, in BoardgameQA only part of the knowledge required to solve the problem is provided as input to the LM; the missing knowledge has to come from the LM itself.
The problems in our dataset are formulated as scenarios of a board game, hence we name it BoardgameQA<sup>1</sup>. A board game theme allows us to create synthetic scenarios with complex defeasible rules to reason about that seem natural when stated in text and hence allows background commonsense world knowledge to also be used. To the best of our knowledge, BoardgameQA is the first dataset for multi-hop reasoning with contradictory inputs. Figure 2 shows a sample example from the dataset where the conflict resolution and missing knowledge have been highlighted.
We benchmark various LMs on BoardgameQA and measure their defeasible reasoning capacity. Most notably, our results reveal that LMs perform poorly when reasoning with conflicting sources, especially in the few-shot setting (compared to the finetuning setting) suggesting that preference understanding and defeasible reasoning capacities do not surface out-of-the-box in pretrained LMs. Secondly, we find that smaller LMs perform poorly when not all of the required information is provided as input. These results highlight a critical gap in the reasoning capacity of current LMs, considering that reasoning over contradicting and incomplete sets of information is a common scenario in many applications, and is key for developing robust AI systems.
Our work spans three dimensions: 1- text-based logical reasoning, 2- reasoning with conflicting sources, and 3- reasoning with incomplete information. In the following section, we briefly summarize the literature on each of these axes that relate to our work.
<sup>1</sup>All dataset variations used in our experiments can be found at: https://storage.googleapis.com/gresearch/BoardgameQA/BoardgameQA.zip. A variation with depth 4 (not used in our experiments) can also be found at https://storage.googleapis.com/gresearch/BoardgameQA/depth4.zip. All variations under https://storage.googleapis.com/gresearch/BoardgameQA/ (including the ones mentioned above) are available under the CC BY license.
Facts: A few players are playing a boardgame. Here is the current state of the game. The dog has  $3. The lion has$ 48. The frog has 81 dollars, and has a knife. [...]
Rules: R1: If the frog has more money than the lion and the dog combined, then the frog builds a power plant close to the green fields of the cat. R2: If something attacks the green fields of the cat, then it does not build a power plant near the green fields of the cat. [...]
Preferences: R2 is preferred over R1. [...]
Question: Does the frog build a power plant near the green fields of the cat?
Proof: We know the frog has $81, 81 is more than 3+48=51 which is the total money of the dog and lion combined, and according to R1 [...], and for the conflicting and higher priority rule R2 we cannot prove the antecedent "the frog attacks the green fields of the cat", so we can conclude "the frog builds a power plant close to the green fields of the cat". So the statement "the frog builds a power plant near the green fields of the cat" is proved and the answer is "yes".
Figure 2: A sample example from BoardgameQA that requires one hop of reasoning. The text in violet highlights conflict resolution and the text in blue highlights the missing information.
Text-based logical reasoning approaches: Earlier works on natural language logical reasoning have finetuned LMs to directly provide answers to logical reasoning questions [11, 4, 40, 18]. Later work showed that explicitly generating the entire proof leads to substantial improvements both in the case of finetuning and in the case of few-shot learning [31, 13, 58, 60]. In addition, modular reasoning approaches where the LM is used as a tool within a reasoning algorithm [23, 12, 51, 24] have been shown to achieve both performance gains and more precise intermediate proof chains. In this paper, we experiment with four types of approaches: 1- finetuning without explicit reasoning steps, 2- finetuning with explicit reasoning steps, 3- prompt-tuning with chain-of-thought (CoT) prompting [52], and 4- few-shot in-context learning with CoT.
Text-based logical reasoning datasets: Many datasets have been created to measure the logical reasoning ability of NLP models [48, 42, 61, 18, 45]. In Table 1, we provide a comparison of (a subset of) these datasets along three desired features in this work. All datasets compared contain only facts and rules that are non-contradicting. The datasets closest to our work are CREPE [30], FalseQA [21] and ConditionalQA [47]; the first two provide false pre-suppositions in the question which can be considered as statements that contradict the ground truth, and in last one the answers to the questions follow a "If  $X$  then yes, if  $Y$  then no" format.
Reasoning with conflicts: From the early days of AI, reasoning with conflicting information has been an important topic and many approaches have been developed to handle such conflicts [34, 33, 37]. The problem we study in this paper is an instance of defeasible reasoning [33, 19, 28] which has applications in various domains (especially in legal reasoning) [43, 16, 7] and has been argued to be one of the most important future directions in a recent survey of LM reasoning literature [56]. In defeasible reasoning, there are preferences over the rules and in the case of conflict between two rules, the conclusion from the higher preference rule is accepted. Previous work on defeasible reasoning with natural language has studied the problem of adjusting the probability of a conclusion based on new (single-hop) evidence [39, 27]. Our work extends this line of work by developing a dataset for multi-hop defeasible reasoning with preferences over sources.
Reasoning with incomplete information: Several existing reasoning benchmarks adopt a setup where part of the required information is missing and needs to come from the model itself [46, 5, 2, 50]. Some datasets also employ a setup in which none of the required rules are provided as input [49, 15, 45, 22]. Our work focuses mainly on cases where part of the knowledge needs to come from the model and another part of the knowledge is provided as input.
3 Background and Notation
We let  $\mathcal{E} = \{e_1, \ldots, e_N\}$  and  $\mathcal{P} = \{p_1, \ldots, p_M\}$  represent a set of entities and predicates. We represent a fact in the logical form using the triple notation  $(e_i, p_j, e_k)$ , where  $e_i, e_k \in \mathcal{E}$  and  $p_j \in \mathcal{P}$ , and a rule as  $r : r_b \to r_h$  where  $r_b$  represents the body of the rule and  $r_h$  represents the head. We use! to indicate negation. A monotonic theory  $\mathcal{T} = (\mathcal{F}, \mathcal{R})$  is a tuple containing a set  $\mathcal{F}$  of (positive or negative) facts, and a set  $\mathcal{R} = \{r_1, \ldots, r_{|\mathcal{R}|}\}$  of rules. We let  $\mathcal{T} \models f$  represent that the fact  $f$  can be derived from the theory  $\mathcal{T}$  using the standard inference rules of logic (See Shoenfield [44]). For a monotonic theory  $\mathcal{T} = (\mathcal{F}, \mathcal{R})$ , if  $\mathcal{T} \models f$ , then for any theory  $\mathcal{T}'$  such that  $\mathcal{T}' = (\mathcal{F} \cup \mathcal{F}', \mathcal{R})$ , we also have  $\mathcal{T}' \models f$  (that is, adding new facts does not change previously derived facts).
Defeasible Theory: A defeasible theory  $\mathcal{T}^{(d)} = (\mathcal{F},\mathcal{R},\mathcal{O})$  is a triple containing a set  $\mathcal{F}$  of facts, a set  $\mathcal{R} = \{r_1,\ldots ,r_{|\mathcal{R}|}\}$  of rules, and a set  $\mathcal{O} = \{r_{t_1} > r_{t_2},\dots,r_{t_3} > r_{t_4}\}$  of pair-wise relative
Dataset → Feature ↓	bAb1 15	CLUTRR	Folio	Proof Writer	PrOntoQA OOD	AR-LSAT	ENWN	Leap of Thought	CREPE	False QA	Conditional QA	Boardgame QA
Contradictory Information	X	X	X	X	X	X	X	X	~	~	~	✓
Incomplete Information	X	~	X	X	X	X	✓	✓	✓	✓	✓	✓
Auto. Diff. Control	✓	✓	X	✓	✓	X	X	~	X	X	X	✓
Table 1: A comparison of BoardgameQA with some of the widely-used logical reasoning datasets (bAbI 15 [53], CLUTRR [45], FOLIO [18], ProofWriter [48], PrOntoQA-OOD [42], AR-LSAT [61], ENWN [46], leap-of-thought [50], CREPE [30], FalseQA [21], and ConditionalQA [47]) in terms of three key features. We use  $\sim$  in the case of incomplete information for CLUTRR because there is only a fixed set of information that needs to come from the model, in the case of automatic difficulty control for leap-of-thought because the depth of reasoning is fixed (difficulty is added through distractors), in the case of contradictory information for CREPE and FalseQA because the contradiction is with the ground truth not with another source of information, and in the case of contradictory information for ConditionalQA because while the answer to a question can be yes under one set of conditions and no under another set of conditions, the two sets of conditions are mutually exclusive.
priorities/preferences between rules. $^2$  The rules hold defeasibly, meaning the conclusion from a rule may be defeated by contrary evidence from a higher priority rule. This happens, for example, when one rule implies something is true but another rule with a higher priority implies it is false; in such cases, we accept the conclusion from the higher priority rule (see Figure 1). We let  $\mathcal{T}^{(d)}\models f$  represent that  $f$  can be derived from a defeasible theory  $\mathcal{T}^{(d)}$  after resolving conflicts. Note that the initial facts  $\mathcal{F}$  are internally consistent and always have priority over the derived facts. We assume the theory is defeasibly consistent, meaning whenever a conflict arises, the preferences can be used to resolve it. An example of a defeasible theory  $\mathcal{T}^{(d)}$  is as follows:
Example 3.1.  $\mathcal{F} = \{\text{Tweety is a penguin.}\}, \mathcal{R} = \{r_1 : \text{Penguins are birds. } r_2 : \text{Birds fly. } r_3 : \text{Penguins do not fly.}\}, \mathcal{O} = \{r_3 > r_2\}$ . From the theory, one can first use  $r_1$  to derive that "Tweety is a bird". Then, one can use  $r_2$  to derive that "Tweety flies". However, one can also use  $r_3$  to derive that "Tweety does not fly", which is in conflict with the previous derivations. Since  $r_3 > r_2$ , we accept the derivation that "Tweety does not fly".
Conflict types: Conflicts can arise for rules whose heads cannot be simultaneously true, e.g., for two rules  $r : r_b \to z$  and  $r' : r_b' \to !z$ . For a theory  $\mathcal{T}^{(d)}$  with these two rules,  $\mathcal{T}^{(d)} \models z$  in two cases: (a)  $r$  has higher priority than  $r'$  and we can prove  $r_b$ , and (b)  $r$  has lower priority than  $r'$  and we can prove  $r_b$  but we cannot prove  $r_b'$ . In the first case, one does not need to take into account  $r_b'$  for conflict resolution, but in the second case it is critical to take  $r_b'$  into account. We name the first type of conflict Type1 conflict and the second type Type2.
4 The BoardgameQA Dataset
We now describe how we create a dataset for measuring the ability of LMs in reasoning with conflicting inputs in a defeasible setup. Our dataset creation follows a backward story generation strategy similar to [55, 23]. Each example in the dataset contains a (defeasible) theory  $\mathcal{T}^{(d)}$  and a question  $q$ . The goal is to predict whether  $\mathcal{T}^{(d)} \models q$ , or  $\mathcal{T}^{(d)} \models !q$ , or neither. Therefore, the label space for each question is  $\{\text{proved}, \text{disproved}, \text{unknown}\}$ . We next describe how we generate examples with the label proved; examples with the label disproved or unknown are created by modifying the examples with label proved.
The facts of each theory describe the current state of a board game, the rules of each theory represent the rules of the board game, and the questions are about the game. In the design of BoardgameQA, we include several variables that can be used to sample examples with varying levels of difficulty with respect to several finer-grained properties (e.g., depth, number and types of conflicts).
2Note: Many types of preferences can be converted into pair-wise relative preferences.
Category	Description	Example Facts	Example Rule
Time Conversion	Compares the age of an entity to a certain age specified with different units.	The dog is 13 months and a half old	If the dog is more than a year old, then ...
Orthography	Asks about the letters in names.	The dog is named Paco. The cat is named Pashmak.	If the dog has a name that starts with the same letter as the name of the cat, then ...
Numeric Comparisons	Some numbers are required to be summed and then compared to other numbers.	The dog has two friends that are nice and five that are not	If the dog has less than 10 friends, then ...
Lexical Entailment	The fact and the rule body are not identical but the fact entails the rule body.	The dog assassinated the mayor	If the dog killed the mayor, then ...
World Knowledge	Some knowledge about the world is needed to connect the fact to the rule body.	The dog is currently in Montreal.	If the dog is currently in Canada, then ...
Event Times	Knowledge about times of events is needed to connect the fact to the rule body.	The dog is watching a movie that was released in 2005.	If the dog is watching a movie that was released after Covid19 started, then ...
Part Of	The fact and the rule body have a part of relation.	The dog is a nurse	If the dog works in healthcare, then ...
Affordance	The rule body is about a certain feature/affordance of the fact.	The dog has a knife	If the dog has a sharp object, then ...
Volumes	Knowledge of what objects fit in what other objects is required.	The dog has a ball with a radius of 15 inches.	If the dog has a ball that fits in a 28 x 35 x 35 inches, then ...
Table 2: Categories, descriptions, and examples of incomplete information in BoardgameQA. For lexical entailment, world knowledge, event times, and affordance, a list of examples is written manually from which the sampling procedure can select. In others, examples are generated automatically.
Entities and predicates: We start with a predefined set of entities  $\mathcal{E}$  (e.g., dog, cat, lion, etc.) and a predefined set of predicates  $\mathcal{P}$  (e.g., invite for dinner, attack the fields, etc.) that we sample from to generate facts and rules. We use the animals as entities and the boardgame-inspired verbs/operations as our predicates. Using these entities and predicates, we can create facts such as the dog attacks the fields of the lion. To make the problem more challenging, we use different entities and predicates across training and test similar to [25]. The full list of entities and predicates is provided in Appendix C.3.
Rule types: We adopt a set of 6 rule templates containing existential and universal quantifiers, conjunctions, and missing information. The rules are as follows:  $1 - \forall X:(X,p_1,e_1)\Rightarrow (X,p_2,e_2)$ $2 - \forall X:(X,p_{1},e_{1})\wedge (X,p_{2},e_{2})\Rightarrow$ $(X,p_{3},e_{3})$  , 3-  $(e_1,p_1,e_2)\Rightarrow (e_2,p_2,e_3)$ $4-(e_1,p_1,e_2)\wedge (e_3,p_2,e_2)\Rightarrow (e_2,p_3,e_4)$ $5 - (e_1,\hat{p},\hat{e})\Rightarrow (e_1,p_2,e_2)$  , and 6-  $\exists X(X,p_1,e_1)\Rightarrow (e_2,p_2,e_3)$  , where  $X$  represents a universally or existentially bounded variable, each  $e_i$  represents an entity, and each  $p_j$  represents a predicate. The fifth rule template corresponds to a rule where the predicate (or object entity) in the rule body may not be an element of  $\mathcal{P}$  (resp.  $\mathcal{E}$  ). For more information, see below.
Selecting a question: To generate each ex
ample, we first sample a question  $q = (e_i,p_j,e_k)$  that should be proved or disproved, where  $e_i$  and  $e_k$  are sampled from  $\mathcal{E}$  and  $p_j$  is sampled from  $\mathcal{P}$ . We also sample the sign of the question (positive or negative). For example, we might sample the question! (dog, attack the fields, lion) asking whether the dog does not attack the fields of the lion. The question is then converted into natural language using a template (see Appendix C.3).
Algorithm 1 GenerateTheory
Input: Question  $q$ , Depth  $d$
1: if  $d == 0$  then
2: addToFacts(q)
3: else
4:  $\mathcal{Q}, r = \text{SampleRuleAndSubq}(q)$ .
5: addToRules(r)
6: if CoinFlip  $(p_{Conf}) ==$  Conflict then
7:  $\mathcal{Q}', r' = \text{SampleRuleAndSub}(!q)$ .
8: addToRules(r')
9: if CoinFlip  $(p_{\text{ConfType1}}) == \text{Type1}$  then
0:  $\mathcal{Q} = \mathcal{Q} + SubSample(\mathcal{Q}')$
11: addToPreferences(r, r')
12: else
13:  $\mathcal{Q} = \mathcal{Q} + \text{RemoveOneSubquestion}(\mathcal{Q}')$
14: addToPreferences  $(r^{\prime},r)$
15: for  $q_{i}$  in  $\mathcal{Q}$  do
16: GenerateTheory(qi, d-1)
Algorithm 2 SampleRuleAndSubq
nput: Question  $q$
1:  $r = \text{SampleQuestion}()$
2: if  $r$  is a rule with incomplete info (type 5) then
3: Sample  $\mathcal{Q} = \{q_1,\dots,q_n\}$  and  $\hat{\mathcal{Q}} = \{q_1',\dots,q_m'\}$  s.t.  $q$  can be derived from  $\hat{\mathcal{Q}}$  and  $r$ , and  $\hat{\mathcal{Q}}$  can be derived from  $\mathcal{Q}$ .
4: else
5: Sample  $\mathcal{Q} = \{q_1,\dots ,q_n\}$  s.t.  $q$  can be derived from  $\mathcal{Q}$  and  $r$
6: return  $\mathcal{Q}, r$