Belief Revision: The Adaptability of Large Language Models Reasoning (2024)

Abstract: The capability to reason from text is crucial for real-world NLP applications. Real-world scenarios often involve incomplete or evolving data. In response, individuals update their beliefs and understandings accordingly. However, most existing evaluations assume that language models (LMs) operate with consistent information. We introduce Belief-R, a new dataset designed to test LMs’ belief revision ability when presented with new evidence. Inspired by how humans suppress prior inferences, this task assesses LMs within the newly proposed delta reasoning ( \Delta R ) framework. Belief-R features sequences of premises designed to simulate scenarios where additional information could necessitate prior conclusions drawn by LMs. We evaluate ~30 LMs across diverse prompting strategies and found that LMs generally struggle to appropriately revise their beliefs in response to new information. Further, models adept at updating often underperformed in scenarios without necessary updates, highlighting a critical trade-off. These insights underscore the importance of improving LMs’ adaptiveness to changing information, a step toward more reliable AI systems.

## introduction human reasoning is characterized by its ability to deal with partial or evolving information. when new information becomes available, we dynamically update our beliefs. we reevaluate and adjust our initial premises or conclusions as necessary in light of this new evidence @xcite . for instance, knowing tweety is a bird, we conclude that it flies since birds usually fly. discovering tweety is a penguin, we retract the conclusion but not the other premises; we still believe tweety is a bird and that birds typically fly, however, we now conclude that it cannot fly since we know that penguins cannot fly. this form of reasoning permits new information to undermine prior beliefs, which necessitates the ability of belief revision @xcite @xcite . the ability to adjust beliefs allows better adaptability of ai systems by enabling them to properly revise prior inferences as further evidence emerges, such as in commonsense inferences @xcite @xcite and decision-making @xcite . despite this, recent reasoning evaluations of state-of-the-art ai technologies, such as language models (lms), primarily focus on its ability to draw conclusions assuming complete information (c.f. @xcite ; @xcite ; @xcite ). while these evaluations useful to demonstrate the reasoning abilities of lms, they fail to capture the concept of belief change. we introduce belief-r, the first-of-a-kind diagnostic reasoning evaluation dataset designed to assess inferences involving belief revision. belief-r is inspired by the concept of the suppression task @xcite which enables the retraction of previously inferred beliefs by the introduction of new contextual premises, mimicking how humans reassess their inferences when presented with additional context. to allow a specific and measurable evaluation on belief revision, we introduce a new reasoning evaluation setting dubbed as delta reasoning (∆r) framework. within ∆r, evaluation is done within two sequential reasoning steps. we start by presenting lms with two initial premises that satisfy basic logical inference rule to assess its basic inference ability. we expect the model to make accurate inferences to establish the prior beliefs. then, we introduce another premise to see if the model adjusts its beliefs or keeps them unchanged, depending on the significance of the newly introduced information to the initial beliefs. belief-r is specifically designed to support the belief revision evaluation through the ∆r framework. each sample in belief-r is equipped with two initial premises that support basic modus ponens or modus tollens inferences, and a new premise that brings in new information that might modify previously held beliefs. we synthetically generate the premises in belief-r leveraging on publicly available dataset, and manually annotate the new information significances along with the ground truth answers through multiple human annotators and majority voting. as illustrated in figure 2 , belief-r uniquely facilitates thorough evaluations of belief revision capabilities. through belief-r, we evaluate the belief revision ability of small and large scale lms using different prompting techniques. our study shows that these models are incapable of revising their prior beliefs. we further reveal a critical limitation: they confront a performance trade-off between updating and maintaining their prior beliefs. models that perform better in the cases where an update is needed, typically faltered on the other. further-more, better prompting methods also fail to significantly enhance this capability. these insights underscore a need for strategies to enhance model's capability to correctly update or maintain its initial beliefs when faced with new evidence to ensure its reliability across evolving scenarios. ## related works belief revision belief revision is the process of changing beliefs to take into account a new piece of information. in ai systems, one of its early implementation is through procedures by which databases can be updated, i.e. for recording and maintaining reasons for system beliefs @xcite @xcite . @xcite created formal frameworks to determine how beliefs should be updated in a rational manner. the core challenge in belief revision is deciding rationally which prior beliefs to modify, retain, or discard when confronted with new evidence @xcite . consequently in this paper, we look at how lms handle belief revision. belief in lms can be thought of as models' output @xcite @xcite . several works revise lms' beliefs through updating its parameter directly or via finetuning @xcite @xcite . however, this process is not a rational process of the model itself @xcite . moreover, it relies on pre-prepared knowledge, which is not ideal if we envision lms to help with discovering new things @xcite . in this work, we assess lms' belief revision capabilities through its response towards queries that neccessitate judgement on whether it needs to update its prior beliefs or keep it. language model reasoning evaluation reasoning is one of the fundamental intelligent behaviors, essential for solving complex real-world tasks @xcite . one works test this behaviour by creating simple tasks to comprehensively check if a system can answer questions by connecting facts or using basic logic @xcite . others design more advanced tests to evaluate inductive, deductive, and abductive reasoning @xcite @xcite . some benchmarks replicate real-world complexities by presenting partial or conflicting informations @xcite @xcite . belief revision focuses on the adaptability problem: whether the model properly revises prior beliefs as new information emerges. we extend the reasoning evaluation by focusing on scenarios where information evolves, presenting queries that require dynamic updates of prior beliefs in light of new evidence. this is distinct from other existing reasoning tasks involving incomplete or contradictory information, i.e. in @xcite @xcite , since they assume a static environment and focus on filling the gap or resolving the contradiction. we further note the comparison in ## belief revision belief revision is the ability to adapt the reasoning process in response to new information. this capability is critical as it ensures rational decisionmaking in the face of incomplete and evolving nature of available information @xcite @xcite . in this section, we introduce the concept of belief revision and its notation, and propose the evaluation framework for belief revision capabilities. ## the belief-r dataset belief-r is designed to specifically assess the belief revision capability through the ∆r framework. to account for this, we adopt a reasoning task that has been extensively studied in cognitive science: the suppression task @xcite . typically, this task employs a trio of premises γ 1 , γ 2 , γ 3 that accompanied by three possible conclusions, i.e. as exemplified in figure 2 for modus ponens: (a) she will study late in the library (q), (b) she will not study late in the library (¬q), and (c) she may or may not study late in the library (3q ∧ 3¬q; here the symbol 3 expresses possibility, 3q can be read as "possibly q"). at step t, we form a query χ t using the first two premises, γ 1 and γ 2 . these two premises are the premises that respectively satisfy the modus ponens or modus tollens conclusion, (@xmath0, p) or (@xmath1, ¬q). these logical rules are basic, and we generally expect that most reasoners can apply them accurately. next, at step t+1, to form the query χ t+1 , we introduce a third premise γ 3 which is another conditional statement @xmath2. the addition of γ 3 brings in new information that might conflict previously held beliefs. the new information in r can be seen either as adding more requirements or providing an alternative pathway, i.e. to reach the same modus ponens conclusion q. for instance, if γ 3 states if the library stays open then she will study late, we now view r: the library stays open as another additional requirement on top of p. in such cases, just knowing p alone isn't enough to conclude q: we also need r to be true, thus the condition now becomes p ∧ @xmath3. in this case, we retract the prior modus ponens conclusion q, and infer the new conclusion 3q ∧ 3¬q. we refer to this subset of dataset as the "belief update" (bu) category. however, in another case, γ 3 could instead states if she has textbooks then she will study late. in this case, r stands as a separate alternative inference path that also leads to q, thus p ∨ @xmath4. here, p still directly leads to q, and the acknowledgement of r doesn't affect this pathway, enabling prior conclusion to still hold. we call this subset as the "belief maintain" (bm) category. in belief-r, the task requires the model to perform multi-step reasoning to manage the relevance of information within r and decide if it needs to update its prior beliefs at step t or not. the model must discern the implicit commonsense and causal links amongst given premises to identify how p and r are related, determining if their interaction is conjunctive (p ∧ r) or disjunctive (p ∨ r). based on the relationships, reasoner needs to determine whether to update its initial conclusion q if the new information r imply an additional requirement for its prior beliefs to hold (p ∧ r), or to maintain its prior beliefs if r simply serves as alternatives (p ∨ r). to quantitatively measure the model's reasoning accuracy, we provide multiple choices and ask it to pick the most plausible conclusion. for instance, in examples shown in figure 2 , we would expect lms to choose options (c) and (a) for each scenario, which aligns with the majority choices made in the original study @xcite . ## result and analysis smaller models fail even on basic logical reasoning tasks. we start by examining the inferences through the first two premises in belief-r. in figure 3 , we find as the number of parameters in lms increases, their ability to handle basic logical inference improves. smaller models, @xmath5, struggle with these tasks, scoring close to the majority baseline. @xmath6, surpassing 75% accuracy. pre-trained lms @xmath7, while instruction-tuned lms show an emerging ability from 2.7b parameters achieving significantly higher performance @xmath8. llms are incapable of revising their prior beliefs. we group our further exploration on these lms that performed well (≥ 75% accuracy) in basic logical inference, and evaluate their average performance in belief maintain (bm) and belief update (bu) subsets. despite being a strong reasoner on simple logic, all larger-scale lms under study fail to perform well on these subsets of belief-r. in evaluation shown in figure 4 , most of the non-api based models perform almost 0% in bu-acc, indicating their inability on performing belief revision. we observe that all larger-scale both open-source and commercial llms perform better on the belief revision tasks, but their performances are still very limited, achieving at most ∼50% on breu. llms confront a trade-off between updating and maintaining their prior beliefs. we discover a trade-off between bu-acc and bm-acc: models performing well on one subset typically faltered on the other, especially in models where the bu-acc is not close to 0% (see fig 4 ). this indicates a potential tension between enhancing specific capabilities, as improving one aspect could inadvertently weaken another. an ideal model would excel at belief revision by consistently making the right decision on whether the new information conflicts with prior beliefs or aligns with them. this underlines the importance of developing strategies that refine the ability to revise beliefs accurately, ensuring its reliability across various scenarios. ## discussion belief revision is harder in a more complex task with modus tollens inferences. we compare llms' belief revision capabilities in average, through tasks with modus ponens and modus tollens rule as the basic logical inferences at step t. as observed in figure 5a , llms show reduced breu score in tasks with modus tollens rule. this is expected, as modus tollens is inherently more difficult relative to modus ponens as it require backward directions of reasoning and it involves reasoning with negations @xcite @xcite . furthermore, in tasks involving modus tollens inference, we observe a notably higher bu-acc compared to a much lower bm-acc. this disparity suggests that executing accurate belief revision becomes more challenging in complex tasks: decisions to update or maintain beliefs are less clear-cut in these scenarios compared to simpler tasks. belief update on abstract concept is more challenging for llms. we examine llms' belief revision capabilities in average, when dealing with scenarios involving causal relationships on events and mental states effect entities and note them in figure 5b . while the breu score is similar, llms demonstrate tendency towards maintaining their beliefs in mental state effects instead of updating them. this may stem from the challenge of recognizing additional requirements implied from the third, mental state-related, premise which is inherently more abstract and less directly observable than a concrete sequential event. better prompting method does not help on belief revision. we explore how different prompting methods affect belief revision abilities of llms on average. figure 5c shows that cot, which encourages llms to elicit reasoning steps, does not significantly enhance belief revision. while this may stem from its vulnerability to missing-step errors @xcite , attempts to correct these errors with the ps prompting offer minimal benefits, improving only by ∼1% of breu. this suggests the ability to revise beliefs could still be absent despite elicitation of reasoning steps. we put a more detailed analysis in appendix c.2. ## conclusion the ability to reason and adapt to changing information is crucial for nlp applications in the real world. most evaluations assume static knowledge environment, which does not prepare models for dynamic real-life scenarios. to address this, belief-r is introduced as a diagnostic dataset for evaluating belief revision capability in lms. through belief-r and a novel evaluation framework for evaluating reasoning in a dynamically evolving environment, ∆r, we reveal that current models struggle with updating their beliefs in response to new information, highlighting the need for improved adaptability and reliability. by exposing these limitations, our work underscores the imperative of developing ai models capable of reasoning adeptly with evolving data, thereby propelling them closer to real-world applicability and robustness.