NLI under the Microscope: What Atomic Hypothesis Decomposition Reveals (2025)

Abstract: Decomposition of text into atomic propositions is a flexible framework allowing for the closer inspection of input and output text. We use atomic decomposition of hypotheses in two natural language reasoning tasks, traditional NLI and defeasible NLI, to form atomic sub-problems, or granular inferences that models must weigh when solving the overall problem. These atomic sub-problems serve as a tool to further understand the structure of both NLI and defeasible reasoning, probe a model’s consistency and understanding of different inferences, and measure the diversity of examples in benchmark datasets. Our results indicate that LLMs still struggle with logical consistency on atomic NLI and defeasible NLI sub-problems. Lastly, we identify critical atomic sub-problems of defeasible NLI examples, or those that most contribute to the overall label, and propose a method to measure the inferential consistency of a model, a metric designed to capture the degree to which a model makes consistently correct or incorrect predictions about the same fact under different contexts.

## introduction atomic decomposition involves breaking sentences down into atomic propositions, or granular facts that are explicitly supported by the original text. this style of decomposition has widespread applications, including assessing the factual precision of generated text @xcite , claim verification @xcite , and multihop qa @xcite , since it allows for careful, finergrained inspection of text. we use atomic decomposition as tool to dive deeper into two types of natural language reasoning: traditional nli @xcite and defeasible inference @xcite , a mode of reasoning where inferences may change in light of new evidence. in both tasks, atomic decomposition of hypotheses into atoms breaks complex atomic hypothesis decomposition natural language inference defeasible inference (δ-nli) updates (u) hypothesis (h) a 1 a 2 a 3 no effect strengthens it begins to snow during their performance. a 1 a 2 a 3 no effect no effect weakens the people are actors. the people are professional. ## atomic sub-problems in nli reasoning about situations often involves weighing multiple pieces of information to draw inferences. consider the last row in hypotheses in snli, and in turn, δ-snli, can be complex sentences, and while solving inference problems, models must weigh all pieces of information in both p and h. we expect humans to make inferences about constituent pieces of information in a manner that is consistent with their overall judgment, an equally desirable property in models. not only does it signal holistic understanding of the situation described in the problem, but it can help pinpoint exactly what types of inferences models struggle with. identifying atomic sub-problems also allows us to understand the granular inferences that are evaluated in benchmark datasets. in turn, this helps to understand the diversity in the dataset: despite there being thousands of examples, certain inferences may come up repeatedly. to identify the constituent sub-problems, we break hypotheses in snli and δ-snli into atomic propositions @xcite to use in subsequent analyses. each atomic decomposition represents a single piece of information. formally, given an snli example with p and h, we generate atomic decompositions of h represented by a 1 ...a n . each atomic sub-problem then involves predicting the relation between a (p , a i ) tuple ( §4.1). given a δ-nli example with p , h, and u , atomic sub-problems involve a (p , a i , u ) tuple where the task is to determine whether u strengthens, weakens, or has no effect on a i ( §5). ## atoms in traditional nli while performing strongly on various benchmarks, llms still struggle with many types of consistency including paraphrastic consistency @xcite , hypothetical consistency @xcite , or even preferential consistency @xcite . when llms make entailment judgments, another desirable property is logical consistency. namely, when an llm itself deems a set of atoms a 1 ...a n entailed by h, we can hold it accountable to maintain consistency between its judgments on each (p , a i ) sub-problem and its overall (p , h) judgment in a logical way. this gives us necessary, but not sufficient, evidence to help signal that it has "understood" the situation. ## atoms in defeasible inference we now turn to defeasible inference to explore how atomic sub-problems can help us better understand the complexities of the task, model performance, and the knowledge evaluated in the δ-snli dataset. in traditional nli ( §4), labels of atomic subproblems function like terms in an equation-the overall relation between p and h can be computed from strict logical rules over relations between p and individual a i . in contrast, defeasible inference functions akin to fuzzy logic @xcite . determining the overall effect of the update u on h involves a softer weighing of the direction and magnitude of its effect on each atom. for example, consider the second row in ## measuring inferential consistency in δ-nli as established, identifying the critical atom(s) of a δ-nli example allows us to pinpoint the knowledge or fact that the example is designed to evaluate ( §5.2). the two different examples in figure 4 share the same critical atom, representing two different contexts under which the model must directly evaluate the fact "the people are friends." a model correctly predicting whether or not people are friends under some contexts but not others may indicate that it has not fully understood the factors that influence the inference. because they contain independent examples, few datasets accommodate measuring a model's inferential consistency (i c ), or the likelihood that its prediction for a particular inference will remain consistently correct or incorrect under different contexts (here, contexts refers to different (p, u ) pairs). correctly drawing an inference under a single context does not guarantee that the model will make a correct prediction for the same inference under a different context. to quantify this, we group examples in δ-snli-test by their critical atom and report the inferential consistency of different models. the 1,761 δ-snli-test examples that contain at least one valid atom (from 193 unique p -h pairs) contain 429 unique atoms. 4 following the procedure above yields 349 unique cliques, or 349 unique critical atoms. 5 we bucket examples by these cliques, resulting in groups of examples that share a critical atom (which we call critical atom in a4, since donning a city uniform is likelier at spill sites. ## conclusion decomposing hypotheses of nli and defeasible nli problems to form atomic sub-problems allows us to measure the logical consistency of models as well as better understand the structure of nonmonotonic reasoning. we find that labels of atomic sub-problems in defeasible reasoning share a more complex relation to the full problem than in traditional nli, even within the same label class. we introduce critical atoms as the primary fact or piece of knowledge evaluated by a defeasible nli example, enabling us to group examples by shared critical atoms and measure the inferential consistency of llms across different contexts.