There’s No Such Thing as Simple Reasoning for LLM s (2025)

Abstract: Large Language Models (LLMs) have been widely found to struggle with logical reasoning, where even fine-tuned models fail dramatically on out-of-distribution problems. However, existing work has focused on relatively complex “many-hop” reasoning problems. In this paper, we analyse the performance of fine-tuned LLMs on simple reasoning problems, all of which can be solved in at most three inference steps. Due to the simplicity of these problems, the model cannot encounter test problems that are fundamentally different from those it has seen during training. Unfortunately, however, we find that the models remain highly brittle, being susceptible to seemingly innocent perturbations, such as the addition of duplicates to the set of premises and shuffling the order in which the premises are presented.

## introduction in its simplest form, logical reasoning involves inferring logical consequences by combining the information that is encoded in a given set of premises. consider, for example, the following premises: (i) alice is not judgmental or alice is sincere; (ii) alice is judgmental or alice is timid; (iii) alice is not timid. premises (i) and (ii) imply: (iv) alice is sincere or alice is timid. combining (iv) and (iii), we can furthermore derive: (v) alice is sincere. in this example, deriving (v) required two derivation steps. in general, logical reasoning problems may involve a large number of such derivation steps. several recent studies have shown that the logical reasoning abilities of large language models (llms) are surprisingly limited @xcite @xcite . while it is possible to fine-tune llms to improve their reasoning abilities, the resulting models struggle with out-of-distribution test examples and they are typically sensitive to language variance @xcite @xcite figure 1 : a simple reasoning problem where the hypothesis can be inferred by combining two premises. however, gpt-4o fails to answer correctly. ## related work evaluating the reasoning abilities of llms has been the subject of an extensive line of work, which has covered, among others, deductive @xcite @xcite , abductive @xcite , commonsense @xcite , and symbolic reasoning @xcite . a common finding is that the reasoning depth, i.e. the number of inference steps that need to be chained to arrive at the answer, is a strong predictor of problem difficulty @xcite . in particular, even fine-tuned llms fail to generalize to problems that require longer inference chains than those in the training set @xcite . chain-of-thought prompting @xcite and providing in-context demonstrations @xcite have been found to improve reasoning abilities. however, such techniques are less effective for smaller models, and they come with their own limitations @xcite @xcite . reasoning models, such as openai o1 and deepseek r1, can overcome some of these limitations, but at the expense of a significant computational overhead @xcite . a practical solution is to use llms to convert problem instances to symbolic inputs for external solvers, and thus avoid relying on the llm to solve the reasoning problems themselves @xcite . while this may indeed be desirable for complex reasoning problems, we would still expect llms to be capable of making simple logical inferences, as this capability may be needed for general language understanding (e.g. for resolving ambiguities). to the best of our knowledge, only few works have focused on simple reasoning problems. @xcite showed that llms can sometimes fail on simple problems, even if they perform well on harder ones. a study on simple math reasoning tasks involving two or three operands further underscores the sensitivity of llms when exposed to variations in numerical and textual framing @xcite . logicasker @xcite evaluates the basic reasoning skills of llms, but their focus is on off-the-shelf models such as gpt-4 and gemini 1.5, whereas we focus on the potential of fine-tuning models on simple reasoning tasks. ## experimental setup the problems we consider involve a set of premises and a hypothesis, all of which are natural language verbalizations of formulas from propositional logic. the task is to predict whether the hypothesis is logically entailed by the premises (w.r.t. the usual semantics of propositional logic). an example of a problem instance is shown in figure 1 . the verbalizations are obtained using simple templates, which makes the statements easy to parse and allows us to focus specifically on the reasoning abilities. training data to fine-tune our llms, we use a dataset that is obtained as follows. 1 we first randomly sample a set of clauses {ϕ 1 , ..., ϕ m }. each clause is a literal or a disjunction of two literals, where a literal is either an atomic proposition or the negation of an atomic proposition. each time a clause is added to the set, we use a sat solver @xcite to check that it remains satisfiable; if not then the clause is discarded and we repeat the sampling process. after the full set of clauses has been sampled, we randomly sample one literal as the hypothesis. further details on the sampling algorithms are explained in appendix a. note that these problem instances are particularly straightforward. for instance, because at most two literals appear in each clause, they fall within the polynomial fragment of propositional logic. we further restrict the difficulty of the problem instances by avoiding problems with an inference depth of more than 3. specifically, for a positive instance, where the hypothesis is logically entailed by the premises, we define the inference depth as the number of inference steps which are needed to derive the conclusion. an inference depth of k means that the hypothesis is entailed by a subset of k + 1 premises. inference depth 0 thus means that the hypothesis is included in the set of premises. for negative examples, the inference depth is defined based on the immediate consequence operator. specifically, starting from the given set of premises, the immediate consequence operator adds all premises that can be derived in one step. the inference depth of a negative problem instance is defined as the number of times that the immediate consequence operator can be applied before the set of premises saturates. for instance, an inference depth of 0 means that all clauses that are logically entailed by the set of premises are already included in the set. intuitively, problems with a higher inference depth are more challenging. we ensure that the training data is balanced in two ways. first, we ensure that there is an equal number of problems of depths 0, 1, 2 and 3. second, for each inference depth, we ensure that there is an equal number of positive and negative examples. the training set consists of a total of 100k problem instances, where the number of atomic propositions in each problem instance varies from 2 to 15, and the number of premises from 2 to 50. a further 10k problem instances are sampled for validation. test data we experiment with three different test sets. first, we sample a set of 1000 problem instances following the same process as for the training data. we will refer to this test set as sat. to test the robustness of the model to out-of-distribution instances, we also generate test sets following the rule priority (rp) and label priority (lp) sampling strategies that were proposed by @xcite . for both strategies, we sample 1000 problem instances. one key difference with sat is that rp and lp only include horn clauses. specifically, the clauses in these two test sets are either positive literals (i.e. atomic propositions), or clauses of the form ¬p ∨ q, where p and q are atomic propositions. ## results and analysis a number of clear observations can be made. first, gpt-4o and the distilled deepseek-r1 model significantly underperform the fine-tuned models. in the case of gpt-4o, using cot prompting somewhat improves results, although the overall performance remains surprisingly disappointing, given the simplicity of the considered problem instances. for example, even in cases where the hypothesis is listed among the premises (i.e., @xmath0, the model still make mistakes in a non-trivial number of cases. furthermore, for the lp dataset with @xmath1, the results for gpt-4o with standard prompting and the distilled deepseek-r1 model are both around random guessing (i.e., 50%). as a more powerful closed-source model, o3-mini performs better, although it still makes mistakes, even in the simplest settings. we noticed a slight improvement in o3-mini with cot, particularly for problems with a higher depth. we discuss the performance of o3-mini further in appendix c.2. the fine-tuned models achieve perfect accuracy for @xmath2. however, for @xmath3, we see a clear deterioration in performance. comparing the different models, we can see that the smallest models (llama-3.2 and phi-4-mini) underperform, while llama-3.1 and ministral-8b generally perform best. it is notable that all fine-tuned models perform considerably worse on rp and lp. while previous work has already highlighted the fact that fine-sat rp lp @xmath4 ## conclusions this paper investigated the ability of llms to reliably and consistently perform simple reasoning. we found that even on problems that only require a single reasoning step, fine-tuned llms make some mistakes. moreover, while such mistakes are rare on in-distribution examples, we found that the performance can deteriorate dramatically when seemingly innocent changes are made to test problems. in particular, we highlighted the effect of duplicated premises and changing the order where literals are positioned. while it is possible to alleviate the specific issues that we identified by adapting the training data, the fact that these issues arose in the first place strongly suggests that fine-tuned models rely on reasoning shortcuts, even for the very simple reasoning problems that we have considered. as such, model performance should be expected to deteriorate whenever out-of-distribution problem instances are encountered.