Out-of-Context Reasoning in Large Language Models (2025)

Abstract: We study how large language models (LLMs) reason about memorized knowledge through simple binary relations such as equality ( = ), inequality ( < ), and inclusion ( \subset ). Unlike in-context reasoning, the axioms (e.g., a < b, b < c ) are only seen during training and not provided in the task prompt (e.g., evaluating a < c ). The tasks require one or more reasoning steps, and data aggregation from one or more sources, showing performance change with task complexity. We introduce a lightweight technique, out-of-context representation learning, which trains only new token embeddings on axioms and evaluates them on unseen tasks. Across reflexivity, symmetry, and transitivity tests, LLMs mostly perform statistically significant better than chance, making the correct answer extractable when testing multiple phrasing variations, but still fall short of consistent reasoning on every single query. Analysis shows that the learned embeddings are organized in structured ways, suggesting real relational understanding. Surprisingly, it also indicates that the core reasoning happens during the training, not inference.

## introduction a large number of works have investigated the reasoning capabilities of large language models (llm), spanning from math @xcite , logic @xcite , planning @xcite @xcite , and, more recently, multi-agent problem solving @xcite . the empirical evidence suggests that the larger a model and its associated training data, the more capable the llm is at handling unseen problems @xcite . complex problem-solving relies on the capabilities of a model to decompose a problem into its sub-components and, similarly to a puzzle, provide the correct answer by integrating the solutions from each sub-task. this principle, known as compositionality @xcite , relies on the assumption that llms possess a core set of capabilities to solve each sub-task with low error probability. most existing benchmarks focus on in-context reasoning, where the necessary information is explicitly provided within the prompt @xcite . this approach offers insights into how models process information presented at inference time. yet, in-context learning does not assess the ability of llms to reason out-of-context, i.e., based on memorised knowledge encountered only during training and that does not appear in the prompt. while several works have explored out-ofcontext learning @xcite @xcite , they primarily focus on complex/high-order tasks, making it hard to identify the reasons behind a model's success or failure. the most closely related work to ours is @xcite , where the authors explored llms' difficulty with reversal relations, summarised by its title "llms trained on 'a is b' fail to learn 'b is a'". the verb "is" can be interpreted as a binary relation or as a verb, introducing a confounder that makes it complex to judge whether a model captures the core properties of transitivity. while in logic, "a is b" implies "b is a", from a linguistic perspective, "students are humans" poses some issues in deriving that "humans are students", and it is thus hard to impute a model's failure to its inability to handle symmetry properly. in addition, they tested whether a model generates a given b, instead of whether "b is a" evaluates true, implicitly assuming that only high-probability predictions are those that a model considers correct. figure 1 (left) illustrates this issue. we are not aware of works that tested llms on other binary relations (beyond 'is') or other properties (beyond symmetry). motivated by this gap in the current literature, we study how well llms handle binary relations, a core concept in math that appears frequently in most problems llms excel at solving when provided with sufficient in-context information @xcite @xcite . our research aims to ground the extent to which llms can reason out-of-context, specifically focusing on logical, relational reasoning. as sketched in figure 1 , we propose an out-of-context representation learning technique that introduces new tokens into a model's vocabulary and trains only their representations while leaving the other parameters unchanged. by training only the representation of unseen tokens, out-of-context representation learning allows us to ( 1 ) understand what reasoning capabilities are present in a model and (2) without relying on external guidance, e.g., in-context learning and/or illustrations @xcite . it also makes analysing the learned parameters much easier. for the rest of the paper, we will refer to our technique as out-of-context representation learning, while (whole model) fine-tuning, which trains the entire model and is also called out-of-context learning, will be referred to as fine-tuning, to avoid confusion. our experiments assess the models capabilities on binary relations and their basic properties, such as reflexivity, symmetry, and transitivity. in summary, in this paper, we: the following sections review the current literature and formally introduce the binary relations and properties that we test. ## related work several papers have explored out-of-context learning in llms. for example, @xcite trained gpt-2 in synthetic biographies and then tested its ability to answer fine-tuning questions about specified details. the model performed well after fine-tuning on such questions for biographies not included in the test set. @xcite advanced this approach by testing questions requiring reasoning, such as determining if someone was born in an even year. while the model performed well on simple tasks (e.g., even or odd birth years) after some fine-tuning, it struggled with more complex questions requiring operations like comparison or subtraction, performing only marginally better than random guessing regardless of fine-tuning. @xcite tested similar capabilities and reported poorer results, potentially due to a lack of fine-tuning or paraphrasing in the training data. @xcite investigated whether llms could make inferences from information spread across distinct training data, concluding that llms can sometimes actually perform better when fine-tuned than with in-context reasoning. @xcite explored llms' difficulty with reversal relations, summarised by its title "llms trained on a is b fail to learn b is a". recent works explain this phenomenon as an intrinsic limitation of transformers architectures at maintaining consistent relation between the subject and the predicted object @xcite . similar findings are reported in the previously mentioned paper @xcite . somewhat differently, but taking a more formal approach, @xcite evaluates the capability of llms to replicate welldefined properties such as symmetry on relevant data (e.g., if a model knows that barack obama is married to michelle obama, does it know that michelle obama is married to barack obama?). however, they do not train the model on new, synthetic data, and it may well be that both directions exist in the training data. more papers took similar approaches, mainly with multi-hop reasoning @xcite @xcite . ## methodology this section introduces the basic notation to describe a binary relation and an llm. we then describe out-of-context representation training methodology and how it differs from standard finetuning and in-context learning. we conclude the section with a brief overview of the dataset format. binary relation. we focus on three binary relations, namely equality ("), inequality (ă), and inclusion (ă). each binary relation satisfies/violates several properties that are the object of our study, for example, reflexivity, symmetry, and transitivity, as well as other properties such as irreflexivity. for a finite set of elements e, the cartesian product e ś e identifies ordered pairs that satisfy a particular relation r. ## experiments in our experiments, we train llama-3-8b, llama-3.2-1b @xcite and mistral-7b-v0.3 @xcite with out-of-context representation learning, as introduced in eq. 1. the experiments for in-context learning are similar, except that the same axioms are introduced in the prompt instead of the training set, and only the minimal setting is used. for each relation, namely strict total order, equality, and proper subset, we craft a training dataset that tests the model's capability to handle one or more properties of such a relation. the llm is then tested on some questions that do not appear in the training, but for which the training set provides sufficient knowledge to solve them correctly. each evaluation contains both true and false statements (i.e., the expected ground truth answer is [yes/no]), expressed with different phrasing to enhance variety. we run each experiment 10 times with different initial random embeddings, then average the results. the out-of-context representation learning paradigm is implemented by introducing a new set of tokens (each paired with a dense representation), in the llm's vocabulary, and by training only these representations. while other works employ chain of thoughts @xcite to test the reasoning capabilities of llms @xcite , we do not employ it as the training does not contain any reasoning chain. future works can address this limitation and check whether a model can generate chains of thought while not being explicitly trained to do so. in the next sections, we first introduce the salient details of each binary relation alongside the implementation details; we then discuss the results of our evaluation. ## results the average results over all experiments are summarised in otherwise, we say that the results are inconclusive. hence, for the minimum variation, the overall models' performance lies between the baseline and random guess, hence are inconclusive; and the overall results for the illustrative settings are better than both the baseline and random guess, for all models. for the illustrative settings, llama-3-8b's accuracy is better than that of llama-3.2-1b, which is better than that of mistral-7b-v0.3. for the minimum settings, llama-3-8b and mistral-7b-v0.3 score the same, and llama-3.2-1b yields slightly worse results. a more detailed analysis of the performance on every experiment follows. ## conclusions this paper explores the ability of llms to reason about binary relationships through out-of-context representation learning. we assessed whether llms can generalise reasoning beyond in-context learning by examining relational properties such as reflexivity, symmetry, and transitivity, on knowledge the model encountered only during training. our findings indicate that out-of-context representation learning allows for better generalisation in most tasks we tested. we show that when the models succeed, they do so by arranging the learned embeddings according to the task. future research will test the robustness of outof-context representation learning against data contamination by repeating the experiments on model trained on plain text version of our experiments.