Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
Authors: Michael A. Lepori
Jennifer Hu
Ishita Dasgupta
Roma Patel
Thomas Serre
Ellie Pavlick
Venue: ICLR 2026 Poster (2026)
Keywords: LLMs
cognitive science
interpretability
common sense reasoning
Primary area: applications to neuroscience & cognitive science

Abstract: Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able to discern the modal category of a sentence (i.e., whether it describes something that is possible, impossible, completely nonsensical, etc.). However, recent studies have called into question the ability of LMs to categorize sentences according to modality. In this work, we identify linear representations that discriminate between modal categories within a variety of LMs, or modal difference vectors. Analysis of modal difference vectors reveals that LMs have access to more reliable modal categorization judgments than previously reported. Furthermore, we find that modal difference vectors emerge in a consistent order as models become more competent (i.e., through training steps, layers, and parameter count). Notably, we find that modal difference vectors identified within LM activations can be used to model fine-grained human categorization behavior. This potentially provides a novel view into how human participants distinguish between modal categories, which we explore by correlating projections along modal difference vectors with human participants' ratings of interpretable features. In summary, we derive new insights into LM modal categorization using techniques from mechanistic interpretability, with the potential to inform our understanding of modal categorization in humans.

Published as a conference paper at ICLR 2026
Is THIS JUST FANTASY? LANGUAGE MODEL REPRESENTATIONS REFLECT HU- MAN JUDGMENTS OF EVENT PLAUSIBILITY
Michael A. Lepori'
Jennifer Hu
Ishita Dasgupta Google DeepMind
Department of Computer Science Brown University
Department of Cognitive Science Johns Hopkins University
Roma Patel
Thomas Serre
Ellie Pavlick
Google DeepMind
Department of Cognitive & Psychological Sciences Brown University
Department of Computer Science Brown University
ABSTRACT
Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplish these tasks, LMs must be able to discern the modal category of a sentence (i.e., whether it describes something that is possible, impossible, completely nonsensical, etc.). However, recent studies have called into question the ability of LMs to categorize sentences according modality (Michaelov et al., 2023 Kaul et al. 2023). In
this work, we identify linear representations that discriminate between modal categories within variety of LMs, or modal difference vectors. Analysis modal difference vectors reveals that LMs have access more reliable modal categorization judgments than previously reported. Furthermore, we find that modal difference vectors emerge in consistent order models become more competent (i.e., through training steps, layers, and parameter count). Notably, we find that modal difference vectors identified within LM activations can be used to model fine-graina human categorization behavior. This potentially provides a novel view into how human participants distinguish between modal categories, which we explore by correlating projections along modal difference vectors with human participants' ratings of interpretable features. In summary, derive new insights into LM modal categorization using techniques from mechanistic interpretability, with the potential inform our understanding modal categorization humans.
INTRODUCTION
Language models (LMs) trained internet-scale data have demonstrated great success in answering questions about our world (Brown et al. 2020), often displaying a surprising understanding of the physical laws governing it (Gurnee & Tegmark 2024). However, much of the content the internet does not accurately reflect the world that we live - overrepresents unlikely events (Gordon & Van Durme 2013). contains innumerable pages of text about fantastical fictional universes, and even contains completely nonsensical sentences (e.g., of the colorless green ideas sleep furiously variety (Gulordava et al., 2019), or in the collected lyrics of The Beatles). This raises the question: How do LMs determine whether sentence describes actual reality, hypothetical scenario, something more inconceivable i... how do LMs map linguistic expressions to their corresponding modal categories?
"Corresponding author; mi chael 1epori @brown, edu

---

Published as a conference paper at ICLR 2026
Understanding how LMs represent modal categories is essential for at least two reasons. First, LMs are increasingly deployed as knowledge bases (Petroni et al. (219), including in high-stakes situations (Magesh et al. 2024 Cheng et al. 2023). Having the ability to distinguish facts about the real world from flights of fancy (and total nonsense!) is a crucial prerequisite for such applications. Secondly, understanding LM's representation of modal categories can be informative for uncovering the underlying "world model" that it has encoded (Mitchell [025 Li et al., (023). This includes coarse- grained knowledge, such as whether a scenario could happen in some possible world, and more fine- tieeggrined knowledge, such as whether scenario is probable. improbable, or impossible in the real world. Modal intuitions have long been used to characterize the folk theories that people employ to understand domains such as physics (McCoy & Ullman 2019 Shtulman & Carey 2007: Shtulman & Morgan 2017). Probing LM representations of the same categories can 1) reveal whether LM representations of modal categories are concordant with human intuitions about these categories, and 2) potentially be used to evaluate whether an LM encodes aspects of the real world that are related to event plausibility.
However, recent work has questioned the ability of LMs to distinguish between modal categories, arguing that their sensitivity to surface-level features makes LM probability estimates poor indicator of a sentence's 's modal category (Kauf et al., 2023] Michaelov et al. (025). This is not unexpected, as wide variety of considerations, aside from modal category, factor into next-token probability judgments (McCoy et al. 2024). The key question remains: are modal categories represented as coherent features in their own right within the LM, or do LMs only represent modality implicitly through unreliable probability estimates?
While Kauf et al. (2023) provides some preliminary indication that models may contain such internal representations using a probing analysis, we substantially elaborate on these results by expanding the range of modal categories and datasets under consideration. We analyze the development of internal representations of modal categories, relate these internal representations to human participants' categorization behavior, and show how we can interpret them in terms of human-understandable features. Specifically, we aim to answer the following research questions:
RQ 1 Do LMs have internal representations of modal categories that go above and beyond merely representing outpul probabilities?
RQ 2 How do LM representations of modal categories develop a) over the course of training b) over consecutive layers c) as model size increases?
RQ 3 Do LM representations of modal categories reflect imeiinrined human categorization deci- sions?
RQ4 4 What interpretable features do these representations correspond to?
To foreshadow our results, we find that LMs often do contain linear representations of the difference between modal categories, or modal difference vectors, which can be used to classify stimuli drawn from variety of existing datasets. Modal difference vectors are often more discriminative than the probability of the sentence (RQ D. We find that modal difference vectors emerge in an intuitive order, with more obvious categorical distinctions emerging earlier in training/layers/ccale than more fine-grained distinctions (RQ For both evaluations, we rely on expert labels of modal categories. However, human participants' intuitions of modal categories not bound reflect such expert labels. Thus, to address 3 we analyze human behavioral categorization data, finding that projections of sentences onto modal difference vectors yield a feature space that reflects human categorization distributions. Intuitively, this feature space clusters stimuli by modal category. Stimuli that lie in between clusters engenders greater disagreement among human participants. Finally, we find that some modal difference vectors correspond to interpretable sets of features, such as subjective event likelihood or imageability (RQ
Overall, our results provide evidence that LMs leam to represent the difference between real life and mere fantasy to a greater extent than implied by previous research. Additionally, these representations appear to capture nuanced aspects of human categorization. These results raise the exciting possibility that one might use an LM's representations of modal categories to gain deeper insight into how and whether they have encoded the causal principles that underlie our world, while retaining the ability to imagine hypothetical realitie!
"ooee available here
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND
Modal Categories The modal category of a statement describes whether that statement could, could not, or must be true (Mallozzi et al.] 2024). The investigation of these categories has a long history in philosophy, where important arguments hinge the validity of modal statements (Hume) 1739
Kripke 1980; Gendler & Hawthorne| 2002). For example, the modal premise that "philosophical zombies are conceivable" can yield the modal conclusion that "it possible that the mind is distinct from the body" (Chalmers, 1997, Descartes] 1641). Modal categories have a shorter (though still substantial) history in the cognitive sciences, where researchers have extensively studied the modal intuitions of children and adults (Shtulman & Carey (007). By probing their intuitions about the (im)possibility of different scenarios, one can obtain a nuanced picture of a participant's s intuitive theories about the causal structure of the world (Griffiths 2015) [Shtulman & Morgan (017). For example, participants' intuitions regarding the difficulty of magical spells tend to be proportional to how much that spell violates their intuitive theories of physics (c.g., conjuring frog out of nothing would be more difficult than teleporting a frog) (McCoy & Ullman 2019). Following previous
computational studies of modality (Kauf et al. 2023 Hu al. 2025a b), we study the following modal categories:
Probable: Scenarios that are both possible and commonplace in our world. E.g., chilling drink using ice
Improbable: Scenarios that are possible, but not commonplace in our world. E.g., chilling a drink using snow
Scenarios that are not possible in our world, as they violate some known law of nature (e.g.. physics, biology, etc.). These scenarios might be true in counterfactual world with different laws of nature. E.g., chilling drink using fire
Impossible:
Inconceivable:
Scenarios that cannot be evaluated for possibility in any possible world, due
to some fundamental semantic or conceptual error (Hu et al. 2025b). We study inconceivable sentences that arise due to selectional restriction violations unmet requirements that a verb imposes its arguments (c.g., animacy, concreteness, etc.) (Chomsky 1965 Katz & Fodor 1963). E.g., chilling a drink using yesterday
Related Work The present study relates a burgeoning literature investigating world models in LMs - underlying sets of causal principles that the LM encodes to represent and make inferences about the world (Mitchell, 2025). LMs have shown reasonably strong capabilities in encoding the
state of a simplified or abstract world (Nanda et al.) 2023 Kim & Schuster 2023 Li et al. 2025 Ivanitskiy et al. 2023), but have struggled when presented with more complex worlds (Vafa et al. 2024). However, world model must be far richer than representation of states - it must represent the principles that explain the dynamics and constraints of the world (Ha & Schmidhuber 2018
Ivanitskiy et al., 2023 Milliere & Buckner 2024). This literature directly connects to previous work behaviorally assessing LMs" commonsense reasoning capabilities, which implicitly or explicitly
assess an LM's understanding of such basic causal principles (Levesque al. 2012 Zellers et al.) 2019
Bisk et al. 2020 Ivanova et al. 2024).
Similar phenomena are studied in the cognitive sciences, where researchers investigate the world models of children and adults through their intuitive theories of physics, psychology, and other domains (Carey) 2000 (Spelke& & Kinzler, 2007] Ullman 2015). Rather than being complete and precise representations of physical laws, these theories comprise the basic principles that human beings use to make sense of the world around them. Notably, these intuitive theories are imperfect,
leading to variety of incorrect physical inferences (Ullman et al. (017). However, they are sufficient for operating in the world under normal circumstances. The causal principles comprising intuitive theories directly determine human intuitions about modal categories: probable and improbable events are consistent with these principles, impossible events are inconsistent with these principles, and inconceivable events might violate the basic conceptual presuppositions underlying these principles (Sosa & Ullman 2022 Hu et al. 2025b). In this work, we analyze LMs' representations of modal categories gain insight into the world models they have encoded.
3

---

Published as a conference paper at ICLR 2026
Table 1: Properties of the datasets analyzed in the study. Prob., Improb., Imposs., Inc. denote whether the dataset contains probable, improbable, impossible, inconceivable sentences, respec- tively. Pair denotes that the dataset contains minimal pairs that vary modal category, allowing for classification using either vectors probability estimation. Adv. indicates that the dataset contains pairs that are adversarial in some way (see main text). Human indicates that we analyze human behavioral data from this dataset.
Name
Prob.
Improb.
Imposs.
Inc.
Pair
Adv.
Human
Hu et 2025b
X
Goulding et al. (2024)
V
X
X
Vega-Mendoza et al. (2021)
V
X
/
Semantic
Kauf et al. (2033)
V
V
X
/
/
Lexical
Hu et al. 2005a
V
/
X
Tuckute et al. 2024)
X
X
X
3 STUDY 1: LMS LINEARLY REPRESENT MODAL CATEGORIES
In this section, we address by first identifying modal difference vectors - linear representations that distinguish between modal categories from one dataset, and then assessing whether modal difference vectors can be used to classify stimuli from other datasets. We compare this method to classification based on sentence probability (among other baselines), and find that modal difference vectors enable more reliable modal categorization.
3.1 METHODS
Datasets We use the Hu et al. (2025b) dataset to identify modal difference vectors, as it contains minimal pairs of stimuli for all pairs of modal categories under consideration. Notably, the impossible stimuli belong to that modal category due to violations of physical laws (c.g., Someone baked a cake inside a freezer). The inconceivable stimuli belong that category because they violate selectional restrictions based on concreteness (e.g.. Someone baked a cake inside a sigh.). Additionally, Hu al. (2025b) estimates that these sentences appear in natural text сorpora with approximately equal frequency across the four modal categories, reducing the plausibility that simple frequency- heuristics can be used to classify sentences. We evaluate the identified modal difference vectors using three other datasets; the Goulding et al. (2024), Vega- Mendoza et al. (2021). and Kauf et al. (2023) datasets. These datasets represent different forms of generalization: the Goulding dataset contains stimuli that are impossible due to other factors (e.g., biology: Someone about to be born with wings.), whereas the Vega-Mendoze et al. (202T) Kauf et al. (202)) datasets contain sentences that are inconceivable due to animacy violations (e.g.. The laptop bought the teacher.). By evaluating modal difference vectors on these datasets, we can understand whether they represent modal categories (such impossibility) in general, whether they instead capture more fine- ine-grained notions like physical impossibility.
(2024)
Furthermore, the Vega- et al. (2021) dataset contains adversarial sentence pairs, where an inconceivable stimulus contains semantically-relate terms and an improbable stimulus contains semantically unrelated terms (e.g., The scientific research was funded by the (microscope/raavler)) sentence pairs, where the incon-
Finally, the Kauf et al.] (2023) dataset contains lexically-adversane ceivable and probable sentences contain the same words a different order (e.g., The teacher bought the laptop.) Every dataset contains expert labels of the modal category of all sentences. See Table for a comparison between the datasets used in this study.
Models We study a variety of models across scales and families including GPT2-(Small, Medium, Large, XL). Llama-3.2-(1B, 3B), 2-Mo---((B,, 7B, 13B). and 2-nma 9B).
Modal Difference Vectors We create linear representations of the difference between modal cate- gories using Contrastive Activation Addition (CAA), a technique used to create línear representations of concepts (Panickssery et al. 2023). These representations simply directions in the hidden state
4

---

Published as a conference paper at ICLR 2026
Creating Modal
Classifying with Diff. Vectors
Diff. Vectors
0.1
Prob.+ Imposs.
Imposs
Prob. papp--ssssssss..
Mean
Hidden State
Hidden State
Hidden Stare
Hidden State
Someone built bridge out of...
Someone chillod drink using...
soup.
steel.
fire.
ice.
Figure 1: (Left)) Diagram describing how we create modal difference vectors. In this example, modal difference vector capturing the difference between probable and impossible stimuli is created by taking the mean over differences in hidden representations. (Right) Diagram describing how modal difference vectors are used to classify novel minimal pairs of impossible/prohale sentences. Hidden representations from each sentence projected onto the modal difference vector, and the magnitudes of these projections are compared.
space of an LM that correspond the difference between pairs of stimuli. Because concerned with the difference between modal categories, we call these linear representations modal difference vectors.
CAA creates modal difference vectors using pairs of stimuli, (x+, xr ). x4 expresses one category, and x- expresses another category. These stimuli are given an LM, MM, separate inference passes, and representations of some token are extracted at particular layer l. r+ (I=x),, r- Mi(r_), Difference vectors for each pair are computed as e r, r.. Modal difference vectors are estimated by averaging over many of these single-pai difference vectors. To classify held-out pairs of stimuli (x',,'') x". using modal difference vector U, we simply check whether x', is larger than x' (Marks & Tegmark 2024). This is analogous to prior work that classifies stimulus pairs based on the overall probability of each sentence (Kauf et al. 2023 Michaelov et al. 2025).
Concretely, we create separate modal difference vectors for every unique pair of categories in (probable, improbable, impossible, inconceivable) by taking the difference between representations of the final "." token at some layer. That layer is found independently for each pair of categories by 5-fold cross-validation, using the classification method described above. there are ties, we select the median layer that achieved the best performance. After identifying the best layer, recompute the modal difference vector over all minimal pairs in the Hu et al. (2025b) dataset. See Figure for an illustration of creating modal vectors and classifying stimulus pairs with them.
Baselines We include three baseline classification methods: Probability, Principal Component, and Random. First, we compare probability-based classifier. We follow prior work (Kauf al. 2024) by calculating sentence probability as the sum of the log- pog-brobyyillty of each token in the sentence. We expect that p( inconceivable) p( (impossible) p( (imprbbbbl) < p( ((probable) (Hu al. 2025b). If minimal pair exhibits this relationship between two stimuli corresponding to different modal categories, then we consider the model to correct. Next, we compare projections along principal components of the hidden states. We compute the first three principal components of the final token of all of the sentences the WikiText validation corpus, for each layer of each model (Merity al. 2016). We then the same cross- coss-validation procedure described above to find the principal component that best partitions stimuli for each pair modal categories. Finally, we repeat this process with randomly sampled vectors from each layer.
classifier that uses
3.2 RESULTS
We present classification accuracies all generalization datasets for all models with at least 2B parameters in Figure 2 We find a qualitative difference in generalization set performance between models that are below this scale, and This is consistent with Kauf et al. (2023). who also noted that modal categorization ability correlated with scale. For models with at least 2B parameters, we see that for all pairs of modal categories
discuss those models separately (see Figure Top Left).
5

---

Published as a conference paper at ICLR 2026
Average Classification Accuracy over Generalization Datasets
Condition
Modal Differance
Probability
0.7
Principal Component
Random
0.5
Probable Sebuuble-mprobbbbee
Probable impossible
Imprebable imprebable-lmpessilae Imposaible
Probablee Probaable-nnanncen enccee-ceeiniiiaiee vible
Improbables bebbbbbbbbbtte inconcelvaaie
Compartson
Figure 2: Classification evaluations for models with at least 2B parameters. Results are averages across models and generalization datasets. Modal difference vectors outperform probability estimates and other projection based classification baselines.
present in the generalization datasets, modal difference vectors match or (sometimes drastically) outperform probability estimates at classifying stimuli by their modal category. Modal difference vectors similarly outperform other projection-base baselines. Notably, this result holds when just considering the adversarial stimuli (See Appendix B
The results are substantially less clear for models with <2B parameters (See Figure Appendix [A]. In all cases, modal difference vectors perform worse for these models than they do for larger models. In the case probable vs. inconceivable distinctions, probability estimates result in higher classification accuracy than modal difference vectors, indicating that there is not one direction that reliably accounts for selectional restriction violations based on both animacy and concreteness. Unless otherwise noted, we will proceed by analyzing only models with least 2B parameters.
One might worry that these modal difference vectors are epiphenomenal (i.e., not causally implicated in model behavior). To address these concerns, provide preliminary evidence that one can successfully steer model generations using modal difference vectors in Appendix
STUDY THE DEVELOPMENT OF MODAL CATEGORIES
Prior human studies have found that the ability to distinguish modal categories arises gradually throughout development, with younger children struggling to distinguish between improbable and impossible events (Shtulman & Carey 2007 Shtulman 2009). In this section, by characterizing how modal difference vectors emerge as a function of various forms of model "development": model scale, layer depth, and training steps. Concordant with prior work (Saxe et al., 2019: [ell et al., (202)). we find that more coarse distinctions emerge in smaller models, in shallower layers, and earlier than more fine-grained distinctions.
address RQ 2
4.1 METHODS
Models For scaling analyses, we study the complete set models described in Section 3 For layer depth analysis, analyze all models with at least 2B parameters. For analyzing training dynamics, we follow Hu et al. (2025b) and study OLMo-2-7B, as this model is accompanied by regular checkpoints throughout training.
Datasets We limit our focus to the Hu al. (2025b) dataset for this section, as it contains all pairs of modal categories, with no adversarial pairs.
Analysis For analyzing model scale and training dynamics, the 5-fold cross-validation pipeline described in Section 3 and report cross-validation performance on the best layer. When analyzing layer depth, report cross-validation performance for each layer.

---

Published as a conference paper at ICLR 2026
Parameter Count vs. Avg. Generalization Acc.
Parameter Count vs, Comparison Acc.
Accuracy
Accuracy
0.8
(a)
(b)
18-28
28-58
18-28
2B-5B
Model Size
Model Size
Training Step vs. Comparison Acc.
Layer VS. Comparison Acc.
Accuracy
Accuracy
0.8
(c)
(d)
0.4
10a
10a
10' Stop
10a
10a
0.0
0.2
0.4
0.6
0.8
1.0
Layer
Probeble-inconceivable
improbabie-inccocceeve
pnrposible/ncorceivvell
Probsbleeimpossible
imppoblle
Improbable Impossible
Figure 3: (a) Average generalization performance vs. parameter count reveals a large performance gap between models with fewer/greater than 2B parameters. At (b) smaller scales, (c) earlier in training. and (d) in earlier layers, models form modal difference vectors that can differentiate inconceivable stimuli from other modal categories. After that, models learn the distinction between probable and impossible, then probable and improbable, and finally improbable and impossible.
4.2 RESULTS
We present results from model scale, layer depth, and training dynamics analyses in Figure We find that there a qualitative difference in model performance on generalization datasets between models with <2B parameters and those with >2B parameters, mentioned in Section Aside from that, we find that models first distinguish the inconceivable modal category from the rest, perhaps by taking advantage of distributional signals of selectional restriction violations (Kauf et al. (003)). Models next distinguish probable and impossible events, then probable and improbable, and finally improbable and impossible. Intuitively, we find that more coarse- grarineg modal distinctions are available earlier than more fine-grained modal distinctions. This pattern of results replicates and expands the results found in Hu et al. (2025b), which analyzes the surprisal assigned to sentences expressing different modal categories over training. We identify the of internal representations. Hu et al. (2025b) also investigates whether the surprisal assigned to sentences expressing different modal categories changes as function of parameter count. Whereas that work does not identify substantial differences with scale, we find that internal representations develop function of parameter count.
pattern, except in terms
STUDY MODELING HUMAN CATEGORIZATION BEHAVIOR
In all previous analyses, assumed that the modal category assigned stimulus by researchers is the gold-standard label. However, we know that modal categories are graded (Shtulman & Morgan 2017 Hu al., 2025a b). and categorization appears rely on fuzzy intuitive theories (McCoy & Ullman 2019). In this study, we address by analyzing whether modal difference vectors can be used capture human participants' categorization behavior, which does not precisely reflect expert labels.
5.1 METHODS
We use human categorization data from Hu et al. (2025b), Goulding et al. (2024). and Hu et al. (2025a). Hu et al. (2025b) contains categorization data for all four modal categories under study. Goulding et al. (2024) contains data from adult and children participants categorizing probable, improbable, impossible stimuli as either possible impossible. We analyze the adult classification
Datasets
7

---

Published as a conference paper at ICLR 2026
data. Hu et al. (2025a) contains data from adult participants categorizing probable and inconceivable sentences as either "total nonsense" " or "not total nonsense". We subset the data to only include stimuli that were classified by four more participants.
Analysis We wish to model human participant's modal categorization intuitions at the stimulus level. To do this, we fit logistic regression models predict the response distribution of population of human participants tasked with categorizing a scenario by its modal category (e.g., the proportions of human participants that labeled scenario as "probable", "improbable", "impossible", and "incon- ceivable"). To derive feature space for the logistic regression models, we take all stimuli within a dataset and project them onto three modal difference vectors: prohable-improhable impossible, and the projections. This defines a 3-dimensional feature space, where stimuli are represented as points within this space. We train logistic regressions to predict the full response distribution for each stimulus using this feature space. Logistic regressions are trained using full batch gradient descent with cross-entropy loss using soft labels. We use the Adam optimizer for 200 epochs Kingma & Ba (217)) and learning rate of 0.01. We provide a qualitative visualization of this feature space in Figure (Left).
improbable- These vectors are chosen to mínimize collinearity between
We use leave-one-out cross-validation to predict the response distribution of each scenario. We then compute several metries to characterize the relationship between predicted and empirical response distributions.
3 summed prgo-bbbbbll,yy projections along principal components, projections along random vectors. We use each of these methods to generate feature spaces on which to logistic regression models. Notably, summed log probability only naturally provides a aaaieess--mmmeeeaa 1-dimensional feature space. However, the other two baselines provide 3-dimensional feature spaces. For each of these feature spaces, we follow the exact same procedure as described above to model the human data.
Baselines We use the same set of baselines as in Section
5.2 RESULTS
In Figure 4 (Right), we present several analyses characterizing how the different logistic regression models fit the human data. First, we compute the overall correlation between the empirical and predicted response distributions. Specifically, we report the correlation between the empirical and predicted probabilities assigned to N 1 of the categories for each stimulus, where N is the number of classes (as there are N degrees of freedom each distribution). This correlation provides a coarse measure of the relationship between probability distributions achieved by merely correctly predicting the response distribution for stimuli that clearly belong to one modal category. Thus, we also characterize the averaged mean squared error between predicted and empirical response distributions. Finally, we correlate the entropy of empirical and predicted response distributions. Across all analyses, we find that feature space derived from modal difference vectors routinely outperforms the baselines. Additionally, we provide qualitative examples of stimuli and their predicted and empirical response distributions in Table Overall, we find that modal difference vectors reflect participants' graded, intuitive notions of these categories better than the baselines.
fairly high value might be
6 STUDY 4: INTERPRETING LINEAR REPRESENTATIONS
One benefit of identifying model-internal representations of modal categories is that they can be directly analyzed in order to understand which features drive modal categorization in LMs. In this exploratory study, we investigate RQ by correlating projections of sentences along modal difference vectors with human ratings these same sentences along a variety of interpretable dimensions. We study projections onto the three vectors used in Section Interpreting these vectors might elucidate the relationship between the different modal categories, which is currently an open question (Hu et al. 2025b).
6.1
METHODS
al. (2025a), and Tuckute et al. (2024). Hu al. (2025h) contains human participant's 's ratings of the subjective event likelihood (i.e., "how
Datasets We use human ratings from Hu et al. (2025)), Hu et
8

---

Published as a conference paper at ICLR 2026
Table 2: Qualitative examples from Goulding al. (2024) showing the probability estimates that each scenario is possible, as assessed by 1) the logistic regression model using projections on modal difference vectors from Gemma-2-9h 2) the logistic regression model using probability estimates from Gemma-2-9B, 3) and the proportion of participants classifying the scenario "possible".
Scenario (Someone is about to...)
Modal Diff. P(Poss.)
Prob. P(Poss.)
Human P(Poss.)
clean
0.99
0.70
1.0
clean a road.
0.94
0.62
0.97
clean a cloud.
0.09
0.57
0.05
stay awake for 5 hours.
0.94
0.59
1.0
stay awake for 5 days.
0.67
0.63
0.53
stay awake for 5 years.
0.25
0.60
0.05
Overall Correlation
Overall Combiation
0.75
(a)
Projections Modal Difference Vectors
0.50
Modal Diference
Expert Labels
Probubility
2.0
Improbable
Prine Companent
1.5
Impossible
0.00
Incenceivable
1.0
-0.25
Hu 2025b
Hu 2025a
Goulding
0.5
Dataset
MSE
Entropy Correlation
0.6
-0.5
0.15
Entroy Conrelation
0.4
-1.0
0.30
0.2
=1.5
0.05
0.0
improbable-mppossibee
Hu 202
2025a
Goulding
Hu. 2025b
2025
Goulding
Dataset
Dataset
Figure 4: (Left) A qualitative example of stimuli from Hu et al. difference vectors. Dots are colored according to their expert label. Background color intensity represents the probability that each point belongs to particular class according logistic regression model fit to this subset of data using these two features. (Right) (a) Pearson correlation between the predicted probability distributions and the empirical proportion of participants that selected each category. (b) Mean squared error between predicted and empirical response distributions. (c) Pearson correlation between the entropy predicted and empirical response distributions. all analyses, we find that featurizing using projections along modal difference vectors leads to better models of human categorization behavior.
2025b) projected along two modal
probable is scenario?") on Likert scale for all sentences in the Hu et al. (2025b) dataset (Event Likelihood in Figure 5 Hu al.) (2025a) contains 12 sentences from Hu al. (2025a) annotated according to their average rank in forced-ranking version of the same subjective event likelihood task (Ranked Inconceivabilityy in Figure 5). Additionally, Tuckute et al. (224) contains 2000 short, diverse sentences, annotated on a Likert scale along variety of dimensions, including how easy a sentence is imagine, how grammatical the sentence is and whether the sentence contains strong emotional valence. This dataset also contains various probability estimates from e.g., another I.M or an N-gram model. See Appendix D for descriptions of each dimension.
6.2 RESULTS
We project all sentences in each dataset onto the three modal difference vectors used Section 5 bobable-improbbble improbable-impon impllsibbees aupossible-inconceivaaal We then correlate these sentence projections with the human participants' annotations of interpretable features, as discussed above.
We present the results of this analysis in Figure 5 Reassuringly, we find that projections along the probable-improbable vector correlate very well with subjective event likelihood, but less well

---

Published as a conference paper at ICLR 2026
Vector Correlation with Interpretable Features
Modal Difference Vector
Probable-improbable
0.6
Improbable- Impossible
Improbable-mpossiblle
0.2
impossibelincooncllabblee
Ranked inconceivabiin cisacieivabiityy
Event Likellhood
Imageatillt
Physical
Places
Serge
Gram.
Arousal
Others Thoughts
Valence
Conversations Invversational
5-Gram Log Prob
Frequency
PCFG Log Prob
GPT2-X Log Prob
Fealure
Figure 5: Absolute correlations between projections along modal difference vectors and interpretable features (averaged over models). Notably, Probable- Probablc-Improbbble correlates with human subjective event likelihood judgments, and aupossibleeIncooceivaall correlates selectively with imageability, the presence of physical objects, and places/environments.
with any of the other dimensions. Projections along the ceaooabble-imossibble vector are correlated with several features, including subjective event likelihood, imagability, whether the sentence makes sense, and grammaticality. This dispersion of correlations makes it harder to interpret exactly what distinguishes possible from impossible scenarios.
Most interestingly, we see that projections along the impossibeeinccoceevabbl vector correlate selectively with features that measure whether a sentence is easy to imagine (either directly or indirectly). This suggests that the ability imagine scenario might be crucial ingredient in distinguishing impossible from inconceivable events. Notably, imagination has been empirically investigated factor in distinguishing impossible from possible scenarios (Shtulman & Carey) 2007 Lane et al.) [016)] Tippee et al., (2024). but not as a mechanism for distinguishing impossible from inconceivable scenarios. However, this finding consistent with classic understanding of conceivability from philosophy (Hume 1739 Yablo 1993).
7 DISCUSSION
We investigate how and whether LMs represent the modal category of a sentence within their hidden states. We find that (1) LMs form representations modal categories, and that these representations more diagnostic than output probability distributions (Section (2) modal difference vectors develop at different points over the course of training, layers, and model size (Section 4 (3) modal
difference vectors provide feature space that reflects human categorization behavior (Section and (4) modal difference vectors may reflect human-interpretable features (Section ())
This investigation lays the foundation for variety of future studies. First, modal difference vectors encoding the difference between, e.g., impossibility and improbability provide a direct means of testing the intuitive theories that LMs derive about the world from raw text input. For example, one might create a controlled dataset of sentences that instantiate different types of physics violations (similar McCoy & Ullman (209) Ivanova et al. (2024) and modal difference vectors to check whether LMs represent each of these scenarios as impossible. This investigation could help reveal the physical constraints encoded by the LMs.
Finally, Sections 5 and T raise an exciting possibility: one might use modal difference vectors to generate hypotheses about human representations of modal categories. Section jestabishhss corre- spondence between modal difference vectors and human categorization behavior, and Section points to a specific, testable hypothesis: that humans distinguish between inconceivable and impossible events the basis of imagination. Imagination has been shown to significantly impact adults' estimation of event likelihood (Koehler) [991). and has been investigated as a strategy that adults and children to distinguish improbable from impossible events (Shtulman & Carey 2007 Lane et al.) 2016; Tipper et al. 2024) Goulding et al., 2022). However, the role of imagination in discerning inconceivable from impossible events remains unknown.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors would like to thank the members the Brown University Language Understanding and Representation Lab and Serre Lab for their valuable feedback this project. Additionally, we would like to thank Jojo Yang for her detailed comments on the manuscript. Finally, we would like to thank Rachel Goepner, for proofreading the manuscript.
REFERENCES
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Susan Carey. The origin of concepts. Journal of Cognition and Development, 1(1):77-41, 2000.
David J Chalmers. The conscious mind: search of a fundamental theory. Oxford Paperbacks, 1997.
Kunming Cheng, Zhiyong Li, Qiang Guo, Zaijie Sun, Haiyang Wu, and Cheng Li. Emergency surgery in the era of artificial intelligence: Chatgpt could be the doctor's right-hand man. International Journal of Surgery, 109()::811-1881) 2023.
Noam Chomsky. Aspects of the Theory of Syntax. MIT Press, 1965.
René Descartes. Meditations first philosophy. In Seven masterpieces of philosophy, pp. 63-108. Routledge, 1641.
Thomas Fel, Louis Bethune, Andrew Lampinen, Thomas Serre, and Katherine Hermann. Under- standing visual feature reliance through the lens of complexity. Advances in Neural Information Processing Systems, 37:69888-69924 2024.
Tamar Gendler and John Hawthorne. Introduction. conceivability and possibility, 2002.
Asma Ghandeharioun, Ann Yuan, Marius Guerard, Emily Reif, Michael Lepori, and Lucas Dixon. Who's asking? user personas and the mechanies of latent misalignment. Advances in Neural Information Processing Systems, 37:125977112003, 2024.
Jonathan Gordon and Benjamin Van Durme. Reporting bias and knowledge acquisition. In Proceed- ings of the 2013 workshop on Automated knowledge base construction, pp. 25-30. 2013.
Brandon Goulding, Emily Elizabeth Stonehouse, and Ori Friedman. Causal knowledge and children's 's possibility judgments. Child Development, 93(3):794-88) 803, 2022.
Brandon W Goulding, Farishteh Khan, Keisuke Fukuda, Jonathan D Lane, and Samuel Ronfard. The development of modal intuitions: test of two accounts. Journal of Experimental Psychology: General, 153(1):184, 2024.
Thomas I. Griffiths. Revealing ontological commitments by magic. Cognition, 136:43-48. 2015.
Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. Colorless green recurrent networks dream hierarchically. Proceedings of the Society for Computation in Linguistics (SCiL), pp. 363-364, 2019.
Wes Gurnee and Max Tegmark. Language models represent space and time. In The Twelfth International Conference on Learning Representations, 2024.
David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv: 2018.
Jennifer Hu, Felix Sosa, and Tomer Ullman. Making sense nonsense. In Proceedings of the Cognitive Science Society, October 2025a.
11

---

Published as a conference paper at ICLR 2026
Jennifer Hu, Felix Sosa, and Tomer Ullman. Shades of zero: Distinguishing impossibility from inconceivability. Journal of Memory and Language, 143:104640, August 2025b. ISSN 0749-596X. doi: 10.1016 jml 104640. URL https:: www. sciencedreet.................... www.:/ieec////ceceeeeeee.e..e.ee
article/ p 011.///7774952205003333
David Hume. A Treatise of Human Nature. Oxford University Press, 1739.
Michael Igorevich Ivanitskiy, Alex F Spies, Tilman Räuker, Guillaume Corlouer, Chris Mathwin, Lucia Quirke, Can Rager, Rusheb Shah, Dan Valentine, Cecilia Diniz Behn, et Structured world representations in maze-solving transformers. arXiv preprint uriiv23312.02566 2023.
Anna A Ivanova, Aalok Sathe, Benjamin Lipkin, Unnathi Kumar, Setayesh Radkani, Thomas Hikaru Clark, Carina Kauf, Jennifer Hu, RT Pramod, Gabriel Grand, et al. Elements of world knowledge (ewok): A cognition-inspired framework for evaluating basic world knowledge in language models. CoRR, 2024.
Jerrold J Katz and Jerry A Fodor. The structure of a semantic theory. Language, 39(2):170-210, 1963.
Carina Kauf, Anna A Ivanova, Giulia Rambelli, Emmanuele Chersoni, Jingyuan Selena She, Zawad Chowdhury, Evelina Fedorenko, and Alessandro Lenci. Event knowledge in large language models: the gap between the impossible and the unlikely. Cognitive Science, 47(1 ((11))ee 1):ee 13366, 2023.
Najoung Kim and Sebastian Schuster. Entity tracking in language models. In The 61st Annual Meeting Of The Association For Computational Linguisties, 2023.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL
https: httpss///rxxiv org/ abs 1412. 6980
Derek Koehler. Explanation, imagination, and confidence in judgment. Psychological bulletin, 110
(3):499, 1991.
Saul Kripke. Naming and necessity, 1980.
Jonathan D Lane, Samuel Ronfard, Stéphane P Francioli, and Paul L Harris. Children's imagination and belief: Prone to flights of fancy or grounded in reality? Cognition, 152:127-140, 2016.
Hector J Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. KR, 20th:13th, 2012.
Belinda Li, Zifan Carl Guo, and Jacob Andreas. (how) do language models track state? arXiv preprint :xXi:22003.02844 2025.
Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Watten- berg. Emergent world representations: Exploring a sequence model trained on a synthetic task. ICLR, 2023.
Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D Manning, and Daniel Ho. Hallucination-freed assessing the reliability of leading ai legal research tools. Journal of Empirical Legal Studies, 2024.
Antonella Mallozzi, Anand Vaidya, and Michael Wallner. The Epistemology of Modality, In Edward N. Zalta and Uri Nodelman (eds.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2024 edition, 2024.
Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, 2024.
John McCoy and Tomer Ullman. Judgments of effort for magical violations of intuitive physics. PloS one, (4(5):20217513, 2019.
R Thomas McCoy, Shunyu Yao, Dan Friedman, Mathew D Hardy, and Thomas L Griffiths. Embers of autoregression show how large language models are shaped by the problem they are trained to solve. Proceedings of the National Academy of Sciences, )21(41)ee322222012 2024.
12

---

Published as a conference paper at ICLR 2026
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models, 2016.
James Michaelov, R. Estacio, Z. Zhang, and B. K. Bergen. Not quite sherlock holmes: Pretrained language models cannot reliably differentiate impossible from improbable events. Findings the Association for Computational Linguistics, 2025.
Raphael Milliere and Cameron Buckner. A philosophical introduction to language models part Continuity with classic debates. ArXiv, abs/2401 000910, 2024. URL https://api
semant lemaanicschoaar.oo ort.Cppuut 2 266844364
Melanie Mitchell. Lims and world models, part 1. AI: A Guide for Thinking Humans, 2025.
Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pp. 16-30, 2023.
Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Tumer. Steering Ilama via contrastive activation addition. arXiv preprint arXiv:2312.06681 2023.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMMNLP-IJCNLP), pp. 2463-2473, 2019.
Andrew Saxe, James L McClelland, and Surya Ganguli. mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116(23): 11537-11546, 2019.
Andrew Shtulman. The development of possibility judgment within and across domains. Cognitive Development. 24:293-309, July 2009, doi: 00.10166..ooddee2200...222 12.066
Andrew Shtulman and Susan Carey. Improbable or impossible? how children reason about the possibility of extraordinary events. Child development, 1803):1015-1022, 2007.
Andrew Shtulman and Caitlin Morgan. The explanatory structure of unexplainable events: Causal constraints on magical reasoning. Psychonomic bulletin & review, 24:1573-1585, 2017.
Felix A Sosa and Tomer Ullman. Type theory in human-like learning and inference. arXiv preprint arXiv:2210.01634, 2022.
Elizabeth S Spelke and Katherine Kinzler. Core knowledge. Developmental science, 10(1):89-96. 2007.
Zoe Tipper, Terryn Kim, and Ori Friedman. Children (and many adults) use perceptual similarity to assess relative impossibility. Developmental Psychology, 2024.
Greta Tuckute, Aalok Sathe, Shashank Srikant, Maya Taliaferro, Mingye Wang, Martin Schrimpf, Kendrick Kay, and Evelina Fedorenko. Driving and suppressing the human language network using large language models. Nature Human Behaviour, 8(3):544 56144-666,,10 2024.
Tomer D Ullman, Elizabeth Spelke, Peter Battaglia, and Joshua B Tenenbaum. Mind games: Game engines as an architecture for intuitive physics. Trends in cognitive sciences, 21(9);649 665, 2017.
Tomer David Ullman. On the nature and origin of intuitive theories: learning, physics and psychology.
PhD thesis, Massachusetts Institute of Technology. 2015.
Keyon Vafa, Justin Chen, Ashesh Rambachan, Jon Kleinberg, and Sendhil Mullainathan. Evaluating the world model implicit in generative model. Advances in Neural Information Processing Systems, 37:26941-26975. 2024.
Marten van Schijndel, Andy Exley, and William Schuler. A model of language processing as hierarchic sequential prediction. Topics in cognitive science, 5(3):522-540, 2013.
13

---

Published as a conference paper at ICLR 2026
Average Classification Accuracy over Generalization Datasets (<28B
Condition
Accuracy
0.8
Mocal Difference
Probability
0.7
Principal Component
0.5
Probable improbable
Probable impossible
Improbaple impossible
Probable- Probable-incanccevvee inteile-iinnnneiiieeee
Improbable Improbabbe--ncanccee inconceiooicvccc aole
Compa rison
Figure 6: Classification evaluations for models with less than 2B parameters. Results are averages across models and generalization datasets. Results are mixed.
Mariana Vega-Mendoza, Martin Pickering, and Mante S Nieuwland. Concurrent use animacy and event-knowledge during comprehension: Evidence from event-related potentials. Neuropsy- chologia, 152:107724, 2021.
Stephen Yablo. Is conceivability guide possibility? Philosophy and Phenomenological Research, 53(1):1-42, 1993.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, Florence, Italy, July 2019. Association for Computational Linguistics. doi:
10.186533v1/P19-1472 /ttp://///////tttttoooooooo////.
A CLASSIFICATION RESULTS FOR MODELS WITH <2B PARAMETERS
We present classification results from models with <2B parameters in Figure 6
We find mixed results across different classification methods, with overall worse performance than with models >2B.
B CLASSIFICATION RESULTS FOR ADVERSARIAL STIMULI
We highlight the performance of all classification methods adversarial stimuli. We include lexically adversarial stimuli from Kauf et al. (2023) and semantically adversarial stimuli from Mega-Mendozal et al. (222)). Lexically adversarial stimuli contain sentence pairs with the same tokens, just in different order (e.g., The teacher bought the laptop/TThe laptop bought the teacher). Semantically adversarial stimuli contain sentence pairs with an improbable stimulus containing semantically- unrelated word and an impossible stimulus containing a semantically-relan research was funded by the (rravelrrmiccroscope)) We find that modal difference vectors distinguish all of these cases reliably, whereas other methods distinguish at most one of these types of adversarial stimuli.
word (c.g.. the scientific
C STEERING WITH MODAL DIFFERENCE VECTORS
In this section, we demonstrate preliminary evidence that the modal difference vectors can be used to steer the generations of language model to produce sentences expressing the intended modal category. We assess this using manually-construun each prefix and model, we generate continuations as follows: First, we generate the top most likely greedily decode 4 more tokens. However,
corpus of 30 novel sentence prefixes. For
next tokens. For each of these 5 continuations, find that models sometimes generate fragments of run-on or syntactically complex sentences. To generate clean qualitative examples, store the overall probability of the period token "" after each generation. We truncate generations after the token position where the period token received the highest probability.
14

---

Published as a conference paper at ICLR 2026
Adversarial Comparisons Highlight (>=28)
Accuracy
0.6
Condition
Modal
Probability
0.4
Principal Component
Random
Lexical
Semantic
Adversarial
Figure 7: Performance of all classification methods just adversarial stimuli. We include lexical adversarial stimuli from Kauf et al. (2023) on the left. These stimuli contain the same tokens, just in different orders. We find that modal difference vectors and probability estimates perform best in the face of this manipulation. We include semantically adversarial stimuli used in Vega-Mendozz et al. (202I) and Michaelov et al. (2025) on the right. We find that modal difference vectors and projections along principal components perform best, while probability estimates are systematically misled.
Following prior work, we intervene models using modal difference vectors while generating (Panickssery et al. 2023] Ghandeharioun et al., (002)). To do so, we add scaled version of the modal difference vectors for probable-improbbble improbable, probable-impossibee
to all residual stream positions at the appropriate layer while generating the next token. We experiment with scalar multipliers of 3 and 5, and find that 5 qualitatively produces better results. As baseline, we repeat this process with no intervention.
We attempt to steer Gemma-2-2B and Llama-3.2-3B. We present a quantitative analysis of steering success in Figure 8 Here, we use the baseline model (without intervention) to measure the surprisal (or negative ng-ppobbbblltty) of the first 5 generated tokens either with without steering. We find that, for both models, surprisal increases in order of basel Line < improbable < iiposssin iible inconceivees inconceivable. This result mirrors the finding from Hu al. (2025b), indicating that steering has the desired impact model generations.
We also present several examples from each model on diverse range of prefixes Tables 3 and 4 While not perfect, these examples show many instances of steering having the desired effect, rendering generations more improbable, impossible, or inconceivable.
D DESCRIPTIONS OF THE FEATURES IN THE TUCKUTE ET AL. (2024) DATASET
In this section, we briefly describe the features the Tuckute et al. (2024) dataset, which we correlate with projections along modal difference vectors in Section 6
Imageability: Ratings on a 7 point Likert scale, answering the question "how easy the sentence visualize, to form image of the sentence's 'ss meaning in your mind?"
Physical: Ratings on 7 point Likert scale, answering the question "how does the sentence make you think of physical objects and/or physical causal interactions?"
Places: Ratings
7 point Likert scale, answering the question "how much does the sentence make you think of places, natural scenes, and/or environments?"
Sense: Ratings on a point Likert scale, where corresponds to "doesn't it make any sense" and corresponds to "makes perfect sense".
Gram.: Ratings on point Likert scale, where 1 corresponds to "completely ungrammati- cal" and corresponds "perfectly grammatical".
Arousal: Ratings on a point Likert scale, answering the question "how much does the sentence make you feel stimulated, excited, frenzied, wide-awake, and/or aroused?"
15

---

Published as a conference paper at ICLR 2026
Surprisal After Steering
Gemma 2-2b
7
Llama-3,2-3b
3-
2
1-
None
Improbable
Impossible
Inconceivable
Steering Vector
Figure 8: Surprisal (with respect to the baseline model) of the next token predictions generated after steering. We find that, for both Gemma-2-2B and Llama-3.2-3B. surprisal values generally increase in order of base 1 ine improbable < impossible < inconcei saannciivabll.... vable. This mirrors the finding from Hu (2025b).
Valence: Ratings 7 point Likert scale, answering the question "how much does the sentence make you feel happy, pleased, content, and/or hopeful?"
Others' Thoughts: Ratings on a point Likert scale, answering the question "How much does the sentence make you think of other people's experiences, thoughts, beliefs, desires, and/or emotions?"
Conversational: Ratings on point Likert scale, answering the question "how likely do you think the sentence is to occur in a conversation between people?"
Ratings on 7 point Likert scale, answering the question "how likely do you think you are to encounter this sentence?"
Frequency:
5-Gram Log-Prob: Probability estimates from 5-gram language model.
PCFG Log-Prob: Probability estimates from the PCFG parser from van Schijndel et al.
2013).
GPT2-XL Log-Prob: Probability estimates from GPT2-XL.
16

---

Published as a conference paper at ICLR 2026
Table 3: Steering Generations using Gemma-2-2B.
Model
Prefix
Remmma-2-2B
Someone measured the furniture using a...
Steering
Generations
None
tape measure, ruler, measuring tape, laser level tape measure, ruler, laser, measuring tape, 3D scanner laser, telescope, microscope, new technique, camera ibajigut, team, different scale, word, new Prefix
Improbable
Impossible
Inconceivable
Model
Gemma-2-22
Someone fed the child with a...
Steering
Generations
None
spoon, bottle of milk, knife, syringe, glass water spoon, istiiza bowl of soup, baby bottle, pizza song, baby, song, video game, snake smile, child, good thing, team, different kind of child
Improbable
Impossible
Inconceivable
Model
Prefix
Gemma-2-2B
Someone fixed the car with a...
Steering
Generations
None
hammer and screwdriver, new engine, lot of effort, 1, screwdriver hammer, piece of wood, rubber band, sledgehammer new car, laser, 3D printer, car, broken mirror team, smile, good team, jem.teamj/emi,, great team
Improbable
Impossible
Inconceivable
Model
Prefix
Gemma-2-4
Someone protected the garden with a...
Steering
Generations
None
fence, net, protective fence, plastic sheet, wire fence fence, 1, giant inflatable, blanket, net new song, cloud, virus, song, flower
Improbable
Impossible
Inconceivable
team, smile, family, new one, jemzteamy/em;
17

---

Published as a conference paper at ICLR 2026
Table 4: Steering generations using Llama-3.2-3B
Model
Prefix
Llama-3.2-3B
Someone destroyed a building using
Steering
Generations
None
sledgehammer, buildozer, wrecking ball, crane, drone flamethrower, bulldozer, sledgehammer, drone, chainsaw boat, car, 9, real-life, and woman term, thesium, weapon, gun, a
Improbable
Impossible
Inconceivable
Model
Prefix
Llama-3.2-3B
Someone hit the ball with a...
Steering
Generations
None
bat, golf club, racquet, racket, stick golf club, bat, baseball bat, racquet, club stick, real, boat, golf course, bow hammer, bat, club, baseball, ball
Improbable
Impossible
Inconceivable
Model
Prefix
Llama-3.2-3B
Someone beat a drum with a...
Steering
Generations
None
stick, drumstick, spoon, big stick, hammer stick, drumstick, spoon, sword, can of beer drum, man in a boat, French, cat, real drum hammer, heart, mind, head, brain
Improbable
Impossible
Inconceivable
Model
Prefix
Llama-3.2-3B
Someone treated the wound using a...
Steering
Generations
None
topical antibiotic, solution of O, technique, mixture honey, tourniquet home remedy, mixture honey, loe vera gel, local anesthetic, homemade remedy
Improbable
Impossible
Japanese, bottle, French, real-life number, series, few, smile, The
Inconceivable
18


Reviews:
[{"review_id": "XWvdyY2QX1", "reviewer": "Reviewer_8BZg", "summary": "This paper investigates whether large language models encode internal representations of modal categories (probable, improbable, impossible, and inconceivable) and whether these internal representations align with human judgments.\nUsing Contrastive Activation Addition (CAA), the authors construct modal difference vectors, representing linear directions in hidden-state space that capture distinctions between modal categories.", "questions": "- The logistic regression uses three modal difference vectors as features. How sensitive are the results to this choice? \n- The authors employ correlation, MSE, and entropy metrics to compare model predictions and human judgments. Whether these metrics capture qualitative alignment as well as quantitative similarity, like category boundaries? For instance, do models misclassify in similar ways as humans do?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- It is very interesting that these internal representations develop systematically with model scale, depth, and training progression, mirroring human developmental trajectories.\n- The work builds on linguistic, philosophical, and cognitive frameworks of modality and imaginability, connecting results to human developmental and philosophical theories \n- The correspondence between model-internal representations and human categorization behavior is compelling and well-analyzed.", "weaknesses": "- The study primarily focuses on short, declarative English sentences describing simple physical or conceptual scenarios. As a result, the findings may not generalize to contextualized modality. Expanding the dataset to include diverse syntactic and pragmatic expressions of modality would provide stronger evidence that the observed internal representations are not tied to a narrow stimulus set.\n- Lack of analysis on confidence and uncertainty alignment. While the paper examines model–human alignment in categorical judgments, it does not analyze how models express confidence in those judgments and contrast between the narrative confidence (what the model says) and the representational confidence, what it encodes internally . It would be valuable to explore cases where the model is confidently wrong or hesitantly correct, and to compare the model’s internal certainty with the explicit “confidence” conveyed in its output probabilities. \n- Some datasets (e.g., Hu et al. 2025a) are small; robustness of correlations might depend on a few stimuli.\n- The reported findings are based on a small family of transformer models. It remains unclear whether the same modal geometry emerges across architectures or initialization seeds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0gpC3sd8WX", "reviewer": "Reviewer_83R5", "summary": "This paper investigates the ability for language models to categorize the modal categories of sentences, i.e. whether an event is possible, impossible, etc. They identify linear representations that separate modal categories, which they call modal difference vectors. They find that modal difference vectors emerge in a consistent order as the number of training steps, parameter count, and layers increase. They also validate these modal difference vectors with human judgments.\n\nIn the first study, they focus on four modal categories, probable, improbable, impossible and inconceivable events and test modal difference vectors vs. a logprob based, PCA based, and random baselines and find that in models above 2B, the modal difference vectors perform better at categorizing the modal category of a sentence, but this is not so for models below 2B parameters. In the second study, they examine accuracy on the classification task for models with different parameter counts, modal category distinctions and throughout the training of a sample model (olmo2 7B). They find that more coarse-grained modal distinctions tend to arise earlier and are more accessible for small models. Lastly, they interpret modal difference vectors by correlating projections with human-rated features.", "questions": "- The paper identifies a stark qualitative difference at 2B parameters, with even the random baseline outperforming modal vectors, but this finding receives minimal analysis. Can you comment more on this and why you believe this to be the case?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**S1**: I appreciate the thoroughness in terms of types of experiments done, as the modal category vectors are analyzed in multiple different settings, from classification accuracy to emergence in scaling to correlation with human judgements. This makes me more confident that the vectors found do correspond to the modal categories, though I still have some doubts (see weaknesses)\n\n**S2**: The paper is overall clear and well written, with the motivation and importance of the problem being addressed up front and each section. I think the results of the paper would be interesting to the community.\n\n**S3**: The topic is interesting to the community and timely, I think the paper would be interesting to those working on hallucination, factuality or related topics.", "weaknesses": "**W1**: This paper falls into a category which I call \"vector for X\". I want to say here that I don't necessarily believe that \"vector for X\" type papers are bad, but I think that to be sound work, I would want to see some causal analysis or intervention applied to the vectors found. Keeping in mind that linear separability between groups of points in high dimensions is the default, this makes me automatically a bit wary of papers that have this as a main contribution. In order to validate that the vectors found are meaningful, I'd want to check that they influence the model behaviour in some way. Appendix D is a good start but I would want to see a more thorough investigation:\n\n- necessity of the direction: for instance by nulling out the modal subspace and checking that performance drops on a modal classification task\n\n- behavioural linkage: it would be nice to show that steering vectors have an effect on some kind of classification task rather than just through surprisal. For example, can steering actually change model decisions on modal classification tasks(e.g. with a fine-tuned head?)\n\n- comparison to alternatives: how do CAA-derived vectors compare to other methods? \n\n**W2**: It seems like there’s the possibility for some conflation with some other linguistic concepts such as negation, sentiment or other concepts. Could you comment on whether this is a concern or if you attempted to control for this? Specifically, for the inconceivable/impossible sentences, is it possible that they have more negation-like linguistic terms or that they are less grammatical? A further thing that I would like for papers like this is a frequency control (as much as possible) -- the inconceivable and impossible sentences probably appear far less in pretraining data compared to the other categories?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tqNWrc0BKy", "reviewer": "Reviewer_swG3", "summary": "The paper investigates whether language models (LMs) encode modal categories—probable, improbable, impossible, and inconceivable—via linear structure in their hidden states. Using Contrastive Activation Addition, it derives modal difference vectors from minimal pairs and shows that, for models ≥2B parameters, these vectors classify modality across multiple datasets (including adversarial ones) and typically outperform sentence-probability baselines. Developmentally, coarser distinctions (e.g., inconceivable vs. the rest) emerge earlier in training, shallower layers, and smaller models, with finer distinctions appearing later and at larger scales. Projecting sentences onto three modal vectors yields a low-dimensional feature space that predicts graded human categorization distributions via logistic regression. Correlation analyses suggest the probable–improbable vector tracks subjective event likelihood, while the impossible–inconceivable vector aligns with imageability/physical concreteness. Preliminary steering results hint these vectors can nudge model generations toward targeted modal categories.", "questions": "Can you demonstrate that injecting/removing these vectors causally mediates modality judgments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1) It shows simple linear method could transfer across datasets and model families, beating stronger-than-naïve baselines on modality classification.\n2) Uses projections as features to predict human response distributions (not just hard labels), capturing graded intuitions.", "weaknesses": "1) Probability is a weak straw baseline for modality; missing stronger comparators (e.g., calibrated logit regressors over tokenwise surprisals, fine-tuned linear probes with controls) and clearer uncertainty estimates/CIs across splits and seeds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g0773LEQRS", "reviewer": "Reviewer_bMoA", "summary": "This paper investigates whether LLMs encode human-like distinctions between possible, impossible, and inconceivable events. Using “modal difference vectors” derived from contrastive pairs (e.g., Someone chilled a drink using ice vs. using fire), the authors claim that these linear directions in activation space reflect graded human judgments of plausibility. The paper reports four main studies analyzing representational structure, training dynamics, correspondence to human categorization, and interpretability. Results show that linear separability of modal categories emerges during training and correlates with human judgments. The work contributes to the growing line of research linking mechanistic interpretability with cognitive semantics.", "questions": "1.  There appears to be a notable discrepancy between the main claim in the Abstract, which suggests that \"LMs\" as a general category possess more reliable modal judgments, and the specific results in Section 3.2. Your analysis explicitly states that for models with <2B parameters, the results are \"substantially less clear\" and that the probability baseline can even \"sometimes result in higher classification accuracy\". This suggests the paper's core finding is an emergent property of scale. Could you clarify whether you see this as a general property of LMs or a scale-dependent one? \n\n2. Could you provide a bit more rationale on the selection of these specific datasets (e.g., Hu et al. 2025b, Kauf et al. 2023, Goulding et al. 2024)? It would be good to know how this particular combination of datasets helps ensure that the model is learning the abstract concept of modality, rather than potentially learning statistical heuristics that might be common across these specific benchmarks. \n\n3. The steering experiment in Appendix D is an interesting proof-of-concept for the causal effect of the vectors. However, the experiment's scale is very limited, based on \"a manually-constructed corpus of 30 novel sentence prefixes\" and only two models. While you correctly label this as \"preliminary evidence\", it is difficult to draw a strong conclusion from such a small sample.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper's primary strength lies in its originality of problem formulation and the clarity of its investigation. The authors compellingly shift the debate on world models in LM: instead of focusing on output probabilities as an indicator of modal understanding (which prior work has shown to be unreliable), this paper poses a more original and nuanced question about how modal categories are encoded in the model's internal representations. The introduction of \"modal difference vectors\" as a specific, linear, and geometrically interpretable tool is a novel methodological contribution. The entire paper is well-structured around four precise research questions, which creates a clear, logical narrative that guides the reader from the vector's existence (Study 1) to its development (Study 2), its link to human cognition (Study 3), and its interpretability (Study 4).\n\nThe paper's contribution (Study 3) is the demonstration that these modal difference vectors, extracted from LMs, can model fine-grained human categorization behavior better than strong baselines. This moves the field beyond simply asking if LMs \"know\" the difference between possible and impossible, and instead investigates whether their internal representational geometry captures the graded and fuzzy nature of human modal judgments. This finding is significant because it provides a novel, quantitative framework for comparing the \"intuitive theories\" of LMs against those of humans.\n\nFinally, the quality of the exploratory analysis in Study 4 is a key strength. The attempt to find interpretable correlates for these abstract vectors is valuable. By correlating vector projections with human ratings of features like \"subjective event likelihood\" and \"imageability,\" the paper provides a crucial first step toward grounding these internal representations in human-understandable concepts.", "weaknesses": "1. There appears to be an inconsistency between the paper's high-level claims and its detailed experimental findings. The abstract makes a general claim that \"LMs have access to more reliable modal categorization judgments...\". This suggests a general property of the language models studied. However, the paper's own results in Section 3.2 (and detailed in Figure 6, Appendix B ) show that this finding does not hold for models with fewer than 2B parameters. For this group of models, the results are described as \"substantially less clear\" and \"mixed\", with the probability-based baseline \"sometimes result in higher classification accuracy\". \n\n2. The experiment presented in Appendix D, while intriguing, is too small in scale to support a conclusion about the causal properties of the modal difference vectors. The authors state this is \"preliminary evidence\" and assess it using only \"a manually-constructed corpus of 30 novel sentence prefixes\" on just two models. This sample size is insufficient to demonstrate a generalizable or reliable phenomenon, providing only anecdotal evidence. \n\n3. A minor point about presentation. The paper's core conceptual framework relies on the distinction between four modal categories (Probable, Improbable, Impossible, Inconceivable). These distinctions, particularly between \"Impossible\" and \"Inconceivable,\" are subtle and central to the paper's thesis. While the text definitions are provided, a reader not already familiar with this specific literature may struggle to immediately grasp the precise boundaries. It could be improved by including an illustrative figure in the Introduction or Background.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)