Truth-value judgment in language models: ‘truth directions’ are context sensitive
Authors: Stefan F. Schouten
Peter Bloem
Ilia Markov
Piek Vossen
Venue: COLM 2025 (2025)
Keywords: mechinterp
mechanistic interpretability
interpretability
truth directions
LLM beliefs
large language model
llm
TL;DR: Investigation of the in-context behaviour of LLM 'truth directions'.

Abstract: Recent work has demonstrated that the latent spaces of large language models (LLMs) contain directions predictive of the truth of sentences. Multiple methods recover such directions and build probes that are described as uncovering a model’s “knowledge” or “beliefs”. We investigate this phenomenon, looking closely at the impact of context on the probes. Our experiments establish where in the LLM the probe’s predictions are (most) sensitive to the presence of related sentences, and how to best characterize this kind of sensitivity. We do so by measuring different types of consistency errors that occur after probing an LLM whose inputs consist of hypotheses preceded by (negated) supporting and contradicting sentences. We also perform a causal intervention experiment, investigating whether moving the representation of a premise along these truth-value directions influences the position of an entailed or contradicted sentence along that same direction. We find that the probes we test are generally context sensitive, but that contexts which should not affect the truth often still impact the probe outputs. Our experiments show that the type of errors depend on the layer, the model, and the kind of data. Finally, our results suggest that truth-value directions are causal mediators in the inference process that incorporates in-context information.

Published as a conference paper at COLM 2025
Truth-value judgment in language models: 'truth directions' are context sensitive
Stefan F. Schouten, Peter Bloem, Ilia Markov, Piek Vossen Vrije Universiteit Amsterdam
(s. schouten, p. bloeemm, markov, p. j.t.j.mm j.m. @oss. nl
Abstract
Recent work has demonstrated that the latent spaces of large language models (LLMs) contain directions predictive of the truth of sentences. Mul- tiple methods recover such directions and build probes that are described uncovering model's "knowledge" or "beliefs". We investigate this phenomenon, looking closely at the impact of context on the probes. Our experiments establish where in the LLM the probe's predictions are (most) sensitive to the presence of related sentences, and how to best characterize this kind of sensitivity. We do so by measuring different types of con- sistency errors that occur after probing an LLM whose inputs consist of hypotheses preceded by (negated) supporting and contradicting sentences. We also perform causal intervention experiment, investigating whether moving the representation of a premise along these truth-oalue directions influences the position of an entailed same direction. We find that the probes we test are generally context sensi- tive, but that contexts which should not affect the truth often still impact the probe outputs. Our experiments show that the type of errors depend on the layer, the model, and the kind of data. Finally, our results suggest that truth-value directions are causal mediators in the inference process that incorporates in-context information.
contradicted sentence along that
Introduction
As Large Language Models (LLMs) enjoy increasing mainstream adoption, becomes more important to understand why they fail in some cases, while excelling in others. Recent findings show that LLM latent spaces contain directions predictive of the truth of sentences (Burns et al., 2023; Marks & Tegmark, 2024). Probes that leverage these directions to assign truth values to sentences are accurate even in misleading contexts where prompting fails. When considering simple declarative sentences on their own, it makes sense to evaluate such truth-oalue probes primarily by their accuracy. For example, we might have a hypothesis: "In New York, days are shortest in December", which we would expect LLM to represent as true. But how should we evaluate this hypothesis is placed in the context of an incorrect premise, like: "December is not during the winter for New York"? In that case, we no longer necessarily have the same expectations. For example, if the premise is understood as setting up a counterfactual scenario, then we would expect an LLM to represent the hypothesis as false. In other words, for multi-sentence inputs, evaluations must prioritize coherence: the degree of logical consistency in truth-value assignments.
When a truth-value probe assigns a sentence S the label TRUE, it is tempting to refer to the model as believing S. Especially because such explanations of L.LM behavior are highly plausible (convincing to humans, Jacovi & Goldberg, 2020). Explaining LLMs by appealing to attitudes like belief has recently been endorsed as propositional interpretability (Chalmers, 2025), and truth-value probes seem like a promising method. Specifically, they could serve to gauge if an
one or more sentences as input-judges them to be true (actively not coherent, it has failed
believes them) or not. But, if a probe's truth-value assignments to reveal (representations of) judgment or belief (Herrmann & Levinstein, 2025), meaning an interpretation that appeals to those concepts would be unfaithful.

---

Published as a conference paper at COLM 2025
Q = 'Syinng that "December is during the winter for New York" is [ coorrett'
determine polarit
H 'Saying that "In New York, days are shortest December" [in correct'
LLM
truh-soluue probe
Experiment
8
PA(H)
p(h q) a( 8)
PACH Q 1)
I PAH -T)
trath-vabue direction
Experiment 2
intervention
ph: d=(q+==0) 0))
ph: at)
0.
1,
2,
3,
4,
5
6
7
8
9,
Figure Overview of our setup. LLM representations and for a premise and hypothesis are extracted and used to train truth-value probes. In experiment 1, the probes are evaluated to determine if and how they incorporate context. In experiment 2, we move a premise's 's representation in the identified truth-value direction, measuring if the probability assigned to the hypothesis changes accordingly.
In this work, we study coherence by probing LLMs on inputs where hypotheses are pre- ceded by premises which appear either affirmed or negated. We take inspiration from the truth-oalue judgment task in language acquisition research where subjects are "asked to make bipolar judgment about whether statement accurately describes particular situation alluded to in some context or preamble" (Gordon, 1996). We find that truth-value assignments are context sensitive, but also sensitive to irrelevant information.
We also investigate if the relevant directions in latent-space causally mediate truth-value judgment, if they only reflect the outcome of that process. Specifically, we establish if representation's position along a truth-value direction determines (in part) where subse- quent statements are positioned along the same direction. Our results suggest that these directions are mediators in the inference process that incorporates in-context information.
In summary, our contributions are: (1) experiments evaluating the context sensitivity of truth-value probes and the consistency with which they incorporate it; we quantify across layers, model sizes and 13 billion), and type of training (pretrained-only vs. instruction- tuned); and (2) an experiment demonstrating that truth-value directions causally mediate natural language inference. We also propose a new variant of CCS (Burns et al., 2023) for which convergence is more stable, and otherwise behaves and performs similarly. Our code available https: //github.
thub. com/s wschuuttottttt
2 Related Work
Probing LLM representations for the truth of sentences has recently received much interest. Burns et al. (2023) introduce Contrast Consistent Search (CCS), an unsupervised probing methods based on the representations of contrasting sentence pairs. Their probes often outperform (zero-shot) prompting approach, even when applied to misleading prompts. Li et al. (2023) shift model activations in the 'truth direction' at inference time, mitigating hallucination. Their interventions use 1) directions from probes trained with logistic re- gression (LR) and CCS; and 2) new method (Mass Mean Shift), which finds the direction as the difference between the means of the true and false sentence activations. Marks & Tegmark (2024) take Mass Mean Shift and use its directions to build probes (Mass-Mean Probing, MMP). They show all probes (based on LR, CCS, and MMP) generalize well be- tween datasets, with MMP performing the best. In causal intervention experiment, they move representations in the identified directions, showing that MMP the best mediator: increases the probability of the model calling false statement true the most.
Previous work has used both data consisting of single facts and longer inputs, such as various ILJ datasets, but did not study the impact of the context. We specifically study the
2

---

Published as a conference paper at COLM 2025
in-context behavior of truth-value probes, analyzing their consistency, and what this means for the way LLMs incorporate contextual information. Like Marks & Tegmark (2024), we also investigate the causal implication of directions in LLM latent space. However, rather than investigate what causes the greatest change in token predictions, we investigate which direction to move a premise in, such that it causes the correct change in the probability of related hypothesis, as evaluated by the same direction.
Recent work has also criticized this type of probing. Truth-value probes might identify properties that correlate with truth (Levinstein & Herrmann, 2024), especially when truth not the most salient feature (Farquhar et al., 2023). We evaluate the probes' coherence, which we do not expect spurious correlations to exhibit.
3 Methodology
We describe our method in three parts: in 3.1 we cover truth-value probes, the necessary assumptions, notation, and methods to construct them; in 3.2 we describe how to construct samples and the possible quantities measured by probes in our setting; and in 3.3 we introduce error scores with which we measure (different kinds of) coherence.
3.1 Truth-value probes
We several probing methods in experiments. These methods datasets of sen- tences, consisting of both true and false statements. We can turn any true statement into false statement (and vice versa) by negating it. We use superscript and to denote the affirmed (X+) and negated (X-) variants of a sentence, respectively. Their LLM vector (see section 4 for how we negate sentences
representations are given in bold, i.e. x x and for how vector representations are extracted). Thus, the dataset used to train probes consist of pairs of hidden states extracted for the positive and negative variants of statements (x+,x .y", yy" D, and their labels indicating which of the two is true (with y 1 y
When using truth-value probes, we assume that the truth of sentences is latently modeled by LLMs. We characterize this latent (A) model as a probability distribution (X). The probes p(x) are assumed to (approximately) recover this distribution. We believe these assumptions are fair, because previous work (Burns et al., 2023; Marks & Tegmark, 2024) has found directions in latent spaces that suggest LLMs do track sentence truth.
Probes are constructed as: p(x) ax 0), where O is the truth-value direction and ø is the sigmoid. Because the probes do not have bias terms, all inputs are mean-normalized (in line with previous work): x # and x - where μ 2D E(x x jED (x+ +x").
Mass Mean Probing (MMP) is a supervised method, which defines the truth-value direc- tion as the difference between the average of the correct and incorrect statements:
6mm Exy|xly 1] Exy | == 0),
(1)
where y is the truth-value (label) for the statement X. We do not include the version of MMP that requires an i.i.d. assumption, because we also evaluate on out-of-distribution data.
Logistic Regression (LR) is also used to train supervised probe. They are trained on x' -x,, i.e. the difference between the negative and positive statements.
0r arg min - Ex [y- In x' + (1 - -- In (1 - o(0 x))].
(2)
8
where yt is the label for the positive variant of the sample, i.e. whether X is true.
Contrast Consistent Search (CCS) is an unsupervised! method with as its objective:
Ccs = arg min Exxx x
[1 111pp(x)) - p[x min p(x ))x
(3)
By unsupervised we mean that knowledge of which sentences are true or false is given.
3

---

Published as a conference paper at COLM 2025
which has two terms: the consistency-loss (encouraging solutions where the probabilities add up to one), and the confidence- (encouraging non-degenerale solutions, i.e. p(x * p(x") # 0.5). The objective can be understood as finding a hyperplane with normal 0 that, for each pair: (1) separates xt from x and (2) is equidistant to x and x
Contrast Consistent Reflection (CCR) proposed here as variant of CCS. Rather than finding hyperplane from which xt and x are equidistant, this method requires x and x to be each other's 's reflection in the hyperplane. It has the following objective:
Ocr
arg arg Ex/x |||x
Px 112,,
(4)
0
where unit vector ô determines the direction of the Householder reflection P= I 200T. This objective does not share the degenerate solution of CCS. This is because for p(x+)= = p(x ) = 0.5, we need x = 0 x = 0, and since [0] = 1 this would imply that A is orth to x and x Thus, while they are equidistant in that scenario (a distance of zero), assuming that x # x they will not be each other's reflection.
On our data CCS does not consistently converge to a good minimum (see Appendix D). CCS finds directions that vary considerably from layer to layer (see Appendix E) making harder to analyze. We see CCR achieve similar performance with more stable convergence. By including CCR, we can see how unsupervised methods compare to supervised methods, without having to worry about observations being artifacts of CCS's instability.
3.2 Probing for truth-value judgment
To probe for truth-value judgment we have setup as displayed in Figure 1, for example:
Q (premise)
December is [not] during the winter for New York. In New York, days are [not] shortest in December.
H (hypothesis)
The bracketed parts in Q and H are omitted or included to produce affirmed (Q+, (++ and negated (Q_, ") sentences. We use truth-value probes to see if the model represents / as true or false, and how this changes when preceded by Q+ or Q
There are different ways in which a belief could interact with the context of a statement. We define three kinds of beliefs in the following way:
prior beliefs, independent of the context, given by PH(H);
conditional beliefs, specially where the context is assumed to be truthful, given by Pa (H)g)) marginal beliefs, where the truth of the premise and hypothesis are modeled jointly, with the effect of the premise summed out, given by Σ, (H,
Each of these types of beliefs are candidates for what is measured by p (h; q), probe applied to LLM activations for hypothesis H when preceded by a premise Q. Note that there could also be beliefs in between conditional and marginal, where the context's truthfulness is not totally assumed, but still biased (e.g. towards being true rather than false).
3.3 Evaluation
To evaluate if probes coherently incorporate in-context information, we include four error scores, each indicating to what degree probe outputs violate desired behavior.
We first define the premise effect (PE) as the difference in probability assigned to the hypothe- sis when preceded with an affirmed premise and probability assigned to the hypothesis on its own: PF = p(h; - phh). We call a method's mean absolute premise effect its premise sensitivity. A value close to zero for this metric would be consistent with prior belief.
The effect of adding the in-context premise can differ in magnitude depending on which probing method we use. In order to make the error scores of different methods comparable,
2We leave this expression unsimplified to emphasize the dependence on the joint distribution, which is what distinguishes marginal from prior beliefs.
4

---

Published as a conference paper at COLM 2025
express the errors in multiples of the premise effect PE. This makes the error scores independent of the overall premise sensitivity of the probing method.
The first two error scores, El and E2 (see Table 1) are based on the fact that expect the probabilities to depend only on factors that are actually capable of influencing the truth value of the hypothesis. Thus, these error scores are proportional to the absolute change in probability that occurs after having the hypothesis preceded by either: 1) a corrupted premise Q or 2) an unrelated premise Q' The truth value of both corrupted and unrelated premises are independent of the truth value of the hypothesis, which why we want the equalities for El and E2 in Table to hold.
E3 and E4 measure when probes fail behave like conditional and marginal beliefs, respec- tively. Consider the example: Q="December is during the winter for New York" and H="In New York, days are shortest in December".
If the model assumes the premise is true (a conditional belief, p(;;q) ~ Px when determining the probability for H, then either: (1) having the context say 'Q is incorrect' should decrease the probability of H, or (2) having the context say 'Q is correct' should increase the probability. This expectation is captured by E3. For the error score, we have: (p(h:q p(h)) PE- = (p(h:g-))
p(h)) - pph))- ) We want the effect of negated premise to be opposite of positive premise: when the numerator is positive, we want the denominator to be negative and vice versa. Taking max{-,0) of this fraction, we can isolate the cases where the numerator and the denominator have the same sign, which are the errors we want to capture in the score.
If the model instead bases itself on its own evaluation of the truth of Q (a marginal belief, i.e. it models Q and H jointly: ph; = L, PA (h, Q)r)), then having 'Q is incorrect' or 'Q correct' should not influence the probability of H at all. This expectation is captured by E4. In that case, the probability assigned to the hypothesis should be the same regardless of whether the premise asserted or denied.
Because low scores for E3 and E4 indicate two equally valid types of truth-value judgment, we do not expect the score to be low for both. See Appendix A for additional details.
Table 1: Expected behavior and corresponding error scores. The subscript e and indicate hypotheses entailed or contradicted by their premise.
(in)equality
error score
E1
(h Q) (h)
Iph:a ) )- p-h()) |PE
E2
(x(h(')) = Pb)
|p(h:g' ) p(h)l |PE-
(a
& (h) & (hellttt)
F3
max (px((p(h; p(h)) PE-1, 0}
2 2 (helgg)
F4
Pa Σ, (h
|p(h:q p(h:q(()))))))) -IPEE=
4
Experiments
In our experiments, we make use of datasets with samples of related sentences whose truth values depend on each other. We use samples from these datasets by creating prompts where the sentences are either affirmed or negated.
We train probes in no-prem and pos-prem setting. For no-prem, the premise Q is left out, and for pos-prem the premise appears in the positive (or affirmed) variant. We include these settings to better understand how truth-values are represented. A direction found in the no-prem setting we might expect to represent prior belief. that direction shows context-sensitivity (when evaluated with premises in-context), that is evidence that the model does not represent the prior and contextual beliefs independently (in orthogonal directions). For pos-prem, the direction found is also influenced by what appears in context. the directions found for pos-prem and no-prem are different, it suggests there a separate (but possibly related) direction used to represent contextual belief.
5

---

Published as a conference paper at COLM 2025
The probe inputs h are the representations of the period following the answer tokens ('correct' / 'incorrect') extracted for each layer. To compare across probing methods we calibrate the probes such that their predictions for the p(h) case have the same variance. We train probes on the following LLMs: Llama2-7b, Llama2-13b (Touvron et al., 2023), and OLMo-7b with and without instruction tuning (Groeneveld et al., 2024).
To measure the premise effect, and error scores described in subsection 3.3, we include the fol- lowing evaluation cases: p(h) (no premise), p(h;q*) (affirmed premise), g (negated premise), p p(h;4') (unrelated premise), and q(:: (corrupted premise). We evaluate both the no-prem and pos-pren in all of these cases. The first two cases are 'in distribution' for the no-prem and pos-pret settings, respectively. The other combinations out of distribution.
When evaluating the probes we use: p(h) 2 (1 p(h ) + +(hh))
Data
We use two existing datasets in our experiments. The first dataset (EntailmentBank, Dalvi et al., 2021) contains hypotheses that are sentences with general world knowledge. These are facts the LLM may have encountered during training and for which it could already have strong prior belief. The second (SNLI, Bowman et al., 2015) contains statements that describe images, to which an LLM has no access. For both datasets, the corrupted sentences are created by replacing the characters in each word of the base sentence with random characters. The polarity of the premises and hypotheses are determined by switching between sentences that say something is 'correct' and saying that it is 'incorrect'. This style of negation avoids some problems that might otherwise arise.
EntailmentBank This dataset contains statements with entailment relationships. The dataset was derived from ARC (Clark et al., 2018), which consists of grade-school level science questions. We combine premises from EntailmentBank with the questions and answers from ARC. The questions are answered correctly or incorrectly to create both entailments and contradictions. For example:
You given the following question:
> In New York, the shortest period of daylight occurs during? (A) December (B) June The statement "New York is located in the northern hemisphere." is [in]eorrect.
Q₂
Q2 The statement "December is during the winter for New York." is [in]correct. Answering the question with "(B) June" is [in]correct.
H
The answer "June" is incorrect, and thus H contradicts the information in Qa, Q2 (when it is not negated), while in the sample with the correct answer would be entailed by Qa Qb- The dataset contains trees of entailing sentences, but we disregard anything but the first level of supporting premises. For the p(h; q) case, we use the distractor premises provided in the dataset. These were ranked as potentially relevant, but during annotation were not selected to be part the entailment tree (Dalvi et al., 2021).
SNLI This dataset is a Natural Language Inference dataset, it consists of premise- hypothesis pairs, which are labeled as: entailment, contradiction, or neutral; describing the meaning relation between the sentences. This dataset was created based on the descriptions of images. To avoid ambiguity, we establish a context as follows:
You are looking at a picture (A) which is placed next to an unrelated picture (B). Describing picture (A/B) as: "Four children playing in some water." [in]correct.
Saying (about picture A) that: "The children are wet." is |in]correct.
The neutral sentences for the ((::4') case are obtained by taking the premise from a dif- ferent, randomly sampled mremisehhypothest that appears in curly brackets is set to to ensure that there is a fully neutral relationship. Without it, the fact that the two sentences (simultaneous) truth less likely. It is also set to B for p(:; q), and set to A for all other cases.
pair. Furthermore, for this case, the 'A/B' /B'
about the same picture could make their
For example, negating "four children are playing in some water" as "four children are not playing in some water", still presupposes the existence of four children. Using negative meta statement leaves open the possibility that the presupposition is false (e.g. the number of children is inaccurate).

---

Published as a conference paper at COLM 2025
Without access to the picture, the model's prior belief should result in 50% accuracy. How- ever, for SNLI is possible to predict the label solely from the hypothesis (Poliak et al., 2018). This makes for interesting scenario when it comes to truth-value probing. A probing method might identify a direction that only encapsulates a statistical pattern, rather than model's truth-value direction. It is also possible that the statistical pattern is absorbed into the model's 's truth-value direction, as simply another reason to believe the sentence. After the addition of premise, we do not expect representation should move (coherently) in a direction which merely encodes statistical pattern. Thus, if a probe trained only on hypotheses does respond coherently the presence of premise at test time, it is further evidence of the probe uncovering truth-value judgment, and not just statistical pattern.
4.1 The effects of altering premises
We evaluate the probes on held-out data, including data from all the other variants. We also include an additional baseline, based on the model's LM-head, where the probabilities assigned to the 'correct' /'incorrect' tokens are rescaled to sum up one.
Table gives an overview of the average probabilities for p(:; p(:; and p(h), split by whether the hyyotse-ypoothesis pair had an entailment or contradiction relation. We observe that the probabilities assigned hypotheses depend strongly on the presence of relevant premises. When the hypothesis is entailed the probabilities are higher, and when the hypothesis is contradicted they are lower. This is true, even for probes trained without the premises present (no-prem), which also achieve good accuracy for the p(;; q*) case. Although the premise sensitivity is lower, it is clear that the directions identified in the no no-prem setting are not encoding specifically prior beliefs.
Table 2: Accuracy of p(h:q" (Acc), mean probabilities (orange=0, gray=0.5, blue=1), and trimmed mean errors scores for probes of each method on both datasets for Llama2-7b. The probes are from layers (L) with: (1) the best accuracy; and (2) the overall lowest error scores (by average error rank E+). The best scores per dataset are in bold, for E3 and E4 the bold values are based on their sum. CCS omitted, full table in Appendix B.
Entailment
Contradiction
Method
Acc
p(h:q") ) p(h:q
p(h)
p(h;q
p(h:q")
E1
E3
LM-head
.80
145.8
.61
.52
.50
.49
.38
.96
.90
.31
1.11
CCR
.63
141.4
.55
.52
.49
.48
.45
1.04
1.22
.99
.62
.58
127.4
.53
.51
.49
.48
.46
.93
1.17
.86
.74
no-prem
I.R
16
.93
160.0
.78
.59
.50
.41
.24
1.04
.90
.21
1.36
EntailmenBBnk
.92
107.6
.75
.61
.50
.39
.25
.89
.85
.28
1.15
MMP
.89
145.2
.71
.54
.49
.46
.31
.68
.79
.20
1.28
22
.86
103.6
.69
.53
.49
.47
.33
.71
.83
.31
1.17
CCR
.87
89.0
.54
.50
.46
.18
.56
.67
.05
1.27
pos-prem
14
.86
70.0
.84
.52
.50
.49
.18
.57
.65
.05
1.27
LR
18
.96
51.6
.92
.60
.50
.40
.10
.52
.58
.08
1.16
.95
.60
.49
.41
.11
.43
.56
MMP
14
.89
60.6
.86
.52
.50
.49
.16
.51
.61
.04
1.26
14
.89
60.6
.66
.52
.50
.49
.16
.51
.61
.04
1.26
LM-head
.62
150.6
.57
.54
,52
.43
.43
.89
.88
.36
1.35
CCR
.57
138.8
.52
.52
.53
.49
.49
.93
1.02
1.16
.26
.52
100.2
.53
.50
.74
no-prem
LR
13
.85
189.8
.67
.75
.50
.24
.32
.91
1.13
.89
1.13
20
.75
103.4
.65
.57
.50
.42
.35
.72
.96
.37
1.21
MMP
.88
178.2
.65
.50
.35
.38
.91
1.06
1.03
SNNS
32
.45
129.0
.48
.51
.51
.49
.52
.92
1.04
.68
.87
CCR
.91
53.8
.68
.50
.28
.14
.42
.53
.47
.60
pos-prem
28
.91
53.6
866
.70
.50
.28
.14
.41
.51
.49
.57
LR
16
.95
95.6
.93
.77
.51
.22
.06
.47
.61
.63
.42
.95
41.8
.68
.50
.29
.11
.38
.48
.44
.61
MMP
.94
90.0
.92
.77
.50
.20
.09
.46
.57
.68
.35
.74
49.6
.69
.63
.50
.34
.27
.39
.50
.62
.44
7

---

Published as a conference paper at COLM 2025
3.6
0.6
0.5
CCS UM-head
ECR
0.5
CER
CCS
Prom Poorist sensttvity
LK-head
0.4
0.4
0.3
Layer
Layer
(a) Trained on no-prem
(b) Trained on pos-prem
Figure Premise sensitivity for Llama2-7b on EntailmentBank.
LogiEMU4)
Mathod
Mothod
[M-tend
(N-head
IR
IR
HNP
НHP
10
15
20
25
30
10
35
20
25
30
Layer
Layer
(a) OLMo-7B
(b) OLMo-7B-Instruc
Figure Log-ratio of E3 and error score for probes trained using the no-prem variants of EntailmentBank on OLMo-7B and OLMo-7B-Instruuc.
Error scores. In Table 2, we can see that especially E1 and E2 are quite high. This suggests that the identified directions no-prem often have ET and E2 close to one. Because the error scores are normalized by the premise effect, value of one means that, on average, corrupted or unrelated premise has an effect with the same magnitude as the original affirmed premise. The error scores improve when probes are trained on pos-prem. Comparing Llama2-7b to Llama2-13b (see Table B.2) shows the scores are not consistently lower for the larger model.
sensitive to irrelevant information. Probes trained
LM-head baseline. Most probes beat the LM-head in terms of accuracy and premise sensitivity. This suggests that inconsistency can occur even when the LLM's representations contain information able to prevent it. This in line with findings for LLM hallucinations.
Premise sensitivity by layer. Figure 2 shows the premise sensitivity across layers for probes of each method when applied to Ilama2-7b. These were trained on the no-prem (left) and pos-prem (right) variants of the EntailmentBank data. All methods show a degree of premise sensitivity, with no-prem showing less than pos-pres There do not seem to be layers where the probe is not sensitive to the premises (approximating PA (H)), while still having above random accuracy (see subsection C.2), Suggesting that LLMs do not represent prior beliefs PA (H) fully independently.
Pretrained-only vs. instruction-tuned. Figure 3 In the later layers of the instruction- tuned model, it leans more toward E4 errors. The instruct-tuned model's behaviour is a lot more sensitive to whether the premise is negated or affirmed. This suggests that instruction-tuning makes the model more likely to represent prior assertions true, which is consistent with the instruction-tunine tuning objective.
Spurious correlations. Looking at SNLI, both LR and MMP show premise sensitivity, suggesting that they find directions indicative of more than just the spurious correlations present in the hypotheses of SNLI. However, for LR the probe's behaviour does seem affected by the spurious correlations. Its average probabilities for samples with negated premises is not between the probabilities obtained for samples with positive premises and premises, resulting in high E3+E4 score.
8

---

Published as a conference paper at COLM 2025
4.2 Intervening premise representations
In this experiment, we alter the LLM's in- ternal representations directly, rather than only altering the input data. We take the directions found by the probing methods in the first experiment, and move the represen- tations of the premises along this direction.
0.10
Methed
OCR
Moan diference
9.00
-2.05
We perform this experiment for the p(:; g+) and p(h;q-) cases on the EntailmentBank data. We move the premise in the direc- tion found during pos-prem training, and use that same direction to evaluate pre- intervention: (h;4), and post-intervention: (h; ((h:do(q ()) We perform the interven- tion using the same method and parameters as Marks & Tegmark (2024). The intervention done on Llama2-136 in layers 8-14, and applied to the representations of the answer tokens (correct, incorrect), and the period after. All interventions have the same magnitude: [6mm
25
Layer
Figure Intervention effect: mean difference in probability dh; layer, for entailments and contradictions.
(h; by
the effect of the causal intervention for the p(h;qt) case. When we move the affirmed premises backwards in the truth-value direction, the probabilities of entailed and contradicted hypotheses decrease and increase, respectively. This shows that truth-value directions causally mediate the incorporation of in-context information. We see that intervening with the direction found by LR has smaller effect than MMP and CCR. The largest change a reduction of around ten percentage points for entailed hypotheses. See Figure C.1 for the results of p (()pd(((()))))))
Results. In Figure 4,
Discussion & Conclusion
We have investigated LLM truth-value judgment, which requires correctly incorporating context when determining the truth value of a sentence. Based on our expectations of how the probability of sentence should or should not change in a supporting, contradicting, or neutral context, we created four error scores. In our experiments, we used several probing methods on four language models, and quantified how they assign probabilities to hypotheses in different contexts. From our results, the following becomes clear:
LLMs incorporate context when representing sentences as more or less (likely to be) true. However, contexts which should have no bearing truth values still have sizable impact on a sentence's position along the direction identified by the probes. Thus, in our evaluation the LLMs exhibited only limited degree of coherence, suggesting attributions of belief to L.MSs based on current truth-value probes are unfaithful.
The positioning of premises along truth-value directions partially determines the posi- tioning of related hypotheses along the same direction. This shows the directions are causal mediators of the inference process, which is likely part of mechanism that, when fully uncovered, will help explain how LLMs tackle reasoning tasks.
Among the tested truth-value probing methods, Logistic Regression failed to reveal levels of coherence and causal mediation that the others did, showing its limitations.
In principle, the lack of coherence can be attributed to both flaws in the probes or flaws in the model. However, we include multiple probing methods, thereby mitigating the risk that the results are due to particular flaw in any single probing method. The sensitivity to irrelevant information we report is also consistent with previous black-box evaluations (Shi et al., 2023). If the probes are at fault, then methodology provides new ways of evaluating, helping to distinguish between good and bad truth-value probes.
Our findings further show the existence of separate (albeit possibly related) directions that can be found depending on whether the probes are based on inputs consisting of individual sentences (no-prem), sentences that occur in context (pos-pren) Recently, Bürger et al.

---

Published as a conference paper at COLM 2025
(2024) showed that truth-values in LLMs occupy two-dimensional subspace: one direction consistently points from true to false, and another is polarity-sensitive and points from false to true for negated statements. Future work should investigate whether there is subspace which encodes truth-values different ways, corresponding more closely to either prior, marginal conditional beliefs.
Future work should also seek to better understand the representations of meaning-relations in LLMs, and the exact mechanisms responsible for incorporating that information into the truth-value directions. For example, by investigating the construction of probes that reveal a model represents two sentences as having a particular meaning relation.
Acknowledgments
This research was supported by Huawei Finland through the DreamsLab project. All content represented the opinions of the authors, which were not necessarily shared or endorsed by their respective employers and/or sponsors.
References
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference Empirical Methods in Natural Language Processing, pp. 632-642, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10.18653 /vl/ D15-1075. URL https:: /twp:////wwwwww///// aclweb............ ttgg/antholog/D15-1110
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering Latent Knowledge in Language Models Without Supervision. In Proceedings of the Eleventh International Conference on Learning Representations, February 2023. URL https: htttpp://opeerreieew..ro forum?1d-ETKGubyhhe..
Lennart Bürger, Fred A. Hamprecht, and Boaz Nadler. Truth is Universal: Robust Detection of Lies in LLMs. Advances in Neural Information Processing Systems, 37:138393-138431, December 2024. URL httpp::/pproceediiggg nwwrips.e.... cc/paper Files//aperr/2224/hahh) html.
David J. Chalmers. Propositional Interpretability in Artificial Intelligence, January 2025. 15740. arXiv:2501.1574) [cs].
URL
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have Solved Question Answering? Try ARC, the A12 Reasoning Challenge, March 2018. URL http:::/aaxx..oor////aaa// wttp://wwwrr/bb////....oo 803.00557. arXiv:1803.05457 [cs].
Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. Explaining Answers with Entailment Trees. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7358-7370 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/0011 www.://1..mmmmmmmmm55.5.. URL
org/ wrgg//220 2021 emnlp-main...mm..mmmmmmmmmmmaa 585.
Sebastian Farquhar, Vikrant Varma, Zachary Kenton, Johannes Gasteiger, Vladimir Mikulik, and Rohin Shah. Challenges with unsupervised LLM knowledge discovery, December 2023. URL
10029. arXiv:2312.10029 [cs].
Peter Gordon. The Truth-Value Judgment Task. In Dana McDaniel, Helen Smith Cairns, and Cecile McKee (eds.), Methods for Assessing Children's Syntax, pp. 206-226. The MIT Press, August 1996. ISBN 978-0-262-27941-3.
doi: 10.7551/ https///dddreettt.ee///o///////o http:///oooo/oooo/////////////// URL https:// iiiecttmmit... -16977/The-Truthhhe -aupghh--Valle---uggneee------s Juskment-Task.
10

---

Published as a conference paper at COLM 2025
Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magmusson, Yizhong Wang, Shane Arora, David Atkin- son, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muen- nighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, William Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah Smith, and Hannaneh Hajishirzi. OLMo: Accelerating the Science of Language Models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Pappers), pp. 15789- 15809, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653 /ttp//1122222222 /22222..cl-oooo..444. URL https://aclantholll hwwpethhooogoooooo///eeee wrg.///////// ac1.4...cl100 acl-long.84111 841/.
Daniel A. Herrmann and Benjamin A. Levinstein. Standards for Belief Representa- tions in LLMs. Minds and Machines, 35(1):1-25, March 2025. ISSN 1572-8641. doi:
com/article/1e, 1007/ s11023-024-09709- 6. Company: Springer Distributor: Springer Institution: Springer Label: Springer Number: 1 Publisher: Springer Netherlands.
Alon Jacovi and Yoav Goldberg. Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4198 4205, Online, July 2020. As- sociation for Computational Linguistics. doi: 10.18655 v1 /tl/22.aclmmmin.335833 URL acl-main. 386.
https://www/ aclweb. org/ orgganthology//e
Benjamin A. Levinstein and Daniel A. Herrmann. Still no lie detector for language models: probing empirical and conceptual roadblocks. Philosophical Studies, February 2024. URL https::/doi.orgg// 1007/
ISSN 1573-0883. doi: s1 511098-023- 00198-023-02094-3.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Watten- berg. Inference-Time Intervention: Eliciting Truthful Answers from Language Model. Advances in Neural Information Processing Systems, 36:41451-41530, De- cember 2023. URL httpss://prooeeeinngs www.riiiiiinnnninni.ieeee httpppperr.fllesppppe///e/// oorp html..
Samuel Marks and Max Tegmark. The Geometry of Truth: Emergent Linear Structure in Representate
Large Language Model Representations of False Datasets. In Proceedings of the First Conference on Language Modeling, August 2024. URL https: httt::://ppenreieww...o net/
Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis Only Baselines in Natural Language Inference. In Maly- ina Nissim, Jonathan Berant, and Alessandro Lenci (eds.), Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pp. 180-191, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/ S18-2023. URL
2023.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrele- vant context. In Proceedings of the 40th International Conference Machine Learning, volume 202 of ICML 23, pp. 31210-31227, Honolulu, Hawaii, USA, July 2023. JMLR.org.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning
11

---

Published as a conference paper at COLM 2025
Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open Foundation and Lune-Tuned Chat Models, July 2023. URL http: wwwaaaiiw......asseeerae
09288. arXiv:2307.0 .09288 [cs].
12

---

Published as a conference paper at COLM 2025
A
Error scores
Here we try to give some (geometric) intuitions for our error scores. Specifically, we make use of the diagrams presented in Figure A.1. These diagrams take as baseline the probability assigned to the hypothesis on its own p(h), and show all other probabilities relative to it. The diagram assumes we are looking at premise-hypothesi pairs with entailment relations. The diagrams for contradictions would be identical, but mirrored vertically.
E1 and consistency errors are shown box in Figure A.1. Both of these errors involve the differ- ence in probability assigned to (a) the hypothesis on its own and (b) the hypothesis preceded with an irrelevant statement, which either:
p(h:q')
p(h:3)
p(h)
E2
A
premise where the characters have been placed by random characters p(; or premise that has been replaced by another randomly sampled premise p(h;q').
p(h:q")
E4
p(h:q")
See Appendix F for examples.
E3 and E4 consistency errors are indicative of two opposing behaviours potentially exhibited by language model. E3 assumes that the context (con- taining the premise) truthful, and that what is asserted should be taken at face value. If con- tradicting premise is (said to be) true this should reduce the probability assigned to the hypothesis, and if a supporting premise is (said to be) true it should increase the probability assigned to the hypothesis. On the other hand, E4 is assumes that the model uses its own evaluation of the context, ignoring if asserted to be true or false. If this the case, then the probability assigned to the hy- pothesis should not depend on the truth value that asserted of the premise. These two are displayed in three different scenarios (B, C, D) in Figure A.1.
E3
p(h)
B
p(h:q")
p(h:q")
E3
p(h)
C
In B, we have ph) < p(;; < p(;;q +t, in this scenario it is always the case that E3 + E4 = (recall that the error scores are given as multiples of PE = (h; overall consistency of the model this is the best score for E3 + E4 that we can expect.
p(h:q
p(h)). When evaluating the
E4
In C, we have p(h) < p(h;q*) < ph; this scenario is 'double wrong', in that there is now part of the probability that is punished by both error scores. Regardless of whether the model trusts that the context is truthful or trusts itself, it should never give higher probability to tailed hypothesis after seeing the premise negated than when saw it affirmed.
P(h)
D
p(h:q")
Figure A.1: Error score diagram.
In D, we have ((;;g))= < p(h) < ph:; now we have E3 equal to zero, since it is perfectly acceptable for the probability of the hypothesis to decrease when preceded by a negated supporting premise. This can occur in two ways, either the supporting premise became a contradicting premise and thus makes the hypothesis less likely, or the premise became neutral, in which case it still takes away one (potentially important) reason to believe the hypothesis.
13

---

Published as a conference paper at COLM 2025
B
Additional Tables
B.1
Llama2-7b
Entailment
Contradiction
Method
L
Acc
E*
p(;qq"
p(h;q
p(h)
ph:;
p(h;
E2
E3
E4
LM-head
.80
214.0
.61
.52
.50
.49
.38
.96
0.90
.31
1.11
CCR
14
141.4
.55
.52
.49
1.04
1.22
29
.58
127.4
.53
.51
.49
48
.46
.93
1.17
.86
.74
CCS
19
.71
241.0
.58
.52
.50
48
.42
.95
1.08
.79
.91
no po-prem
22
.34
170.6
.45
.49
-50
-00
.55
.87
.97
.89
.50
LR
16
.93
160.0
.78
.59
.50
.11
.24
1.04
.90
.21
1.36
EntailmentBank
14
.92
107.6
.75
.50
.25
.28
1.15
MMP
19
.89
145.2
.71
.54
.49
46
.31
.68
.79
.20
1.28
22
.86
103.6
.69
.53
.49
.47
.33
.71
.83
.31
1.17
CCR
16
.87
89.0
.86
.54
.50
.18
1.27
14 .86
70.0
.84
.52
.50
.49
.18
.57
.65
.05
1.27
pos pos-pren prem
CCS
28
.91
121.4
.86
.56
.50
.4
.15
.48
.55
.05
1.20
14
.89
83.0
.87
.54
.50
.46
.15
.54
.63
.06
1.21
LR
18
.96
51.6
.92
.60
.50
440
.10
.52
.58
.08
1.16
14
43.6
.60
.49
1.16
MMP
14
.89
60.6
.86
.52
-50
.49
.16
-51
.61
.04
1.26
14
.89
60.6
.86
.52
.50
.49
.16
.51
.61
.04
1.26
LM-head
.62
150.6
.57
.54
.52
1.35
CCR
7
.57
138.8
.52
.22
.53
.49
.49
.93
1.02
1.16
.26
12
.52
100.2
.51
.53
.51
.47
.50
.74
.95
.99
.27
CCS
12
.73
164.8
.55
.53
.48
prem
18
.34
162.2
.48
.49
-11
-51
.52
.78
.91
.96
.22
LR
13
.85
189.8
.67
.75
.50
.24
.32
.91
1.13
.89
1.13
20
.75
103.4
.65
.57
.50
.42
.35
.72
.96
.37
1.21
SIIL IT
MMP
13
.88
178.2
.61
.65
.50
.35
.38
.91
1.06
1.03
.54
32
129.0
.48
.51
.52
1.04
CCR
26 .91
53.8
.87
.68
.50
.28
.14
.42
.53
.47
.60
28 .91
53.6
.86
.70
.50
28
.14
.41
.51
.49
.57
CCS
13 .95
159.2
.97
.79
.50
.33
.08
.52
.65
.66
.36
26
.88
65.4
.85
.74
.51
25
.15
.38
.50
.62
.43
16
95.6
.93
.77
.22
26
.95
41.8
.88
.68
.50
.29
.11
.38
.48
.44
.61
MMP
17
.94
90.0
.92
.77
.50
.20
.09
.46
.57
.68
.35
6
.74
49.6
.69
.65
.50
.34
.27
.39
-50
.62
.44
Table B.1:
Accuracy (Acc), mean probabilities (orange=0, gray=0.5, blue=1), and errors scores for probes of each method on both datasets. The probes are from layers (L) with: (1) the best probe accuracy; and (2) the overall lowest error scores (by average error rank E*).
14

---

Published as a conference paper at COLM 2025
B.2 Llama2-13b
Entailment
Contradiction
Method
L
Acc
E*
p(bg
p(b,
p(h)
p(h;
p(h;q"
E1
E2
E3
E4
LM-head
.88
233.8
.61
.58
.49
.22
.37
1.38
1.18
.60
1.50
CCR
21
.94
232.0
.71
.55
-50
.45
.31
1.67
1.38
.69
1.42
.58
135.8
.52
.52
.49
1.01
1.16
.25
porpeem
LR
17
.93
250.8
.70
.61
.50
.40
.31
1.80
1.45
.63
1.34
EntailmentBank
9
.63
125.0
.56
.57
.49
40
.42
1.04
1.06
.66
.84
MMP
20
.94
207.4
.72
.57
.50
.3
.30
1.48
1.20
.49
1.39
9
.63
123.4
.55
.55
.48
.43
.43
.93
1.11
.83
.41
CCR
19
.92
98.4
.85
.59
.50
1.35
pos perpprenn pos-prem
15
-90
60.2
.84
.59
.50
.41
.17
.65
.61
.08
1.27
LR
17
.98
63.8
.90
.67
.50
.34
.12
.54
.48
.13
1.00
15
.97
36.4
.90
.66
-51
.35
.12
.56
-51
.12
1.02
MMP
17
.93
98.2
.86
.58
.50
.42
.17
.70
.60
.07
1.33
15
56.6
.59
.50
1.24
LM-head
.87
247.0
.59
.61
.49
.66
.35
1.25
1.10
.83
.85
CCR
21
.82
163.6
.58
.54
.49
.66
.41
.87
1.03
.89
.44
13
.69
154.0
.53
.51
.51
49
.47
.89
.97
1.00
.27
no-prem
LR
19
.87
229.4
.68
.66
.50
.31
.29
1.07
1.07
.70
1.02
4
.58
143.8
.54
.55
.50
.44
.45
.78
1.04
.79
.47
SNNS IN
MMP
19
189.4
.64
.55
.50
.34
24
.88
140.6
.65
.57
-11
.42
.32
.79
.89
.67
.77
CCR
15
.92
115.6
.91
.69
.51
.28
.10
.40
.53
.49
.55
pos- prem
8
.70
73.6
.68
.63
.52
.38
.33
.38
.48
.56
LR
18
.98
93.0
.93
.73
.51
.26
.06
.39
.54
.47
.57
17
.98
51.6
.94
.70
.29
.06
MMP
18
.94
109.4
.89
.66
.51
.32
.11
.50
.64
.40
.70
4
.69
68.2
.64
.53
.50
.47
.34
.40
-50
.08
1.13
Table B.2: Accuracy (Acc), mean probabilities (orange=0, gray=0.5, blue=1), and errors scores for probes of each method on both datasets. The probes are from layers (L) with: (1) the best probe accuracy; and (2) the overall lowest error scores (by average error rank E+).
15

---

Published as a conference paper at COLM 2025
C
Additional Figures
C.1
Causal experiment moving negated premises toward truth-value direction
0.10
Mean difference
0.05
Method
CCR
0.00
LR
MMP
-0.05
-0.10
5
10
15
20
25
30
35
40
Layer
Figure C.I: Mean difference in probability p(h; + --0)) - p(h;q ) after moving
negated premises the positive truth-value direction.
C.2 Premise sensitivity and accuracy
Llama2-7b
0.8
Acouncy@no
0.6
0.6
Accursy
Method OCR
Method
COR
CCS
OCS
LM-hood
SMP
KHP
0.0
Figure C.2: Llama2-7b EntailmentBank Accuracy no-prem. Probes trained no-prem (left) and pos-prem (right).
1.0
1.0
0.8
0.6
0.4
Mothod CCR OCS
Mehhod
CCR
CCS
LMhead
hod
LR
MMP
NHP
00
00
Layer
Figure C.3: Llama2-7b EntailmentBank Accuracy pos-prem. Probes trained no-prem (left) and pos-prem (right).
16

---

Published as a conference paper at COLM 2025
Kethod CCR
Method
0.5
ECS
0.5
COR
DCS
Premis soroties
LM nead
Lishead
0.4
0.4
HN?
VNF
Premire
02
01
Layer
Figure C.4: Llama2-7b EntailmentBank Premsise sensitivity. Probes trained on no-prem (left) and pos-prem (right).
OLMo-7b
1.0
10
Accurorrrr prom
Sccura//ree
0.4
6.4
Method
10
15
20
25
30
5
20
15
20
25
50
Layer
Layer
Figure C.5: OLMo-7b Banoo--eb-EtaallmeeBBnnnk Accuracy on no-prem. Probes trained on no-prem (left) and pos-prem (right).
1.0
0.8
0.0
Acorsc srom
9.8
02
Layer
Layer
Figure C.6: OLMo-7b EntailmentBank Accuracy on pos-prem. Probes trained on no-prem (left) and pos-prem (right).
0.5
Promise censtrie
8.4
0.1
0.
Method
Method
CCR
CCR
CCS
CCS
LM4 baad
LW-hred
0.1
SMP
KHP
0.
Figure C.7: OLMo-7b EntailmentBank Premsise sensitivity. Probes trained on no-prem (left) and pos-pren (right).
17

---

Published as a conference paper at COLM 2025
D Llama2-7b Accuracy for 30 different seeds CCR vs. CCS
[m4]
snil
[=20]
|-24|
+f
0.9
accurey
1-4 ent bank
I- ent bank
I-12| ent bank
I-16| ent. bank
I-20 | ent bank
|-24 ent bank
0.9
Pyp
0.8
acorary
methed
methedd
Figure D.1: After 200 steps of full-batch gradient descent.
After 200 steps of full-batch gradient descent with learning rate of 0.001, CCR probes have already converged to much greater extent than CCS probes. All probes are trained in the pos-prem setting.
[md] | snf
Im8 snli
[-12]
[-16 snli
[=20]
|-24|
0.8
accuacy
0.7
1-4 ent bank
I- ent bank
1-12 | ent bank
|-16 I ent. bank
1-20 | ent bank
I-24 | ent bank
accunryy
0.7
Figure D.2: After 1000 steps of full-batch gradient descent.
After 1000 steps, the CCR probes have converged to a point where their accuracy scores are identical. CCS shows no continued convergence.
We leave a thorough analysis of CCS's failure modes and how to address them for future work.
18

---

Published as a conference paper at COLM 2025
E
Llama2-7b Cos Similarity 0 CCR (left) CCS (right)
0.55
030
15
0.80
0.00
-050
-1.50
-1000
-1.00
25
19
25
29
1yer
1aye2
Figure E.1: EntailmentBank no-prem Cosine similarity between directions of all layers.
1.00
0.75
025
0
-345
-0.25
3.35
0.75
-1.00
-1.30
Figure E.2: EntailmentBank pos-prem- Cosine similarity between directions of all layers.
050
050
33
35
0.00
0.00
3.30
0.50
-335
-3.75
-1.00
-1.00
15 23
25
27
31
19
21
20 29
1sye1
1aye2
Figure E.3: SNLI - no-prem- Cosine similarity between directions of layers.
075
075
025
0
3.25
0.25
-3.50
-3.0
-335
-1.00
-1.00
Figure E.4: SNLI pos-prem Cosine similarity between directions of all layers.
19

---

Published as a conference paper at COLM 2025
F
Data samples
Each variant of five samples from each dataset. Highlighted in red is the text that is inserted to convert positive sample X into a negative sample X
E1
EntailmentBank no-prem
You are given the following question:
> In clear weather, a bright light can be seen for a long distance. In conditions of heavy fog, the visibility is greatly reduced. Which of the following explains the reduced visibility? (A) Light is absorbed by water vapor near the ground (B) Light is scattered by water droplets in the Auswering the question with "(B) Light scattered by water droplets in the air." incorrect
You are given the following question:
> The main function of a tree's trunk is to provide (A) air (B) support
Answering the question with "(B) support" is incorrect
You given the following question:
The temperature in hot is high enough pull electrons away from atoms. What state of matter results from this process? (A) plasma (B) gas
Answering the question with ((B) gas" is incorrect
You are given the lollowing question: > Which measurement is best expressed in light years? (A) the time it takes for planets to complete their orbits (B) the distance between stars in the Milky Way Auswering the question with "(B) the distance between stars the Milky Way" incorrect
You are given the following question:
>
Some sinkholes and caves are created when water dissolves certain rocks and minerals below
ground. Which two parts of the water cycle are most directly responsible for the formation of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltration Answering the question with "(B) precipitation and infiltration" incorrect
F2 EntailmentBank original-neg-prrr
You given the following question:
In clear weather, bright light can be seen for a long distance. In conditions of heavy fog, the visibility is greatly reduced. Which the following explains the reduced visibility? (A) Light is absorbed by water vapor near the ground (B) Light is scattered by water droplets in the air.
The statement "Water droplets scattering light decreases the visibility." " incorrect.
The statement "Fog made of water droplets." " is incorrect.
Answering the question with "(B) Light is scattered by water droplets in the air." incorrect
You are given the following question:
> The main function of a tree's trunk is to provide (A) air (B) support
The statement PProvdding support is kind of function." " is incorrect.
The statement " trunk a part of tree for supporting the tree." is incorrect.
Answering the question with "(B) support" is incorrect
You are given the following question:
>
The temperature in a hot star is high enough to pull electroms away from atoms. What state of matter results from this process? (A) plasma (B) gas
The statement "Plasma will be formed by high temperature pulling electrons away from atoms." incorrect.
The statement "Plasma is a kind of state matter." is incorrect.
Auswering the question with "(B) gas" is incorrect
20

---

Published as a conference paper at COLM 2025
You given the lollowing question:
Which measurement best expressed in light-years" (A) the time it takes for planets to complete their orbits (B) the distance between stars in the Milky Way
The statement " "Light year is used to measure the distance between stars." is incorrect.
4 The statement "The milky way is made of stars." is incorrect.
Answering the question with "(B) the distance between stars in the Milky Way" is incorrect
You given the following question:
Some sinkholes and are created when water dissolves certain rocks and minerals below ground. Which two parls of the water cycle are most directly responsible for the formation
of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltration
3
The statement "Infiltration is slage in the water cycle process." " is incorrect.
The statement "Precipitation is stage in the water cycle process." is incorrect.
The statement "Sinkholes and caves formed by precipitation and infiltration." " is incorrect. Answering the question with "(B) precipitation and infiltration incorrect
F3 EntailmentBank oriiaaaa--eeiiiinnl-----eeee
You given the following question:
In clear weather, bright light can be seen for long distance. In conditions of heavy fog, the visibility is greatly reduced. Which the following explains the reduced visibility? (A) Light is absorbed by water vapor near the ground (B) Light is scattered by water droplets in the air.
3 The statement "Water droplets scattering light decreases the visibility." correct.
The statement "Fog made of water droplets." " is correct. Answering the question with "(B) Light is scattered by water droplets in the air." incorrect
You are given the following question:
> The main function of a tree's trunk is to provide (A) air (B) support
3 The statement "Providing support is kind of function." " is correct.
The statement "A trunk is part of tree for supporting the tree." is correct.
Answering the question with "(B) support" is incorrect
You given the following question:
The temperature in a hot star is high enough pull electrons away from atoms. What of matter results from this process? (A) plasma (B) gas
The statement "Plasma will be formed by high temperature pulling electrons away from atoms." is correct.
The statement " "Plasma kind of state of matter." is correct.
Answering the question with "(B) gas" incorrect
You are given the following question:
Which measurement best expressed light years? (A) the time lakes for planets to complete their orbits (B) the distance between stars in the Milky Way
3 The statement "Light year is used to measure the distance between stars." is correct.
The statement "The milky way is made of stars." " is correct.
Answering the question with "(B) the distance between stars the Milky Way" incorrect
You given the following question:
Some sinkholes and caves are created when water dissolves certain rocks and minerals below
ground. Which two parts of the water cycle are most directly responsible for the formation of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltratinn
The statement "Infiltration is stage in the water cycle process." " is correct.
The statement "Precipitation is stage in the water cycle process." " is correct.
The statement "Sinkholes and caves are formed by precipitation and infiltration." correct.
Answering the question with (B) precipitation and infiltration" incorrect
E4 EntailmentBank random neg-prem
You given the following question:
21

---

Published as a conference paper at COLM 2025
In clear weather, bright light can be seen for long distance. In conditions of heavy log, the visibility is greatly reduced. Which of the following explains the reduced visibility? (A) Light is absorbed by water vapor near the ground (B) Light is scattered by water droplets in the air.
3 The statement "Wpbjd qixtdxox Imhpuxdozz yulge veowqufns upb ujyodevflv." is incorrect.
The statement "Biy ax pxss mh cqbsx kmashuhk." is incorrect. Answering the question with "(B) Light scattered by water droplets in the air." incorrect
You are given the following question:
> The main function of a tree's trunk is to provide (A) air (B) support
3 The statement "Oyniagdvm esmktbg qo idpv ptmxrqog." is incorrect.
The statement "Y iguwd my eekb wi p owwr ntxmmvckwn krh sdrf." is incorrect.
Answering the question with "(B) support" is incorrect
You given the following question:
The temperature in a hot star is high enough to pull electrons away from atoms. What state of matter results from this process? (A) plasma (B) gas
3 The statement "Ttcimk ptdw kd fdxler sv chzh sfrptoxtptf scimart civpzttyb vywt xify qppgb." is incorrect.
The statement "Tspffit nvv i ilti tw kkapy kd riqigm." " incorrect. Answering the question with "(B) gas" is incorrect
You are given the following question:
>
Which measurement is hest expressed in hight-years" (A) the time it takes for planets to complete their orbits (B) the distance between stars in the Milky Way
3. The statement "Uchbk muic ql qbft olgiref iat fkhamshg venepxz cloni." is incorrect.
The statement "Yld vvsig lpd je ihmu ye xunns." incorrect. Auswering the question with "(B) the distance between stars in the Milky Way" is incorrect
You are given the lollowing question:
> Some sinkholes and caves are created when water dissolves certain rocks and minerals below
ground. Which two parts of the water cycle most directly responsible for the formation of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltration
The statement "Kbfjcebziplr yd n cleyi gf hine ntiww tdedl hgztuvy." incorrect.
The statement "Qywstpjnndzzunr ix v nyvun bj xlq vjrhb csiyj zmmqafy.." is incorrect.
5 The statement "Nbmdezjfs noa sxkwm oli ivrenv gq irehugwadltbe hwj bkktzxhkvdbb." is incorrect.
Answering the question with "(B) precipitation and infiltration" incorrect
F55 EntailmentBank pos-pre
You are given the following question:
> In clear weather, a bright light can be seen for a long distance. In conditions of heavy fog, the visibility is greatly reduced. Which of the following explains the reduced visibility? (A)
Light is absorbed by water vapor near the ground (B) Light is scaltered by water droplets in the air.
The statement "Wpbjd qixidxox Imbinxduzza yulge veowqufns upb ujyedevflvv" is correct.
4 The statement "Biy ax pxss mh cqbsx kmasluhk." is correct. Answering the question with "(B) Light is scattered by water droplets in the air." is incorrect
You given the following question:
The main function of a tree's trunk is provide (A) air (B) support
The statement "Oyniagdvm esmktbg qo idpv eg ptmxrqog." " is correct.
The statement "Y iguwd my eekb wi p owwr zen ntxrmvckwn krh sdrf." is correct.
Answering the question with "(B) support" is incorrect
You are given the following question:
The temperature in hot star is high enough to pull electrons away from atoms. What state of matter results from this process? (A) plasma (B) gas
'hee statement "Ttcimk ptdw kd fdxlær sv chzh sfrptoxtptf scimart cjvpzttyb vywt xjfy qppgb." is correct.
4 The statement " "Tspfft mv i ilti tw kkapy kd rtqigm." " is correct.
Answering the question with "(B) gas" is incorrect
22

---

Published as a conference paper at COLM 2025
You given the following question:
Which measurement best expressed
sight-years? (A) the time it takes for planets to complete their orbits (B) the distance between stars in the Milky Way
The statement "Uchbk muic ql qbft ew olglref iat fkhamshg venepxz ctoni." is correct.
4 The statement "Yld vvstg lpd je ihmu ye xnnns." is correct.
Answering the question with "(B) the distance between stars in the Milky Way" incorrect
You given the lollowing question:
Some sinkholes and caves are created when water dissolves certain rocks and minerals below
ground. Which two parls of the water cycle are most directly responsible for the formation
of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltration
The statement "Kbfjeebziplr yd cleyi gt hme ntiww tdedl hgztuvy." is correct.
The statement " "Qywstpindgzmm ix v nyvun bj xlq vjrhb esiyj zumqafy." correct.
The statement "Nbmdezjfs noa sxkwm oli ivreny gq ireluqwadltbe hwj bkktzxhkvdbh."
correct.
6 Answering the question with "(B) precipitation and infiltration" is incorrect
F6 EntailmentBank shuffle-neg-peen
You given the following question:
In clear weather, bright light can be seen for a long distance. In conditions of heavy fog, the visibility is greatly reduced. Which the following explains the reduced visibility? (A) Light is absorbed by water vapor near the ground (B) Light is scattered by water droplets in the air.
3.
The statement " "Clouds dusts block visible light." is incorrect. The statement "If an object reflects light toward the eye then that object can seen."
incorrect.
The statement "Difficulty seeing means visibility decreases." is incorrect.
Answering the question with "(B) Light is scattered by water droplets in the air." is incorrect
You given the following question:
The main function of tree's trunk is provide (A) air (B) support
The statement "Bark is a protective covering around trunk of branches of a tree." is incorrect.
4
The statement "The function of something is what that something is used to do." is incorrect. The statement " "Role means function." " is incorrect.
Answering the question with "(B) support" is incorrect
You given the following question:
The temperature in a hot star is high enough to pull electrons away from atoms. What state of matter results from this process? (A) plasma (B) gas
The statement "State of matter means physical state." is incorrect.
The statement "State of matter is kind of physical property." " is incorrect.
The statement "Physical state means state of matter." " is incorrect.
Answering the question with "(B) gas" is incorrect
You are given the following question:
> Which measurement is best expressed in light years? (A) the time it takes for planets to complete their orbits (B) the distance between stars in the Milky Way of how far object moves." is
The statement " "Distance moved] / distance travelled is incorrect.
The statement " "Measuring sometimes requires recording / learning an amount." is incorrect.
The statement "Light is kind of nonliving thing." is incorrect. Answering the question with "(B) the distance between stars in the Milky Way" is incorrect
You are given the following question: Some sinkholes and
are created when water dissolves certain rocks and minerals below ground. Which two parts of the water cycle are most directly responsible for the formation of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltratinn
The statement "In the water cycle infiltration can follow runoff." is incorrect.
4 The statement As the amount of rainfall increases the rate of chemical weathering will increase." " incorrect.
23

---

Published as a conference paper at COLM 2025
The statement "Rainfall means precipitation." is incorrect.
Answering the question with "(B) precipitation and infiltraion" incorrect
F.7 EntailmentBank- shuffle-pospppeeeeee
You are given the following question:
>
In clear weather, a bright light can be seen for a long distance. In conditions of heavy fog, the visibility is greatly reduced. Which of the following explains the reduced visibility? (A) Light is absorbed by water vapor near the ground (B) Light is scattered by water droplets in air.
The statement "Clouds / dusts block visible light." is correct.
The statement "If an object reflects light toward the eye then that object can be seen." is
correct.
The statement " "Difficulty seeing means visibility decreases." " is correct.
Auswering the question with "(B) Light scattered by water droplets the air." incorrect
You are given the following question:
> The main function of a tree's trunk is to provide (A) air (B) support
3 The statement "Bark is a protective covering around the trunk of branches of a tree." is correct.
The statement "The function of something what that something used do." correct.
The statement "Role means function." " is correct.
Answering the question with "(B) support" is incorrect
You are given the following question:
The temperature in a hot star is high enough to pull electrons away from atoms. What state of matter results from this process? (A) plasma (B) gas
The statement "State of matter means physical state." " is correct.
The statement "State of matter is a kind of physical property." is correct.
The statement "Physical state means state of matter." is correct.
Answering the question with ((B) gas" is incorrect
You given the following question:
Which measurement is hest expressed in light years?? (A) the time it takes for planets to complete their orbits (B) the distance between stars in Milky Way
The statement "Distance moved/ / distance travelled is a measure of how far an object moves." is correct.
The statement " "Measuring sometines requires recording / learning amount." is correct. The statement "Light is kind of nonliving thing." correct. Answering the question with "(B) the distance between stars in the Milky Way" incorrect
You are given the following question:
>
Some sinkholes and caves are created when water dissolves certain rocks and minerals below
ground. Which two parls of the water cycle most directly responsible for the formation of sinkholes and caves? (A) evaporation and transpiration (B) precipitation and infiltration
The statement the water cycle, infiltration can follow runoff." " is correct.
The statement As the amount rainfall increases the rate of chemical weathering will increase." is correct.
The statement " "Rainfall means precipitation." " is correct.
Answering the question with "(B) precipitation and infiltration" incorrect
E88 SNLI no-prem
You are looking at picture (A) which is placed next an unrelated picture (B). Saying (about picture A) that: AA man is rocking out on his guitar, while wearing funky costume." is incorrect
You are looking at picture (A) which is placed next to unrelated picture (B). Saying (about picture A) that: "the men at the restaurant eating" is incorrect
You are looking at picture (A) which is placed next to unrelated picture (B). Saying (about picture A) that: "The men playing badmitton." " is incorrect
24

---

Published as a conference paper at COLM 2025
You are looking at a picture (A) which is placed next an unrelated picture (B). Saying (about picture A) that: "The person is showing affection towards the dog." is incorrect
You looking picture (A) which is placed next unrelated picture (B).
Saying (about picture A) that: "The young girl isn't holding any flowers." " is incorrect
E9 SNLI - Loii-oorgiaal--e-preee
You are looking at a picture (A) which is placed next an unrelated picture (B).
Describing A as A man dressed in a funky outfit is playing guitar." is incorrect. Saying (about picture A) that: "A man is rocking out on his guitar, while wearing funky
costume." is incorrect
You are looking at picture (A) which is placed next to an unrelated picture (B).
Describing A "A quarterback is looking to set up a pass from the end zone, while teammate provides some blocking." " is incorrect.
Saying (about picture A) that: "the men are at the restaurant cating" is incorrect
You are looking at picture (A) which is placed next to unrelated picture (B). Describing A "Two athletes wrestle the floor of gymnasium several others stand near." " is incorrect.
Saying (about picture A) that: "The men are playing badmitton." " is incorrect
You looking at picture (A) which is placed next unrelated picture (B). Describing as "An elderly person holds a white doge and kisses their check,"" incorrect. Saying (about picture A) that: "The person showing affection towards the dog." is incorrect
You looking at picture (A) which is placed next unrelated picture (B). Describing as "A young girl holds flowers one hand and basket with a bow in another."
incorrect.
Saying (about picture A) that: "The young girl isn't holding any flowers." is incorrect
E10 SNLI- SNLI-orrignal--os-prees
You are looking at picture (A) which is placed next to unrelated picture (B). Describing A "A man dressed in a funky outfit is playing guitar." is correct. Saying (about picture A) that: man is rocking out on guitar, while wearing funky costume." " is incorrect
You looking at picture (A) which is placed next unrelated picture (B).
Describing as "A quarterback is looking to set up a pass from the end zone, while a teammate provides some blocking." is correct.
Saying (about picture A) that: "the men are at the restaurant eating" is incorrect
You are looking picture (A) which is placed next an unrelated picture (B). Describing A as "Two athletes wrestle on the floor of a gymnasium as several others stand near." is correct.
3 Saying (about picture A) that: "The men playing badmitton." is incorrect
You are looking at picture (A) which is placed next an unrelated picture (B). Describing A as An elderly person holds a white doge and kisses their cheek." correct. 3 Saying (about picture A) that: "The person is showing affection towards the dog." " is incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). Describing A as A young girl holds flowers in one hand and a basket with a bow in another." correct.
Saying (about picture A) that: "The young girl isn't holding any flowers." " is incorrect
25

---

Published as a conference paper at COLM 2025
E11 SNLI -enee
You are looking al a picture (A) which is placed next to an unrelated picture (B).
Describing B as "C okw dlhktsj wn z edplx fauzlg ft yrhixbt ozuhmf." is incorrect. Saying (about picture A) that: " "A man is rocking out on his guitar, while wearing funky costume." " incorrect
You are looking at picture (A) which is placed next to an unrelated picture (B). Describing B "R obvvillugee ztnesvg nt esl jo ilgh nuto mnv dhe qben, denyf Iltugint spshpmas xpbxewdy." " is incorrect.
Saying (about picture A) that: "the men are at the restaurant eating" is incorrect
You are looking at picture (A) which is placed next to unrelated picture (B). Describing B "Stg tbikesfy grznqix ule sgigy yc k qywzomiwx ey imiacty wjyobs usmom xmpb." " is incorrect.
Saying (about picture A) that: "The men are playing badmitton.." " is incorrect
You are looking at picture (A) which is placed next to unrelated picture (B). Describing B "Qt Ihndsef kknyzz patiu eepov rwdn lix lejowk jjtyq tifmp." " is incorrect.
Saying (about picture A) that: "The person showing affection towards the dog." " is incorrect
You are looking at picture (A) which is placed next to an unrelated picture (B). Describing B "H nnnvt lwnl poakr ljwgvyl na klc stxy hda i eqfhhd wqeo bea tz axqhavi."
is incorrect.
Saying (about picture A) that: "The young girl isn't holding any flowers." is incorrect
F12 SNLI - random-pos-prem
You are looking at picture (A) which is placed next an unrelated picture (B). Describing B as "C okw dlhktsj wn z edplx fauzlg ft yrhlxbt ozuhmf." is correct. 3 Saying (about picture A) that: "A man is rocking out on his guitar, while wearing funky costume." is incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). Describing B as "R obvvillugec ztnesvg nt esl jo u ilgh nuto mnv dhe qben, denyf) lltugint spshpmas uuza xpbxewdy." is correct.
Saying (about picture A) that: " "the men at the reslaurant eating" is incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). 2 Describing B as "Stg tbhkesfy grznqix xx ule sgigy k qywzomiwx ey imiacty wjyobs usmom xmpb." " is correct.
Saying (about picture A) that: "The men are playing badmitton." " is incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). Describing B "Qt Thndsef kknyzz patiu eepov rwdn liz lejowk jjtyq tifmp." is correct.
Saying (about picture A) that: "The person is showing affection towards the dog." is incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). 2 Describing B as "H nnnvt Iwnl poakr ljwgvyl na klc stxy hda i cqfhhd wqeo bea tz axqhavi." is correct.
Saying (about picture A) that: "The young girl isn't holding any flowers." is incorrect
E13 SNLI - shuffle-neg-pree
You looking picture (A) which is placed next unrelated picture (B). Describing as A bald man wearing black using a fan made of feathers, walking down the street," incorrect.
3 Saying (about picture A) that: AA man is rocking out on his guitar, while wearing funky costume." " is incorrect
26

---

Published as a conference paper at COLM 2025
You looking al piclure (A) which is placed next unrelated picture (B). Describing "Children all dressed the same are standing outside a building." incorrect. Saying (about picture A) that: "the men are at the restaurant eating" is incorrect
You are looking at picture (A) which is placed next to an unrelated picture (B). Describing B "There is one man in the foreground with hammer, another is in the background, possibly doing the
work the in the foreground." is incorrect. Saying (about picture A) that: "The men are playing badmition." " is incorrect.
You are looking at a picture (A) which is placed next to an unrelated picture (B). 2 Describing B as "Man walking by a corner market with gralfiti." is incorrect. Saying (about picture A) that: "The person is showing affection towards the dog." " is incorrect
You looking picture (A) which is placed next Describing as "Two men by the lake one dressed in a penguin costume while his friend runs along side of him." is incorrect.
unrelated picture (B).
3 Saying (about picture A) that: "The young girl isn't holding any flowers." is incorrect
E14 SNLI - shuu-ssuuelee-p-preet
You are looking al picture (A) which is placed next an unrelated picture (B). Describing B as "A bald man wearing black using a fan made of feathers, walking down the street." is correct.
Saying (about picture A) that: "A man is rocking out on his guitar, while wearing funky
costume." " incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). Describing B as "Children all dressed the same are standing outside a building." correct. 3 Saying (about picture A) that: "the men are at the restaurant eating" is incorrect
You looking at picture (A) which is placed next Describing "There is one man the foreground with a hammer, another the background, possibly doing the same work as the man in the foreground." correct. Saying (about picture A) that: "The men are playing badmitton." is incorrect
unrelated picture (B).
You are looking at picture (A) which is placed next to unrelated picture (B). Describing B "Man walking by a corner market with graffiti." " correct. Saying (about picture A) that: "The person showing alfection towards the dog." is incorrect
You are looking at a picture (A) which is placed next to an unrelated picture (B). 2 Describing B as "Two men by the lake one dressed in a penguin costume while his friend runs along side of him." " is correct.
Saying (about picture A) that: "The young girl isn't holding any flowers." " is incorrect
27


Reviews:
[{"review_id": "6BKKI2uO0x", "reviewer": "Reviewer_8KB7", "summary": "The paper studies the context-sensitivity of truth value judgments as estimated from truth direction probes. The author(s) introduce a framework to evaluate for context effects, distinguishing different types of errors. They run experiments using different probing methods, applied to different LLMs, and run extensive analyses on EntailmentBank and SNLI. They find that truth directions are generally context sensitive, but that there is also sensitivity to irrelevant context.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well-written and structured. The methods are clearly explained, adopting a rigorous notation.\n- The paper addresses a generally important topic for the NLP community (factuality in LLMs), focusing on a less studied angle: whether the truth value assigned to sentences is appropriately revised based on context information. This is a crucial aspect for general text understanding and instruction-following capabilities. \n- The author(s) consider a variety of probing methods, including highlighting limitations of some of them. On top of testing for the effect of altering premises, they also run a study where they modify the direction of context premises, providing evidence for a causal relation between truth directions and context incorporation.", "reasons_to_reject": "- Based on the current discussion of the results, it doesn't emerge as fully clear what results we should interpret as limitations of the probes (therefore of existing analyses methods) or of the LLMs. \n- To clarify the above, it might help to run a comparison of the context-sensitivity exhibited when probing for truth judgments based on truth direction probes vs. other methods (for instance, based on model outputs -- e.g., asking for truth values to the LLM directly). \n- I generally encourage the author(s) to expand on the Conclusions section with a wider discussion on the implications of their findings and directions for future research (for instance on the sensitivity to irrelevant information).", "questions_to_authors": "-  What results should interpret as limitations of the probes (therefore of existing analyses methods) or of the LLMs?\n- Have you run any analyses comparing the context-sensitivity of truth values as estimated by looking at model outputs?", "extra_scores": {}}, {"review_id": "nyiU1Rfrg6", "reviewer": "Reviewer_Gm35", "summary": "The paper seeks to interpret LLM latent spaces as including \"truth\ndimensions\" which can be acivated by promptong/probing with consistent\nor counterefactual entailing contexts during inference in QA tasks for\nLLaMa2 &B and 13B nad OLMo 7B models.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The method uses entailment paira from EntalimentBank and SNLI datasets\nin a hypothesis-QA task, when probed with negated vs. original premise\nand a neutral random premise against a hypothesis-only baseline.\nProbes are trained on premise and no-premise data using a variety of\nmethods including a novel CCR version of CCS, and tested on held-out\ndata.  Various measures of premise and hypothesis strength at the\nmodel head and levels are observed.  The method seems very sound.  The\nwork is clearly presented.", "reasons_to_reject": "Results are somewhat mixed.  There seems to be a strong premise\neffect, with positive premises boosting activation of the\nhypothesis and negated premises depressing it, supporting the\nhypothesised/predicted truth dimensions.    However, the sensitivity\nto irrelevant or corrupted premises under measures E1 and E2 suggests\nthat the truth value probes are unfaithful, so the relation of truth\ndimensions to inferential capability in LLM remains unclear.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SDuoAkCuSu", "reviewer": "Reviewer_PGfW", "summary": "This paper investigates how Large Language Models (LLMs) represent and judge the truth of sentences, particularly about context. The authors examine \"truth directions\" in LLM latent spaces that have been found to predict sentence truth. This is built upon previous research that has used these directions to probe LLM \"knowledge\" or \"beliefs.\" \n\nThe authors introduce four error scores to evaluate the coherence of truth-value judgments. They conduct experiments across different model architectures (Llama2-7b, Llama2-13b, and OLMo-7b) and training paradigms (pretrained vs. instruction-tuned). They also perform causal intervention experiments by moving representations along truth-value directions to test their influence on related hypotheses. \n\nThrough carefully designed experiments, the submission establishes how probes trained on these truth directions are sensitive to context and measure different types of consistency errors, especially when LLMs process hypotheses preceded by supporting, contradicting, or neutral premises. Based on this, the author proposes a new variant of the Contrast Consistent Search method called Contrast Consistent Reflection (CCR) with more stable convergence.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The submission presents a novel and important research problem of how LLMs represent truth in context, moving beyond single-sentence analysis to examine how context affects truth judgments.\n\nThe authors introduce a rigorous framework for evaluating the coherence of truth-value probes with clear error metrics. In addition, the causal intervention experiments provide valuable insights into how truth-value directions mediate natural language inference. \n\nThe proposed CCR method offers a more stable alternative to existing truth-value probing techniques. Extensive experiments have been conducted to provide a good insight into the proposed method.", "reasons_to_reject": "There are some writing issues with the submission, listed as follows:\n   1. **Confusing notations:** The section. 3.1 uses $\\textbf{x}$ to denote the LLM vector representation of a sentence. However, these are not further employed in the following discussion, instead of using $H$. The notation of $\\hat{\\mathcal{Q}}$ and $\\mathcal{Q}'$ is not introduced before Table 1. \n   2. **Ambiguous demonstration:** It will be easier for the reader to understand the content by explicitly giving the natural language demonstration with clear notation. For example, the author could consider giving demonstrations like: **Permise** (in-context information): December is not during the winter for New York; **Hypothesis** (assertion to be verified by LLM given the premise):  In New York, days are shortest in December.\n   3. It would be better to have the figure lies above the result discussion section (section 4.1), for better readability.\n\nThe experimental setup is somewhat artificial, relying on specific negation patterns that may not reflect how contradictions typically appear in natural text.\n\nThe designed experiment doesn't fully explore how the identified issues might impact downstream applications of truth-value probes in real-world LLM deployments.", "questions_to_authors": "1. Does the powerful model acquire the ability to handle premise sensitivity? For example, the closed-source model like GPT-4o, Claude-3.7-Sonet, or the latest reasoning model including QwQ-32B, DeepSeek-R1, and its distill variants like DeepSeek-R1-Distill-Llama-3.1-8 B.\n2. How might your findings about context sensitivity in truth directions inform techniques for making LLMs more truthful or reducing hallucinations?\n3. Did you observe any notable differences in how instruction-tuned models handle negation compared to base models beyond what was reported?", "extra_scores": {}}, {"review_id": "4z6k5UWTdI", "reviewer": "Reviewer_BqG9", "summary": "This paper investigates how context or premises interact with several different kind of truthful probes in LLM. The probes are trained with or without premises, and are tested with affirming or conflicting premises. The discovery is that there are interferences between the context and truthful directions.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Before attempting to apply truthful probes in real-life LLM monitoring or intervening, there should be more careful study to test how it generalizes out of the toy environment they were developed. And this work is exactly working on this with the emphasis on context. This is important directions to study. \n\nExperiment-wise, I like how the authors devices 4 different error forms and calibrate results by ratio in error rates. The selection of datasets and models also seem comprehensive enough to draw conclusions.", "reasons_to_reject": "I'm not entirely sure prior works claim that the truthful probes are supposed to focus only on the last sentence or the hypothesis. I certainly expect it to not do so, as discovered by authors. If I have read the paper correctly, L157 should be revised.\n\nThough the paper carefully examines the property, a reason that I can think of against this paper is that it doesn't fully surprise me or shed light on the mechanism of why the context following happens, and how can we fix that (discussed in future works).", "questions_to_authors": "NA", "extra_scores": {}}]

Author rebuttal:

Meta-review: Past work has found the existence of a "truth feature" in LMs that distinguishes true statements from false ones; this has sometimes been used to argue for the existence of belief-like states or internal "world models" in LMs. The current paper studies the extent to which truth features really function like beliefs---i.e. whether they remain stable in contexts containing unrelated information, or update appropriately in contexts that should modify a models' credence in the statement being evaluated. As noted by reviewers, results are somewhat mixed, but in some sense this is the point of the paper: not only are truth features context-sensitive; they are also sensitive to information that shouldn't bear on truth values at all. Overall I think this paper makes a focused, useful contribution to our understanding of knowledge representation in LMs.
Decision: Accept