Abstract Counterfactuals for Language Model Agents
Authors: Edoardo Pona
Milad Kazemi
Yali Du
David Watson
Nicola Paoletti
Venue: NeurIPS 2025 poster ()
Keywords: causal inference
counterfactual inference
language models
agents
causality
interpretability
Primary area: probabilistic_methods

Abstract: Counterfactual inference is a powerful tool for analysing and evaluating autonomous agents, but its application to language model (LM) agents remains challenging. Existing work on counterfactuals in LMs has primarily focused on token-level counterfactuals, which are often inadequate for LM agents due to their open-ended action spaces. 
Unlike traditional agents with fixed, clearly defined action spaces, the actions of LM agents are often implicit in the strings they output, making their action spaces difficult to define and interpret. 
Furthermore, the meanings of individual tokens can shift depending on the context, adding complexity to token-level reasoning and sometimes leading to biased or meaningless counterfactuals.
We introduce \emph{Abstract Counterfactuals}, a framework that emphasises high-level characteristics of actions and interactions within an environment, enabling counterfactual reasoning tailored to user-relevant features. 
Our experiments demonstrate that the approach produces consistent and meaningful counterfactuals while minimising the undesired side effects of token-level methods.
We conduct experiments on text-based games and counterfactual text generation, while considering both token-level and latent-space interventions.

Abstract Counterfactuals for Language Model Agents
Edoardo Pona
Milad Kazemi King's College London mi llld. kazemi@kcl. ac. uk
King's s College London edoardo. pona@kcl.accccc .uc
Yali Du
David Watson
Nicola Paoletti
King's College in London yali du@kcl. ac. uk
King's 's College in London david. watson@kel.cc ac uk
King's College in London micola. ptw.lla.palett@kk1111... ac
Abstract
Counterfactual inference a powerful tool for analysing and evaluating au- tonomous agents, but its application to language model (LM) agents remains challenging. Existing work on counterfactuals in LMs has primarily focused on token-level counterfactuals, which are often inadequate for LM agents due to their open-ended action spaces. Unlike traditional agents with fixed, clearly defined action spaces, the actions of LM agents often implicit in the strings they output, making their action spaces difficult to define and interpret. Furthermore, the mean- ings of individual tokens shift depending the context, adding complexity to token-level reasoning and sometimes lcading to biased or meaningless coun- terfactuals. We introduce Abstract Counterfuctuals, framework that emphasises high-level characteristics of actions and interactions within an environment, en- abling counterfactual reasoning tailored user-relevant features. Our experiments demonstrate that the approach produces consistent and meaningful counterfac- tuals while minimising the undesired side effects of token-level methods. We conduct experiments on text-based games and counterfactual text generation, while considering both token-level and latent-space interventions.
Introduction
The recent successes of Large Language Models (LLMs) have paved the way for a novel approach to developing autonomous agents. Previously, agents were typically trained isolated environments with limited knowledge. Language Model (LM) Agents (37] instead leverage their vast background knowledge (owing to their training on internet-scale datasets) to solve increasingly general tasks, including web browsing and research [38], multi-modal robotics [4, and navigating open- ppdeeended
environments [36] 39]. As such, LM Agents have also been studied for high-risk domains such as
medicine, law, and diplomacy [7 20 29 32 33) This, however, has important safety implications due to the (well-known) issues surrounding LLM safety, such as social biases and opaque reasoning
11 13 21
Causal and counterfactual explanations effective techniques to enhance the explainability and reliability of Al models. These techniques can answer how a model would have behaved in alternative (counterfactual) settings, given observations of its behaviour in factual setting [15] 24 34 This ability to reason about "what if"" scenarios is crucial in understanding responsibility and blame in
autonomous systems 6 j4 and deriving counterfactual policies-i.. i.e.. policies which, in hindsight,
would have been optimal with minimal interventions [2] [9] Recently, Ravfogel et al. [27] and Chatzi et 7 have proposed methods for counterfactual inference on LLMs based on structural causal models (SCMs) 24]. These two methods are the first apply SCMs for LLM counterfactuals.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Factual Context $
Counterfactual context S'
Token-level Counterfactual
taken lovel upproach agnostic the meaning of the action. makes courageoual actiona more thdy.. desp having deserved "auttous addon.
You the
walking through the jungle when lion behind You.
PLA' d. 2):
notion bear starinn
I:hrecce
02
P(Y" do(s"), A
P[A| e)
P(A| do(+)))
Cour courageous
0.5
stand your ground
0.5
I freeze
0.3
3 slowly wilk back
0.3
hide
fore Lsnn
S run for your lial
0.
observation
2: slilylly walt back
CA
Abstract Counterfactual
Dur abstrrctt countedaatuall are swore the meaning entooed by Y. Having choerand a 'csutiuu action, similar actions become more lotky.
P( de(+'), m]:
P(Y 8)
P(Y | do(se)) )))
fromn
04
P/Y" do(o"),
courageous
couragenuu
0.3
race the
01
cour agoous
01
cautious
cautious
dindty walt back
03
cautious
Figure 1: Abstract Counterfactuals overview. Our method considers the meaning of the observed action-encodeed in the abstraction Y "ndnd in the counterfactual setting, updates actions' probabilities according to the observed value of Y rather than the observed token A.
They define a token-level SCM, that models the sampling of individual tokens. We call these approaches token-level counterfactual (TLCF).
We argue that applying token-level inference to LM agents is inadequate for two main reasons. In open-text environments, where agents issue actions as arbitrary strings, the lack of a predefined action space requires the environment interpret language model (LM) outputs. Here, token-level inference may be inadequate for capturing the high-level semantics, emphasizing token-level syntax instead. Additionally, the high-level meaning of actions can vary with the agent's 'ss context; identical symbols may correspond to different actions across contexts, particularly in choice-based environments where actions are selected from predefined strings. For example, the meaning of tokens such as "choice 1" can differ widely across situations, an issue token-level methods fail to address (as illustrated in the subsequent example). In general, when LM agents serve as 'algorithmic models' [3] of real-world systems- for example, in agent- based simulations such Vezhnevets et al. B5 it may be preferable to compute counterfactuals at a higher conceptual level, abstracting away from the low-level (token-level) details of their computational implementations. We may also wish to leverage expert knowledge to explicitly define or inform this conceptual abstraction level. This challenge arises not because the token-generation function Sa fails encode context, but because counterfactual inference at the token level conditions on surface tokens whose meaning may shift or vanish under a changed context.
In this paper. we introduce an approach to LM agents' counterfactuals that overcomes the above limitations of token-level methods. The key idea is to introduce an abstraction Y of the LLM action A, so that Y A captures the high-level semantics of A and remains valid across contexts. Then, to derive counterfactual action a', instead of performing counterfactual inference directly on the tokens of A, our method does it at the level of Y. The resulting counterfactual abstraction Y" is then "mapped back" into the action space, i.e., we derive a' so that its abstraction Y a' is compatible with Y'. This way, we obtain the desired counterfactual (a') but override the token-level mechanism that generates the action. This also means our method requires only black-box access the LLM.
To better understand our approach, consider the hypothetical besed adventure game illustrated in Figure | In the factual setting, the LM agent's policy assigns distribution P(A s) over the possible action choices (i.e., the game options), encoded by tokens $1', '2', and '3', given the context S = s. Suppose the agent samples action '2', which, in s, corresponds to hiding after seeing bear in the woods, instead of running away (action '3') or facing the bear (action '1'). Now consider a counterfactual context s' where lion in the jungle is nearing the agent. Having observed our agent hiding from the bear in the woods, we want to predict what they would do in the jungle.
If we follow the token-level approaches of 26 and 7 counterfactual inference would increase the counterfactual probability of action (the action token previously observed). The problem that action '2' in context s' means the agent will face the lion, while in s, the same action token indicates

---

cautious behaviour. A sensible counterfactual should rather increase the probability of cautious behaviour in s'.
In this example, our technique would introduce an abstraction Y to represent the high-level semantics of the actions. For instance, Y can be binary variable that represents courage. So, in the factual context s, action '2' maps to Y = (not courageous); in the counterfactual context s', action '2' maps Y (courageous) while actions '1' and '3' map to Y O. In our inference approach, mediated by the abstraction Y, observing Y = 0 leads to increasing the likelihood of Y' = in the counterfactual world, and with that, the likelihoods of the action(s) compatible with that abstraction value (the cautious actions '1' and '3', in this case).
In summary. introduce abstract counterfactuals (ACF), approach for LM agents' counterfactuals that overcomes the issues of token-level action generation mechanisms by leveraging a semantics- and context-aware proxy of the actions. Unlike TLCF approaches, inference method crucially does not require white-box access to the LLM internals. Moreover, our abstractions can be defined either in an unsupervised fashion, with an auxiliary LM discovering and classifying the relevant abstractions, or through supervised classifiers informed by expert-defined categories.
We evaluate our approach on three benchmarks: MACHIAVELLI 22
choice-based game for evaluating agents' social decision making, and two open-text tasks, involving the generation of short biographies [8 and Reddit comments [9], respectively. Results demonstrate that our abstract counterfactuals consistently outperform token-level ones by improving the semantic consistency between counterfactual and factual actions and also within multiple counterfactual realizations.
Preliminaries
Structural causal models and counterfactuals [24] Structural causal models (SCM) provide mathematical framework for causal inference. An SCM C (S, UtPt) consists of set S of (acyclic) structural assignments of the form X; = (PA, U2) for sis |S), where X, are the endogenous (observed) variables, PA, S {X1, ( M (Xx)} denote the parents of variable X,, and Pu = Pu, ...., x Pu is the joint distribution over the so-called exogenous (unobservable) variables U. The exogenous distribution Pu and the assignments S induce unique distribution over the endogenous variables X, denoted P
An intervention on E corresponds to replacing one or more structural assignments, thereby obtaining SCM Ć. The entailed distribution of this SCM, P is called interventional distribution and allows us to predict the causal effect of the intervention on the SCM variables.
P counterfactual inference corresponds predicting the hypothetical value of x had we applied an intervention on C. This process consists in inferring the value of that led (known abduction step), by deriving the posterior
Given observation
FuJx=x(n) P(X U u) Pu(u)
P(x
Then, counterfactual statements correspond to statements evaluated on the SCM obtained from after performing the target intervention and updating Pu with the posterior Pu x=x-
Structural causal models for reto-regressive token generation To perform causal inference in the context of language modelling, Chatzi et al. 77], Ravfogel et al. [27] have recently proposed an SCM to describe the process of auto-regressive token generation, summarised below.
Let V be the LM vocabulary (set of available tokens). Given a token sequence (or prompt) x U Vi, where K denotes the maximum sequence length, the next token generated X by the LM follows categorical distribution X ~ Cat (xxttmmax((x*x1)))aax( where À refers to decoder (e.g.. a transformer) which outputs logits over V given an input sequence. obtain a structural assignment for the categorical variable X Chatzi et al. [7] and Ravfogel et al. 27 employ the Gumbel-Max
code MIT; data rescarch-only
2 MIT
PCC BY 4.0
3

---

SCM [22]. so that X can be expressed as deterministic function of the prompt x-=1, -- the language
model X, and the exogenous noise! U:
[x* 02 = arg max (A(x* ) + Un)
VEV
where U = (U.) Ev is vector of i.i.d. Gumbel random variables and {(x), denote the logits output by the language model for token e given the prompt At each time step of the sequence generation, the sampled token - is appended to the current sequence. This procedure continues until sampling of an 'end-of -eddoo--equeenee token, or the sequence exceeds the maximum length K.
Given a sampled (factual) token x² i.e., realisation of the distribution entailed by the SCM [ token-level counterfactual (TLCF) is derived from (1) as described in the previous section, where the abduction step corresponds inferring the Gumbel noise posterior U U x* xx= using the observation z² and the prompt x* Since the mechanism uniquely identified and so requires approximate inference [18], resulting in a stochastic U and a stochastic counterfactual. Interventions may involve altering the LM decoder X, e.g., by changing its weights, or by manipulating the tokens in the prompt x²
is non non-invertible, U' cannot be
It is essential to notice that TLCFs always increase the counterfactual probability of the observed token z² Indeed, if x was observed, the posterior Gumbel U will increase the probability of those noise values where Uz* is high enough to maximize X(x*=1)=** + Ugg across all possible tokens; if instead u is not high enough to yield x², then its posterior probability will be zero. We argue that this property of TLCFs represents limitation that hinders the semantic consistency of LM agents counterfactuals, as we explain next.
3 Abstract Counterfactuals Method
We now present SCM for LM Agent operating in sequential Dociison-mmakin environment,
similar to the SCMs proposed by [2] The state at step is given by pair S; = (X,, 0), where X, represents the agent's 'ss current prompt and represents the LM's parameters (c.g. weights, sampling strategy, latent manipulations [ The agent uses a stochastic, state-conditional policy to select an
action A4. After deploying the action, the environment transitions into new state S1+1. This process is summarised by the following SCM:
A, = U,A) S. As.
(2)
where U.A and U. are the exogenous factors associated to the action and state-update mechanisms.
In an LM agent, the action A; consists sequence of tokens sampled from the model, i.e., it is the result of applying the autoregressive token generation SCM ( for multiple steps. In this case, TLCF approach would perform posterior inference of the sequence of Gumbel exogenous variables U.A given the observed token sequence a, and the state st. Hence, discussed previously, TLCF would assign a higher counterfactual probability to the observed tokens, irrespective of whether the tokens remain relevant in the counterfactual context. This behaviour leads to two different categories of failure cases for TLCF:
Choice-based environments: in such environments, the token-level representation of the action may change its meaning across different contexts; this was discussed earlier in the Figure [ Texample. where the observed action token '2' entails a cautious action in the factual context and a reckless action in the counterfactual one. Another example of this failure case is given in Figure 3 of Section 4.1
Open-text environments: in this case, the action space consists of arbitrary token sequences. TLCF approaches would ignore the high-level meaning the generated action text, focusing instead on token-level utterances. The result is that the inference procedure would carry no or very little semantic information from the factual/observed context into the counterfactual one. An example of this issue is shown in Appendix [] where, after a gender-steering intervention on the model, TLCF fails generate short bio which is consistent with the profession observed in the factual setting.
*We treat 0 as part of the state because we may wish to intervene on it just we would intervene on any other variable in the SCM.

---

In other words, we cannot trust the token-level mechanism fA alone to perform counterfactual inference (as done instead previous approaches (7): quoting [TO, we do want "conflate the uncertainty of the [Ianguage] model over the meaning of its answer with that over the exact tokens used express that meaning".
Our abstract counterfactuals (ACF) method relies on a simple yet effective idea: introduce in the above LM agent SCM (2) an abstraction variable Y, that represents the high-level meaning of action A, in state St, in way that the meaning expressed in Y, remains consistent across contexts. This is illustrated in Figure and described by the following structural assignment:
= fr U.
We stress that Y, depends on both state and action, allowing dependent meaning. Our abstraction acts as a proxy for the token-level action A,, a proxy which retains those high-level features that matter when reasoning about counterfactual outcomes. For instance, in the Figure exxmpll. Y describes whether the agent is cautious or courageous; in our experiments with the MACHIAVELLI benchmark (Section 4.1) is used to classify the agent's ethical behaviour, e.g., its intention to cause physical harm; or, Y can represent the notion of 'job" in a biography -egeerrtion task (see Section
capture the action's context-
Supervised and unsupervised abstractions. The ACF method leaves the choice of how to construct this abstraction. Where the user has 'expert knowledge' about the domain or an opinionated view on what features of the samples should be considered meaningful, we can use supervised abstrac- tion. That is, either using annotations directly (as we do in Section 4.1 using classifier trained in supervised manner (see Appendix D Alternatively, we can use an unsupervised approach to discover semantic groups (i.e. the support of Y,) and estimate the distribution of Y; Ar, St. For unsupervised abstractions,
UP
U/A
UfF1
S
A
Sc41
U
Y
UY
auxiliary LLM for automated concept discovery and classification, as described in Appendix
Figure 2: SCM of an LM agent with abstraction variable Y
3.1 Inference method
Given a factual state let a A s be the observed action (from now, we omit the time-step indices for brevity). The goal of ACF is to compute a counterfactual action A' in different state s' given a, but without performing abduction on the token-level mechanism fA (which is semantics cemantic--gnostti)). To do so, ACF derives a counterfactual Y' for the observed abstraction value by performing abduction over the combined mechanism fy o fA. Note that ACF's abduction step is conditioned only on the abstraction y. not the action a. Such obtained Y' represents the abstraction of the, yet unknown, counterfactual action A'. The latter is found by mapping back Y' into the action space, i.e., by deriving the posterior distribution of A given Y' and s'. In summary, ACF's s inference procedure consists of the following three steps:
1. Abduction: derive the posterior distribution of the exogenous noise for Y. U3 = Uy given the observation 8, y. Formally, this is given by
Pur, (uly s,y) p(y 8, mly)) Puy (uly
P(y (s)
where P(y s, a/y-) ) is the probability, induced by UA (the randomness in action sampling), that and u/y result in the observation y:
Ply s. s,uby) = P - (
and Ply s) = Eu, PP s, J.
2. Counterfactual inference of Y': For given counterfactual state s', we plug in the above posterior
UY to obtain distribution of the counterfactual abstraction Y' = Y s', Uy follows:
Py ss p (f (TA(ss,UA))55 U').
5

---

3. Mapping Y' back into the action space: in the final step, we derive the counterfactual action A'' way that its distribution is consistent with the distribution of the counterfactual abstraction Y" derived in step 2. First, we compute the posterior
(a 1' Pr s,,d') Pa(a') I Py (s))
(3)
where Pr a')) "weighs" the probability action by the probability that a' leads to y/, with Pa(d' o')=P(() (a' - [s(s', UA)) Pr s s',,a) =P (y' fy fr sy,
and (y 8¹) E. Py a' PA(a s¹). Finally, we obtain the desired distribution of '''
by marginalizing 33) over Y
(ar(a' (8') [Pa(d Y',
We stress that this approach allows us to perform counterfactual inference on but without performing inference on UA i.e., by-passing the (token-level) mechanism SA during the abduction step. This allows us to treat the LLM as a black box from which we only need to take samples.
In the above three steps, we normally estimate fa UA) empirically by autoregressive sampling of the language model. However, if A consists of only one token (as choice-based environments), we can use the precise softmax probabilities computed by the model.
Interventional consistency. Any well-defined counterfactual inference method should consistent from an interventional viewpoint: a counterfactual represents the "individual-eeell" outcome of an intervention, and so, averaging the counterfactual distributions for each individual should yield the interventional distribution, i.e., the "population-leve"" outcome. We prove that our ACFs enjoy this consistency property, as stated below.
Proposition 1 For some action d', let Pa(a') s') be its interventional distribution and PA (a' |s') be its ACF distribution given we observed and i.e., obtained by using the posterior Uy = Uy 8, y. Then, it holds that
Puy [Par(a' | Pa(a' |&))
where the expectation is taken over Puy, the prior distribution of Uy.
The proof is provided in Appendix A
3.2 Interpretation and desiderata of the abstraction distribution
As discussed, we can think of the abstraction distribution as a context-dependent proxy which allows us to generate more meaningful counterfactuals. Additionally, the abstraction distribution can be seen as (soft) partition of the outcomes into groups (e.g., cautious vs courageous actions in Figure [ In this sense, ACF infers group condicoondiiional outcomes rather than individual ones.
While method is compatible with arbitrary distributions for Y, it is important that they satisfy certain intuitive criteria for meaningful counterfactuals. First, we want some degree of statistical dependency: Y J.. (s, a), for Y meaningfully characterise the action and the context. Second, the abstraction should not be too coarse or too fine-grained: with a small number of abstraction classes, we may oversimplify and omit important individual nuances; with too many classes, risk introducing unnecessary complexity and reducing interpretability. In the case of supervised abstraction, achieving this balance relies on expert judgment, whereas in the unsupervised case, depends on the concept discovery method employed.
4 Evaluation
In this section, we evaluate ACF against the token-level counterfactual inference approach of (27. The primary goal of this evaluation is to assess how well ACF maintains high-level semantic consistency across factual and counterfactual scenarios. Our evaluation conducted on three datasets: the
MACHIAVELA ELLLI benchmark (23]. the Bios dataset (8), and the GoEmotions dataset [9]. Due to

---

the deterministic nature of the MACHIAVELLI environment, we only present illustrative cases demonstrating our method's effectiveness.
In text-generation settings (Sections
4.2 and
we evaluate the following metrics (explained in
detail in appendix B
1. Abstraction Change Rate (ACR): The proportion of instances where the most probable counterfactual abstraction value differs from the observed one. A low rate indicates that the semantic content between factual and counterfactual generations remains consistent.
2. Counterfactual Probability Increase Rate (CPIR): The proportion of instances where the counterfactual probability for the observed abstraction value exceeds its interventional probability. This metric evaluates whether observing a particular abstraction value increases its counterfactual probability, as desired.
3. Semantic Tightness (ST): The semantic similarity among different counterfactual samples generated from the same factual setting (measured via the cosine similarity of their embed- dings). High semantic tightness indicates that counterfactual samples remain similar. To compare the ST values of ACF and TLCF, report the proportion of times ACF has better ST than TLCF (the win rate) and the t-statistic of a paired T test.
4.1 MACHIAVELLI
This consists of collection of text-based 'Choose- Your-Own-Adventure games, extensively annotated with behavioural ten- dencies displayed in each scenario. In particular, these annotations measure the agents' tendencies towards unethical (Machiavellian) behaviour. Game scenarios (states) are presented as strings of text. Available actions are defined in each scenario, and are presented the agent as multiple-choice selections. Our agent is implemented by the OLMo- IB LLM [12]. Given a scenario and a compatible action, the transition function is deterministic, so we can evaluate the annotations of this state-action pair by looking at those of the induced next state. We use a subset of the annotations available as "abstractions" for our method, characterising the agent's actions in terms of its tendency towards physical harm, dishonesty or power seeking, to name few (for the full list, see Appendix G.. These annotations are fixed, making PY 88 a) degenerate distribution.
Our first case study focuses on the 'MACHIAVELLI" benchmark (23)
Figure 3 compares abstract vs. token-level counterfactuals on an extracted scene from Machiavelli [23. showing how the abstract counterfactual derived distribution more consistent with the observed annotation. (More such examples are included in Appendix H As we can see, performing counterfactual inference at the token level ignores the different meanings of the presented options, and instead focuses on the action labels presented (i.e., the tokens '0', '1' ... which are mapped to the options). This complicates the counterfactual inference, especially in scenarios where factual and counterfactual scenarios might present us different action spaces, with no exact correspondence between options. In addition, token level methods are not well defined when the cardinality of the presented action space varies. This is because the Gumbel noise associated with options that are not present in the factual setting is undefined. For our comparison, we pad the Gumbel noise vector with 0 values when the counterfactual action space is larger than the factual one, and we truncate them when the opposite true.
4.2 Latent space interventions gender steering
Following Ravfogel et al. 27 we investigate gender steering interventions in GPT-2-XL. by modify- ing its latent representations. Using the MiMiC intervention [3I), we learn a linear transformation that aligns the mean and covariance of male-focused biographies (source distribution) with those of female-focused biographies (target distribution), training on the gender-annotatee [8. While Ravfogel et al. [7 report that such transformations can unintentionally change other attributes- mrorbutes-mosttt notably protagonists' professions- due to biases in both the language model and training data, our ACF method addresses these side effects by conditioning high-level semantic concepts and explicitly maintaining consistency in the targeted abstraction.
Bios dataset
In this setting, an intervention consists in replacing the factual state with (x, 0') where the model parameters have been modified according to the MiMiC [I transformation and the prompt has been left unchanged. This prompt corresponds to the first 8 tokens of the biography the model
7

---

Factual
Counterfactual
You creep down the corridor, every sound razor-sharp. floorboard ahead you freeze. corner, you spot a grim-faced man studying notice board. you, in your favour. Or you could talk you handle right. A threat might keep tine, but won't make
wriggle against ropes. Angel "Sadly, this is goodbye"" One of them hooks your bound legs to chain, the You're dragged across
gestures goons.
upside
on him.
down over dark pit. Below, water ripples several crocodiles moving. The winch turns. Two men, one near the edge.
Below, weoden platforms and
door. Think fast.
0.68
tohen levd countarfettull
phoviccl hurn
0.82
abarrett couterectaa
phayiical
0,23
0.03
0.02
0.1
0.02
0.07
O: surprise attack him
thresten him your weapon.
t: to talk him
C: persuade them to
myself
greb edge, force them to let me free
3: drop in the
observation
Figure 3: MACHIAVELLI case study. Action distributions for factual (left) and counterfactual (right) settings. The latter are obtained via the abstract counterfactual method, and the token level method. The observed abstraction value is 'physical harm: 1', associated with action "0" in the factual setting. Our method correctly increases the counterfactual probabilities associated with actions which also lead to 'physical harm: 1'. Token level counterfactual inference simply increases the probability associated with the observed action token '0', without considering its high-level meaning. The Gumbel noise for token '3', not present in the action factual action space, is undefined.
to complete, following [27]. As described above, an action a ~ A represents the full continuation of the biography generated by sampling from the LM until an end-of- -f--enn--eueceee token or fixed maximum length. We run our method both with an abstraction learned an unsupervised manner (described in appendix (F) well as supervised one. We define the supervised abstraction categorical distribution over the set of professions available (taken from the Bios dataset [8]), and we train classifier Py-(Y a, s) on these labels (more details supervised abstractions in appendix
Using this learned abstraction in the procedure defined above results in counterfactual generated texts that may shift in aspects such as phrasing stylistic details, but ensure consistency in their higher-level semantic content. We evaluate our method on a random sample of 250 biographies, and observe in table T ACF exhibits much higher abstraction value consistency from factual to counterfactual settings both with supervised and unsupervised abstractions compare to the token-level alternative.
Our method can be seen as 'steering' text generation towards samples that yield specific abstraction distribution. A potential concern is that may reverse the intervention's 's effects, steering the generation back to male-focused setting to match the factual (male-focused) distribution. However, this occurs rarely in practice. We compare the intervention's 's effectiveness with the method from Ravfogel et al. (27), demonstrating similar gendered pronoun distributions across both methods over the sample (Figure
4.3 Token level interventions emotion tracking
In the context of counterfactual text generation, one of the simplest interventions we can perform is replacing one or more tokens in the prompt x <-x'. x'. We want to obtain a counterfactual continuation for some alternative prompt x' (perhaps differing only factual continuation and its abstraction value for the factual prompt x. Concretely, we replace last (non-padding) token of the provided prompt with the most likely token (predicted by the model) excluding the one present in the prompt. A fitting case study for this setting, is the generation of text which conveys a specific sentiment emotion which we can capture through Y. For this, use the GoEmotions [9] dataset. This consists of manually annotated dataset of 58k English
specific token), having observed the

---

Table 1: Metries for latent space interventions Bios dataset, comparing abstract counterfactuals (ACF) with token-level counterfactuals (TLCF). For the ST metric, all reported values have p 0.001. The paired t-statistic for the unsupervised abstraction is t 13.04; the supervised abstraction 10.95. In the ST row, we report the win rate ACF over TLCF.
Supervised Abstraction
Unsupervised Abstraction
(profession)
Metric
GPT2-XL
GPT2-XL
ACF
TLCF
ACF
TLCF
ACR + CPIR
0.04
0.40
0.12
0.38
0.98
0.59
0.98
0.73
ST t
0.78
0.81
Reddit comments, each labelled with one more of 27 emotion categories or 'neutral'. In line with the framework defined above, we learn an abstraction distribution Py that captures the distribution of emotions in the generated text. This is implemented as classifier trained over the GoEmotions dataset, which gives us probabilities over the 28 categories available. More details on the architecture of this classifier is available in appendix E We evaluate our method on the GPT2-XL [25) and
LLama3.2- IB [T] LLMs. Similarly to the latent space intervention study 4.2 we measure the rate
of abstraction value change described in appendix B Table 2 show that method achieves
higher abstraction consistency than the token level one adapted from Ravfogel et al. [27], significantly
decreasing the rate of change in abstraction values. We also perform the
pairwise comparison
of semantic tightness as in the previous section, observing in all cases higher scores for ACF.
Table 2:
Metrics for token level interventions on GoEmotions dataset comparing abstract counterfactuals (ACF) with token-level counterfactuals (TLCF). For the ST metric, all reported values have < 0.001. With GPT2-XL, the paired t-statistic for the unsupervised abstraction variant 13.6, and the supervised variant is 10.16. For Llama-3.2- IB, the unsupervised variant gives f = 11.8, and the supervised method gives = 8.4. In the ST row, we report the win rate of ACF over TLCF.
Supervised Abstraction (emotion)
Unsupervised Abstraction
Metric
GPT2-XL
Llama-3.2-1B
GPT2-XL
Llama-3.2-1B
ACF
TLCF
ACF
TLCF
ACF
TLCF
ACF
TLCF
ACR
0.02
0.32
0.05
0.37
0.27
0.54
0.41
0.67
CPIR 1
0.96
0.68
0.97
0.67
0.87
0.48
0.75
0.47
ST
0.76
0.72
0.82
0.80
5
Conclusion
In this paper, we introduced abstract counterfactuals, a novel framework tailored for generating meaningful counterfactuals for language model agents. Our approach overcomes limitations of token- level methods by leveraging high-level semantic abstractions that capture user-relevant features. By reasoning through abstracted concepts rather than individual tokens, our method ensures consistent and interpretable counterfactual reasoning across varying contexts. Experimental evaluations on text- based games and text-generation tasks with latent-space and prompt-level interventions demonstrated the effectiveness of ACF in a wide range of settings.
Limitations As our method requires sampling from a black-box language model, the most important limitation is the computational cost of taking several samples. In addition to that, another limitation

---

stems from the necessity of defining an abstraction distribution, which might be tricky for certain settings. We mitigate this limitation by introducing unsupervised abstractions (see appendix
6 Funding Acknowledgement
Authors of this work were supported by the Engineering and Physical Sciences Research Council grants numbers EP/Y003187/1, UKRI849 and MCPS-VeriSee EP/W014785/2.
References
[1] Kenza Amara, Rita Sevastjanova, and Mennatallah El-Assady, Challenges and Opportunities in Text Generation Explainability, May 2024. URLhttp://arxin htee//bbs/4000... 2405 08468 arXiv:2405.0844 [cs].
[2] Onur Atan, William R. Zame, Qiaojun Feng, and Mihaela van der Schaar. Constructing Effective Personalized Policies Using Counterfactual Inference from Biased Data Sets with Many Features, July 2018. URL wtwp:/wwww.rggbbb//....o. [stat].
arXiv:1612.08082
[3] Leo Breiman. Statistical Modeling: The Two Cultures (with comments and rejoinder by the author). Statistical Science, 16(3):199-231, 08 2001, doi: 10. 1//////2/// URL.
http://dx doi. org/ 10. 1214/
[4]
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang- Bean Edward Lee, Sergey Levine, Yao Lu, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. RT-2: Vision-Language-Actinn Action Models Transfer Web Knowledge to Robotic Control, July 2023. URL http://arxiv. org/ abs/ 2307 15818 trXiv:2307.15818 [cs].
[5] Lars Buesing, Theophane Weber, Yori Zwols, Sebastien Racaniere, Arthur Guez, Jean- Caan-Baptiste Lespiau, and Nicolas Heess. Woulda, Coulda, Shoulda: Counterfactually--uutt Guided Policy Search. arXiv preprint 1XX.::8811.0622. 2018.
[6] Ruth M. J. Byrne Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning. In Proceedings of the Tieety-Eighth International Joint Conference on Artificial Intelligence, pages 6276-6282, Macao, China, August 2019. International Joint Conferences Artificial Intelligence Organization. ISBN 978-0-9992411 1--1. doi: 10.24963/
1jcai.2019/876 URI. :/wp../:////ww.... www.:wwjca.i.oo //f.aa...r//prrccee//g//////////
[7] Ivi Chatzi, Nina Corvelo Benz, Eleni Straitouri, Stratis Tsirtsis, and Manuel Rodrizuz. Counterfactual Token Generation in Large Language Models, September 2024. URL http:
wwtervi.org/bb//240/00 17027 arXiv:2409.1 17027 [cs].
[8] Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexan- dra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in Bios: Case Study of Semantic Representation Bias Stah-Stakes Setting, January 2019.
URL http::ttpp......ooo htttpp//aaxii.orrrbbb///00....00
abs/
#rXiv:1901.0900 109451.
[9] Dorottya Demszky, Dana Movshovitz-Attiae Jcongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. GoEmotions: A Dataset of Fine-Grained Emotions, June 2020. http:
//arxiv. wwwgg/bb///00/// 00547
arXiv:2005.00547 [cs].
[10] Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting Hallucinations in Large Language Models using Semantic Entropy. Nature, 630(8017):62--330 June 2024. doi: 10. 10.1100011486666 -86-002-00422--0 URL https:: ://ps:///wwww.. /twp://waturreeeeeeeee/.//twwwwe com/
ISSN 1476-4687.
articles ///ee///// s41586 024-4118-24--0000 - Publisher: Nature Publishing Group.
10

---

[11] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 Herd of Models, arXiv preprint urXiv:2407.21783 2024.
[12] Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, lan Magnusson, Yizhong Wang, Shane Arora, David Atkin- son, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Flazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo. Luca Soldaini, Noah A. Smith, and Hannaneh Hajishirzi. Olmo: Accelerating the Science Language Models, 2024. URL https: wtwg/ba///aaa. 00838
[13] Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. Large Language Model based Aulti- A Survey Progress and Challenges, April 2024. URL http: /tpp://axxxvi wwwpriiv.org//aao wtwgg//bb//22.....go [es].
http://rrw....r///bb////222..10. 01680 arXiv:2402.01680
[14] Joseph Y. Halpern. Cause, Responsibility, and Blame: A Structural-Model Approach, December 2014. URL http::/tttx/iii//
2985 arXiv: 1XXv:1412.22985 [cs].
[15] Joseph Y Halpern and Judea Pearl. Causes and Explanations: A Structural-Mode Approach. Part II: Explanations. Br. Philos. Sci., 56(4):889 911, 2005.
[16] Milad Kazemi, Jessica Lally, Ekaterina Tishchenko, Hana Chockler, and Nicola Paoletti. Coun- terfactual Influence in Markov Decision Processes, February 2024. URL http://arxiv org/ abs/ 0bs/2402.oo 08514 arXiv:2402.08514.
[17] Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and Philip S. Yu. Large Language Models in Law: A Survey. 2023. doi:
URL /tttps://arxxt org/
Abs/2312.088 Publisher: arXiv Version Number:
[18] Chris Maddison, Daniel Tarlow, and Tom Minka. A sampling, Advances in neural information processing systems, 27, 2014.
[19] Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. Explainable Reinforcement Learning Through Causal Lens, November 2019. URL http::/aaxxvvorr//aaaaa 10958 arXiv: 19055 1995510958 [cs].
[20] Nikita Mehandru, Brenda Y. Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J. Butte, and Ahmed Alaa. Evaluating Large Language Models as Agents in the Clinic. upj Digital Medicine, 7(1):84, April 2024. ISSN 2398-6352. doi:
URL
https://wwes http://wwwwww.....ttet.t
nature.
[21] Roberto Navigli, Simone Conia, and Bjöm Ross. Biases in Large Language Models: Origins, Inventory, and Discussion. J. Data and Information Quality, 100)::0: June 2023. ISSN 1936- 1955.
doi: 10. 10.1145/3597307. | htttps://d/ acm. http:////aa..aacooor////////.../ /ttp://600/101111//5/7777777
[22] Michael Oberst and David Sontag. Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models. In International Conference on Machine Learning, pages 4881-4898 PMLR, 2019.
[23] Alexander Pan, Jun Shern Chan, Andy Zou, Nathaniel Li, Steven Basart, Thomas Woodside, Jonathan Ng, Hanlin Zhang, Scott Emmons, and Dan Hendrycks. Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark. ICML, 2023.
[24] Judea Pearl. Causality. Cambridge University Press, New York, 2009.
[25] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language Models are Unsupervised Multitask Learners. OpenAl blog. 1(8)99, 2019.

---

[26] Shauli Ravfogel, Anej Svete, Vésteinn Snæbjarnarson, and Ryan Cotterell. Counterfactual Generation from Language Models, November 2024. URL htthttp::///// http:://axiii org/ argg/bbs/2111 arXiv:2411.00 107180.
07180
[27] Shauli Ravfogel, Anej Svete, Vésteinn Snæbjarnarson, and Ryan Cotterell. Gumbel Counter- factual Generation From Language Models. In The Thirteenth International Conference on Learning Representations, 2025. URL htttps://openrerill
[28] Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural
Language Processing. Association for Computational Linguisties, 11 2019. URL https
/ //arxiv.org hargg/ab//iee 1etp/1998.... 10084
[29] Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel, Max Lamparth, Chandler Smith, and Jacquelyn Schneider. Escalation Risks from Language Models in Military and Diplomatic Decision-Making. In The 2024 ACM Conference on Fairness, Accountability, and Trans- parency, pages
Rio de Janeiro Brazil, June 2024. ACM. ISBN 9798400704505 doi:
10.1145/33301066366888
https:/::/// acm. att.a..rr//.o//0.111//3300000110 0t01//00..14/333000066664444
[30] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter, 2020. URL https://axxiv org/
abs/1910. 01108
[31] Shashwat Singh, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, and Pon- nurangam Kumaraguru. Representation Surgery: Theory and Practice of Affine Steering, July 2024. URL /tt wttp://..wrraa/////////// 09631 arXiv:2402.09631 [cs].
[32] Zhongxiang Sun. A Short Survey of Viewing Large Language Models Legal Aspect. 2023. doi: 144.5500ARXXIV22330001..... URLhttpss:////
arg/abs/2303 00.333...1366 09136
Publisher:
arXiv Version Number: 1.
[33] Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Ar- man Cohan, and Mark Gerstein. MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning, June 2024. URL/ttpp://axii/ 1xXiv2231111055 [es].
2311.
[34] Jin Tian and Judea Pearl. Probabilities of Causation: Bounds and Identification. Ann. Math. Artif. Intell., 28-1-888188--00088 -4)2877-333 2000.
[35] Alexander Sasha Vezhnevets, John P Agapiou, Avia Aharon, Ron Ziv, Jayd Matyas, Edgar A Duzmtzz William A Cunningham, Simon Osindero, Danny Karmon, and Joel Z Leibo. Generative Agent-Based Modeling with Actions Grounded in Physical, Social, or Digital Space using Concordia. arXiv preprint arXiv:2312.0366. 2023.
[36] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An Open-Ended Embodied Agent with Large Language Models, October 2023. URL http:://axxiii.....///. http//rxxvvvoo..///////// abs5 16291 arXiv:2305.166) 16291 [es].
[37] Lei Wang. Chen Ma. Xueyang Feng, Zeyu Zhang, Hao Yang. Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science, 18(6): 186345, December 2024. ISSN 2095-2228, 2095-2236. doi: 10.1007/s1 1704-024-40231-1. URI. htttp://axxiv.ot/// abs/ 11432
arXiv:2308.1 1432 [cs].
[38] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing Reasoning and Acting Language Models. International Conference on Learning Representations (ICLR). 2023.
[39] Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang. Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, and Jifeng Dai. Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory, June 2023. htt.:ttp: arXiv:2305.1 7 144 [cs].
org/abs/ 2305. 17144
12

---

A Proof of Proposition
We want to show that the expectation of our counterfactual distribution w.r.t the exogenous values distribution is equal to the interventional distribution. Formally,
Buy-Poy [Pa | Pa |
For simplicity of notation, omit conditioning s'. Also, for simplicity of notation, assume all variables are discrete. The proof below would work by replacing sums with integrals for continuous variables. So, the LHS of the above statement can be rewritten as
ΣΣ P(u)) P(y) 1 w') P(v/ a') P(a')
P(y")
P(y' | Ply')
P(u' l1) P(u)
and want show this is equal to P(a'). By noting that (by Bayes rule) then the above expression simplifies to
ΣΣ P(u' (w') P(y la') P(a')
which is equivalent to
Σ P(l d') P(d)) E
By Bayes rule, rewrite E. P( v/) Ply') Ev P(u', yn) and it's easy see that this
equals 1. So, the above expression simplifies to
P(d P(a) = Σ Pl)'aa')
which is equal to P(a').
B Metrics
This metric calculates the proportion of instances where the most probable counterfactual abstraction value differs from the observed abstraction value. Formally, it calculates the number of instances where:
Abstraction Change Rate:
argmax y' ED" a
Σ Pes (1' aa, DEa
Where is the abstraction value observation. A low abstraction change rate indicates that the semantic content of the counterfactual generation remains consistent with the initial generation.
2. Counterfactual Probability Increase Rate:
This metric measures the proportion cases where the counterfactual probability for the observed abstraction value is greater than its interventional probability. Formally, calculates the number instances where:
(vf d',s') > P(y a,, 8/)
This metric evaluates whether the counterfactual probability derived from counterfactual samples (a' ~ (a' s')) exceeds that of interventional samples (a PA (a 8')).
Semantic Tightness:
also evaluate the semantic similarity between different counter- factual samples generated from the same factual setting. Formally, given set of strings a {aj 0g...... ) and a semantic embedding model X, we can compute the semantic tightness as:
semantic_tightness(41
cos Aaa)
a
where we measure the average cosine similarity between all pairs of embeddings from the strings in a. As our embedding model X, we use the call-mpnet-base-v2 model from the ceeneece-transfoomee library 28). High semantic tightness indicates that counterfactual samples remain similar, even when generated independently from the same factual context.
13

---

C Counterfactual sample semantic tightness
Figure 4: Scatterplots showing the semantic tightness of counterfactual samples generated with abstract counterfactuals and token-level counterfactuals for the same initial state s. Each point represents initial state.
Semantic Tightness
Gender Steenin (Bos)) Supervised Abstraction, gpt2-a)
Gender Steering (Wios), Unsupervised Abstraction, got2-al
Lavel Tuphtmes
0.4
244
0.6
Abdract Courterfactual Tightneex
Counterlattus Sementie Tightness
Replacement (Gofimatione),
gpt2-xl
Replacement (Gofmmttoons), Unsupervised gpt2-xl
0.6
24
Taken Countert
Counterfactual
Counterfactual
Tokeen Rep iscoment (SoEmotions), Supervised Aostraction, 1lama-3.2-18
Tolsen Repl replacement (GoEmetons), Unsupervised Abstraction, 1ama-3.2-18
0.6
2.8
Toleen Coumartaattuat
0.4
2.4
Abstrcct Coutterfactua Semretic Tightnass
Abstract Counterlactaa Sumantic Tightnass
D Gendered pronoun distributions
14

---

Figure 5: Pronouns distribution counterfactual samples generated with method and token
level method [27]
for the gender steering intervention on GPT2-XL
0.6
Rate
0.4
abstract counterfactuals
token level (Ravfogel ct. al.)
0.2
0
Only Male
Only Female
Mixed
No Pronouns
Category
E Supervised LLM abstraction
For both text-generation tasks we run experiments with supervised abstractions. Here, a classifier models the distribution Py (Y a), after being trained on annotated dataset. This classifier implemented by fine-tuning a DistilBERT 33 language model on the respective dataset. For the 'gender steering' latent space interventions profession, using the BiosBias 8 dataset, resulting in a model with an fl score of 0.85. The available profession categories from this dataset are: professor, physician, aftorney, photographer, journalist, nurse, psychologist, teacher, dentist, surgeon, architect, painter, model, poet, filmmaker software engineer, accountant, composer, dietitian, comedian, chiropractor, pastor, paralegal, yoga teacher, dj. interior designer, personal trainer, rapper.
fine-tune the model predict the protagonist's
In the case of the token replacement interventions, we fine-tune the classifier the GoEmotions dataset (9), assigning the generated text to one the following categories: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief. joy, love, nervousness, neutral, optimism, pride, realization, relief, remorse, sadness, surprise, neutral. The emotion classification pipeline has an score of 0.63.
Unsupervised LLM abstractions
Our goal is discover effective sample space for Y which captures the meaning of generated text completions. This similar the semantic clustering performed by [10] to compute entropy semantic level. For semantic grouping. however, we do not use the bi-directional entailment algorithm proposed by [10), as it is more suited for specific claims (i.e. Q&A format) rather than prompted text generation. Instead, topic assignment. Given factual and interventional states s, s', we sample text continuations from both, and join them in single set This set is fed to language model prompted to identify list of topics that partition the set. Each sample a E a then fed individually to a language model prompted to assign the most likely topic from the previously discovered set. We use the model's logits over topic indices to construct the distributions Y a.
two-stage process, which consists in topic discovery phase, and
For the experiments throughout the paper use 'gpt-4o-mini language model. As the openai API (as of this writing) only provides logits the 20 most likely tokens, we filter out invalid token id responses and add an 'Other' category to the sample space of Y after topic discovery.
15

---

Unsupervised LLM Abstraction Prompts
Topic Discovery Prompt:
"You helpful assistant, and you output python code Your task is to come up with groupe that partition the following list strings at the semantic level.
<formatted string list> You must
exclusively with valid list of tuples, where element 0 is category title (string), and element category"
very brief description of that
Topic Assignment Prompt:
"You are helpful assistant, and you only output integers within the range specified by the number of topics.
Given set topics, their descriptions, and their (0-indexed)
new string, assign the string to the most appropriate topic. Reply only with the ID of the topics.
Topica: <formatted topics>
String be classified: <formatted string?
topic
G MACHIAVELLI annotations
We are interested in the annotations for the 'morality" category from the MACHIAVELLI 23 benchmark. The available (binary) annotations, for each scene, are the following: killing, physical harm, non physical harm, intending harm, deception, manipulation, promises, stealing, trespassing, spying, vandalism, fairness, faaiesss and other.
H MACHIAVELLI further examples
Figures 8 show further examples of abstruct counterfactuals in MACHIAVELLI 2 settings
with 'morality' annotations as abstractions.
0.28
How you going try win the card
Fln Tiran
observation
Counterfactual
comentt antuval
manby
Facuall Intorennto sal
When
rodiding he
had pelioe
artroduce yourself
0.21
hud undod salking whole You leanned that ho was Putronkl personal Micsal
aedicantt-nn and nneered that Mitenal Petrovich was not the most pleacont people to work these dirc.
can fere
fits Now Nenvle oater
iiumrtion in breaking
madiredd Dard Prok Patrurs
Figure Scenes from the 'Aegis Project Files 180' game, abstraction 'manipulation'
16

---

Factual
lights
Ange-mg
guesswork up of
thing name, very
shead makes you freeze, Holding your breath, you inch forwards peer around the corner. The grim- facadd man has his back you Hi seurts to be inspacting on old notice board. You're in the pertect onsitio to get the drop on him betore he realizes you're here. That could make all difference fight Then could talk, Maybe
here But from inmmndiattll threaseninng appreech-ves try anything also
whole
have probably
D: surpriss wttack him
your weapon.
t:y to talk him
(Jaey to conperate.
observation
Counterfactual
wriggle, tosting
usually sove gaing send third She grina mischivausly.
the someone, Just the thing help with the nosy cop. Two binds, one stone. It's been fun, realls: Sadiy" she mations the goons, "our acquaintanon is gring to ba short one. One the guords tims, and you notice what was hidden behind himm hanging from the calling, back dangling from end, He picks attaches
Yikas lwel countrrbrsoo
no pheiiiiiat horm
anotact conteerectaal
pheskcal
loopect end
hook the
struggle dragged
sidsening you hoisted ait, dangling upside down over
pit Therels targe pond drnw theen, the water, somethinng moxiing.
Siveral somothings.
excer this is Not Goodtrese
"Oid bell you about Putrowich's crocadiess? Angel perversely upbeat. "Too
chenon to mise expecisllty sincw point Hiam him. You did
clearly
going me?"
through
mathing your hraed hang around the
kiow
chips you all. You fattine and they're ring to send more in investigato. So puan to be lang way away the time Patrowett's pera gett you She gives tistle wave "Have fun She sounters away from the and hear her to the
The "Mreloe there's nothinn left"" going
turning
men up
standing pretty
0.03
0.07
edge. Below you, arcangements its wood rock. strike you that there might be door down there well, some kind of maintanance entrunce for the encicsure. Think hoothoow you going to get out one?
green
persudde them lst me
grah odge. foree
di the
Figure 7: Scenes from the 'Aegis Project Files 180' game, abstraction 'physical harm'.
17

---

Factual
dusis and dejo
with pocuie
derue divedin deloc
norling wac deretic ii sou rought ohonee uo deeo of Polee he cinch the of
cak whec home anjanidl auamaac que for
durddng in the mesrdd acioe
siraul
right we momou lave busty
0.22
Falerry
lie macie
dinay hrunding ter henge vedur te lind aine deeen tor
Weshoodd
inteckke Ptalemyy Natertary
WWshhold bank befurtari.
(hi fuckrie
Scilks rerm heee en lqpal santo we mesut "Vyher thetf Poleeyy wets prest you by and you for
observation
Counterfactual
weta lovel coul coatterlacuul
raxiting
cuntarbeuuul
krkng
toat can enogge mng mane can Trxck Lokk sor proclde" peciue Ldtuaten
take resualle
te Phasery
0.74
Smaps optar the all stheugh on martle remsi's aloes boom the yo coloon hel
grua poocarte the
0.47
to placr Pemae schi you mediatly booe servos. "Mlln Now
seco itr to Voeaar, Nooi
tegleess tross moder you
0.28
at aich
to lid them Show ligh
0.25
riuer'' of atua
Kncan Horenn mesosse rotal
0.10
The lauu suute di dena hataraal drean day beiry mosilly moll aductte lise
0.02
0.04
haen word meak me foyetie aeoals Buntl on snregggee an
the aeaaly Detitee cuieac
C unlll Palary meviigute with
Spruliin Siatinnniiinng Pailowr ante lan
Figure 8: Scenes from the 'Alexandria' game, abstraction 'killing'.
18

---

I
Gender steering examples
Gender Steering Example
Factual Text:
Also a traditional reporter, this New Jersey native went on to more than 40 stories for the News of the World, then founded the news webaite MuckRock to croudsource information and ultimately went on to co-found the tech-oriented think tank, the Future of Privacy Forum. Now he's helping others get their stories out by sharing his He's full member of the PRWeb team, as we are proud be [...] Predicted Abstraction: 'Journalist"
Token-Level Counterfactual Text:
Also traditional reporter, this New Jersey native Boston College and USC before earning law degree. is currently managing editor of the weekly column "Sex and Shared Desires," and occasionally contributes to "TribLive" on the Huffington Post, LGET Nation! Predicted Abstraction: 'Attomey"
Abstract Counterfactual Text:
Also a traditional reporter, this New Jersey-born reporter among those chosen to xepresent KQED's public television reporting. Ms. Williams has reported from across the country, from Israel and Moscow, and eight continents. She has been with the Public Insight Network for decades. Before joining KQED, WITN in Washington, DC [...] Predicted Abstraction:
Williams was senior producer member station
'Journalist"
J Compute Resources
We run all the reported experiments on a server equipped with an x86_6e 66, 128-core CPU with 405.2 GB of RAM and an NVIDIA A40 GPU with 48GB of VRAM. The server runs Ubuntu 20.04.6 LTS.
Counterfactual sample generation pipelines take around hours each (for each configuration of model, abstraction type and dataset). Fine-tuning of the supervised abstraction models takes around Ih for each abstraction.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The method is introduced in section and experimental results are shown in section 4
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Limitations are discussed in section 5 an abstraction distribution) addressed and mitigated by the introduction of unsupervised abstractions (see appendix F).
One of the limitations (the choice of
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
20

---

Answer: [Yes]
Justification: The only theoretical result (proof of consistency of our counterfactual method) is presented in appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The method we introduce is explained detail in multiple steps in section Furthermore, we include practical details about the construction of the supervised and unsupervised abstractions (including LLM prompts) respectively appendices and F
3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived
well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
21

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: We provide a zip file as part of the submission with the full codebase required for running and evaluating the results, as well as documentation how to use it.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips. wwtpcc/pccppcpcccceooooooop / sesiissionnie ssinnoliccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide extensive details about experimental settings, configurations, models and data in the paper appendices, as well as in the provided anonymized codebase.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: While we do report the statistical significance of the semantic tightness metric, we do not do so for the other metrics, due to limits in compute budget and API costs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources and time of execution for experiments are detailed in appendix !
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have reviewed and adhered to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss social impacts of language model agents and specifically coun- terfactual inference the introduction of our work. We not foresee negative societal impacts arising from counterfactual inference. Our work addresses the possibility of biased data and language models resulting biased counterfactuals, and our research aims at limiting this.
23

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We not releasing any new data models. We do not foresee any misuse risk from the released method and experiment code.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-uss should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original owners of all assets and code we build upon, and report their licenses in footnotes.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
24

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pa se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The only new asset produced part of this paper is the associated code. This is provided an anonymized .zip file, including documentation about its functioning and usage.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our reserach didn't inlcude any crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
25

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Aside from studying LLMs themselves throughout the paper, we use LLMs as components of our method, implementing the 'abstraction' distributions. We describe their usage this regard in detail in appendices and
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standart components.
Please refer to our LLM policy (https://neurii c for what should or should be described.
26


Reviews:
[{"review_id": "87ER326ft8", "reviewer": "Reviewer_5RqV", "summary": "This paper proposes a framework for counterfactual text generation in language models. The main idea is that instead of performing counterfactual inference at the token or syntactic level, the abduction step in Judea Pearl's three-step counterfactual inference is carried out at a more abstract, semantical level. Experimental results show that on several benchmarks, the proposed method improves semantical consistency across factual and counterfactual contexts, over the token-level counterfactual generation methods.", "questions": "In addition to the questions raised earlier in connection with \"weaknesses\", another puzzling point is the claim that the proposed approach can treat a LLM as a black box. It makes sense for interventions that only manipulate prompts. But why is this the case when the interventions also manipulate the decoder \\lambda?", "limitations": "Depending on the authors' responses to the questions raised earlier, the section on limitations may need to be expanded or adjusted.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The motivations of the approach are sensible and interesting, and the proposal is quite novel\n\n2. The technical presentation is largely clear and cogent.\n\n3. The experimental results are interesting and fairly convincing.\n\nWeaknesses:\n\n1. It is not entirely clear if the alleged flaws of the token-level counterfactual generation are due specifically to the Gumbel-Max setup or reflect general problems with token-level counterfactuals. For example, one problem highlighted by the authors is that \"TLCFs always increase the counterfactual probability of the observed token x^k\", but the argument for this claim is specific to Gumbel-Max SCMs, and for general SCMs, it is not true that observing a certain value will inevitably increase this value's counterfactual probability. This seems to indicate that the problem is confined to Gumbel-Max models. On the other hand, however, the authors also tried to motivate their work by noting the general phenomenon of context-dependence of meaning, which suggests that the work is not just intended to improve counterfactual generation with Gumbel-Max models. A clarification of this issue would be appreciated.\n\n2. Related to this ambiguity, and more importantly, if the claim is that there is a general problem with token-level counterfactuals going beyond the particular Gumbel-Max setup, the present paper does not yet provide a compelling demonstration of this general problem. In particular, it is unclear to me why f_A in general cannot be expected to properly represent the dependence of actions/generated texts on states/prompts that take into account the context-dependence of meaning. (Is the thought that we need an abstraction mechanism f_Y to tease out some context-invariant component of A?)\n\n3. On p. 5, in step 2 \"Counterfactual inference of Y'\", it appears to require a joint probability law of U_A and U'_Y. It is unclear how the joint distribution is supposed to be determined? In particular, are they assumed to be independent, even though U'_Y is updated?", "was_revised": true, "final_justification": "The authors' responses to my concerns are reasonable. My evaluation remains positive after taking into account the authors' feedback and other reviews.", "rebuttal": ">It is not entirely clear if the alleged flaws of the token-level counterfactual generation are due specifically to the Gumbel-Max setup or reflect general problems with token-level counterfactuals. For example, one problem highlighted by the authors is that \"TLCFs always increase the counterfactual probability of the observed token x^k\", but the argument for this claim is specific to Gumbel-Max SCMs, and for general SCMs, it is not true that observing a certain value will inevitably increase this value's counterfactual probability. This seems to indicate that the problem is confined to Gumbel-Max models. On the other hand, however, the authors also tried to motivate their work by noting the general phenomenon of context-dependence of meaning, which suggests that the work is not just intended to improve counterfactual generation with Gumbel-Max models. A clarification of this issue would be appreciated.\n\n\nThank you for the thoughtful comment. You're right that the argument we presented regarding the increased counterfactual probability of the observed token under TLCFs is specific to Gumbel-Max Structural Causal Models (SCMs). \nHowever, our broader motivation extends beyond the Gumbel-Max framework. The main issue we aim to highlight is that token-level counterfactuals, regardless of the SCM instantiation, are tightly bound to the syntactic form of the model and often fail to reflect the underlying semantic dependencies in language.\n\nWe will revise the manuscript to clarify this distinction between model-specific analysis and general motivation.\n\n\n>Related to this ambiguity, and more importantly, if the claim is that there is a general problem with token-level counterfactuals going beyond the particular Gumbel-Max setup, the present paper does not yet provide a compelling demonstration of this general problem. In particular, it is unclear to me why f_A in general cannot be expected to properly represent the dependence of actions/generated texts on states/prompts that take into account the context-dependence of meaning. (Is the thought that we need an abstraction mechanism f_Y to tease out some context-invariant component of A?)\n\n\nThank you for the insightful comment. It is true that in general, $f_A$ does reflect context dependence, since it is implemented by a language model that sees the full prompt or preceding sequence. However, the issue arises in how counterfactual outcomes are computed. In standard token-level approaches, the 'abduction' step defines the exogenous factors over specific observed tokens. While these tokens make sense in the original context, their meaning can become ill-defined or unsupported in a different, counterfactual prompt. \n\nFor example, suppose the model sees the prompt “He studied hard to become a” and generates the completion “journalist”. If we now intervene to change the prompt to “She is a”, the token-level exogenous posterior associated with the next token would be that of “to” (the fourth token in both sentences, which will have no support in the counterfactual context), rather than the profession. We want to define the exogenous factor over the high-level meaning of the sentence (e.g., profession), not the dynamics of the individual tokens. \n\nOur approach addresses this by abstracting the state-action pair into a higher-level variable $Y$, which captures semantic meaning independent of specific surface forms. We can then derive a posterior over the exogenous noise at the abstract level, allowing for meaningful counterfactual reasoning even when the context changes. This abstraction preserves interpretability and semantic consistency across factual and counterfactual scenarios.\n\nWe will revise the paper to better articulate this limitation of token-level conditioning and to include an example like the one above for clarity.\n\n>On p. 5, in step 2 \"Counterfactual inference of Y'\", it appears to require a joint probability law of U_A and U'_Y. It is unclear how the joint distribution is supposed to be determined? In particular, are they assumed to be independent, even though U'_Y is updated?\n\nThank you for raising this point. \n\nIn brief, $U'_Y$ is determined in the abduction step by conditioning $U_Y$ on the observed state $S$ and abstraction $Y$. That is, we compute a posterior $p(U_Y \\mid S, Y)$, and sample $U'_Y$ from this distribution. In Step 2, we then obtain the counterfactual abstraction $Y'$ by evaluating the combined mechanism $f_Y \\circ f_A$ at the intervened state $S'$, using $U'_Y$ in place of $U_Y$. This allows us to propagate the context-dependent semantic content of the original observation into the counterfactual setting.\n\nRegarding the joint distribution: Following from the structure of our causal diagram, $U_A$ and $U_Y$ are independent a priori. Since only $U_Y$ is updated during abduction, we treat $U_A \\sim p(U_A)$ as independent of $U'_Y \\sim p(U_Y \\mid S, Y)$. This results in an effective joint distribution $p(U_A) \\cdot p(U'_Y \\mid S, Y)$ used during the counterfactual rollout. We will revise the manuscript to clarify this assumption and make the mechanics of the abduction and inference steps more explicit.\n\n>In addition to the questions raised earlier in connection with \"weaknesses\", another puzzling point is the claim that the proposed approach can treat a LLM as a black box. It makes sense for interventions that only manipulate prompts. But why is this the case when the interventions also manipulate the decoder $\\lambda$?\n\n\nYes, when we say our method is black-box, we refer to the method of computing counterfactuals, given arbitrary interventions performed to the models (whether black-box or white-box). We study latent space interventions (gender steering) specifically, as they were interesting case studies presented in works we build upon."}, {"review_id": "gl5bl2sJA7", "reviewer": "Reviewer_SCN4", "summary": "The paper introduces Abstract Counterfactuals (ACF), which is a framework for generating counterfactuals from LLM agents.\nPrevious approaches used token level counterfactuals, which didn't capture the semantic aspects of the LLM agent's actions.\n\nThe main contribution is the introduction of a semantic-level abstraction variable, Y, that captures the context and meaning, e.g., intent, profession, or emotion. As Y depends on the (factual) state and action, it is used for the counterfactual reasoning to obtain a counterfactual action that is semantically consistent with the factual action. The paper offers supervised and unsupervised methods to obtain Y.\n\nThe paper does experiments on ACF: a text-based game (MACHIAVELLI), a biography generation task, and emotion tracking in Reddit comments. Results show that ACF creates more semantically consistent counterfactuals than previous token-level approaches.", "questions": "In Appendix A, Proof of Proposition 1, the paper states: \"We want to show that the expectation of our counterfactual distribution with respect to the exogenous values distribution is equal to the interventional distribution.\" \n- How does the proof differs from the known result in causal inference that the expectation over the counterfactuals (individual level)  is the interventional probability.\n\n-  What is an estimate for the time complexity of the sampling method?\n\n- Are there formal approaches to know how coarse or fine-grained the abstractions are?", "limitations": "Nothing to add to the limitations the paper mentions", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality\nStrengths:\n- The paper provides a well-defined approach based on structural causal modeling and counterfactual reasoning\n- The paper validates the theory by doing experiments on different benchmarks\n- The paper sets up evaluation metrics suitable to the counterfactual semantic consistency: Abstraction Change Rate, Counterfactual Probability Increase Rate, and Semantic Tightness.\n\nWeaknesses:\n- The creation of the abstraction Y depends on dataset, classifiers used, or results from unsupervised methods.\n- The computational costs (e.g., sampling) could be challenging for practical real-world scenarios\n\nClarity\nStrengths:\n- The paper is clearly written and well-structured. The reader can easily follow the exposition for the SCM and the LLM Agent operating in a sequential decision-making environment.\n- The examples are intuitive, e.g., the bear and lion scenario\n- The details in the appendix are helpful\n\nWeaknesses:\n- Appendix B Metrics could benefit from more details\n\nSignificance\nStrengths:\n- The paper addresses issues in token-level counterfactual reasoning of LLM agents\n- The ACF approach can be impactful for LLM agent safety and fairness\n\nWeaknesses:\n- The quality of the abstraction Y may vary in real-world scenarios as it depends on many factors\n\nOriginality\nStrengths:\n- The paper presents a new method, ACF, that is different from past approaches, e.g., token-level counterfactual reasoning\n- The use of structural causal modeling with semantic abstraction Y for counterfactual reasoning is original.\n\nWeaknesses:\n- The creation of the abstraction", "was_revised": true, "final_justification": "I'm satisfied with the authors' answers. No change of score from my side.", "rebuttal": ">How does the proof differs from the known result in causal inference that the expectation over the counterfactuals (individual level) is the interventional probability.\n\nThe proof mirrors the standard result in causal inference, which states that the expectation over individual-level counterfactuals recovers the interventional distribution. What we're doing here is adapting that proof to our specific method of computing counterfactuals. In essence, we're verifying that our approach preserves those results. \n\n>What is an estimate for the time complexity of the sampling method?\n\nThank you for the question. The overall time complexity of our method is linear in the number of sampled realizations $N$. Each realization involves one forward pass through a language model, which has complexity $O(L \\cdot d \\cdot n^2)$, where $L$ is the number of layers, $d$ the hidden size, and $n$ the sequence length.\n\nSampling from the latent space is efficient and negligible in comparison. We can use standard concentration bounds in principle to determine the number of realizations $N$ that achieves the desired statistical guarantees. \n\n>Are there formal approaches to know how coarse or fine-grained the abstractions are?\n\n\nThank you for the question. One obvious metric for abstraction granularity is the number of abstraction classes; more classes typically indicate a finer-grained abstraction. However, this alone may not be sufficient, as many classes might be rarely or never used.\n\nA more informative measure is the mutual information between $Y \\mid S$ and $A \\mid S$. High mutual information suggests that the abstraction $Y$ retains a strong statistical relationship with the original action $A$, meaning $A$ can be nearly recovered from $Y$. In contrast, a coarser abstraction will lead to lower mutual information, as multiple actions will map to the same abstract representation.\n\nWe mention this in the paper in section 3.2, line 232, when referring to the need for “statistical dependency” between the two."}, {"review_id": "89wsJ6ap43", "reviewer": "Reviewer_NR3E", "summary": "The paper proposes modeling abstract counterfactuals building on the use of gumbel counterfactuals for autoregressive language models. Compared to the prior method the abstract counterfactuals are categorically valued and dependent on sampled sequences, rather than counterfactual at the step wise token level. A structural causal model is given, an inference procedure introduced and it shown to be consistent. Empirical evaluations are done on three datasets using three suggested measures. The results show that the abstrat counterfactuals change the original distribution better than only relying on token level counterfactuals.", "questions": "* Do you feel the framing around language model agents is necessary?\n* Do you see a way to sample from the models constrained on the abstract counterfactual?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n* The point of needing higher level counterfactuals than the token based autoregressive ones is intuitive\n* The paper is well written and clear\n* The three step inference approach is clear\n* Nice evaluation metrics\n\nWeaknesses\n* The preliminaries might need a slight bit of tightening, see smaller notes below\n\nNotes\n* The text in figure 1 is hard to read when printed\n* Colors in figures were flat gray when printed\n* I am not sure the use of numbers to indicate the actions in figure 1 and the introduction is better than typing out the action names in some typeface.\n* Some might want to include the endogenous variables in the SCM tuple, and mention the exogenous ones are independent\n* You might want to mention why the equation in line 100 is always well defined\n* In line 105 you say LMs are always autoregressive, this is not correct.\n* In line 109, K is not defined until line 117.\n* The type of the functions in equation 1 and 2 does not strictly match the preliminaries, should the parents be clarified?", "was_revised": true, "final_justification": "I stand by my original review and am pleased with the author's response.", "rebuttal": ">The text in figure 1 is hard to read when printed. Colors in figures were flat gray when printed\n\nThanks for pointing this out. we will fix it in the camera-ready version.\n\n>I am not sure the use of numbers to indicate the actions in figure 1 and the introduction is better than typing out the action names in some typeface.\n\nWe used numbers to indicate actions rather than typing them out to show them from the same perspective of the models, highlighting the ambiguity that results from the same symbols (numbers) being associated with different actions across the factual and interventional settings.\n\nOne of the weaknesses of the token-level approach is precisely that the conditioning is done over the tokens output by the policy (the number, or action id) rather than the action’s meaning or full representation. \n\n\n>Some might want to include the endogenous variables in the SCM tuple, and mention the exogenous ones are independent\n\nThanks for the suggestion, we will include the variables in the definition of the SCM for the camera ready version of the paper.\n\n>You might want to mention why the equation in line 100 is always well defined\n\nThank you for the suggestion, we will explain that we assume that all $P(\\mathbf{X}=\\mathbf{x}) > 0$ for all observations. \n\n\n>In line 105 you say LMs are always autoregressive, this is not correct.\n\nThank you for pointing this out. We will correct it in the camera-ready version of the paper.\n\n>In line 109, K is not defined until line 117.\n\nThanks for noticing this. We will define the notation in line 108. \n\n>The type of the functions in equation 1 and 2 does not strictly match the preliminaries, should the parents be clarified?\n\nThank you for your comment. We will re-order the arguments in the structural assignments displayed in these equations to match more closely the notation outlined earlier in the paper. \n\n>Do you feel the framing around language model agents is necessary?\n\nWe find that the LM agent formalisation is more flexible, and encompasses the text-generation setting by capturing multi-token generation steps as a single action. We explain this connection in section 4.2. While it is true that most of the case studies we consider are concerned with the special case of text-generation, our method is more general and can be applied beyond these settings, hence our choice to use the more general LM agent framework. \n\n>Do you see a way to sample from the models constrained on the abstract counterfactual?\n\nThis is an interesting point. \n\nOur approach corresponds to a form of rejection sampling based on the samples’ consistency with the counterfactual abstraction distribution. We constrain samples on the abstract counterfactual ‘a posteriori’. \n\nHowever, we think that constraining the language generation process in a direct / a-priori manner may not be feasible. One possible approach to this would require some sort of ‘model inversion’, and a probabilistic latent space. The abstraction could be defined as the embedding vector of the observation itself, with associated distribution in the associated probabilistic embedding space. Then, a counterfactual sample would be taken in this embedding space, which would be inverted back into text representation. \n\nThere would be several difficulties with this approach, however. Firstly, we are not aware of any (large) language models that operate on a probabilistic embedding space, especially when considering representations of whole sentences. This would also limit our method to white-box models or models for which the providers allow access to their embeddings, which isn’t very common. \n\nSecondly, mapping backwards from an embedding to the corresponding generating text generally requires access to several supervised pairs of embeddings and prompts, and usually involves training a soft-token prefix for a frozen white-box model (e.g. GPT2) such that the completion matches the supervised un-embedded text. Even ignoring the fully deterministic nature of the embeddings used in these methods, the overall approach seems unlikely to yield text of the same quality as that produced by an arbitrary LLM."}, {"review_id": "bWvsW3jQfk", "reviewer": "Reviewer_X1N5", "summary": "The paper works on the issue of counterfactual generation of language models at the token level. The issues arise because token-level actions might not be able to capture semantic meaning at a high-level. This might worsen in different environment. The paper works on this issue by proposing the idea of abstraction for an action from language models. The proposed method is model-agnostic, treating LM as a black-box model. \n\nAt high level, the propsoed method learns the abstraction concepts from the factual actions from the LM. Through the abstraction, the fraemwork generates the counterfactual profile in the abstraction space. Then it is mapped back the action space as the counterfactual action. \n\nThe proposed method is done by introducing another SCM that depends on observable, unobservable variables and actions. The support space of abstaction can be learned through either supervised or unsupervised setting. \n\nThe experimental section demonstrates the effectiveness of the proposed method on three benchmark datasets. The blackbox models are GPT2-XL and LLama3.2-1B. The author cliam that the token-level prediction tends to predict the observed token, while the abstaraction approach tends to predict the actual counterfactual action at the high level.", "questions": "1. How difficult is it to learn the abstraction space? In the paper, the abstraction can be learned in supervised or unsupervised setting. How is the support space defined in each case?\n2. Is $f_y$ shared for every single $t$? If so, how do you make sure that the support of Y is consistent across different $t$? If not, does that mean we need to learn each independent SCM for every $t$?", "limitations": "The only limitation that I can see is that only two LLM models are compared, which lack of convincing evidence for the claim made in the paper.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The issue that the paper is working on is significant, allowing the LM to be more explainable and transparent. \n- The proposed method seems effective from the experimental results. \n- The proposed method is fairly intuitive and easy to follow. \n\nWeaknesses\n- The main concern that I have is the novelty. It seems that the key contribution is the additional SCM that models the abstraction variable. - The intuition of why the abstraction variable works is not fully explained. \n- Only two models are compared. Adding some of the recent models might be more convincing.", "was_revised": true, "final_justification": "I have read the comments from the authors and updated my score accordingly.", "rebuttal": ">The main concern that I have is the novelty. It seems that the key contribution is the additional SCM that models the abstraction variable. - The intuition of why the abstraction variable works is not fully explained. \n\nOur main contribution is the introduction of an SCM which models the abstraction explicitly, and a method for generating counterfactual actions while performing counterfactual inference at the abstraction level. This allows us to perform counterfactual inference in a more meaningful manner where actions are identified with sequences of tokens, and whose meaning is context-dependent and may differ between factual and counterfactual settings. \n\nFor example, suppose the model sees the prompt “He studied hard to become a” and generates the completion “journalist”. If we now intervene to change the prompt to “She is a”, the token-level exogenous posterior associated with the next token would be that of “to” (the fourth token in both sentences, which will have no support in the counterfactual context), rather than the profession. We want to define the exogenous factor over the high-level meaning of the sentence (e.g., profession), not the dynamics of the individual tokens. \n\nOur approach addresses this by abstracting the state-action pair into a higher-level variable $Y$, which captures semantic meaning independent of specific surface forms. We can then derive a posterior over the exogenous noise at the abstract level, allowing for meaningful counterfactual reasoning even when the context changes. This abstraction preserves interpretability and semantic consistency across factual and counterfactual scenarios.\n\n>How difficult is it to learn the abstraction space? In the paper, the abstraction can be learned in supervised or unsupervised setting. How is the support space defined in each case?\n\nIn the supervised case, the support of the abstraction variable is defined by the user (explicitly, or based on available labelled datasets). Predicting the abstraction then becomes a traditional classification task, whose performance will depend on factors like data availability and quality. \n\nIn the unsupervised case, the procedure is broken down in two steps, both consisting of auxiliary language model calls. Firstly, we define the support of the random variable Y (using first of the prompts in appendix F). Then, we can use the result of this step to classify arbitrary strings over this support, using another language model call (second prompt in appendix F). \n\n>Is $f_y$ shared for every single $t$? If so, how do you make sure that the support of $Y$ is consistent across different $t$? If not, does that mean we need to learn each independent SCM for every $t$?\n\nThe mechanism $f_y$ remains the same across timesteps. In the supervised setting it is the modeller’s responsibility to ensure $Y$’s support is reasonable and spans the full range of relevant concepts. In practice, this set of concepts is usually easy to determine (e.g. professions for biography generation tasks). In the unsupervised setting, an auxiliary LLM call is used to discover the relevant concepts. For this, it is important to show this LLM a sufficient number of representative samples. \n\n>The only limitation that I can see is that only two LLM models are compared, which lack of convincing evidence for the claim made in the paper.\n\nThank you for your comment. For easier comparison with previous work, in the text generation setting, we used the same models (and intervention checkpoints) as Ravfogel et. al., the token-level baseline. Furthermore, in the text-based games setting (MACHIAVELLI), we implement the policy using the OLMo-1B model. \n\nOur method is not tied to any specific implementation detail of any of the language models, and, by virtue of its black-box nature, is designed to work with any policy which can be easily sampled, combined with an abstraction."}]

Author rebuttal:

Meta-review: This paper proposes Abstract Counterfactuals for language model agents, intervening at a higher action or interaction level rather than at the token level. The idea is well motivated, with clear potential to make counterfactual reasoning more semantically meaningful for agents. The writing is generally clear and the empirical results are encouraging across the reported tasks.

I believe the conceptual scope could be made clearer. The paper should more precisely define what counts as an abstract counterfactual and clearly separate its claims from token-level counterfactual methods, including a worked non Gumbel-Max example. It should also test the sensitivity of results to how the abstract variables are constructed and report failure cases.
Decision: Accept (poster)