Title: Abstract Counterfactuals for Language Model Agents
Venue: NeurIPS 2025
Accepted: yes
Keywords: causal inference

counterfactual inference

language models

agents

causality

interpretability
Abstract: Counterfactual inference is a powerful tool for analysing and evaluating autonomous agents, but its application to language model (LM) agents remains challenging. Existing work on counterfactuals in LMs has primarily focused on token-level counterfactuals, which are often inadequate for LM agents due to their open-ended action spaces. 
Unlike traditional agents with fixed, clearly defined action spaces, the actions of LM agents are often implicit in the strings they output, making their action spaces difficult to define and interpret. 
Furthermore, the meanings of individual tokens can shift depending on the context, adding complexity to token-level reasoning and sometimes leading to biased or meaningless counterfactuals.
We introduce \emph{Abstract Counterfactuals}, a framework that emphasises high-level characteristics of actions and interactions within an environment, enabling counterfactual reasoning tailored to user-relevant features. 
Our experiments demonstrate that the approach produces consistent and meaningful counterfactuals while minimising the undesired side effects of token-level methods.
We conduct experiments on text-based games and counterfactual text generation, while considering both token-level and latent-space interventions.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper works on the issue of counterfactual generation of language models at the token level. The issues arise because token-level actions might not be able to capture semantic meaning at a high-level. This might worsen in different environment. The paper works on this issue by proposing the idea of abstraction for an action from language models. The proposed method is model-agnostic, treating LM as a black-box model. \n\nAt high level, the propsoed method learns the abstraction concepts from the factual actions from the LM. Through the abstraction, the fraemwork generates the counterfactual profile in the abstraction space. Then it is mapped back the action space as the counterfactual action. \n\nThe proposed method is done by introducing another SCM that depends on observable, unobservable variables and actions. The support space of abstaction can be learned through either supervised or unsupervised setting. \n\nThe experimental section demonstrates the effectiveness of the proposed method on three benchmark datasets. The blackbox models are GPT2-XL and LLama3.2-1B. The author cliam that the token-level prediction tends to predict the observed token, while the abstaraction approach tends to predict the actual counterfactual action at the high level.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The issue that the paper is working on is significant, allowing the LM to be more explainable and transparent. \n- The proposed method seems effective from the experimental results. \n- The proposed method is fairly intuitive and easy to follow. \n\nWeaknesses\n- The main concern that I have is the novelty. It seems that the key contribution is the additional SCM that models the abstraction variable. - The intuition of why the abstraction variable works is not fully explained. \n- Only two models are compared. Adding some of the recent models might be more convincing.\n\nQUESTIONS:\n1. How difficult is it to learn the abstraction space? In the paper, the abstraction can be learned in supervised or unsupervised setting. How is the support space defined in each case?\n2. Is $f_y$ shared for every single $t$? If so, how do you make sure that the support of Y is consistent across different $t$? If not, does that mean we need to learn each independent SCM for every $t$?\n\nLIMITATIONS:\nThe only limitation that I can see is that only two LLM models are compared, which lack of convincing evidence for the claim made in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes modeling abstract counterfactuals building on the use of gumbel counterfactuals for autoregressive language models. Compared to the prior method the abstract counterfactuals are categorically valued and dependent on sampled sequences, rather than counterfactual at the step wise token level. A structural causal model is given, an inference procedure introduced and it shown to be consistent. Empirical evaluations are done on three datasets using three suggested measures. The results show that the abstrat counterfactuals change the original distribution better than only relying on token level counterfactuals.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n* The point of needing higher level counterfactuals than the token based autoregressive ones is intuitive\n* The paper is well written and clear\n* The three step inference approach is clear\n* Nice evaluation metrics\n\nWeaknesses\n* The preliminaries might need a slight bit of tightening, see smaller notes below\n\nNotes\n* The text in figure 1 is hard to read when printed\n* Colors in figures were flat gray when printed\n* I am not sure the use of numbers to indicate the actions in figure 1 and the introduction is better than typing out the action names in some typeface.\n* Some might want to include the endogenous variables in the SCM tuple, and mention the exogenous ones are independent\n* You might want to mention why the equation in line 100 is always well defined\n* In line 105 you say LMs are always autoregressive, this is not correct.\n* In line 109, K is not defined until line 117.\n* The type of the functions in equation 1 and 2 does not strictly match the preliminaries, should the parents be clarified?\n\nQUESTIONS:\n* Do you feel the framing around language model agents is necessary?\n* Do you see a way to sample from the models constrained on the abstract counterfactual?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Abstract Counterfactuals (ACF), which is a framework for generating counterfactuals from LLM agents.\nPrevious approaches used token level counterfactuals, which didn't capture the semantic aspects of the LLM agent's actions.\n\nThe main contribution is the introduction of a semantic-level abstraction variable, Y, that captures the context and meaning, e.g., intent, profession, or emotion. As Y depends on the (factual) state and action, it is used for the counterfactual reasoning to obtain a counterfactual action that is semantically consistent with the factual action. The paper offers supervised and unsupervised methods to obtain Y.\n\nThe paper does experiments on ACF: a text-based game (MACHIAVELLI), a biography generation task, and emotion tracking in Reddit comments. Results show that ACF creates more semantically consistent counterfactuals than previous token-level approaches.\n\nSTRENGTHS AND WEAKNESSES:\nQuality\nStrengths:\n- The paper provides a well-defined approach based on structural causal modeling and counterfactual reasoning\n- The paper validates the theory by doing experiments on different benchmarks\n- The paper sets up evaluation metrics suitable to the counterfactual semantic consistency: Abstraction Change Rate, Counterfactual Probability Increase Rate, and Semantic Tightness.\n\nWeaknesses:\n- The creation of the abstraction Y depends on dataset, classifiers used, or results from unsupervised methods.\n- The computational costs (e.g., sampling) could be challenging for practical real-world scenarios\n\nClarity\nStrengths:\n- The paper is clearly written and well-structured. The reader can easily follow the exposition for the SCM and the LLM Agent operating in a sequential decision-making environment.\n- The examples are intuitive, e.g., the bear and lion scenario\n- The details in the appendix are helpful\n\nWeaknesses:\n- Appendix B Metrics could benefit from more details\n\nSignificance\nStrengths:\n- The paper addresses issues in token-level counterfactual reasoning of LLM agents\n- The ACF approach can be impactful for LLM agent safety and fairness\n\nWeaknesses:\n- The quality of the abstraction Y may vary in real-world scenarios as it depends on many factors\n\nOriginality\nStrengths:\n- The paper presents a new method, ACF, that is different from past approaches, e.g., token-level counterfactual reasoning\n- The use of structural causal modeling with semantic abstraction Y for counterfactual reasoning is original.\n\nWeaknesses:\n- The creation of the abstraction\n\nQUESTIONS:\nIn Appendix A, Proof of Proposition 1, the paper states: \"We want to show that the expectation of our counterfactual distribution with respect to the exogenous values distribution is equal to the interventional distribution.\" \n- How does the proof differs from the known result in causal inference that the expectation over the counterfactuals (individual level)  is the interventional probability.\n\n-  What is an estimate for the time complexity of the sampling method?\n\n- Are there formal approaches to know how coarse or fine-grained the abstractions are?\n\nLIMITATIONS:\nNothing to add to the limitations the paper mentions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a framework for counterfactual text generation in language models. The main idea is that instead of performing counterfactual inference at the token or syntactic level, the abduction step in Judea Pearl's three-step counterfactual inference is carried out at a more abstract, semantical level. Experimental results show that on several benchmarks, the proposed method improves semantical consistency across factual and counterfactual contexts, over the token-level counterfactual generation methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The motivations of the approach are sensible and interesting, and the proposal is quite novel\n\n2. The technical presentation is largely clear and cogent.\n\n3. The experimental results are interesting and fairly convincing.\n\nWeaknesses:\n\n1. It is not entirely clear if the alleged flaws of the token-level counterfactual generation are due specifically to the Gumbel-Max setup or reflect general problems with token-level counterfactuals. For example, one problem highlighted by the authors is that \"TLCFs always increase the counterfactual probability of the observed token x^k\", but the argument for this claim is specific to Gumbel-Max SCMs, and for general SCMs, it is not true that observing a certain value will inevitably increase this value's counterfactual probability. This seems to indicate that the problem is confined to Gumbel-Max models. On the other hand, however, the authors also tried to motivate their work by noting the general phenomenon of context-dependence of meaning, which suggests that the work is not just intended to improve counterfactual generation with Gumbel-Max models. A clarification of this issue would be appreciated.\n\n2. Related to this ambiguity, and more importantly, if the claim is that there is a general problem with token-level counterfactuals going beyond the particular Gumbel-Max setup, the present paper does not yet provide a compelling demonstration of this general problem. In particular, it is unclear to me why f_A in general cannot be expected to properly represent the dependence of actions/generated texts on states/prompts that take into account the context-dependence of meaning. (Is the thought that we need an abstraction mechanism f_Y to tease out some context-invariant component of A?)\n\n3. On p. 5, in step 2 \"Counterfactual inference of Y'\", it appears to require a joint probability law of U_A and U'_Y. It is unclear how the joint distribution is supposed to be determined? In particular, are they assumed to be independent, even though U'_Y is updated?\n\nQUESTIONS:\nIn addition to the questions raised earlier in connection with \"weaknesses\", another puzzling point is the claim that the proposed approach can treat a LLM as a black box. It makes sense for interventions that only manipulate prompts. But why is this the case when the interventions also manipulate the decoder \\lambda?\n\nLIMITATIONS:\nDepending on the authors' responses to the questions raised earlier, the section on limitations may need to be expanded or adjusted.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}