Title: Causal Differentiating Concepts:  Interpreting LM Behavior via Causal Representation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: LM interpretability

causal representation learning

concept discovery
Abstract: Language model activations entangle concepts that mediate their behavior, making it difficult to interpret these factors, which has implications for generalizability and robustness. We introduce an approach for disentangling these concepts without supervision. Existing methods for concept discovery often rely on external labels, contrastive prompts, or known causal structures, which limits their scalability and biases them toward predefined, easily annotatable features. In contrast, we propose a new unsupervised algorithm that identifies causal differentiating concepts—interpretable latent directions in LM activations that must be changed to elicit a different model behavior. These concepts are discovered using a constrained contrastive learning objective, guided by the insight that eliciting a target behavior requires only sparse changes to the underlying concepts. We formalize this notion and show that, under a particular assumption about the sparsity of these causal differentiating concepts, our method learns disentangled representations that align with human-interpretable factors influencing LM decisions. We empirically show the ability of our method to recover ground-truth causal factors in synthetic and semi-synthetic settings. Additionally, we illustrate the utility of our method through a case study on refusal behavior in language models. Our approach offers a scalable and interpretable lens into the internal workings of LMs, providing a principled foundation for interpreting language model behavior.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a method for discovering interpretable, behaviorally-relevant concepts from language model (LM) activations, called “causal differentiating concepts”. Causal differentiating concepts are defined as latent directions whose change are necessary and sufficient to alter the model’s output behavior. The method operates in an unsupervised manner, relying on a constrained contrastive learning objective and a key sparsity assumption: that behavioral differences between examples can be attributed to sparse shifts in latent concept space. The authors provide theoretical identifiability guarantees under these assumptions, and validate the method on synthetic, semi-synthetic, and real LLM-generated behaviors (e.g., refusal behavior in LLaMA-3.1–8B), showing superior disentanglement performance compared to sparse autoencoders and other baselines.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well-motivated and addresses an important challenge in interpretability: extracting concise, causal explanations of LLM behavior without relying on predefined labels or concepts. The formalization of \"causal differentiating concepts\" is elegant, and the method is both novel and theoretically grounded. The empirical evaluations are good.\n\nHowever, there are two main concerns. First, the method extracts latent directions from the final layer of the LM, where representations may have already collapsed into task-specific, highly abstract forms. This design choice potentially limits the interpretability of the recovered concepts, since semantic richness is often better preserved in earlier layers. In practice, it is very likely that at least some of the differentiating concepts would collapse at the final layer, especially in more elaborate cases, where such interpretability methods would be most required.\n\nQUESTIONS:\na) Could the authors comment on whether applying the method to intermediate layers (rather than the final LLM layer) could yield more semantically meaningful concepts, even if less directly behavior-predictive?\n\nb) How sensitive is the method to violations of the 1-sparse assumption? Have the authors tested scenarios with 2-sparse or distributed causal shifts?\n\nc) In practice, how does one choose the number of latent dimensions? Is performance sensitive to this choice?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of extracting interpretable variables from LLM activations, under various assumptions. Given a dataset of paired texts x and discrete labels c (coming from an LLM), it's an important problem to be able to extract out the concepts that are responsible for the labels (assuming such a structure exists). The work considers the setting that there exist a set of mediating concepts z that are a function tau(g(x)) of the LLM activations g(x). Furthermore, the work assumes\n1. z is mediating, i.e. c is indepenent of x given z\n2. c is related to z via a logit-linear function.\n3. Every concept is a 1-sparse causal differentiating concept: There exist a pair of classes such that changing this concept is necessary and sufficient to change from one class to the other.\n\nUnder these assumptions, it's shown that z is identifiable up to linear transformations. Moreover, the authors propose a simple loss function to learn these latents, which is a variant of the contrastive loss. Experiments on synthetic data, BiasBios data and LLM text data show the effectiveness of their approach, compared to some prior works such as SAEs, when we look at disentanglement and MCC metrics. The target audience are people interested in interpretability of LLMs.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n- The paper approaches the interesting and relevant problem of extract interpretable variables from LLM representations. This is important for many reasons, such as to figure out inherent biases in LLM systems.\n\n## Weaknesses:\n\n- The assumptions seem a bit too strong. For instance, assuming that every relevant concept for a given set of labels is a 1-sparse causal differentiating concept is a very unique structure that is unlikely to hold in practice. It seems like the proof or technique may break if we have 2-sparsity (for an appropriate definition) instead of 1-sparsity. Could the authors comment on this?\n\n- The assumption that the mediating concepts are linearly related to the output variables is also very strong, since it essentially abstracts away the nonlinearity and turns the problem into a standard linear regression setting.\n\n- The theorems are fairly straightforward given the strong assumptions, e.g. theorem 1 follows by simple manipulations once linearity is assumed and theorem 2 essentially follows the proof ideas from Ahuja et al. 2022, so theoretical novelty is quite limited.\n\n- The number labels studied in the experiments seem very small, compared to what the approach or method can handle. It would be nice to have experiments on slightly more complicated real-world datasets with multiple classes.\n\nQUESTIONS:\nSome questions were raised above.\n\n- It seems like for the proof to go through, it maybe possible to replace 1-sparsity to some sort of \"full rank\" condition on concepts <-> label structure. Could the authors comment on this?\n\nLIMITATIONS:\nLimitations have been discussed.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 1, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel causal representation learning object which aims to recover the latent factors underlying model behaviour using an labelled dataset. Unlike previous approaches (e.g. causal abstract), this method does not need the hypothesis about the latent factors up front, but rather it uses key properties of the data and assumptions about the latent variables to learn the factors by itself. The authors provide results on both synthetic and real data which validates their method against alternatives without their causal objective, and baselines like sparse autoencoders.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The paper is very mathematically clean and offers a new approach to a real thorny issue in causal abstraction: generating hypotheses about underlying latent variables in the causal model is currently a fully manual process. If we could learn the latent variables that would make causal abstraction far more scalable, which this work seems to make progress towards doing.\n- The experiments involve several levels of complexity (from clean synthetic data to a more complex collection of real-world data on a relevant topic: refusal behaviour) and compare against well-formed baselines. It is great to see the inclusion of currently practical and in-vogue methods like SAEs. Outperforming these baselines builds convincing evidence for the validity of the method.\n\nWeaknesses\n- The real-world dataset is quite cleanly organised. It would be nice if this method could be applied to far messier real-world data like the baseline SAEs that were compared against. But this is not a significant weakness since this may require a wholly new method; this approach seems suitable for the problem the authors put in the scope of the paper.\n\nQUESTIONS:\n- The bottleneck is currently applied to the last layer hidden representations as justified by assumption 1. Is there any setup under which this method could be applied to internal representations, or does assumption 1 render this unjustifiable? This would enable identifying latent factors in model-internal components, which is useful for interpretability. It would also enable an alternative to SAEs.\n- If the dimensionality of the bottleneck doesn't correspond well to the true number of factors, what happens? E.g. if only 1 dimension or 5 dimensions are given to the harmfulness task, are the learned factors still cleanly basis-aligned? I'd be curious if it results in interesting decomposition of the topic factor (latent 2 in figure 2). Some experiments or at least theoretical analysis of this would be informative.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

comments: SUMMARY:
The authors claim 3 contributions:
1. "Causal differentiating concepts" - factors that much change to eliciting different model behavior - and a sparsity assumption for their identification.
2. A contrastive learning algorithm to identify the above concepts which is proven recover disentangled features.
3. Synthetic experiments and a real-world case-study supporting the above results.

STRENGTHS AND WEAKNESSES:
Originality: The work introduces a novel but simple latent factor model for describing categories of behaviors in LLMs, under certain modeling assumptions.

Quality & significance: The theory appears to be sound, and the work improves significantly upon the baselines in disentanglement and completeness, another measure of disentanglement. However, it would be desirable to see more evaluation on the validity of the underlying assumptions.

Clarity: the work is generally clearly written.

QUESTIONS:
1. Is there evidence to support the claim that the true conditional probability distribution of concepts can be modeled with the logit-linear function, as in Assumption 1? Are such conditional distributions universally expressive, or does the assumption impose constraints on the distribution that can be learned?
2. Does Assumption 2 require that every pair must be differentiated by some $z^r$? Does it require that each $z^r$ must differentiate exactly one pair of concepts?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03