Title: Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions
Venue: NeurIPS 2025
Accepted: yes
Keywords: Interpretability

Concepts

Causal Representation Learning
Abstract: Machine learning is a vital part of many real-world systems, but several concerns
remain about the lack of interpretability, explainability and robustness of black-box
AI systems. Concept Bottleneck Models (CBM) address some of these challenges
by learning interpretable concepts from high-dimensional data, e.g. images, which
are used to predict labels. An important issue in CBMs are spurious correlation
between concepts, which effectively lead to learning “wrong” concepts. Current
mitigating strategies have strong assumptions, e.g., they assume that the concepts
are statistically independent of each other, or require substantial interaction in
terms of both interventions and labels provided by annotators. In this paper, we
describe a framework that provides theoretical guarantees on the correctness of
the learned concepts and on the number of required labels, without requiring any
interventions. Our framework leverages causal representation learning (CRL)
methods to learn latent causal variables from high-dimensional observations in
a unsupervised way, and then learns to align these variables with interpretable
concepts with few concept labels. We propose a linear and a non-parametric
estimator for this mapping, providing a finite-sample high probability result in the
linear case and an asymptotic consistency result for the non-parametric estimator.
We evaluate our framework in synthetic and image benchmarks, showing that the
learned concepts have less impurities and are often more accurate than other CBMs,
even in settings with strong correlations between concepts.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work utilizes causal representation learning methods to first learn causal variables related to a given task, and then maps those causal variables into interpretable concepts for use in a CBM. By first learning causal variables, the authors propose that with few concept labels they are able to train a CBM with less impurity and higher accuracy than prior work (CBM, CEM, HardCBM).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Concept leakage is a very important problem in CBMs, and learning disentangled concept representations is a very interesting idea for ensuring concepts mean what they say.\n- Results are strong on both synthetic datasets evaluated. \n\nWeaknesses\n- Both datasets are synthetically created and fairly toy problems. Do the causal representation learning methods just not scale? Why can the authors not evaluate on a classic dataset such as CUB-200? If the theoretical analysis is grounded in the fact that the CRL approach can learn to disentangle the factors of variation within the dataset, then how is learning the causal variables not enough already? Current CBMs offer a solution for when the learning task is too complicated for simpler methods to work effectively, but if this approach is limited to small-scale datasets then how could it be useful in practice?\n- From what I understand, the approach has many hyperparameters compared to typical concept bottleneck models. Is performance resistant to the selection of these hyperparameters? How were the parameters in Table 3 of the Appendix selected?\n\nQUESTIONS:\n- Could the authors explain Figure 20 in the appendix as well as the general computation cost in more detail? From what I understand, the proposed approach is a two-stage approach, yet is more computationally efficient than CBM which should naively learn concepts from annotated data. Similarly, if I utilize a more powerful backbone for CBM, can I outperform the proposed approach without needing to perform the additional complexity?\n- What do the authors mean by 'few concept labels' throughout the paper? Are fewer concepts used? I don't see this claim substantiated in the main results, as it seems each method is tested over a different number of training points.\n- Could the authors explain lines 295-297: \"Although CITRIS-VAE provides the correct permutation of the groups, we ignore it and perform a random permutation on the variables. We train an MLP for each of the 32 dimensions that predicts all causal variables. Based on the R2 scores of these regressions, we learn the group assignments.\" From what I understand the authors are saying the CRL method perfectly learns the true factors of variation within the dataset, is this the case?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes using Lasso regression to map from a pretrained representation $M$, stemming from some exisitng VAE-based model that has been trained to discover disentangled causal factors, into a predefined set of concepts $C$, via a sparse mapping $\\alpha$. Sparsity is very important in this task, since each $M_i$ only becomes semantically interpretable if it maps to a single $C_j$. This is specifically formalized using Group Lasso, in order to allow for multidimensional factors in $M$ to be active together (for instance, 3D position as a factor).\nThe results on several synthetic datasets, where the underlying factors are known by design, show the advantage of the proposed approach with respect to regular concept bottlenecks.\n\nSTRENGTHS AND WEAKNESSES:\nThe main strength of this submission lies in the numerical results, which show the clear advantage of using disentangled features for downstream tasks.\n\nOn the weaknesses side:\n1. I found it tricky to understand what the contribution is. This may be due to my low familiarity with some of the elements in the paper. In the beginning, based on the title and abstract, I thought that the main contribution would be around CRL, but the it turns out that the CRL component is taken as a pretrained representation. This means that the novel aspect should lie with the learning of $\\alpha$, for which two variants of Group Lasso are used. I apologize if I misunderstood this aspect, but it would be helpful to state more clearly which element is novel. Is it the application of Group Lasso to this specific task that is novel? Are some of the provided proofs novel?\n2. This issue is reflected in the title itself, where the three lines suggest three different elements where the main contribution is (sample efficiency? Theoretical guarantees? No need for interventions?). By reading the rest of the paper, I could not make up my mind about this.\n3. As noted by the authors, CITRIS-VAE already recovers interpretable factors. What is then the contribution of the proposed approach with respect to that?\n4. The numerical comparisons are against several CBMs. However, there are no comparisons against the raw factors discovered by the various VAEs used. Does the proposed matching between $M$ and $C$ lead to differences in the performances?\n5. I guess the use of Group Lasso is due to the existence of structured, multidimensional causal factors. Is this so? I could not find a clear motivation in the paper.\n6. The authors claim that their method performs “better than the theory predicted”. I would like to seem some insights into how this is the case, since I could not find any in the discussion section.\n\nQUESTIONS:\nMy main request for the authors would be a clear delineation of where the claimed novelty lies, including a title that clearly states it (or at least an abstract).\nIn addition, I would also like to see the response of the authors to the raised weakness points.\n\nLIMITATIONS:\nSome of the main limitations are mentioned in the conclusions section, specifically related to the fact that an actual alignment between the real causal factors and human-understandable concepts may not exist. It is also mentioned that the method is not meant to work with correlated factors, although it is claimed it works better than expected in this setting. The one limitation that is not mentioned is the fact that human-annotated concepts are generally hard to obtain. In addition, all datasets used are synthetic, which raises questions about the usability of the approach on real data.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a two-stage framework for *sample-efficient concept learning* that couples recent Causal Representation Learning (CRL) encoders with an *alignment stage* that maps latent causal variables to user-defined concepts.  Stage 1 leverages any off-the-shelf CRL method to obtain latent variables that – under suitable identifiability conditions – recover the ground-truth causal factors up to permutation and element-wise transformations.  Stage 2 estimates this unknown permutation and the per-variable transforms with only a small number of concept labels.  Two estimators are introduced:\n\n* **Linear estimator** based on a Group-Lasso regression that comes with finite-sample, high-probability guarantees on both parameter error and exact permutation recovery (Thm 4.2) and an explicit sample-complexity bound.\n* **Kernelized estimator** that removes linearity restrictions and remains consistent under general invertible per-variable diffeomorphisms (Thm 5.2).\n\nExperiments on synthetic toy data, Action/Temporal-Sparsity time-series benchmarks, and the image dataset *Temporal-Causal3DIdent* show accurate permutation recovery and competitive downstream classification accuracy with substantially fewer concept labels than Concept Bottleneck Models (CBMs) and their variants.\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths\n\nThe work addresses an important bottleneck of CBMs – heavy dependence on concept supervision – by showing how unsupervised CRL can substitute for most concept labels.  Theoretical analysis is unusually thorough: the linear estimator enjoys *finite-sample* guarantees with an explicit tuning rule for the regulariser, while the kernelised version attains asymptotic consistency.  The presentation of assumptions, proof sketches, and ablation studies on the effect of regularisation, dimension, correlation, and data size is clear and technically solid.  Empirically, the method achieves lower Oracle Impurity Scores and higher label accuracy than strong CBM baselines when concept labels are scarce.  The code-free alignment step is light-weight and could be dropped into existing CRL pipelines, potentially broadening the impact of CBMs in domains where labels are expensive.\n\n#### Weaknesses\n\n1. **Over-idealised core assumption.** All theoretical guarantees hinge on the *pre-trained CRL achieving almost perfect disentanglement*, i.e. recovering each latent $G_i$ up to permutation + one-dimensional invertible transform.  In practice, available CRL methods on complex images or temporal data often yield *partially* disentangled variables and are sensitive to permutation/grouping instability; once this fails, the subsequent Group-Lasso or kernel step may learn mis-aligned coefficients, yet the paper provides no error bounds or robustness analysis beyond a brief empirical remark that “it still works”.\n2. **Correlation assumptions mis-match practice.** Finite-sample theory requires the *weak-correlation* condition (Assump. 4.1) for Group-Lasso. The authors admit that the empirical $\\rho$ in experiments “far exceeds” the theoretical regime, but no new convergence analysis is given, leaving the most central finite-sample bound without guarantees in realistic high-collinearity regimes.\n3. **Equating human concepts with causal variables is too strong.** The framework *assumes* that target “concepts” coincide with the data-generating latent causes (or monotone transforms).  In many domains (medical imaging, NLP) human-level concepts are abstractions over multiple causal factors.  While future work on “causal abstraction” is mentioned, the present proofs, estimator design and all experiments rely on the stronger assumption.\n4. **Narrow experimental coverage.** Three out of four datasets are synthetic and the sole real-data benchmark (Temporal-Causal3DIdent) is a rendered scene dataset.  There is no evaluation on natural images, text, or genuine human-annotated concept benchmarks, so interpretability remains at the metric-level (OIS/NIS) rather than user utility.\n5. **Discrete concepts lack theory.** Although the paper states that the method “also handles binary concepts” via a logistic Group-Lasso variant, there is *no* consistency or sample-complexity analysis for discrete or categorical variables; multi-class concepts such as “object shape” in Causal3DIdent are ignored in practice, leaving an important gap for real-world deployments.\n\nQUESTIONS:\n1. Can you provide quantitative evidence (or theory) for how permutation recovery degrades when CRL variables contain mixtures of causal factors?  \n2. Given that empirical $\\rho$ exceeds Assump. 4.1, do you observe systematic failures when correlations approach 0.9–0.99?  Could a debiased estimator or decorrelation pre-processing tighten the finite-sample guarantees?\n3. Have you considered extending the alignment to *sets* of latent variables (blocks) mapping to a single concept?  \n4. Would the method scale to CUB or CheXpert concepts if CRL encoders were available?  \n5. Do you foresee extending Theorem 4.2 to the logistic case, or providing a finite-sample excess-risk bound for binary concepts?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the problem of extracting interpretable concepts from data. This problem is broken into two steps. The first step is to extract latent causal generative factors from data, which is abstracted away as causal representation learning and is not addressed in this work. The paper focuses on the second step, which is to recover interpretable concepts from these latent factors. For this latter step, two approaches are proposed with theoretical guarantees. The first one is to use group regularized linear regression and the work presents sample complexity guarantees (w.h.p). The second one is to kernelize this procedure using basic ideas from RKHS theory and the paper obtains asymptotic guarantees borrowing results from high dimensional statistics. Experiments on synthetic and image benchmarks show the validity of their results.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n- While the field of causal representation learning is rapidly advancing, sample complexity guarantees are far less studied, so this work is a relevant and important direction in progressing the field.\n\n- The main claims seem to be supported well with proofs, and the experimental details are well hashed out, and the paper also probed various aspects of the experimental settings. I also appreciate the detail in the proofs which make them easy to follow, as well as the extensive plots and tables.\n\n## Weaknesses:\n\n- The assumption that the ground truth causal variables correspond to the concepts that we would like to learn about (even the minor modifications in appendix A that blocks of causal variables correspond to a concept), seems very strong. It's not clear this will be the case and indeed, the field of causal representation learning aims to recover these causal factors regardless of their interpretability.\n\n- Similarly, assuming that the mixing function can be recovered by CRL techniques (assumption 3.2) is abstracting away the difficulty of the problem at hand. It's this precise step that requires various assumptions such as availability of interventional data, therefore this limits the technical contributions of the results of this paper.\n\n- The experimental results validate the theory to some extent, however the main drawback is that they're still somewhat simple and limited -- they mostly work with toy data (including misspecification) and semi-synthetic setups like causal3DIdent. However, this is in line with most prior works in this direction.\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nLimitations have been discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}