Title: The third pillar of causal analysis? A measurement perspective on causal representations
Venue: NeurIPS 2025
Accepted: yes
Keywords: Causal Analysis

Representation Learning
Abstract: Causal reasoning and discovery, two fundamental tasks of causal analysis, often face challenges in applications due to the complexity, noisiness, and high-dimensionality of real-world data.
Despite recent progress in identifying latent causal structures using causal representation learning (CRL), what makes learned representations useful for causal downstream tasks and how to evaluate them are still not well understood.
In this paper, we reinterpret CRL using a measurement model framework, where the learned representations are viewed as proxy measurements of the latent causal variables. Our approach clarifies the conditions under which learned representations support downstream causal reasoning and provides a principled basis for quantitatively assessing the quality of representations using a new Test-based Measurement EXclusivity (T-MEX) score. 
We validate T-MEX across diverse causal inference scenarios, including numerical simulations and real-world ecological video analysis, demonstrating that the proposed framework and corresponding score effectively assess the identification of learned representations and their usefulness for causal downstream tasks. Reproducible code can be found at https://github.com/shimenghuang/a-measurement-perspective-of-crl.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors aim to clarify the role of CRL in causal analysis, particularly when the causal variables are not directly observable. The paper introduces a new evaluation metric, the Test-based Measurement EXclusivity (T-MEX) score, to assess how well learned representations align with the underlying causal structure. The T-MEX score is validated through both theoretical analysis and empirical experiments in numerical simulations and ecological video analysis.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The T-MEX score is a unique and practical contribution, offering a new way to quantify the alignment between learned representations and their causal counterparts. This metric is applied effectively in both simulated and real-world settings.\n\n2. The paper thoroughly addresses the existing challenges in causal analysis and representation learning, proposing a novel and interpretable solution.\n\n\n**Weaknesses**\n\n1. While the framework and T-MEX score are valuable, they may be challenging for practitioners unfamiliar with the underlying mathematical foundations or causal inference techniques. The paper could benefit from clearer explanations or visual aids for those new to the field.\n\n2. The paper discusses its framework’s ability to generalize across various causal tasks, but it remains to be seen how well it handles more complex or diverse causal environments beyond the tested scenarios.\n\nQUESTIONS:\n1. How well does this method generalize to more complex causal environments with multiple hidden variables and interactions? Could you provide examples of this and does T-MEX scale efficiently to high-dimensional datasets or when the number of causal variables increases significantly? Are there any known limitations in scaling the methodology to large datasets or real-time applications?\n\n2. How does T-MEX perform in comparison to existing evaluation metrics in causal representation learning in cases where causal dependencies are weak or non-linear?\n\n3. You discuss extensions to noisy data in your measurement model framework. Can you elaborate on how T-MEX behaves when noise levels are high or when data is missing?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose T-MEX, a new score to evaluate the quality of learned causal representations. Compared to existing metrics such as $R^2$ and MCC, the authors show that T-MEX is more adequate to assess how good the representation will be for causal downstream tasks. The idea behind this metric, based on conditional independence tests, is to measure the difference between 1) the graph linking the ground-truth and the learned representations and 2) the assumed graph between the ground-truth and the measurement model. Finally, the authors validate their score on synthetic and real-world data, demonstrating that a lower value of T-MEX is indeed associated with a smaller average treatment effect bias.\n\nSTRENGTHS AND WEAKNESSES:\n__Strengths:__\n- The authors propose an original score that is needed and will be useful to researchers in causal representation learning.\n- The paper is clear and well-written.\n\n__Weakness:__\n- The empirical validation of the score compared to existing methods is limited and could be more convincing.\n\nQUESTIONS:\n- While I understand the usefulness of the measurement model when considering models learning block-identifiable representation and specific causal tasks, the usefulness of the metric is less clear to me when considering models with identifiable representation (up to permutation & scaling), which is the most common case. In this scenario, the measurement model is simply mapping each variable to its equivalent measurement. \n\n- Besides the comment about the inadequacy of MCC, no experiments on the paper show this case and how MCC performs compared to T-MEX. I’d be interested to see this comparison since this is the most common case in CRL.\n\n- Considering the same scenario, T-MEX seems like a crude score: since T-MEX assesses in a discrete manner the existence/absence of conditional independence, one could have the same T-MEX score for two highly different representations that lead to different results on downstream tasks. One that strongly correlates with the ground-truth latent, and another one that weakly correlates, but is enough to be dependent, according to the CI test used.\n\n- For the experiment in Section 5.1, while it is said that $R^2$ fails to show a clear correspondence to the ATE bias, the average $R^2$ leads to the same ordering of the model as T-MEX.\n\n- Finally, while I recommend the authors for using a real-world dataset to validate their method, I find this experiment limited. Since there is only one measure variable and one ground-truth latent, the only two possible graphs are the empty graph or $Y$ causes $\\hat{Y}$. The empty graph means that the learned representation is independent of the ground-truth which seems like an overall bad model. Despite this, the differences observed in Figure 5 are pretty modest.\n\n__Typos:__\n- Line 121: diffeomporphism\n- Line 265: the MCC would obtain the highest value 1 although…\n- V is both used to denote the variables and the graph between them. I would use a different notation.\n- In Section 5.1, both sd and std are used to represent standard deviation. I would choose one notation and stick to it.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper treats causal-representation learning as a measurement problem: each learned code dimension should be an exclusive, CI-verifiable proxy for a single latent cause. It proposes T-MEX, a conditional-independence score that quantifies this exclusivity and proves a finite-sample link between low T-MEX and accurate causal-effect estimates. Simulations and an ecological-video benchmark show T-MEX tracks ATE bias better than common metrics like $R^2$ and MCC.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper formalises CRL as a measurement model and provides sound theoretical analysis.\n2. The authors conduct wll-designed small-scale studies.\n3. T-MEX is algorithm-agnostic and needs only observational samples, potentially valuable where ground-truth ATE is unavailable. It connects identification theory with downstream estimands via the notion of causal validity of a measurement model.\n\nWeaknesses:\nMy major concern is that the main contribution of T-MEX is that it can be potentially used to evaluate CRL methods when we know the ground truth causal structure and variables. While in the current paper, there is no explicit guidance on how to conduct this. For a random CRL method, how would T-MEX automatically find the block matching? Can T-MEX surrogate be used if we do not have access to the true variables? How would the grid of the CI test affect the performance? Is T-MEX efficient for large block sizes?\nHere are some minor concerns.\n1. There are multiple typos in the current version (for example, 'diffeomporphism' in line 121).\n2. The empirical evaluations are on small datasets. The simulations have only 5 variables. \n3. T-MEX relies on a valid, powerful CI oracle which may hinder its utility in real-world applications.\n4. The evaluations do not show the confidence intervals for T-MEX.\n5. The reliacne on the test choice is not shown.\n6. The relation between T-MEX and recent works on self-compatibility/falsification can be discussed more in detail in section 4 to highlight the difference.\n\nQUESTIONS:\n1. Can you compare T-MEX against CI-based graph validation metrics (for example, self-compatibility (von Kügelgen ’21; Ahuja ’23) or falsification/compatibility score (Marx & Vreeken ’22)) on the same representations?\n2. Would other CI tests give consistent rankings? Can you verify with experiments?\n3. Can you analyze the trade-off between T-MEX and the size of the CI oracle? Something like a worst-case or average-case bound on the number of CI tests T-MEX performs as a function of dimensions would suffice?\n4. Can you provide a recommendation of the $\\alpha$-level and sample-size heuristics?\n5. Is it possible to scale the experiments to aournd 20 latent variables with 100 observable variables? How does the runtime evolve witht he dimensions?\n\nI would be happy to raise my socre if my questions can be addressed.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors recast causal representation learning (CRL) as a third pillar of causal analysis by treating learned latent codes as measurements of hidden causal variables. Within this formal “measurement-model” framework they introduce T-MEX—Test-based Measurement EXclusivity—an evaluation score that counts conditional-independence violations of the exclusivity relations implied by a chosen measurement model. They prove finite-sample bounds showing T-MEX is controlled by the Type-I / Type-II error of the underlying CI tests and empirically demonstrate that the score predicts downstream ATE bias far better than correlation- or disentanglement-based metrics.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper is well-written and easy to follow.\n- The paper thoroughly explains why previous evaluation metrics and disentanglement scores can be misleading when latent variables are causally related.\n- The experimental results are interesting and solid.\n\n\n**Weaknesses**\n\n- **Strong reliance on exclusivity alone.** IIUC, the framework checks only whether any pair of learned features violates exclusivity given the others; it does not test other measurement-model axioms such as invertibility, monotonicity, or functional form. A representation that passes T-MEX could still be useless for causal inference if, for example, several causal variables collapse into a single latent factor.\n\n- **Linear-Gaussian focus** and **sample-complexity limits of CI testing.** All theoretical derivations assume additive Gaussian error terms (for analytic tractability of CI tests) and all simulations instantiate linear DAGs. Real-world problems often involve nonlinear, heteroskedastic, or discrete mechanisms; whether T-MEX still correlates with causal-effect bias under such violations is unknown. Each exclusivity claim is evaluated via a multivariate conditional-independence (CI) test whose power decays quickly with dimensionality and conditioning-set size. While Proposition 3.1 upper-bounds the expected T-MEX under the null, the paper does not translate this into concrete guidance on how many samples are needed for typical vision or language embeddings with hundreds of features, leaving practitioners without a way to judge feasibility.\n\nI am less familiar with the current progress in causal representation learning, so any clarifications on the questions above would help me to understand the background better.\n\nQUESTIONS:\nSee my comments above.\n\nLIMITATIONS:\nSee my comments above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}