The third pillar of causal analysis? A measurement perspective on causal representations
Authors: Dingling Yao
Shimeng Huang
Riccardo Cadei
Kun Zhang
Francesco Locatello
Venue: NeurIPS 2025 poster ()
Keywords: Causal Analysis
Representation Learning
Primary area: probabilistic_methods

Abstract: Causal reasoning and discovery, two fundamental tasks of causal analysis, often face challenges in applications due to the complexity, noisiness, and high-dimensionality of real-world data.
Despite recent progress in identifying latent causal structures using causal representation learning (CRL), what makes learned representations useful for causal downstream tasks and how to evaluate them are still not well understood.
In this paper, we reinterpret CRL using a measurement model framework, where the learned representations are viewed as proxy measurements of the latent causal variables. Our approach clarifies the conditions under which learned representations support downstream causal reasoning and provides a principled basis for quantitatively assessing the quality of representations using a new Test-based Measurement EXclusivity (T-MEX) score. 
We validate T-MEX across diverse causal inference scenarios, including numerical simulations and real-world ecological video analysis, demonstrating that the proposed framework and corresponding score effectively assess the identification of learned representations and their usefulness for causal downstream tasks. Reproducible code can be found at https://github.com/shimenghuang/a-measurement-perspective-of-crl.

The third pillar of causal analysis? A measurement perspective on causal representations
Dingling Yao"!, Shimeng Huang"!, Riccardo Cadei', Kun Zhang?. and Francesco Locatello'
IIstitute of Science and Technology Austria
Carnegie Mellon University
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) Equal contribution.
Abstract
Causal reasoning and discovery, two fundamental tasks of causal analysis, often face challenges in applications due to the complexity, noisiness, and high- dimensionality of real-world data. Despite recent progress in identifying latent causal structures using causal representation learning (CRL), what makes learned representations useful for causal downstream tasks and how to evaluate them are still not well understood. In this paper, we reinterpret CRL using a measurement model framework, where the learned representations are viewed as proxy measure- ments of the latent causal variables. Our approach clarifies the conditions under which learned representations support downstream causal reasoning and provides principled basis for quantitatively assessing the quality of representations using a new Test-based Measurement EXclusivity (T-MEX) score. We validate T-MEX across diverse causal inference scenarios, including numerical simulations and real-world ecological video analysis, demonstrating that the proposed framework and corresponding score effectively assess the identification of learned represen- tations and their usefulness for causal downstream tasks. Reproducible code can found at
Introduction
Causal analysis rests reasoning operates under the assumption that the causal structure is known or can be assumed, and leverages data to make quantitative causal statements, for example, about the average effect of one variable on another. As causal structures are often unknown, causal discovery aims to uncover this structure, assuming that the causal variables of interest are readily observed. In many real-world settings, however, the causal variables may not be directly observable. While originally formulated mostly enable causal capabilities in machine learning models, Causal Representation Learning (CRL, Schölkopf et al., 2021) has the potential to serve as third pillar of causal analysis: enabling applications of causality involving unstructured data. For this, we reinterpret causal representation learning using the formalism of "measurement models" (Silva et al., 2006), wherein the learned representations serve as proxy measurements for latent causal variables. This perspective of CRL allows us to better characterize when representation supports downstream causal reasoning, and also provides principled basis for quantitatively evaluating the quality of identification.
foundational pillars: causal reasoning and causal discovery. Causal
Methodologically, CRL tackles a more challenging task compared to independent component analysis (ICA) and disentanglement, where the latent variables are assumed be independent of each other (Hyvärinen and Pajunen, 1999; Hyvarinen et al., 2019; Higgins et al., 2017: Locatello al.. 2019). Instead, CRL aims unmix a of causally related latent variables. Many recent causal representation learning works have provided different theoretical results for causal variable
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

identification compiling various problem settings (von Kügelgen et al., 2021, 2024; Zhang et al., 2024b; Ahuja al., 2024, 2022; Varici et al., 2024; Zhang et al., 2024a; Yao et al., 2024b; Kong et al., 2022; Lippe et al., 2022b; Xie et al., 2024; Dong et al., 2024; Lachapelle et al., 2022, 2023; Yao al., 2022; Zhang al., 2024a; Squires al., 2023; Buchholz et al., 2024; Kong et al., 2023), recently unified by (Yao et al., 2025) into single general methodology. Although most of the results have been theoretical in nature, machine learning models explicitly empowered with identified causal structure have been shown to be more robust under distributional shifts and provide better out-of-distribttion generalization (Fumero et al., 2024; Ahuja et al., 2021; Bareinboim and Pearl, 2016; Zhang al., 2020; Rojas-Carulla et al., 2018). From an Al for science perspective, CRL has shown its potential in understanding climate physics from raw measurement data (Yao et al., 2024a), answering causal questions in the scope of ecology experiments (Cadei et al., 2024, 2025; Yao et al., 2025), psychometric studies (Dong et al., 2024), and countless more applications related to biomedicine (Zhang et al., 2024a; Sun et al., 2025; Ravuri al., 2025; Jain al., 2024).
Despite recent progress in identifying latent causal structures within causal representation learning, it remains unclear what makes learned representations useful for downstream causal tasks and how to best evaluate them. Building on the proposed measurement model framework, we introduce a new evaluation metric, the Test-based Measurement EXclusivity (T-MEX) Score, which effectively quantifies how well the learned representation aligns with the underlying measurement model. This underlying measurement model can be specified by, for instance, identifiability theory of a CRL algorithm (Fig. 1), assumptions for a particular causal reasoning task (Figs. 2 and 4), or ground truth knowledge. In contrast to commonly used CRL evaluation metrics, which suffer from clear limitations ($ 4), we demonstrate that T-MEX reliably assesses both the identifiability (Defn. B. and causal validity (Defn. 2.2) of learned representations, as shown in a wide range of causal reasoning tasks across numerical simulations and real-world ecological video analysis ($ 5). We summarize the main contributions of this paper as follows:
We reinterpret CRL using measurement model framework, wherein the learned representations serve as proxy measurements for latent causal variables ($ 2). This formalism provides a clearer characterization of both the identification quality of learned representation and its usefulness for causal downstream tasks.
Alpordio
Algoribo
We propose new evaluation metric (T-MEX) that quantifies the alignment of the representations and the underlying measurement model ($ 3), and we demonstrate its advantages over widely used CRL evaluation metrics that suffer from notable limitations ($ 4).
(b)
Figure 1: (Left) A measurement model where X is a fully mixed measurement of the causal variables. X is often termed the observables in CRL literature, representing the observed data. (Right) Two measurement models speci- fied by different CRL identification algorithms: (a) Algorithm guarantees one-to-one corre- spondence between the learned representation and causal variables; (b) Algorithm 2 guaran- tees that 2A, corresponds to Z₁ while 2 represents mixing of Zg and Z3.
Supported by theoretical analysis, our empirical evaluations confirm that T-MEX maintains validity and effectiveness across diverse causal reasoning scenarios, including treatment effect estimation and covariate adjustment in both numerical simulations and real-world ecological experiments ($ 5).
2 CRL from A Measurement Model Perspective
Throughout, we write [N] as shorthand for the set (1...., N). Random vectors are denoted by bold uppercase letters (c.g. Z) and their realizations by bold lowercase (e.g., z), indexed by superscripts. For instance, samples of Z are written as {zf hee[a]. A vector Z can be sliced either by single index E [dim(Z] Z)] via Z, or index subset A C [dim (Z)] with Z, == {Z, E A). Pz denotes the probability distribution of the random vector and pz(z) denotes the associated probability density function (We omit the subscription and write p(z) when the context is clear). By default, a "measurable" function measurable w.r.t. the Borel sigma algebras and is defined w.r.t. the Lebesgue measure. A more comprchensive summary of notations is provided in App. A.
Notation.

---

2.1 The Measurement Model Framework
We formulate causal representation learning using a measurement model framework inspired by the formalism of (Silva et al., 2006).
Definition 2.1 (Measurement model). Let V -Z, Z) be a collection of variables that can be partitioned into two sets: a set of latent causal variables Z = {Z ... ZNl } with Z, taking values in IR for all i E [N], and a set of observed measurement variables Z : .ZAu> J where for all j E [M]. AA, takes values in IR Ps with Dj E N4. and it holds that Z n Z
A measurement model M = (2, 2, (hg دررر specifies that 2 follows a deterministic structural causal model
M {2A, h2 (ul2 |==
where pa(2 pa,z,,) S [A] for all j E [M], and Z pa(2,,) S Z are called the causal parents of AA,- The functions hy for all j E [M] are called the measurement functions. If for some j E [M]] Ipa (a,)),)) and the function hy is the identity map, then the causal variable pa (Z,,) is said to be measured directly.
Remark 2.1 (Difference from (Silva et al., 2006)). While we borrow the concept of a measurement model from Silva et al. (2006), our framework differs in two key aspects. First, Silva et al. (2006) aims to uncover relationships among latent causal variables by searching for pure measurements, i.e., a tree- structure in which latent nodes have fixed, noisy, low-dimensional observed children (measurements). In contrast, we interpret a given causal representation produced by a CRL algorithm as measurement variables and focus evaluating their usefulness for specific causal tasks, which requires specification of a causal model. Second, Silva et al. (2006) assumes a linear latent structural causal model, whereas our framework imposes no parametric structural assumption the latent causal variables. Rather, we specify the relationship between the causal variables and their measurements according to certain hypotheses, such as identification guarantees, prior knowledge, assumptions for specific causal downstream tasks. As we will see in $ 3, this also allows us to properly evaluate a learned CRL model.
Remark 2.2. While we treat the measurement variables Z as noise-free nonlinear mixing of their causal parents, we can easily extend our framework noisy measurements by considering the noise variables as additional latent causal variables.
Example 2.1. Assume by the identifiability theory of a specific CRL method that each ZA, block- identifies (see Defn. B.1 (von Kügelgen et al., 2021, Defn 4)) a subset of latent variables Zs. (S, S (N). Then for the measurement model M = (Z, Z, {h, 3 it holds that: Σ = h, (Zs.), with hy: R | + R: a diffcomporphism for all j E [M].
The measurement model induces a partial directed acyclic graph (DAG), that is, for any latent variable q that is Moock-identified (Defn. B. by Aj. there is an edge from the latent causal variable Z, to the measurement variable A, and the measurement function hy is a diffeomorphism. Illustrative examples are shown in Fig. for different identifiability guarantees.
Discussion. Note that a measurement model specified by certain identifiability theory (see Fig. 1) is necessary but not sufficient condition for drop-in replacement of variable with its identified counterpart in a causal inference engine (Pearl and Mackenzie, 2018) or a downstream causal estimand like average treatment effect (Robins et al., 1994). To this end, we introduce causally valid measurement model.
Definition 2.2 (Causally valid measurement model). The measurement model (Defn. 2.1) is "causally valid" with respect statistical estimand g that identifies target causal estimand, if the measure- ment Z is drop-in replacement in g for the true causal variables Z, i.e., (Z) - =Z().
Discussion. Causal validity of a measurement model with respect to a specific estimand boils down to the estimand being invariant with respect to the measurement function. As (von Kügelgen et al., 2024) already pointed out, identification of a latent causal variable up a non-linear parameterization (i.e., ilock-idenifiiaiiiity (Defn, B)) does not allow average treatment effect estimation if either the treatment outcome is latent causal variable without additional information. For that,
3

---

direct measurement (see Defn. 2.1) as in (Cadei et al., 2024, 2025) is necessary; alternatively. one can choose an estimand that is invariant to non-linear invertible parameterizations, e.g.. (conditional) mutual information (Janzing et al., 2013). As another example, non-linear invertible parameterization enough to model confounding variables (Yao et al., 2024a) and instruments, see F for extended discussions and examples. Finally, note that the causal validity of the measurement models does not always require one-to-one correspondence between the measurement variables and latent causal variables: When an estimand concerns a searse-graining of subset of variables, then a measurement model mixing the right subset of variables can still be causally valid. For example, the valid adjustment set W in Fig. 11 can contain two more variables, which remain entangled with each other in the learned representation W := h(W) as long as the measurement function is invertible, see App. F for detailed derivations.
When is measurement model "true"? Note that any causal model between learned representation can always be trivially formulated as a measurement model, with each identified representation variable corresponding to a latent causal variable (i.c., 2, - Ze implicitly implies measurement model Z, + Zi + Z2 + Zz). Sometimes, by means of other assumptions, the latent causal model may not match one-to-one with the measurements; for example, see Fig. 1 (b). Our discussion on the measurement model only specifies the dependency between learned representation and an (implicitly) assumed latent causal model. Following (Peters et al., 2014), we intend the latent causal model to be true if it agrees with the results of randomized studies in practice. If the latent causal model is true, then a causally valid mcasurement model is trivially also true.
3 Evaluating Causal Representations using Measurement Models
This section explains how the measurement model formalism introduced in 2 serves natural tool to evaluate causal representations. A causal representation is defined as a set of measurement variables output from an encoder
parameterized function that maps the observables X to the measurement variables Z. Each CRL method specifies a measurement model, either through its iden- tifiability guarantees or the particular causal task it addresses. This measurement model defines which causal variables a representation should exclusively measure. Given paired samples of the true causal variables Z and their corresponding measurement variables Ž from a trained CRL model, evaluation boils down to comparing the measurement model against the observed joint distribution P2.22 Before presenting our proposed evaluation metric, we introduce the following additional notation.
Additional notation. Let Z,, Zz, and Zy be three absolutely continuous random variables taking values in Rd Rd and Rdz respectively. We say that Z, and Z₂ are conditionally independent given Zg if (ZZ,, Ze Zz) p(Z) Zz)p(Z2 Za), and it is denoted as Z, 1l. Z2 Zg. A statistical
test o a function that maps data to (0, 1}, e.g. yp Rxaxd x Raxxz x R Ruxdzz -> {0. 1), where denotes the number of samples. The test yp rejects a null hypothesis Ho if (Z, Zz. Zs) = 1 and
does not reject if (Z, Zz, Zs) = 0. Given a significance level a E (0, a test said to be valid
if it holds that supp P(y Zi, Zz, Z3) = 1) < a, and it is said to have power 3 E (0, 1) against
an alternative distribution & Ho if (pp ( ZZ,, Zz, Za) = 1) =
Exclusivity of measurements. A measurement model describes the relationship between the causal and the measurement variables. Specifically, it tell us for each measurement variable, which causal variables it should exclusively measure. We formally define this concept below.
Definition 3.1 (Exclusivity of measurement variable). Let M (Z, Ž, (he be measure- ment model, if a measurement variable A j E [M] only has one causal parent Z; for some i E [N]. then we say A exclusively measures Z,-
Given samples of the causal and measurement variables {(z², (2,2 ]<e())) we can check whether the measurement variables do satisfy the exclusivity property in the data by testing the following null hypotheses:
Ho(i,)) : ZA, 11 Z, Z N]N()
(3.1)
for all E [N] and j E [M]. For numerical summary of the overall exclusivity of the measurement variables, propose the following Test-based Measurement EXclusivity (T-MEX) score.

---

Definition 3.2 (Test- based measurement exclusivity score). Let V E 10 be the adjaceney matrix corresponding to the conditional independencies according to a measurement model M. such that for all j E [M] and i E [N]. Vj= = 1 if a causal variable Z, is causal parent of a measurement
variable AA, according to the measurement model, and Vj, = 0 otherwise. Let W E {0. 1] 1 be the matrix constructed according to the test results of the conditional independencies in eq. (3.1) based on the samples of (Z, Z). such that for all j E [M] and E [Av] W5+ 1 if Ho (i, j) is rejected,
and W = 0 otherwise. Then the test-based measurement exclusivity (T-MEX) score is defined as the hamming distance between V and W:
M ΣΣ
T-MEX W)
1(55 + Wj)).
where 1 denotes the indicator function.
Details for computing T-MEX is given in Alg. 1. As T-MEX score is based on conditional indepen- dence testing, its value depends the randomness in the samples, and the properties of the statistical tests being used. In Prop. 3.1, we show the upper bound of the expected T-MEX score when the joint distribution Pz 2 of the causal variables Z and output measurement variables Z from a CRL model does align with a measurement model.
Proposition 3.1. Let (EEN)EEENNJEE be a family of tests for eq. (3.1) where for all i E [N] and j E [M,, Pij is valid with level a E (0,1) and has power at least 3 E (0, 1). Given an adjacency ma- trix V E NNX based on a measurement model, if the joint distribution Pz, of the causal and mea- surement variables does align with the measurement model, and each entry in W is computed based on an independent set of samples ((z), 2*) 2*) (eeul Maj N., then the expected T-MEX satisfies
E[T-MEX(V, W)l S a- (MN- (IVIh) + +1 (1- (1-8))1[Vila,
where ||V|| NN E Viz is the L1 -nom V.
Remark 3.1. Prop. 3.1 assumes that each null hypothesis in eq. (3.1) is tested using an independent set of samples. When there is only one set of samples available for a large number of tests, using the same sample set can lead to inflation of the false positive rate, and may inflate the T-MEX score. In this case, we recommend doing a multiple comparison adjustment when constructing W, for example, the Bonferroni- -Holm correction (Holm, 1979), which controls the family-wise error rate while it does not make assumptions on the dependencies of the multiple p-values.
Remark 3.2. In this section, we focus on the exclusivity perspective of a measurement model via an approach similar to the idea of falsification of causal graphs (e.g., Kook, 2025; Faller et al., 2024). This is non- non-parametric approach which is agnostic to the measurement functions. In certain cases, however, a measurement model may contain not only the conditional independence structure, but also other parametric assumptions through specifications of the measurement functions (hi) JEMM- Then, one may extend T-MEX to also take these constraints into account.
4 Related Work: Flaws of Existing Evaluation Metrics for CRL
In this section, we cover the metries that have been used by most papers proposing causal represen- tation learning approaches (von Kügelgen et al., 2021, 2024; Zheng et al., 2022; Ahuja et al., 2024, 2022: Varici et al., 2024; Zhang et al., 2024a,b; Yao et al., 2024b; Lippe et al., 2022a,b; Lachapelle et al., 2022, 2023; Yao et al., 2022; Zhang et al., 2024a; Squires et al., 2023; Buchholz et al., 2024; Yao et al., 2025) to name a few. We highlight how these metrics are not immediately suitable to evaluate identification results in the presence of causal relations, making it diffficult to compare models and requiring great care in the interpretation of the results that is often missed (Gamella et al., 2025).
Standard evaluation for latent variable identification in existing CRL works employs coefficient of determination R2 (Defn. 4.1), and mean correlation coefficient (Defn. 4.2). However, when the latent variables are causally related, a high score of these two metrics does not indicate that the learned representations align with the measurement model we expect from the identifiability theory. Example 4.1 illustrates this limitation of these two metrics under the presence of causal dependencies.
5

---

Example 4.1. Assume that the latent causal variables Z in Fig. (b) follow a linear Gaussian additive noise model. Specifically, the latent variables Z and Zg are generated based on the following
structural equation:
Z = a Z +
(4.1)
with Pe. Fle] and II. Z, Suppose that the measurement model which induces Fig. 1 (b)
specifies that the measurement function h R is diffeomorphism such that ZA, h (Z)), that is, A, identifies 1 while AA, should not contain any additional information about Zz.
Coefficient of determination. Rº measures the proportion of the variation in the dependent variables explained by the regression model (Draper and Smith, 1998), formally defined as
Definition 4.1 (Population R2 score). Let (Z, ZA,) be pair of random variables both taking values in R, E [N],j E [M]. The coefficient of determination R² score for predicting Z, from Zn, is defined as
R² (Z, V ZA,1)
V(Z,)
where E and V denote the expectation and variance operators, respectively.
Problem of R in Example 4.1: Let R2 ZA.) denote the R² score defined in Defi, 4.1.
Following the linear mechanism in eq. (4.1), R (Z2, ZA,) can be expressed
(Zz,2AA,) = V(E Zz Ax,1) N(BjaZ, + e (),,))
((Z2)
V(aZı +e)
(4.2)
(E [A.))
(Z)( R R2 A..)
QZV(Z,) + V(e) a2V(Z,) + V(e)
Depending on the noise level V(e). (Z,, ZA,) can be either close to RR Za,) when V(e) < (Z)(Z,) or close to 0 when Vee) is significantly higher than (Z,; in either case it does not reflect whether AA, identifies Ze or not, in the sense of Defn. B.I. Ultimately, H² is metric for predictability, not for identifiability. Using as an identifiability metric under causal dependency can lead to misinterpretation (Gamella et al., 2025).
Remark 4.1 (Other problems of R2 score). R22 is designed to measure how well linear model fits between two random variables. When the fitted model is nonlinear, R² can yield values outside [o, 1]. which can be misleading. See also Cameron and Windmeijer (1997) for more details.
Mean correlation coefficient (MCC). Intuitively, MCC measures the component-wise correspon- dence between the learned representation Ž and the ground truth latent variables Z. When using MCC, it is required to have the same latent and encoding dimensions. We restate the definition the MCC as follows.
Definition 4.2 (Mean correlation coefficient).
MCC
max [Epert[N]
Corr(Z; (Z, Zz(())
N
1=1
where Corr(;,) refers to the Pearson correlation under linear relationship and Spearman correlation in the nonlinear case.
However, we notice that MCC cannot capture how well the representations are disentangled, mis- aligning with its original purpose of measuring component-win correspondence. Assume in Fig. 1 (b) that ZA, = Z, and ZA [Zg, Za]. The learned representations ZA, are linear mappings of their causal parents Z 2a(2,)))
2, = s Z1;
2, = a a- + Zz:
2 = c Zg + d Zz.
where s, a, b, c, d A 0. this case, the MCC would obtain the highest value 1, although Zg, Z3 are still entangled in the learned representation 2, demonstrating that MCC is inadequate in evaluating clement-wiss identification under causal relations.

---

Evaluation of causal relations. Causal relations are usually evaluated with the standard metrics Structural Hamming distance (SHD). We remark that evaluating causal discovery on the learned rep- resentations should always be done in conjunction with latent variable identification, as it is possible to achieve a perfect SHD (i.e, zero) with entangled representations, using e.g., LiNGAM (Shimizu et al., 2006), as shown numerically in App. D.3.
Evaluation of disentangled representation. Evaluating disentangled representations (where the ground truth latent variables are assumed to be mutually independent) is comparatively easier. In the disentangled case, the main objective is assess how well the learned representation aligns one-to- one with the ground truth latents. Commonly used evaluation metrics for disentangled representations include the BetaVAS VAE Score (Higgins al., 2017), FactorVAE Score (Kim and Mnih, 2018), Mutual Information Gap (MIG Chen et al. (2018)). DCI-disentanglement (Eastwood and Williams, 2018 88), Modularity (Ridgeway and Mozer, 2018) and SAP (Kumar al., 2017). Broadly, evaluating learned representations can be viewed as two-stage procedure, first estimating the relationship between latent variables and representations, and then aggregating this information into a single score (Locatello et al., 2020). In some way, our test can be seen following the same strategy, although evaluating variable-level correspondence is less straightforward given underlying causal relationships, making fundamentally more challenging and understudied problem.
5 Experiments
This section demonstrates the validity of the proposed T-MEX score in various causal reasoning set- tings. We first focus on covariate adjustment in numerical simulations, using T-MEX to evaluate both identifiability (Defn. B.1) and causal validity (Defn. 2.2) of the representations ($ 5.1). Next, we move on to treatment effect estimation in high- high-dimensional ecological video analysis, where we demonstrate that T-MEX effectively characterizes how well the learned representation supports answering down- stream causal questions ($ 5.2). For both experiments, we estimate T-MEX based on the projected covariance measure (PCM) test (Lundborg al., 2024) implemented in the python package pycome ts (Huang and Kook, 2025), which is an algorithm-agnostic test for conditional independence (see App. for more explanations). Further experiment details and additional results deferred to App. D.
5.1 Numerical Simulation
This experiment validates our proposed T-MEX evaluation metric through controlled numerical simulation. We leverage CRL model confounders and perform backdoor adjustment estimate the average treatment effect (ATE). We report both R² and the ATE bias, demonstrating that T-MEX closely aligns with the absolute ATE bias and provides a reliable measure of representation quality, whereas R2 fails to yield consistent or meaningful conclusions.
X
0.20
Z
Z
Zs
Z+
Z
JATE Bias
0.15
0.10
Zu
0.05
Figure 2: Measurement model containing the latent causal variables Z, Zz, and Z3 (white nodes) and observed (also termed "directly mea- sured" in Defn. 2.1) causal variables Z4 and Z; (gray nodes). The entangled observable X is shown as a dashed oval. ZA, denotes the exclu- sive measurement (Defn. 3.1) of Zi.
0.00
T-MEX
Figure 3: T-MEX tracks the absolute bias of the ATE estimates of Za on Zs where Z, is conditioned on as the back door adjustment.
Experiment settings. We generate five causal variables, Z; for E (5] according to a linear structural causal model (see App. D.1), where two of the causal variables, Z4 and Zs. are observed (also termed "directly measured" in Defn. 2.1). The entangled observations X == f(Z,, Zz Za) are
7

---

generated by applying a diffeomorphism f R - R2, implemented as an invertible MLP, on the causal variables. Our targer causal task is to estimate the ATE Z4 on Zs. As the true causal relationship between Z4 and Zs is linear we can construct consistent causal estimator where Z adjusted using linear regression, which is invariant up to bijective transformations of Z1 (App. F). Although Z is latent and cannot be directly adjusted for, one can measure it through bijective transformation ZA, (Zz) which is obtained from the entangled observation X. Note that in this case, ZA, exclusively measures (Defn. 3.1) the confounder Z1, as depicted in Fig. 2. We train three different CRL models based on the identifiable learning algorithm proposed by Yao et al. (2024b)
and obtain samples of the measurement variable ZA:
Model A: a sufficiently trained model from which we expect the learned representation A (where by slight abuse of notation, the superscript represents the model indicator) to exclusively measure Z1;
Model B: an insufficiently trained model with unclear latent tient-mmaasurement correspondence;
Model C: a corrupted version of Model A where the representation 2 is defined as a Ar
linear mixing of the identified representation A and Z2, Zs-
Results. Tab. 1 summarizes the T-MEX scores together with the coefficient of determination R² for all three models A. B and C, presented mean=sd. For statistical validity, we compute the results using 50 simulated datasets from each model, with each dataset containing 4096 observations. Further details about the test results provided in App. D. Tab. shows that sufficiently trained model (Model A) achieves low T-MEX score, indicating that the learned representation ZA exclusively measures the latent variable Z, In contrast, the insufficiently trained and corrupted models (Models and C) exhibit high T-MEX scores, demonstrating misalignment between the learned representation and the hypothesized measurement model (Fig. 2). Fig. 3 presents the absolute ATE bias estimated from the learned representations of all three models. We observe a strong correlation between T-MEX and the absolute bias of the ATE, validating T-MEX as reliable indicator of the causal validity of the learned representation (Defn. 2.2). In contrast, as shown in Tab. 1, R² is relatively high for all three latent variables, failing to show clear correspondence with the ATE bias.
Table 1: T-MEX, R² scores, and Spearman correlation coefficients of the learned representations (presented as medn of model A (sufficiently trained, i.e., Z, exclusively measures Z1). model B (insufficiently trained model with unclear latent- ment correspondence) and model C (manually corrupted representation by linearly mixing Zz, Zy with the representation of model A) based on 50 simulated datasets, where each dataset contains 4096 observations.
R²
Spearman Cor, Coef.
Model T-MEX
Z,
Z,
Z₁
A 0,12010 + 0.3283
0.9984 1: 0,0001
0.7516 1 0.0064
0.8001 1: 0.000
1.0000 1: 0.0000
0.8568 a: 0.0044
0.8864
O. 0.0040
1.1800
0.6865
0.8303
0.0027
0.0061
0.9602
0.9908
0.0001
C
2,0000 1. 0.0000
0.9394 1 0.0016
0.5421
0.0096
0.6827 1 0.0084
0.9673 1. 0.0013
0.7215
0.0076
0.8016
0.0062
5.2 Real-world Ecological Experiment: ISTAnt
T
Y
This experiment validates the T-MEX score on ISTAnt (Cadei et al., 2024), a real-world ecological bench- mark designed for treatment effect estimation. We show strong correlation between T-MEX and the absolute bias of the ATE, demonstrating that T-MEX can reliably evaluate the causal validity of learned representations under the challenge of nigh-dimensiooal real- world data.
X
Y
Figure 4: Measurement Model for the causal task in ISTAnt. T denotes the treatment (chemical exposure) and the latent outcome Y represents the ant's grooming behavior. Observable X (video recordings) is represented using a dashed oval. The measurement Y exclu- sively measures (Defn. 3.1) Y.
Experiment settings. ISTAnt consists of video recordings of ant triplets with occasional grooming behavior. The goal is to extract a per-frame representation for supervised behavior classification (grooming or not) to estimate the ATE of an intervention (exposure to certain pathogen).
8

---

Retrieving causally valid representations in this case is challenging as we have more non annotated than annotated data, described by (Cadei al., 2024). Fig. depicts the hypothesized measurement model for this particular causal task, note that the treatment T and outcome Y are unconfounded because the data is collected through a randomized controlled trial (RCT), meaning that the binary treatment T is randomly assigned.
Results. We compute the T-MEX score for 2.400 different models at significance level of a 0.05, and compare both classification accuracy and absolute ATE bias against T-MEX. A full description of the considered models and training details is reported in App. D.2. We only focus on the models that yield an accuracy over 80% for meaningful statements. We observe that models with T-MEX = 0 achieve higher mean and lower variance for both accuracy and absolute ATE bias, demonstrating that T-MEX effectively and reliably evaluates the quality of learned representations in terms of both classification performance and causal validity (Defn. 2.2).
Statistical validation. To further assess the statistical significance between the T-MEX = 0 and T-MEX groups, we conduct Mann-Whitney U test (Mann and Whitney, 1947) with the null hypothesis
Ho: E |ATE Bias|
T-MEX = =1] E |ATE Bias
T-MEX = 0].
The resulting p-value of 0.0047 leads us to reject Ho. providing strong evidence that the average absolute bias of the ATE for models with T-MEX 1 is significantly higher than for those with T-MEX = O. Overall, T-MEX shows strong correlation with the absolute bias of the ATE, validating its reliability as evaluation metric for the causal validity of learned representations (Defn. 2.2).
Real-world implications of T-MEX. We emphasize that the proposed T-MEX score can be computed using only observational data, possibly with selection bias, as long as this selection bias does not change the conditional independence between measurements and causal variables. Instead, calculating the ATE bias as in (Cadei et al., 2024) requires a validation set that closely approximates the underlying population of the randomized controlled trial, a significantly stronger assumption that often difficult to satisfy in real-world settings. Overall, T-MEX offers convenient and accessible evaluation metric that reliably quantifies the usefulness of the learned representation for a causal downstream task, without the need for additional identifying assumptions.
1.00
100
0.95
Accuracy
[ATE Bias|
75
0.90
0.85
25
0.80
T-MEX
T-MEX
Figure 5: T-MEX reflects model performance in terms of both classification accuracy and causal validity (Defn. 2.2). Compared to their counterparts, models with lower T-MEX achieve consistently high accuracy (Left) and low absolute ATE bias with reduced variance (Right).
5.3 Evaluation of T-MEXX Properties using Synthetic Data
In this section, we outline additional experiments designed to assess specific properties of T-MEX. including its scalability, robustness across different test choices, and behavior under weak or nonlinear causal relations well noisy measurements. The corresponding experimental details and results are provided in App. D.4.
Reliability and scalability with higher-dimensional latent variables. It is well known that the statistical power of conditional independence (CI) tests deteriorates as the dimensionality of the conditioning set increases- limitation shared by most CI methods (e.g., Shah and Peters, 2020; Strobl al., 2019; Zhang et al., 2012). We examine the reliability and scalability of T-MEEX given data generated from a non-linear location-scale SCM with up to 50 latent nodes in Tab. 3 in App. D.4.

---

Since our proposed framework is agnostic to CI test choices, T-MEX can be easily scaled to larger dimensions by adapting more powerful and scalable test methods they are developed.
Consistency of T-MEX under different test choices. CI testing is an important component in our framework, and choosing valid and powerful test is essential. We chose PCM test in our experiments for its theoretical guarantees on the validity (type I eгror control) and power of the test, making it reliable choice of test for the exclusivity claim. We also compare T-MEX scores using different conditional independence tests in Tab. 4 in App. D.4, based on the same experimental setup as in $ 5.1. In this experiment, we see that T-MEX remains consistent across different test choices.
Consistency of T-MEX under noisy measurements. As discussed in Remark 2.2, noise in the measurement functions can be treated additional independent latent variables. This means that if the noise is independent of the existing causal variables, the conditional independencies required for T-MEX remain intact. In such cases, can safely compute T-MEX without explicitly modeling the noise; the score will remain valid. The empirical T-MEX value shown to be robust under noise, as shown in Tab. 7 in App. D.4.
T-MEX under weak causal relations. Building on the experiment in $ 5.1, we also examine how T-MEX behaves when the causal relationships between the latent causal variables are weak and compare it with the commonly used CRL identifiability metrics R² and MCC. See Tab. 5 in App. D.4. In this case, T-MEX correctly reflects the entanglement in the representation, while both R and MCC give a misleading score suggesting perfect element wise identification.
Alignment between T-MEX and ATE under nonlinear SCM. Extending the results in 5.1,, we further examine the correspondence between T-MEX and ATE under a nonlinear setting. As shown in Tab. 6 in App. D D.4, the results are highly similar to the linear case $ 5.1, T-MEX closely aligns with the absolute ATE bias, effectively evaluating causal representations for downstream inference tasks with nonlinear causal relations.
6 Conclusion and Limitations
This paper introduces a novel perspective on Causal Representation Learning (CRL) based on a measurement model framework, in which causal representations are treated proxy measurements of latent causal variables ($ 2). This perspective provides flexible framework that unites CRL identification theory with downstream task assumptions via measurement functions, yielding principled way to evaluate representation quality. More specifically, we propose a new evaluation metric, Test-based Measurement EXclusivity (T-MEX) score, which quantifies the discrepancy between a given measurement model (specified by a CRL algorithm, a causal task, or ground truth knowledge) and the joint distribution of causal and measurement variables (representation outputs of CRL model) using conditional independence tests ($ 3). Because these conditional independence tests impose parametric assumptions, our T-MEX score remains broadly applicable. However, like any statistical procedure, these tests are subject to sampling variability and potential statistical errors, so the reliability of T-MEX depends which test chosen. By remaining agnostic about the specific test, we empower practitioners to tailor the score to whatever assumptions they are willing make (e.g., parametric or non-mmaraetrric.. We demonstrate, using both simulations 5.1) and real-world video analysis ($ 5.2). that our proposed T-MEX score effectively quantifies the identification and causal validity of the learned representation (Defn. 2.2). This provides convenient and practical evaluation scheme for representation quality in real-world scenarios, especially when the true treatment effect bias is unavailable, such as in the absence of randomized studies.
A Acknowledgment
This research was funded in whole or in part by the Austrian Science Fund (FWF) 10.55776/COE.1. For open access purposes, the author has applied a CC BY public copyright license to any accepted manuscript version arising from this submission.
10

---

References
K. Ahuja, Caballero, Zhang, J.-C. Gagnon-Audet, Bengio, Mitliagkas, and Rish. Invariance principle meets information bottleneck for out-of-distribution generalization. Advances in Neural Information Processing Systems, 34:3438-344 -455,, 2021.
K. Ahuja, S. Hartford, and Bengio. Weakly supervised representation learning with sparse perturbations. Advances in Neural Information Processing Systems, 35:15516-1552 2022.
K. Ahuja, A. Mansouri, and Y. Wang. Multi-domain causal representation learning via weak distributional invariances. International Conference on Artificial Intelligence and Statisties, pages 865-873. PMLR, 2024.
C. Ai, L.-H. Sun, Z. Zhang, and L. Zhu. Testing unconditional and conditional independence via mutual information, Journal of Econometrics, 240(2):105335 2024.
M. Arjovsky, Bottou, I. Gulrajani. and Lopez-Paz. Invariant risk minimization, 2020.
E. Bareinboim and J. Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 13(27))7345-7355, 2016.
Billingsley. Probability and Measure. Wiley Series in Probability and Statistics. John Wiley & Sons, Hoboken, NJ, 4th edition, 2008.
S. Buchholz, G. Rajendran, E. Rosenfeld, B. Aragam, B. Schölkopf, and P. Ravikumar. Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems, 36, 2024.
R. Cadei, L. Lindorfer, S. Cremer, C. Schmid, and F. Locatello. Smoke and mirrors in causal downstream tasks. Advances in Neural Information Processing Systems, 37, 2024.
R. Cadei, I. Demirel, De Bartolomeis, L. Lindorfer, S. Cremer, C. Schmid, and F. Locatello. Causal lifting of neural representations: Zero-shot generalization for causal inferences. arXiv preprint arXiv:2502.06344. 2025.
A. C. Cameron and A. Windmeijer. An R-squared measure of goodness of fit for some common nonlinear regression models. Journal of econometrics, 77(2):329-344 1997.
G. Casella and R. Berger. Statistical inference. CRC Press, 2024.
R. Т. Chen, X. Li, R. B. Grosse, and D. K. Duvenaud. Isolating sources of disentanglement in variational autoencoders. Advances neural information processing systems, 31, 2018.
V. Chernozhukov, D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21 21(1):C1-C6) 2018.
X. Dong, B. Huang, Ng. X. Song, Y. Zheng, S. Jin, R. Legaspi, Spirtes, and K. Zhang. A versatile causal discovery framework to allow causally-related hidden variables. In The Twelfth International Conference Learning Representations, 2024.
N. R. Draper and H. Smith. Applied regression analysis, volume 326. John Wiley & Sons, 1998.
C. Eastwood and C. K. Williams. A framework for the quantitative evaluation of disentangled representations. 6th International Conference on Learning Representations, 2018.
M. Faller, L. C. Vankadara, A. A. Mastakouri, F. Locatello, and Janzing. Self-compatibility: Evaluating causal discovery without ground truth. In International Conference Artificial Intelligence and Statistics, pages 41 1132-4140. PMLR, 2024.
T. Fernández and N. Rivera. A general framework for the analysis of kernel-based tests. Journal of Machine Learning Research, 25(95):1 2024.
M. Fumero, F. Wenzel, L. Zancato, A. Achille, E. Rodolà, S. Soatto, B. Schölkopf, and F. Locatello. Leveraging sparse and shared feature activations for disentangled representation learning. Advances in Neural Information Processing Systems, 36, 2024.

---

J. L. Gamella, S. Bing, and J. Runge. Sanity checking causal representation learning on a simple real-world system. arXiv preprint 2rXiv:2502.20099 2025.
I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerch- ner. beta-VAE: Learning basic visual concepts with constrained variational framework. In International conference on learning representations, 2017.
S. Holm. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pages 65-70, 1979.
Huang and L.. Kook, pycomets. http://ggthhbaooommiieeeeeggngne 2025. Accessed: 2025-05-15.
A. Hyvärinen and P. Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429-433) 1999.
A. Hyvarinen, H. Sasaki, and R. Turner. Nonlinear ICA using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859-868. PMLR, 2019.
M. Jain, A. Denton, S. Whitfield, A. Didolkar, B. Earnshaw, J. Hartford, et al. Automated discovery of pairwise interactions from unstructured data. arXiv preprint vrXiv:2409005544 2024.
D. Janzing, D. Balduzzi, M. Croose-Wentrrup and B. Schölkopf. Quantifying causal influences. The Annals of Statistics, 415522334-2 2013.
H. Kim and Mnih. Disentangling by factorising. In International conference on machine learning, pages 2649-2658. PMLR, 2018.
L. Kong, S. Xie, W. Yao, Y. Zheng, G. Chen, P. Stojanov, V. Akinwande, and K. Zhang. Partial disentanglement for domain adaptation. In International Conference on Machine Learning, pages 11455- 11422. PMLR, 2022.
L. Kong, B. Huang, F. Xie, E. Xing, Y. Chi, and K. Zhang. Identification of nonlinear latent hierarchical models. arXiv preprint arXiv:2306.07916 2023.
L. Kook. Falsifying causal models via nonparametric conditional independence testing. In Proceed- ings of the 39th International Workshop on Statistical Modelling (IWSM), 2025. (to appear).
L. Kook and A. R. Lundborg. Algorithm-agnostic significance testing in supervised learning with multimodal data. Briefings in Bioinformatics, 25(6)bbbec475, 2024.
D. Krueger, F. Caballero, J.-H. Jacobsen, A. Zhang, Binas, Zhang, R. Le Priol, and A. Courville. Out-of-distributien generalization via risk extrapolation (rex). In International conference on machine learning, pages 5815-5826. PMLR, 2021.
A. Kumar, P. Sattigeri, and A. Balakrishnan. Variational inference of disentangled latent concepts from unlabeled observations. arXiv preprint /1t1://711000443 2017.
S. Lachapelle, R. Pau, Sharma, Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA, In First Conference on Causal Learning and Reasoning, 2022.
Everett, R. Le Priol, A. Lacoste, and Lacoste-Julien.
S. Lachapelle, T. Deleu, D. Mahajan, I. Mitliagkas, Y. Bengio, S. Lacoste-Julien. and Q. Bertrand. Synergies between disentanglement and sparsity: Generalization and identifiability in multi-task learning. International Conference on Machine Learning, pages 18171-18206. PMLR, 2023.
P. Lippe, S. Magliacane, S. Löwe, Y. M. Asano, T. Cohen, and E. Gavves. Causal representation learning for instantancous and temporal effects in interactive systems. In The Eleventh International Conference on Learning Representations, 2022a.
P. Lippe, S. Magliacane, S. Löwe, Y. M. Asano, T. Cohen, and S. Gavves. Citris: Causal identifiability from temporal intervened sequences. International Conference on Machine Learning, pages 13557-13603, PMLR, 2022b.
12

---

F. Locatello, S. Bauer, M. Lucic, G. Raetsch, S. Gelly, B. Schölkopf, and O. Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In International Conference on Machine Learning, pages 41 14244... PMLR, 2019.
F. Locatello, S. Bauer, M. Lucic, G. Rätsch, S. Gelly, B. Schölkopf, and O. Bachem. A sober look at the unsupervised learning of disentangled representations and their evaluation. Journal of Machine Learning Research, 21 (209)): 2020.
A. R. Lundborg, I. Kim, R. D. Shah, and R. J. Samworth. The projected covariance measure for assumption- variable significance testing. The Annals of Statistics, 52(6):2851-2877) 2024.
H. B. Mann and D. R. Whitney. On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statisties, pages 50-60, 1947.
M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.00 019,9, 2023.
J. Pearl and D. Mackenzie. The book of why: the new science of cause and effect. Basic books, 2018.
J. Peters, J. M. Mooij, D. Janzing, and B. Schölkopf. Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 2014.
J. Peters, D. Janzing, and B. Schlkopf. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press, 2017. ISBN 0262037319.
A. Ravuri, K. Ulicna, J. Osea, K. Donhauser, and J. Hartford. Weakly supervised latent variable inference of proximity bias in crispr gene knockouts from single-cell images. In Learning Meaningful Representations of Life (LMRL) Workshop at ICLR 2025. 2025.
K. Ridgeway and M. C. Mozer. Learning deep disentangled embeddings with the F-staistiic loss. Advances in neural information processing systems, 31, 2018
J. M. Robins, A. Rotnitzky, and L. P. Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89(427):8468666, 1994.
M. Rojas-Carulla, B. Schölkopf, R. Turner, and Peters. Invariant models for causal transfer learning. Journal of Machine Learning Research, 19(36): 2018.
J. Runge. Conditional independence testing based on nearest-neighhor estimator of conditional mutual information. In A. Storkey and F. derez-Cruz, editors, Proceedings of the Twenty-Finn First International Conference Artificial Intelligence and Statistics. volume 84 of Proceedings of Machine Learning Research, pages 938-947, PMLR, 09-11 Apr 2018
B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612-634, 2021.
R. D. Shah and J. Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics, 4833)1511-15388, 2020.
S. Shimizu, P. O. Hoyer, A. Hyvärinen, A. Kerminen, and M. Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
R. Silva, Scheines, C. Glymour. Spirtes, and D. M. Chickering, Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7(2), 2006.
C. Squires, A. Seigal, S. S. Bhate, and C. Uhler. Linear causal disentanglement via interventions. In International Conference on Machine Learning, volume 202, pages 22560-32560. PMLR, 2023.
E. V. Strobl, K. Zhang, and S. Visweswaran. Approximate kernel-based conditional independence tests for fast non -on-aarmettiic causal discovery. Journal of Causal Inference, 7(1):0088017, 2019,
Y. Sun, L.. Kong, G. Chen, L. Li, G. Luo, Z. Li, Y. Zhang, Y. Zheng, M. Yang, P. Stojanov, et al. Causal representation learning from multimodal biomedical observations. In The Thirteenth International Conference on Learning Representations, 2025.
13

---

B. Varici, E. Acartürk, K. Shanmugam, and A. Tajer. General identifiability and achievability for causal representation learning. International Conference on Artificial Intelligence and Statistics, pages 2314-2322. PMLR, 2024.
J. von Ktigelgen, Y. Sharma, L. Gresele, W. Brendel, B. Schölkopf, M. Besserve, and F. Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in Neural Information Processing Systems, 34:16451-16467, 2021.
J. von Kügelgen, M. Besserve, L. Wendong, L. Gresele, A. Kekić, E. Bareinboim, D. Blei, and B. Schölkopf. Nomparametric identifiability of causal representations from unknown interventions. Advances in Neural Information Processing Systems, 36, 2024.
L. Wendong, A. Kekić, von Kügelgen, S. Buchholz, M. Besserve, Gresele, and B. Schölkopf. Causal component analysis. Advances in Neural Information Processing Systems, 36, 2024.
F. Xie, B. Huang, 7. Chen, R. Cai, C. Glymour, Z. Geng, and K. Zhang. Generalized independent noise condition for estimating causal structure with latent variables. Journal of Machine Learning Research, 25(191): 1-,,,, 2024.
D. Yao, C. Muller, and F. Locatello. Marrying causal representation learning with dynamical systems for science. Advances in Neural Information Processing Systems, 37, 2024a.
D. Yao, D. Xu, S. Lachapelle, S. Magliacane, P. Taslakian, G. Martius, J. von Kügelgen, and F. Locatello. Multi-view causal representation learning with partial observability. In The Twelfth International Conference on Learning Representations, 2024b.
D. Yao, D. Rancati, R. Cadei, M. Fumero, and F. Locatello. Unifying causal representation learning with the invariance principle. The Thirteenth International Conference on Learning Representa- lions, 2025.
W. Yao, G. Chen, and K. Zhang. Temporally disentangled representation learning. Advances in Neural Information Processing Systems, 35:26492-26505. 2022.
J. Zhang, K. Greenewald, C. Squires, A. Srivastava, K. Shanmugam, and C. Uhler. Identifiability guarantees for causal disentanglement from soft interventions. Advances in Neural Information Processing Systems, 36, 2024a.
K. Zhang, J. Peters, D. Janzing, and B. Schölkopf. Kernel-based conditional independence test and application in causal discovery. arXiv preprint arXiv: 1202.375, 2012.
K. Zhang, M. Gong. P. Stojanov, Huang, Q. Liu, and C. Glymour. Domain adaptation problem of inference graphical models. Advances in Neural Information Processing Systems, 33, 2020.
K. Zhang, S. Xie, I. Ng, and Y. Zheng, Causal representation learning from multiple distributions: A general setting. Internatinal Conference on Machine Learning. 2024b.
Y. Zheng, I. Ng, and K. Zhang. On the identifiability of nonlinear ICA: Sparsity and beyond. Advances in neural information processing systems, 35:16411-16422 2022.
Y. Zheng. Huang, W. Chen, Ramsey, M. Gong, Cai, S. Shimizu, P. Spirtes, and K. Zhang. Causal-learn: Causal discovery in python. Journal of Machine Learning Research, 25(60):1-8, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We formalize the measurement framework in $ 2, define the evaluation metric T-MEX in $ 3 and show its effectiveness in $ 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations the work is discussed in $
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

We summarize the property of the proposed T-MEX score in Prop. 3.1 and provide the proof in App. C.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Details about data generating pipeline, model architecture and training setup are provided in App. D for all experiments. Experimental results can be reproduced following the Readme md file provided in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
16

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We include the experimental and implementation details for both ex- periments ($ 5) in App. D. The dataset we used in $ 5.2 is publicly available Code for generating the numerical datasets ($ 5.1), training, and evaluation for both experiments all included in the supple- mentary material, following NeurlPS code and data submission guidelines. Curated code will be published upon acceptance.
https://doi.org/ hagg/1..008//9.ffiggio 2egshare.26484344.22
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please
the NeurIPS code and data submission guidelines for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines Sotpubliegguiiee/oooeuuuubiueeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental details (including experiment setup, training, and testing details) for both experiments are specified in App.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We repeat our numerical simulations across multiple datasets to showcase the statistical validity of our proposed T-MEX score (see $ 5.1.. In real-world scenarios where additional simulated datasets are unavailable, we apply additional statistical tests to demonstrate that experimental findings are statistically significant, see $ 5.2.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative етror rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute-related information is provided in App. D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The authors carefully reviewed the NeurIPS Code of Ethics and believe none of the concerns from NeurIPS Code Ethics applies this work.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
18

---

This paper proposes a new theoretical perspective on causal representation learning; thus, there no immediate positive or negative societal impact.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not release a new dataset propose models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We carefully cited all datasets and works that were used in this paper. Also, the causal representation learning algorithm we used in App. D.1 and the statistical testing software (pycomets) are open-sourced under an MIT license.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
19

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that souree should be provided.
assett are released, the license, copyright information, and terms use the package should be provided. For popular datasets, paperswithcode. wtw.siithooocccccdatteeeeee has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: This work focuses on new formalism and evaluation scheme of causal representation learning. Thus, we do not release new assets other than the source code reproduce the experimental results. This code attached to the submission, and all corresponding training details are documented in App. D for reproducing.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
20

---

Justification:
This work does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy httt:://obuiis..ccoootcrrre/ssss for what should or should not be deseribed.
21

---

Appendix
Table of Contents
A Notation and Terminology
22
B Preliminaries
22
Proofs and Algorithms
22
D Experiment Details and Additional Results
24
D.1 Numerical Simulation
24
D.2 World Ecological Experiment: ISTAnt
25
D.3 Caveats of Using SHD to Evaluate Causal Representations
26
D.4 Additional Numerical Experiments Showcasing the Properties of T-MEX.
27
E Background on Conditional Independence Testing
29
F Extended Discussion
29
E.I Representations of Treatment and Outcome
30
F2 Representations Confounders or Instruments
30
A
Notation and Terminology
This section summarizes the symbols used throughout the paper.
Z
Causal variables
X
Observables
D,
Dimension of the representation Za,
N
Dimension of the causal variables Z
Number of samples for the statistical tests for T-MEX
P(O)
Probability operator
B(-
Expectation operator
1(+)
Indicator function
B
Preliminaries
Definition B.1 Block-identifiabllit (von Kügelgen et al., 2021)). A set of latent variables Z E Rd block-identified by a representation 2 e dRds if there exists a bijection h IRd= - Rdd such that = (Z).
C Proofs and Algorithms
This section includes the proof for Prop. 3.1 and the algorithm compute the T-MEX score. Proposition 3.1. Let (PI)) be family of tests for eq. (3.1) where for all i [N] and j E [M], Pij is valid with level (0,1) and has power at least 3 E (0, 1). Given an adjacency ma- trix V E XMX based on measurement model, if the joint distribution P 2,2 of the causal and mea- surement variables does align with the measurement model, and each entry in W is computed based
22

---

Algorithm 1: Compute T-MEX score from one set of samples
Input: Paired samples of causal variables and measurement variables {z, RA1..... ZAM } where 2 E TR and A, E DR for j [M], adjacency matrix of the measurement model of statistical tests for (fr)) ( for (3.1), where for all i [ E [M]. Pij TRxx x RR R=x(N-1) {0, 1}
V {0. 1) N M
Output: T-MEX score of the given sample
W
NNas
for E N] do
for j [M] do
Wi - (Zi A, (NT)))))
end
end
return
M De D
1 (V: + Wi))
on an independent set of samples {(z", 2²t) (eEm maj N4, then the expected T-MEX satisfies
E[T-MEX(V, W)) Sa (MN ([V]b) + 31 (1-3))|IVVI)
where || = E EE Vig is the In norm of V.
Proof. Suppose the joint distribution of (Z, Z) aligns with the conditional independencies indi- cated by the adjacency matrix V, that is, for all i E [N] and E [M]. if VúJ = 0, it holds that
ZA, 1L Z, (M) if Viz 1, it holds that Za, P. Z, Z [M
Fix significance level (0, 1). Suppose for all i [N] and all [M]. the statistical test {0, 1) is valid at level a and has power at least E [0, 1] against the alternative distribution where Zn, 1 Z ZNMM
Pi _ D x [=*
Then given independent sets of samples {z 2t ]ee[mu) for i E [M] and j E [M], and Wyy = PPy (z, 2A, (I)) ). it holds that
if Vij then P(We = 1) S a;
if Vúj = then P(W.; = S 3.
Therefore, the expected value of T-MEX score is given by
E[T-MEX(V,
Σ 106, 0) + (1-11 Σ 1V1j =
< aa (MN |1 (1 - 3) 1|V1|
where ||V| is the 1-norm of V. The second inequality is implied by the that each test Piy is valid with level a and has power 2 8.
Remark C.I. Proposition 3.1 tells us that the measurement model does hold for the joint distribution of the causal variables and the output representations from a trained CRL model, we would expect to see a "low" T-MEX score given that we employ valid statistical tests that are also powerful enough to reject the null under alternatives. A "low" T-MEX score does not in general refer to a 0 score, as it depends on V. the chosen significance level o, and the power the test B. For example, let 0.05, we consider valid statistical test that has the highest power, i.e., f = 1, additionally, assume the number of Os V is then the expected value of the T-MEX score is no larger than 0.05 x 2 0.1.
23

---

D Experiment Details and Additional Results
This section elaborates on the experiment settings of $ 5. We include further information regarding the data-generalie generating process for the simulated experiment ($ 5.1) and the ISTAnt dataset (Cadei et al., 2024) used in the ecological case study ($ 5.2), as well as additional experimental results.
D.1 Numerical Simulation
Experiment setting. We consider five causal variables (Zi.... Zs) generated based on linear structural causal model (Peters et al., 2017)
= BZ + +
0
0
0
1
0
0
1
where Z (Z,, Z2, Z3, Z4. Zo), takes values in R No(0,, I). and B
0
1
0
0
0
0
0
which induces the partial DAG depicted in Fig. 2. Two of the causal variables (Z4 and Z5) are
observed (i.e., directly measured as Defn. 2.1), and the other three (Z,, Z2, and Zg) are latent and we observe only a bijective mixing X of them.
For the purpose of latent variable identification, we consider the multiview scenario in (Yao et al., 2024b) where two views X1, X2 are generated from different subsets of latent variables. Formally, we have
X; = f1 (Z, Zz)) X, = Zz)
(D.I)
where f1, f2 IR22 R22 are diffeomorphisms, implemented using invertible MLPs as suggested by Yao et al. (2024b).
Implementation details. We employ the latent variable identification algorithm proposed by Yao et al. (2024b), which guarantees that the shared latent variables among different views can be identified up diffeomorphism in the sense of Defn. B.I. Thus, by utilizing X X2, bijective transformation of their shared latent variable Z, This allows us to construct a measurement model M = (Z. A {hr }} (see Fig. 2), where Z = (Z1, Zs} and ZA, hhZz) for some (unknown) smooth invertible map + R.
obtaín nonlinear
We train three CRL models following the implementation settings in (Yao et al., 2024b, Tab. 4),
Model A: sufficiently trained model (trained for 50001 steps) from which we expect the learned representation ZA, (where by a slight abuse of notation, the superscript represents the model indicator) to exclusively measure Zı;
Model B: insufficiently trained model (trained for 51 steps) with unclear latent- measurement correspondence;
Model C: corrupted version of Model A where the representation 2 21, 0.22% 0.1Z3. i.e., a linear mixing of the representation ZAA, from Model A, and Zz, Zz.
For each of the three trained models, we generate 50 independent datasets, each containing 4096 paired samples of Z, ZA,- We compute the respective T-MEX scores based on these generated datasets for all three models, using the the projected covariance measure (PCM, Lundborg al., 2024) implemented in py cometss (Huang and Kook, 2025) using linear regression models to estimate the conditional means (see App. E).
Additional results. Since T-MEX relies on statistical testing, we further assess its statistical validity by examining the underlying p-values that lead to the test results and the T-MEX score. Fig. 6 shows the p-values resulted from testing each of the three null hypotheses:
(1) ŻA, 1L Z, Day for i [3].
We omit Z₁ and Z3 since they are not involved in generating the two views X, and X2.

---

0.8
Model
0.6
B
p-value
0.4
0.2
a
0.0
Z
Zz
Z3
Figure 6: Violin plots of p-values from testing the conditional independencies ZA, -L Z, Z ; for i E [3] based the PCM tests (Lundborg et al., 2024). The black dashed line is at the significance level 0.05. A p-value < for Z; means there is edge from Ze the measurement Z
Fig. shows that Model A aligns with the measurement model in Fig. 2, evidenced by (i) small p-values for (1) and (ii) approximately uniformly distributed p-values for both Ho (2) and Ho (3). given a valid test (see App. for further explanations). p-values smaller than a, leading to rejections of the null hypotheses, which indicates that the learned representation Z Ar is mixture of all three causal variables Z Z2. Z3, and thus fails to exclusively measure Z,.
contrast, Models B and C, nearly all
Computational resources. We train the CRL models (model A, B, C) using single node GPU (NVIDIA GeForce RTX1080Ti) with 10GB RAM, 4 CPU cores for less than one GPU hour. ATE estimation and T-MEX computation take less than minute standard CPU.
D.2 Real-World Ecological Experiment: ISTAnt
Experiment Setting, ISTAnt is real-world ecological benchmark designed to evaluate learned representations on downstream causal inference tasks from high-dimensional observational data. It comprises 44 ant-triplet video recordings collected through a randomized controlled trial. This benchmark adopts the problem formulation introduced by Cadei causal effect of specific treatments (e.g., chemical exposure) on ants' social behavior, particularly grooming events. The experimental design and recording setup are shown in Fig. 7; for further details, refer to (Cadei et al., 2024, App. C).
(2024), aiming to estimate the
In ISTAnt, each observation (video recording) is associated with treatment assignment T, and set of experimental covariates W, (including experiment day, time of the day, batch, position within the batch, and annotator). However, only a subset of videos is annotated with the outcome of interest Y, (i.e., grooming events), which hinders reliable causal inference at population level, such as treatment effect estimation. To address this challenge, Cadei al. (2024) proposes to train classifier on top of pre-trained feature extractor (e.g., DINOv2, Oquab et al., 2023) using this limited set of annotated samples, impute missing labels while still enabling valid causal inference at the population level; specifically, for estimating the Average Treatment Effect (ATE).
Implementation details. Following (Cadei al., 2024), we train 2,400 classification heads top of DINOv2 (Oquab et al., 2023), varying the architecture and training settings, and estimate the causal effect using all video samples together with the predicted labels Ys by AIPW estimator (Robins et al., 1994). The hyperparameter configurations are summarized Tab. 2, with all other implementation details following (Cadei et al., 2024, App. C).
By contrasting with the measurement model depicted in Fig. 4, we compute the T-MEX scores for all 2,400 models. Since we focus on models with more than 80% prediction accuracy ($ 5.2), the null hypothesis Y J.Y Y T is rejected in all cases, consistently indicating Y Y. Thus, only focus
25

---

Table Hyperparameters for the weal-world ecological experiment ($ 5.2 and App. D.2), giving rise to 2,400 model configurations in total. All other settings follow (Cadei et al., 2024, App. C).
Hyperparameter
Value(s)
Input Preprocessing
YES NO 1,
Number of Hidden Layers
Batch Size
64, 128, 256
Adam: learning rate
5e-2, 1e-2, 5e-3, 1e-3, 5e-4
Empirical Risk, Invariant Risk (Arjovsky et al., 2020), vREx (Krueger et al., 2021), Deconfounded Risk (Cadei al., 2025) 0.1,.., 9
Training objective
# Seeds
on the following null hypothesis:
Ho Y TT L
where Y denotes the predicted label and Y the ground truth one. misalignment with the mea- surement model in Fig. 4 leads to rejecting Ho. resulting T-MEX=1, whereas causally valid representation Y that exclusively measures Y gives rise to T-MEX-0. We summarize all results in Fig. and provide extended discussions in $ 5.2.
Computational resources. We run all the analyses in $ 5.2 using 48GB of RAM, 20 CPU cores, and a single node GPU (NVIDIA GeForce RTX2080Ti) for 24 GPU hours. Data preprocessing and feature extraction using DINOv2 account for the majority of the computational time, whereas classifier training, AIPW estimation, and the T-MEX test contribute negligibly by comparison.
(a) Filming box
(b) Batch example
Figure 7: Visualization of ISTAnt recording set-up (Cadei al., 2024).
D.3 Caveats of Using SHD to Evaluate Causal Representations
Experiment Setting. This experiment explores the poten- tial pitfalls when directly using SHD to evaluate causal rep- resentations without properly evaluating the element-wise latent variable identification. Specifically, we consider set of causal variables generated through the following structural equations:
Z₂
Za
Z
Z
Z
Z = E1
(D.2) Figure 8: Example measurement model, where ZA, block-identifies Z, Zz. Z and 2a, identifies Zy. Zy respectively.
Z 012
Z 32
Z3 = 013
Z + 0.2
Ze + B3 -E,
26

---

80
Count
40
20
SHD Value
Figure 9: Structural Hamming Distance Values (SHD) of 100 structure and measurement models following eqs. (D.2) and (D.3), where the measurement Za, is a mixing of the ground truth latent Zi, Zz. SHDs computed between the discovered graph Z and the ground truth one.
Assume the learned representation corresponds to the ground truth causal variable as follows:
2 71 Zz 7/1 Z2
ZA 7/2 Z2
(D.3)
4 7/3 Zi
where ΣA, remains mixing of Z₁ and Zz. The corresponding measurement model is shown in Fig. 8.
Implementation details. We generate 100 different structure and measurement models follow- ing eqs. (D.2) and (D.3), with all coefficients as and ys sampled from Unif[1, 10) and the Is sampled from Uni5[0.005, 0.02]. We run LiNGAM (Shimizu et al., 2006) from causal-learn (Zheng al., 2024) to discover the causal relationships between the measurements ZA, Z ZAg.
Results. Fig. shows the structural hamming distance between the discovered graph on Z and the ground truth one. Despite being entangled between Zi, Zz, Z still yield the correct causal graph in most of the cases (77%), as shown by the first bar in the plot. Hence, causal relations between the measurement variables should always evaluated in conjunction with the variable identification. Otherwise, it can lead misinterpretations as showcased by Fig. 9.
Computational resources. Data generating and causal discovery for App. D.3 in total takes less than 10 minutes on a standard CPU.
D.4 Additional Numerical Experiments Showcasing the Properties of T-MEX.
In the following, T-MEX Oracle is computed based on an oracle CI test with zero type I error and power 1. We We compute the oracle T-MEX score demonstrate desirable properties of the metric- such as its consistency and robustness across different conditional independence tests. This is feasible the simulated setting, where the measurement functions are known by design. In contrast, for empirical representations learned by CRL. model, the true measurement functions are unknown, making oracle computation infeasible in practice.
Scalability given higher-dimensiona. latent variables. We simulate the causal variables based on nonlinear SCM a location-scale SCM implemented by Wendong al. (2024). Then, the measurement variables are simulated as direct copy of each of the corresponding causal variables. For different numbers of latents (n-latent), we report the T-MEX score based on the Generalized Covariance Measure (GCM) test with linear regression (see App. E for more details about GCM), along with their standard error based on random causal DAGs for the latent causal variables, each with 30 repetitions and 1000 observations. In Tab. 3, we see that T-MEX remains closely aligned with the T-MEX oracle in all cases, and can be efficiently computed up to latents within a reasonable time, validating its applicability in moderate high dimensions.
27

---

Table T-MEX for higher- digees-aimeniional latents generated from nonlinear SCM
n-latent
T-MEX
T-MEX Oracle
time (sec)
5
0.0483 + 0.2223
0.0359 + 0.0011
10
0,0033 + 0.0577
0.1559 + 0.0017
20
0.0550 0.5187
0.8080 + 0.0094
50
0.0917 + 0.9582
10.8450 + 0.1004
Consistent results when using different CI tests. Following the same settings in $ 5.1, consider PCM test (see App. E) with random forest (RF) and Kernel Conditional Independence (KCI) test (Zhang al., 2012). We see that T-MEX ranks the models consistently with the results in $ 5.1.
Table 4: T-MEX under different CI Tests
CI test
Model
T-MEX
PCM (RF)
A
0.0000 0.0000
PCM (RF)
B
0.8000 = 0.7559
PCM (RF)
C
2.0000 0.0000
KCI
A
0.0400 0.1979
KCI
B
0.1600 0.4218
KCI
2.0000 0.0000
T-MEX under weak causal relations. Building on the experiment in $ 5.1,, we next examine how T-MEX behaves under weak causal relations, and compare it with the previous CRL identifiability metrics such as R² and MCC. The data is generated from a linear SCM with three latent causal variables following the measurement model described in Fig. 9. The linear coefficients between the causal variables are sampled uniformly between 0.01 and 0.1.
Table 5: T-MEX under weak causal relations
T-MEX Oracle
T-MEX
MCC
1.0100 + 0.0995 1.0000 + 0.0000 1.0000 + 0.0000
Under identifiability assumptions that guarantee element-wise correspondence, T-MEX correctly detects the mixing effect and gives a score near one (note that T-MEX = 0 indicates perfect aligmment). In contrast, both Fta and MCC fail to reflect this violation. Despite the entanglement in Z,, they still assign the maximum score of misleadingly suggesting perfect element-wise identification (see Tab. 5).
Alignment between T-MEX and ATE under nonlinear SCM. Extending $ 5.1, we further examine the correspondence between T-MEX and ATE under the more general nonlinear setting. We consider the same causal graph as given in $ 5.1, where Z₁ confounds Ze and Zg. and needs be adjusted for a valid treatment effect estimation. "perfect" model means Z, exclusively measures Z,, whereas mixed model indicates an entangled representation, i.e., 2, mixes Z₁ and Zz.
Table 6: T-MEX v.s. ATE bias under nonlinear SCM
Model
T-MEX
abs(ATE bias)
Perfect 0.0100 + 0,1000
0.1532 + 0.0283
Mixed 1.0000 + 0.0000
0.5357 + 0.0540
28

---

As shown in Tab. 6, the results are highly similar to the linear case $ 5.1, T-MEX closely aligns with the absolute values of the ATE bias, effectively evaluating causal representations for downstream inference tasks with nonlinear causal relations.
Consistency of T-MEX under noisy measurements. We compare the empirical T-MEX value with/without noise. The noisy measurements are generated by adding Gaussian noise to the original latents. Le., the measurement function writes
h(z)) - +
with independent Gaussian noise. Results are evaluated over 100 datasets with 1000 samples each. Tab. 7 shows that the empirical T-MEX score largely remains consistent under noisy measurements, validating its practical usability.
Table 7: T-MEX w/wo noise
T-MEX Oracle
without noise
with noise
0.0100 0.0100
0.3900 0.6497
E
Background on Conditional Independence Testing
Testing conditional independence of random variables X and Y given third random variable Z known to be a difficult problem a continuous variable (Shah and Peters, 2020). The goal of conditional independence test is to test the null hypothesis
Ho
X
11 Y Z.
Shah and Peters (2020) have shown that there is valid test (i.e., test that guarantees a Type I error rate to be no larger than the given significance level a)) that has power against all alternatives.
Consider univariate variables X, Y, Z, the generalized covariance measure (GCM) test proposed in Shah and Peters (2020) aims to test an implication conditional independence which can be written as the following null hypothesis:
HOCM E((YY Y (
E X
Z
0.
The validity of the GCM test thus relies that the conditional means E| Z] and E|X Z] can be learned at sufficiently fast rates. It turns out that GCM does not have power against any alternative for which E[Cov(X, Y Z)] 0 but X Y Z (Lundborg al., 2024).
The projected covariance measure (PCM) proposed by Lundborg et al. (2024) improves the power issue of GCM by testing different implication of conditional independence:
HPCM E X, Z = E Z].
Similar to GCM, to ensure its validity, PCM also requires that the conditional means can learned sufficiently fast, which is satisfied in our experiments ($ 5).
There are other conditional independence tests such as mutual information based methods (Ai et al., 2024; Runge, 2018) and kernel-based methods (Fernández and Rivera, 2024; Strobl et al., 2019; Zhang et al., 2012). We opted for PCM in our experiments for its computational advantage and theoretical guarantees on its validity under flexible, model-agnostic framework. More discussions on the usage of PCM and GCM can be found Kook and Lundborg (2024). Notably, T-MEX is general evaluation metric for causal representations that does not specify any particular type of tests, allowing practitioners to choose other testing methods that are more suitable for their problem settings.
F Extended Discussion
This section elaborates on the implications of learned representations for downstream causal tasks. As briefly discussed the main paper (following Defn. 2.2), representation causally valid (Defn. 2.2) with respect to statistical estimand if and only if the statistical estimand remains unchanged when
29

---

plugging in the measurement variables correspond to the causal variables. More concretely, we illustrate the implications of nonlinear invertible reparameterizations of causal variables in two commonly encountered scenarios: when representations serve as proxies of (i) the treatment or outcome variables, and (ii) the confounders instrumental variables.
F.1 Representations of Treatment and Outcome
Assume in Fig. 10 that A AA are element-wisg wise non- linear invertible reparametrization of Z Ze respectively; ie., Vi E [1, Za. h; (Z.) for some diffeomorphism h4 R + R. We aim to estimate the treatment effect of Z + Z2 using the learned representations ZA, and A
Zi
Z,
Za
Assume the Z2 is generated following eq. (4. i.c.,
A
ZA
2 = a Z +
with Per Ele] and Il. Z, Given there is observed confounding, the ground truth average treatment effect written
Figure 10: AA, measures Z: through
nonlinear bijection for both i 1,2.
JB Z2
dZ z1)l
Z₁ 21
aE[ + c
AZ Z2)
(F.1)
θz
θz1
Ozi
We assume measurement function h2 for all i {1,2) to be linear, i.c.,
A = Z1,
2 02 Zz. and 01, + 0.
(F2)
The ATE estimand from the learned representations yields:
ZA. + -A,)= OE Z
ZA, 2Ar
O2
DE L2
aq a a121
(F3)
0a,i2
Zi = 21 02 0.
011 01021
01
As shown by eq. (F.3), the ATE estimand using the learned representation ZA, and ZA, can be arbi- trarily scaled by the factor of az/a:. Thus, measurements that bijectively transform the causal latent variables cannot naively support estimating the treatment effect, violating causal validity (Defn. 2.2); requires direct supervision or observation on both treatment and outcome variables, as also pointed out by (von Kügelgen et al., 2024, Sec. 4).
On the other hand, information-theoreiic measures for quantifying causal influence remain invariant under bijective transformation, such the mutual information I (Z,; Z2) Iim AZ AAz), as shown by Janzing et al. (2013).
F.2 Representations of Confounders or Instruments
Measuring confounding. We first show an example where an observed treatment T and an observed
outcome Y is confounded by third variable W which is measured by W = h(W) through deterministic invertible function h.
Formally, the measurement model is defined as Mumi (Z, Z, (b}) with Z (T, Y, W and
where T, Y are directly measured (Defn. 2.1). The corresponding DAG is given in Fig. 11. We show the following that this measurement model is indeed causally valid (Defn. 2.2) with respect to the statistical estimand for the Average Treatment Effect (ATE) of T on Y.
30

---

W
T
Y
W
Figure 11: ATE remains invariant under bijective transformation of confounders. The treatment T and outcome Y are directly measured (i.e., observed) whereas confounder W is measured by W through a nonlinear bijection.
Figure 12: ATE remains invariant under bijective transformation of instruments. I measures the in- strument variable through a nonlinear bijection. The treatment T and outcome Y directly measured (i.e., observed), and U denotes unob- served confounding.
Under the standard assumptions for backdoor adjustment, follows that E dT T t)) Ew [Y W. T t)]
B(Y W, T = PdWWw
/ B(Y h (W): -1)(())p-((>))
dh (ww) dw
dw
(F.4)
EEY W. T t)P
= Ew E(Y W T = b)
where we used the change of variable formula and the fact that E(Y E(Y (W), =).. This is because h-1 hw) is sufficient statistic for W (Casella and Berger, 2024, Ch. 6.2) following h is invertible.
=
Under the same assumptions, the ATE for binary treatment can then identified by the following statistical estimand
ATE(T + Y) = E|y do = 1)] - E do T = 0)]
(F.5)
Ew [E Y
W, T 1) Y W, T 0)]
Following eq. (F.4), we have
ATE(T Y)= Ew
W
E T 0)
indicating that the identified statistical estimand ATE(T Y) remains invariant for the measurement W. Similarly, ATE also remains invariant when the treatment is continuous:
ATE(T - Y) OE Y do = t) DE Y W. T =
EY W.T
T
dl
dl
dl
(F.6)
where the last equality holds because of eq. (F.4). Therefore, we have shown that invertible reparame- terizations of the confounders can be drop-in replacement of the true confounding variables in the statistical estimand for ATE, for both discrete and continuous treatments, and thus this measurement model Moonf is indeed causally valid for ATE.
Measuring instrumental variables. We now give second example of ATE estimation under
instrumental variable setup. We assume that the instrument is measured by Î h(I) through bijective transformation h. We show that under certain assumptions, the statistical estimand does not
change when using I as drop-in replacement of the true instrument I. We focus on the case where the
31

---

instrument I, the treatment T, and the response Y are all univariate continuous variables; further dis- cussion on multivariate and discrete valued variables is beyond the scope of this paper. Formally, the measurement model is defined as M = (z, Z, {b}) with causal variables Z = {I, T, Y and mea- surement variables Ż {I). The treatment T and outcome Y are directly measured (Defn. 2.1) and confounded by unknown hidden confounders U. Fig. 12 shows the DAG of this measurement model.
We show in the following that the instrument remains a valid instrumental variable under a bijective transformation, i.e., the measurement variable I = (I) also satisfies the standard IV assumptions, which are listed as follows:
Relevancy: I IATU
Unconfoundedness: Ill.U U
Exclusion restriction criteria: I Y U
Following standard probability theory (see e.g.. Billingsley, 2008), if h is bijective function, all three conditions still hold when replacing I by hI)). This means that if the ATE identified by statistical estimand when using I an instrument, it is also identified when using T as an instrument. In other words, the measurement model MM is causally valid with respect an identified statistical estimand because can serve as a drop-in replacement for (Defn. 2.2).
As specific example, consider the case where the causal mechanism of Y is partially linear (a commonly studied setup in the eemi-parametric inference literature, see e.g., Chernozhukov al. (2018)), i.e., Y = T33 + g(U,,) e), for some measurable function where E E[g(U,, c)] 0 and where E ~ Ps an independent noise variable, the ATE
Y do =t] 3 + g(U,e))
ATE(T Y)
ôt
can be identified by the statistical estimand
Cov(Y,I)
ATE(T + Y)
(F.7)
Cov(T,
We show in the following that the statistical estimand ATE(T - Y) in eq. (E7) remains invariant when using I as drop-in replacement for I. Plugging in I in the numerator
Cov(Y, 1) YI
E Y Y
TÉ ETTEE) 3:Coo(TTT)) I),
have Cov(Y, 3 T)
Cov( Y
Therefore, have shown another example where the
CTv
measurement can serve a drop-in replacement for the latent instrumental variable for downstream causal inference tasks.
32


Reviews:
[{"review_id": "XMWoYc0JQY", "reviewer": "Reviewer_ns4w", "summary": "The authors recast causal representation learning (CRL) as a third pillar of causal analysis by treating learned latent codes as measurements of hidden causal variables. Within this formal “measurement-model” framework they introduce T-MEX—Test-based Measurement EXclusivity—an evaluation score that counts conditional-independence violations of the exclusivity relations implied by a chosen measurement model. They prove finite-sample bounds showing T-MEX is controlled by the Type-I / Type-II error of the underlying CI tests and empirically demonstrate that the score predicts downstream ATE bias far better than correlation- or disentanglement-based metrics.", "questions": "See my comments above.", "limitations": "See my comments above.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is well-written and easy to follow.\n- The paper thoroughly explains why previous evaluation metrics and disentanglement scores can be misleading when latent variables are causally related.\n- The experimental results are interesting and solid.\n\n\n**Weaknesses**\n\n- **Strong reliance on exclusivity alone.** IIUC, the framework checks only whether any pair of learned features violates exclusivity given the others; it does not test other measurement-model axioms such as invertibility, monotonicity, or functional form. A representation that passes T-MEX could still be useless for causal inference if, for example, several causal variables collapse into a single latent factor.\n\n- **Linear-Gaussian focus** and **sample-complexity limits of CI testing.** All theoretical derivations assume additive Gaussian error terms (for analytic tractability of CI tests) and all simulations instantiate linear DAGs. Real-world problems often involve nonlinear, heteroskedastic, or discrete mechanisms; whether T-MEX still correlates with causal-effect bias under such violations is unknown. Each exclusivity claim is evaluated via a multivariate conditional-independence (CI) test whose power decays quickly with dimensionality and conditioning-set size. While Proposition 3.1 upper-bounds the expected T-MEX under the null, the paper does not translate this into concrete guidance on how many samples are needed for typical vision or language embeddings with hundreds of features, leaving practitioners without a way to judge feasibility.\n\nI am less familiar with the current progress in causal representation learning, so any clarifications on the questions above would help me to understand the background better.", "was_revised": true, "final_justification": "I have read the response. I am slightly leaning towards accepting the paper.", "rebuttal": "We thank the reviewer for their time and valuable feedback. We will address the remaining concerns as follows:\n\n&nbsp;\n\n## Additional Experimental Results\n\n**Table 7: T-MEX v.s. ATE bias under nonlinear SCM** \n\n|  Model  |        T-MEX      |  abs(ATE bias)    |  \n| -------:|------------------:|------------------:|  \n| Perfect |$0.0100 \\\\pm 0.1000$| $0.1532 \\\\pm 0.0283$|   \n|  Mixed  |$1.0000 \\\\pm 0.0000$| $0.5357 \\\\pm 0.0540$| \n\n* **Experimental Setup:** We consider the same causal graph as given in **`Section 5.1`**, where $\\\\mathbf{Z}\\_1$ confounds $\\\\mathbf{Z}\\_2$ and $\\\\mathbf{Z}\\_3$, and needed to be adjusted for a valid treatment effect estimation. A *perfect* model means $\\\\widehat{\\\\mathbf{Z}}\\_1$ exclusively measures $\\\\mathbf{Z}\\_1$, whereas a *mixed* model indicates an entangled representation, i.e.,  *$\\\\widehat{\\\\mathbf{Z}}\\_1$ mixes $\\\\mathbf{Z}\\_1$ and $\\\\mathbf{Z}\\_2$*   \n* **Results:** Similar to the linear case (**`Section 5.1`**), T-MEX closely aligns with the absolute values of the ATE bias, effectively evaluating causal representations for downstream inference tasks with nonlinear causal relations. \n\n&nbsp;\n\n## Response to Weaknesses\n\n***W1: Strong reliance on exclusivity alone.***\n\n* Thank you for this thoughtful comment\\! We fully agree that additional properties of the measurement functions $h\\_{ij}$, such as invertibility or parametric functional form, are important for further investigating the causal validity of a representation for a specific causal task.   \n* As discussed in **`Remark 3.2`, our framework intentionally operates in a non-parametric setting to allow broad applicability, agnostic to the form of** $h\\_{ij}$​. That said, the framework is easily extensible: once exclusivity is verified, one may perform additional tests for properties like linear or nonlinear invertibility (e.g., via $R^2$ or correlation metrics) on the detected latent-measurement pairs. **Since the form of $h\\_{ij}$ can vary across use cases (e.g., diffeomorphism or affine transformations), we chose not to include such tests in the general-purpose T-MEX score**.  \n* Importantly, **exclusivity serves as a necessary foundation for evaluating such higher-order properties.** Without a reliable one-to-one correspondence between latent variables and learned representations, questions of invertibility or functional form become ill-posed.  \n* However, we fully agree that for a specific causal downstream task, it is important to evaluate both exclusivity and additional functional constraints such as invertibility. This is why **we also report high $R^2$ (`Section 5.1`) and classification accuracy (`Section 5.2`) to confirm that the learned representations retain enough task-relevant information beyond structural exclusivity.** We will add a paragraph to the paper to clarify this point further.\n\n*“A representation that passes T-MEX could still be useless for causal inference if, for example, several causal variables collapse into a single latent factor.”* \n\nThere are a few subtleties about this statement in addition to the discussion on the measurement function above.\n\nThe measurement model details which causal inference questions can be answered from the measurements, and T-MEX tests alignment between the representation and the model. We interpret the reviewer's concern as that T-MEX may pass, but the resulting measurement model is still insufficient for certain causal inference estimands. In our view, this is a **modeling issue, and not an evaluation issue.** In this case, one needs to refine the measurement model (e.g., by collecting more data with specific interventions) and train a new causal representation learning method that allows identification of a more fine-grained model. \n\n***W2: Linear-Gaussian focus and sample-complexity limits of CI testing.***\n\n***T-MEX with nonlinear SCM***\n\n* Thank you very much for suggesting this new experiment. We include new experimental results with nonlinear SCM, following the causal graph structure in **`Section 5.1`** (see **`Table 7`**).   \n* As shown by the results, **the T-MEX score also closely aligns with the absolute ATE bias under non-linear SCM**, showcasing the general applicability of T-MEX under a wide range of CRL scenarios.\n\n   \n***Sample-complexity limits of CI testing***\n\n* The sample complexity of T-MEX depends on the sample complexity of the chosen CI test. For the Projected Covariance Measure (PCM) test we used in our experiments, we refer to **`Theorems 5 and 7`** in **`[Lundborg et al., 2024]`** for asymptotic analysis. The theorems show that the power of the test converges to 1 uniformly for a set of alternatives that are sufficiently separated from the null distributions. We will more explicitly acknowledge the challenges of conditional independence tests, but we remark that this is an active research field in statistics and our score is agnostic to the specific test that is used.  \n* Regarding your concern about the feasibility in vision or language settings, it is crucial to distinguish between the latent causal variables—whose identifiability is the goal of CRL—and the typically high-dimensional learned embeddings from raw data. Most real-world applications can be framed around a small number of causal variables that are descriptive for a specific downstream task. These can be predicted or extracted from high-dimesnional representations learned by foundation models (see **`Section 5.2`** for a real-world example).\n\n*A. R. Lundborg, I. Kim, R. D. Shah, and R. J. Samworth. The projected covariance measure for assumption-lean variable significance testing. The Annals of Statistics, 52(6):2851–2878, 2024.*"}, {"review_id": "LLfgsWdDqU", "reviewer": "Reviewer_k71J", "summary": "The paper treats causal-representation learning as a measurement problem: each learned code dimension should be an exclusive, CI-verifiable proxy for a single latent cause. It proposes T-MEX, a conditional-independence score that quantifies this exclusivity and proves a finite-sample link between low T-MEX and accurate causal-effect estimates. Simulations and an ecological-video benchmark show T-MEX tracks ATE bias better than common metrics like $R^2$ and MCC.", "questions": "1. Can you compare T-MEX against CI-based graph validation metrics (for example, self-compatibility (von Kügelgen ’21; Ahuja ’23) or falsification/compatibility score (Marx & Vreeken ’22)) on the same representations?\n2. Would other CI tests give consistent rankings? Can you verify with experiments?\n3. Can you analyze the trade-off between T-MEX and the size of the CI oracle? Something like a worst-case or average-case bound on the number of CI tests T-MEX performs as a function of dimensions would suffice?\n4. Can you provide a recommendation of the $\\alpha$-level and sample-size heuristics?\n5. Is it possible to scale the experiments to aournd 20 latent variables with 100 observable variables? How does the runtime evolve witht he dimensions?\n\nI would be happy to raise my socre if my questions can be addressed.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. This paper formalises CRL as a measurement model and provides sound theoretical analysis.\n2. The authors conduct wll-designed small-scale studies.\n3. T-MEX is algorithm-agnostic and needs only observational samples, potentially valuable where ground-truth ATE is unavailable. It connects identification theory with downstream estimands via the notion of causal validity of a measurement model.\n\nWeaknesses:\nMy major concern is that the main contribution of T-MEX is that it can be potentially used to evaluate CRL methods when we know the ground truth causal structure and variables. While in the current paper, there is no explicit guidance on how to conduct this. For a random CRL method, how would T-MEX automatically find the block matching? Can T-MEX surrogate be used if we do not have access to the true variables? How would the grid of the CI test affect the performance? Is T-MEX efficient for large block sizes?\nHere are some minor concerns.\n1. There are multiple typos in the current version (for example, 'diffeomporphism' in line 121).\n2. The empirical evaluations are on small datasets. The simulations have only 5 variables. \n3. T-MEX relies on a valid, powerful CI oracle which may hinder its utility in real-world applications.\n4. The evaluations do not show the confidence intervals for T-MEX.\n5. The reliacne on the test choice is not shown.\n6. The relation between T-MEX and recent works on self-compatibility/falsification can be discussed more in detail in section 4 to highlight the difference.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns during the rebuttal. Some minor points are partially addressed, which is reasonable. I believe the additional discussions and experiments will greatly improve the paper.", "rebuttal": "Thank you for your thoughtful comments and constructive feedback. We address your questions below.\n\n&nbsp;\n\n## Additional Experimental Results\n\n**T-MEX Oracle:** T-MEX score computed based on an oracle CI test with 0 type I error and power 1.\n\n**Table 1**: Due to character limit, we kindly invite you to find this table in the rebuttal for Reviewer **`yCFv`**.\n\n**Table 2: T-MEX under different CI Tests**\n\n| CI test  |  Model |       T-MEX          |  \n|:---------:|:-------:|---------------------:|  \n| PCM (RF) |    A   | $0.0000 \\\\pm 0.0000$    |  \n| PCM (RF) |    B   | $0.8000 \\\\pm 0.7559$    |  \n| PCM (RF) |    C   | $2.0000 \\\\pm 0.0000$    |  \n| KCI      |    A   | $0.0400 \\\\pm 0.1979$    |  \n| KCI      |    B   | $0.1600 \\\\pm 0.4218$    |  \n| KCI      |    C   | $2.0000 \\\\pm 0.0000$    |\n\n* **Experimental Setup:** We follow the same settings in **`Section 5.1`**. We consider PCM test (see **`Appendix E)`** with random forest (RF) and Kernel Conditional Independence (KCI) test **`[Zhang et al. 2012]`**.   \n* **Results:** T-MEX ranks the models consistently with the results in **`Section 5.1`**.\n\n*Zhang, K., Peters, J., Janzing, D., & Schölkopf, B. (2011). Kernel-based conditional independence test and application in causal discovery. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence (pp. 804-813).*\n\n**Table 3: Comparison of T-MEX and self-incompatibility (SI) score \\[Faller et al., 2024\\] using three causal discovery methods (PC, FCI, RCDLINGAM)**\n\n|T-MEX Oracle | T-MEX \t| SI score (PC) \t| SI score (FCI) \t| SI score (RCDLINGAM) \t|  \n|:----------------:|-------:|:-------------:|:--------------:|:--------------------:| \n|         1             | $1.1000 \\\\pm 0.3606$   \t| $0.0 \\\\pm 0.0$ \t| $0.0 \\\\pm 0.0$  | $0.0 \\\\pm 0.0$      |\n\n* **Experimental Setup**: We follow the example in **`Appendix D.3`** with 3 causal variables and 3 measurements. Specifically, \\\\( \\\\widehat{\\\\mathbf{Z}}\\_2 \\\\) and \\\\( \\\\widehat{\\\\mathbf{Z}}\\_3 \\\\) correspond to \\\\( \\\\mathbf{Z}\\_2 \\\\) and \\\\( \\\\mathbf{Z}\\_3 \\\\) respectively, while \\\\( \\\\widehat{\\\\mathbf{Z}}\\_1 \\\\) remains a ***mixture*** of \\\\( \\\\mathbf{Z}\\_1 \\\\) and \\\\( \\\\mathbf{Z}\\_2 \\\\)*,* as shown in **`Figure 8`**. We assume the **element-wise identification as the ground-truth measurement model** (a common identification guarantee by many CRL methods).   \n* **Results**: T-MEX correctly detects exclusivity violations, consistently yielding a score close to the oracle 1 . **In contrast, the unsupervised SI score fails to identify the mixing effect, assigning zero incompatibility even when the representation is entangled.** \n\n*Faller, P. M., Vankadara, L. C., Mastakouri, A. A., Locatello, F., & Janzing, D. (2024). Self-compatibility: Evaluating causal discovery without ground truth. In International Conference on Artificial Intelligence and Statistics (pp. 4132-4140). PMLR.*\n\n&nbsp;\n\n## Response to Weaknesses\n\n***W1: Assumption on knowing the ground truth measurement model***\n\n* **T-MEX does not require knowledge of the causal graph among latent variables.** Instead, it only assumes a known correspondence between each ground-truth causal variable and its intended measurement .   \n* **Assuming knowledge about the latent variables aligns with common practice in CRL evaluation**. Metrics like $R^2$ and MCC also rely on access to the ground-truth latent variables for benchmarking.  \n* **What sets T-MEX apart is its ability to account for causal dependencies** between latent variables — a key difference from disentanglement metrics, which typically assume independent latent factors. This makes the evaluation more challenging but also more aligned with real-world causal tasks.  \n* **T-MEX is not designed for validation/model selection, but it can be used this way if supervision is available.** This is reasonable especially in structured scientific settings (e.g., randomized trials, task-specific modeling), where it is clear which variables must be measured for downstream causal reasoning. We demonstrated this in **`Section 5.2`** with a real-world ecology causal task.\n\n***W2: Block-matching***\n\n* T-MEX can provide the matching between each learned representation block and the underlying causal variables. Importantly, **it does *not* require a fixed index alignment and can learn the matching up to permutations when there is an element-wise correspondence between the measurements and the causal variables,**  as we explain next.  \n* For a random CRL method where the indexing of representation blocks is arbitrary, **one can compute the T-MEX score after constructing the empirical measurement model under all valid permutations of the latents and select the assignment that yields the *lowest* score.** This best-matching procedure ensures that T-MEX reflects the true degree of exclusivity, regardless of how the CRL method arranges its outputs.  \n* This is analogous to how existing metrics (e.g., MCC) are typically computed under the best matching permutation between latent and learned components.\n\n\n***W3: T-MEX with large latent dimensions***  \n\t\n\n* We fully agree that CI testing in large dimensions is a well-known statistical challenge. In our experiments in **`Section 5`**, we employ the PCM test which is theoretically solid and implemented efficiently in the **`pycomets`** package. We show in **`Table 1`** that T-MEX remains computationally tractable and efficient up to **50 causal variables and 50 measurement variables**.   \n* That said, we would like to emphasize that **an extremely large number of latent causal variables is often undesirable in practice**. Most real-world applications naturally decompose into a small number of interpretable causal factors. Accordingly, the majority of existing CRL benchmarks and methods focus on **small- to medium-scale latent spaces** (e.g., 5–20), which is also the regime where downstream causal reasoning (e.g., ATE estimation, adjustment) is most tractable. We agree, however, that this is a limitation, and we encourage future works in this direction (e.g., moving to ML-driven dependence measures that may be less reliant on dimensions).\n\nResponse to minor concerns\n\n1. **Typos***:* Thank you for carefully reviewing our paper. We will correct them in the new version.  \n2. **Evaluate on a larger dataset***:* **`Table 1`** shows additional T-MEX experiments with an increasing number of latents.   \n3. **Reliance on CI tests**:\n    * We fully agree that **CI testing is an important component** in our framework, and choosing a valid and powerful test is essential.  We chose PCM test in our experiments for its theoretical guarantees on the validity (type I error control) and power of the test, making it a reliable choice of test for the exclusivity claim.  \n    * Despite known challenges, CI testing remains a **flexible and principled approach for evaluating exclusivity**—an aspect that standard metrics like MCC or $R^2$ cannot capture under causal dependencies. We will add this discussion to the paper.\n\n\n4. **Confidence intervals for T-MEX:**   \n   * **`Table 1` `in the main paper`** shows the standard deviation of T-MEX over 50 different simulations.  \n   * Additionally, **`Figure 6`** in **`Appendix D.1`** reports the p-values for **`Section 5.1`**, which indicates the confidence level of all individual tests that result in the T-MEX score.\n\n5. **Reliance on the test choice**: **`Table 2`** shows that T-MEX remains consistent across different test choices.  \n6. **Related work on self-compatibility/falsification:**   \n   Self-incompatibility (SI) score **`[Faller at al. 2024]`** is an unsupervised score for measuring inconsistency between different subgraphs quantified by SHD (We chose to compare with this work as it is, to the best of our knowledge, the most recent one regarding self-compatibility), and thus  can be viewed as an unsupervised proxy of SHD. However, as we show in **`Table 3`** and **`Appendix D.3`**, both SI and SHD failed to detect mixing in the learned representations.\n\n&nbsp;\n\n## Response to Questions\n\n***Q1: Compare T-MEX with other graph validation metrics***  \n**`Table 3`** evaluates **T-MEX and the self-incompatibility (SI) score proposed by Faller et al. (2024)**.\n\n***Q2: T-MEX scores under different CI tests***  \n**`Table 2`** presents additional results for the simulation in **`Section 5.1`** using different CI tests.\n\n***Q3: Number of CI tests as a function of dimensions***\n\n* For $N$ causal variables and $M$ measurement blocks, we need $M \\\\times N$ tests; also see **`Algorithm 1`** in the **`Appendix`**.  \n* **`Table 1`** provides a **runtime analysis** for different numbers of latents to show the practical applicability of T-MEX.\n\n***Q4: Recommendation of the alpha-level and sample-size heuristics***    \nT-MEX compares the adjacency matrix implied by the measurement model to the one estimated from data using CI tests, both false positives and false negatives can affect the score. \n\n* For valid CI tests such as PCM and GCM (see **`Appendix E`**), larger sample sizes generally improve the power of the tests.  \n* Moreover, there is a trade-off between power and false positive rate: at fixed sample size, increasing alpha improves power but also raises the Type I error rate.   \n* **We recommend choosing alpha and sample size based on the sparsity of the measurement model**: in sparser settings, controlling false positives is more important, and a smaller alpha is preferable. For practical guidance, we refer to recommendations in statistical testing, such as **`[Lakens 2022]`**.\n\n*Lakens, D., 2022. Sample size justification. *Collabra: psychology*, *8*(1), p.33267.*\n\n***Q5: Scaling up experiments*** \n\n* We would like to clarify, **T-MEX evaluates the correspondence between learned representations and ground-truth latent variables**; it does not perform representation learning or dimensionality reduction itself.  \n* Following your suggestion, we report additional results in **`Table 1`**."}, {"review_id": "NI6k3ERzUl", "reviewer": "Reviewer_MeWB", "summary": "The authors propose T-MEX, a new score to evaluate the quality of learned causal representations. Compared to existing metrics such as $R^2$ and MCC, the authors show that T-MEX is more adequate to assess how good the representation will be for causal downstream tasks. The idea behind this metric, based on conditional independence tests, is to measure the difference between 1) the graph linking the ground-truth and the learned representations and 2) the assumed graph between the ground-truth and the measurement model. Finally, the authors validate their score on synthetic and real-world data, demonstrating that a lower value of T-MEX is indeed associated with a smaller average treatment effect bias.", "questions": "- While I understand the usefulness of the measurement model when considering models learning block-identifiable representation and specific causal tasks, the usefulness of the metric is less clear to me when considering models with identifiable representation (up to permutation & scaling), which is the most common case. In this scenario, the measurement model is simply mapping each variable to its equivalent measurement. \n\n- Besides the comment about the inadequacy of MCC, no experiments on the paper show this case and how MCC performs compared to T-MEX. I’d be interested to see this comparison since this is the most common case in CRL.\n\n- Considering the same scenario, T-MEX seems like a crude score: since T-MEX assesses in a discrete manner the existence/absence of conditional independence, one could have the same T-MEX score for two highly different representations that lead to different results on downstream tasks. One that strongly correlates with the ground-truth latent, and another one that weakly correlates, but is enough to be dependent, according to the CI test used.\n\n- For the experiment in Section 5.1, while it is said that $R^2$ fails to show a clear correspondence to the ATE bias, the average $R^2$ leads to the same ordering of the model as T-MEX.\n\n- Finally, while I recommend the authors for using a real-world dataset to validate their method, I find this experiment limited. Since there is only one measure variable and one ground-truth latent, the only two possible graphs are the empty graph or $Y$ causes $\\hat{Y}$. The empty graph means that the learned representation is independent of the ground-truth which seems like an overall bad model. Despite this, the differences observed in Figure 5 are pretty modest.\n\n__Typos:__\n- Line 121: diffeomporphism\n- Line 265: the MCC would obtain the highest value 1 although…\n- V is both used to denote the variables and the graph between them. I would use a different notation.\n- In Section 5.1, both sd and std are used to represent standard deviation. I would choose one notation and stick to it.", "limitations": "The authors adequately addressed the limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "__Strengths:__\n- The authors propose an original score that is needed and will be useful to researchers in causal representation learning.\n- The paper is clear and well-written.\n\n__Weakness:__\n- The empirical validation of the score compared to existing methods is limited and could be more convincing.", "was_revised": true, "final_justification": "The authors addressed most of my concerns and, I think, most of the other reviewers'. They added new experiments and were thorough. I keep my score and recommend the acceptance of the paper.", "rebuttal": "We sincerely appreciate your positive feedback and would like to address your questions point by point.\n\n&nbsp;\n\n## Additional Experimental Results\n\n**Table 4: Spearman correlation coefficients for Section 5.1** \n\n|  Model \t| $\\\\mathbf{Z}\\_1$      \t            | $\\\\mathbf{Z}\\_2$      \t            | $\\\\mathbf{Z}\\_3$      \t\t|  \n|:---:|---------------------|---------------------|---------------------|  \n| A \t| $1.0000 \\\\pm 0.0000$      \t            | $0.8568 \\\\pm 0.0044$ \t| $0.8864 \\\\pm 0.0040$ \t|  \n| B \t| $0.8434 \\\\pm 0.0061$ \t| $0.9602 \\\\pm 0.0017$ \t| $0.9908 \\\\pm 0.0004$ \t|  \n| C \t| $0.9673\\\\pm 0.0013$  \t| $0.7215 \\\\pm 0.0076$ \t| $0.8016 \\\\pm 0.0062$ \t|\n\n* **Experimental Setup**: The results are computed using the same datasets as in **`Section 5.1`**,  where the representation $\\\\widehat{\\\\mathbf{Z}}\\_{A\\_1}$ is supposed to exclusively measure $\\\\mathbf{Z}\\_1$. Note that the learned representation has dimensionality 1, while the latent space has dimensionality 3\\. Therefore, we compute correlation coefficients individually, i.e., \\\\( \\\\text{corrcoef}(\\\\widehat{\\\\mathbf{Z}}\\_{A\\_1}, \\\\mathbf{Z}\\_i) \\\\) for \\\\( i \\= 1, 2, 3 \\\\).  \n* **Results:** As shown, similar to \\\\( R^2 \\\\), the correlation coefficients fail to provide a clear correspondence between the representation and the latent variables. This illustrates why the Mean Correlation Coefficient (MCC) can yield deceptively high scores, even when the learned representation is misaligned with the identifiability theory.\n\n&nbsp;\n\n## Response to Weaknesses\n\n***W1: Empirical validation of the score compared with other existing methods*** \n\n* Thank you for this suggestion. We have included several additional results to further validate our approach, including **`Table 4`** (evaluating MCC), **`Table 3`** (comparison to self-compatibility score, see rebuttal for Reviewer **`k71J`**), and **`Table 5`** (comparison to $R^2$ and MCC under weak and nonlinear causal relations, see rebuttal for Reviewer **`yCFv`**).\n* We deeply apologize that we can not include these results here due to the character limit. Please refer to other rebuttals for more explanation.\n\n&nbsp;\n\n## Response to Questions\n\n***Q1: Usefulness of the measurement model under one-to-one correspondence of representations***\n\n* Thank you for this thoughtful question. We agree that many CRL methods offer theoretical guarantees of identifiability up to permutation and scaling. However, in practice, such guarantees often **do not fully translate** into the learned representations due to **finite sample sizes**, **model misspecification**, or **optimization issues**.  \n* For example, theory may guarantee that\n\\\\[\\\\widehat{\\\\mathbf{Z}}\\_1 \\= a \\\\cdot \\\\mathbf{Z}\\_1,\\\\quad \\\\widehat{\\\\mathbf{Z}}\\_2 \\= b \\\\cdot \\\\mathbf{Z}\\_2 \\\\quad \\\\text{with } a, b \\\\neq 0,\\\\] yet in practice, one might observe entangled representations such as \\\\[\\\\widehat{\\\\mathbf{Z}}\\_1 \\= \\\\mathbf{Z}\\_1 \\+ \\\\mathbf{Z}\\_2,\\\\quad \\\\widehat{\\\\mathbf{Z}}\\_2 \\= \\\\mathbf{Z}\\_2 \\\\]\n\n   This leads to a discrepancy between: **(a)** the *ideal measurement model* induced by the identifiability theory (a one-to-one correspondence between latents and measurements), and **(b)** the *empirical measurement model* reflected by the learned representation (where $\\\\widehat{\\\\mathbf{Z}}\\_1$ mixes both $\\\\mathbf{Z}\\_1$​ and $\\\\mathbf{Z}\\_2$​​). **T-MEX is designed to quantify exactly this kind of misalignment.**\n\n***Q2: Compare T-MEX and MCC in the experiments.***   \nThanks for your suggestion. We include this additional result in **`Table 4`** and will update our manuscript accordingly.\n\n***Q3: Discrete nature of T-MEX***  \nThank you for this insightful observation. You're absolutely right that T-MEX, by design, focuses on **exclusivity violations in a discrete manner**,  indicating whether a learned representation incorrectly encodes information from unintended causal variables. However, we would like to offer a few clarifications:\n\n* While T-MEX itself returns a discrete score (counting conditional independence violations), the underlying **CI tests produce p-values**, which provide a *graded measure of evidence* against the null. This enables practitioners to assess the *strength* of each exclusivity relation beyond a binary decision.\n\n* The **discrete nature of T-MEX is similar to metrics like Structural Hamming Distance (SHD)** used in causal discovery, which also quantify edge-wise structural mismatches without measuring correlation strength. We see this as complementary to *predictivity-focused metrics*.\n\n* In fact, **T-MEX and predictivity-focused metrics like $R^2$ or MCC  can and should be used together**. While T-MEX evaluates whether the learned structure aligns with the expected measurement model (from a *causal* perspective), metrics such as $R^2$ and MCC assess *how well* the representation predicts the ground-truth latent variable (from a *predictive* **non-causal** perspective).\n\n* As noted in **`Remark 3.2`**, the measurement model can be further extended to incorporate additional assumptions, such as combining exclusivity checks with constraints on correlation strength or parametric function classes.\n\n***Q4:*** ***$R^2$ interpretation in Section 5.1*****:** \n\n* While the average $R^2$ scores happen to match the T-MEX ordering in Section 5.1, this is coincidental and not generally reliable.  \n* If a model is expected to disentangle $\\\\mathbf{Z}\\_1$​, one might look at its $R^2$ score on $\\\\mathbf{Z}\\_1$​. However, **Model C** shows a **high R² on $\\\\mathbf{Z}\\_1$**, even though its representation additionally mixes **$ \\\\mathbf{Z}\\_2$​ and $\\\\mathbf{Z}\\_3$​**, leading to substantial ATE bias (**`Figure 3`** in **`Section 5.1`**). This misalignment is correctly flagged by T-MEX but not by $R^2$.  \n* More broadly, $R^2$ **does not indicate which latent variable is being captured** by each representation dimension, and it is unclear how to rank models consistently when representations are entangled. **T-MEX addresses this by explicitly testing measurement exclusivity**, making it a more principled tool for causal representation evaluation.\n\n**Q5: Interpretation of the measurement model for the ISTAnt experiment (Section 5.2):**\n\n* The **videos are generated by both the treatment $\\\\mathbf{T}$**, the **outcome $\\\\mathbf{Y}$, and other experimental settings that we do not see, but may correlate the predictions with the treatment (not $\\\\mathbf{Y}$, as we have a Randomized Controlled Trial).** Since we focus only on models with **classification accuracy above 0.8** (see **`Figure 5`** in **`Section 5.2`**), the learned representation $\\\\widehat{\\\\mathbf{Y}}$ must encode meaningful information about $\\\\mathbf{Y}$,  ruling out the trivial “empty graph” case. However, we need to test whether the predictions contain spurious correlations with the treatment assignment $\\mathbf{T}$, which T-MEX captures (see **`Appendix D.2`** for more details).  \n    \n* This leads to two relevant causal structures:  \n    \n  1\\.  \\\\( T \\\\rightarrow \\\\widehat{Y} \\\\leftarrow Y \\\\): the representation is entangled with both treatment and outcome (T-MEX \\= 1),    \n  2\\. \\\\( Y \\\\rightarrow \\\\widehat{Y} \\\\): the representation exclusively captures the outcome (T-MEX \\= 0).  \n    \n* These correspond exactly to the two T-MEX levels we analyze. As shown in **`Figure 5`**, T-MEX discriminates between models with high and low ATE bias and classification performance, offering a meaningful evaluation for real-world scientific applications.\n\nThanks for pointing out the typos. We will correct them in the updated manuscript."}, {"review_id": "HH5cnZxmoG", "reviewer": "Reviewer_yCFv", "summary": "The authors aim to clarify the role of CRL in causal analysis, particularly when the causal variables are not directly observable. The paper introduces a new evaluation metric, the Test-based Measurement EXclusivity (T-MEX) score, to assess how well learned representations align with the underlying causal structure. The T-MEX score is validated through both theoretical analysis and empirical experiments in numerical simulations and ecological video analysis.", "questions": "1. How well does this method generalize to more complex causal environments with multiple hidden variables and interactions? Could you provide examples of this and does T-MEX scale efficiently to high-dimensional datasets or when the number of causal variables increases significantly? Are there any known limitations in scaling the methodology to large datasets or real-time applications?\n\n2. How does T-MEX perform in comparison to existing evaluation metrics in causal representation learning in cases where causal dependencies are weak or non-linear?\n\n3. You discuss extensions to noisy data in your measurement model framework. Can you elaborate on how T-MEX behaves when noise levels are high or when data is missing?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n1. The T-MEX score is a unique and practical contribution, offering a new way to quantify the alignment between learned representations and their causal counterparts. This metric is applied effectively in both simulated and real-world settings.\n\n2. The paper thoroughly addresses the existing challenges in causal analysis and representation learning, proposing a novel and interpretable solution.\n\n\n**Weaknesses**\n\n1. While the framework and T-MEX score are valuable, they may be challenging for practitioners unfamiliar with the underlying mathematical foundations or causal inference techniques. The paper could benefit from clearer explanations or visual aids for those new to the field.\n\n2. The paper discusses its framework’s ability to generalize across various causal tasks, but it remains to be seen how well it handles more complex or diverse causal environments beyond the tested scenarios.", "was_revised": true, "final_justification": "Most of all my concerns are resolved.", "rebuttal": "Thank you for carefully assessing our work and for considering our paper as “*a unique and practical contribution”*. We provide detailed responses to the questions as follows.\n\n&nbsp;\n\n## Additional Experimental Results\n\n**T-MEX Oracle:** T-MEX Oracle is computed based on an oracle CI test with 0 type I error and power 1. We compute the oracle T-MEX score to demonstrate desirable properties of the metric—such as its consistency and robustness across different conditional independence tests. This is feasible in the simulated setting, where the measurement functions are known by design. In contrast, for empirical representations learned by a CRL model, the true measurement functions are unknown, making oracle computation infeasible in practice.\n\n**Table 1: T-MEX for higher-dimensional latents generated from nonlinear SCM**\n\n| n\\_latent | T-MEX                        | T-MEX Oracle |  time  (sec)   |\n|:-------------------:|:----------------------------:|:------------:| -----------------------:|\n| $5$                 | $0.0483 \\pm 0.2223$          | $0$          | $0.0359 \\pm 0.0011$    |\n| $10$                | $0.0033 \\pm 0.0577$          | $0$          | $0.1559 \\pm 0.0017$    |\n| $20$                | $0.0550 \\pm 0.5187$          | $0$          | $0.8080 \\pm 0.0094$    |\n| $50$                | $0.0917 \\pm 0.9582$          | $0$          | $10.8450 \\pm 0.1004$    |\n\n\n* **Experimental Setup:** We simulate the causal variables based on a nonlinear SCM – a location-scale SCM as implemented by **`Liang et. al. 2023`**. Then, the measurement variables are simulated as a direct copy of each of the corresponding causal variables. For different numbers of latents (n\\_latent), we report the T-MEX score based on the Generalized Covariance Measure (GCM) test with linear regression (see **`Appendix E`** for more details about GCM), along with their standard error based on 20 random causal DAGs for the latent causal variables, each with 30 repetitions and 1000 observations.   \n* **Results:** T-MEX remains closely aligned with the T-MEX oracle in all cases, and can be efficiently computed up to 50 latents within reasonable time, validating its applicability in moderate to high dimensions.\n\n*Liang, W., Kekić, A., von Kügelgen, J., Buchholz, S., Besserve, M., Gresele, L., & Schölkopf, B. (2023). Causal component analysis. Advances in Neural Information Processing Systems, 36, 32481-32520.*  \n*Reference*\n\n**The data-generating process for Table 5** follows the example in **`Appendix D.3`** with minor modifications. Here we provide a brief introduction of the example for convenience:\n\n* This example consists of three causal variables \\\\( \\\\mathbf{Z}\\_1,\\\\mathbf{Z}\\_2,\\\\mathbf{Z}\\_3 \\\\) and we obtain three entangled measurements  \\\\( \\\\widehat{\\\\mathbf{Z}}\\_1, \\\\widehat{\\\\mathbf{Z}}\\_2 \\\\) and \\\\( \\\\widehat{\\\\mathbf{Z}}\\_3 \\\\)  for the latent causal variables.  \n* Specifically, \\\\( \\\\widehat{\\\\mathbf{Z}}\\_2 \\\\) and \\\\( \\\\widehat{\\\\mathbf{Z}}\\_3 \\\\) directly correspond to \\\\( \\\\mathbf{Z}\\_2 \\\\) and \\\\( \\\\mathbf{Z}\\_3 \\\\), while \\\\( \\\\widehat{\\\\mathbf{Z}}\\_1 \\\\) remains a ***mixture*** of \\\\( \\\\mathbf{Z}\\_1 \\\\) and \\\\( \\\\mathbf{Z}\\_2 \\\\)*,* as shown in **`Figure 8 in Appendix D.3`**.    \n* We assume the element-wise identification as the ground-truth measurement model (a common identification guarantee by many CRL methods). In this case, the T-MEX Oracle yields one.\n\n**General clarification for Mean $R^2$ and MCC computation**: Both Mean $R^2$ and MCC are computed under the best index assignment, i.e, the permutation that gives the highest score, following the standard implementation in CRL. \n\n**Table 5:  T-MEX, $R^2$, and MCC under weak or nonlinear causal relations**\n\n|Causal Relations| T-MEX Oracle |   T-MEX  |       Mean $R^2$      |   MCC   |  \n|:-------------:|:-------------:|:---------:|:-------------------:|:--------:|  \n| weak | $1$          |$1.0100 \\\\pm 0.0995$|$1.0000\\\\pm 0.0000$|$1.0000 \\\\pm 0.0000$|  \n| nonlinear| $1$          |$1.0000 \\\\pm 0.0000$|$0.9711 \\\\pm 0.0015$|$0.9852 \\\\pm 0.0008$|\n\n* **Experimental Setup:**   \n  * **Weak**: We generate data from a linear SCM with three latent causal variables following the DGP described above. **The linear coefficients between the causal variables are sampled uniformly between 0.01 and 0.1.**   \n  * **Nonlinear**: We generate data from a **partially non-linear SCM** with three latent causal variables following the DGP described above. The causal relationship between $\\\\mathbf{Z}\\_1$ and $\\\\mathbf{Z}\\_2$ as well as between $\\\\mathbf{Z}\\_1$ and $\\\\mathbf{Z}\\_3$ is nonlinear (a leaky tanh function) with additive noise, while the causal relationship between $\\\\mathbf{Z}\\_2$ and $\\\\mathbf{Z}\\_3$ is linear.   \n  * In both cases, we repeated the experiments 100 times each with 1000 observations.   \n* **Results:** In both cases, T-MEX yields a score of 1 (up to some variance), indicating exclusivity violation in $\\\\widehat{\\\\mathbf{Z}}\\_1$, whereas Mean $R^2$ and MCC provide misleading results (with the highest score of 1\\) that suggest perfect element-wise identification. See **`Q2`** for a more detailed explanation.\n\n**Table 6: Consistency of T-MEX under noisy measurements** \n\n|T-MEX Oracle|      witout noise     |      with noise   |  \n|:------------:|:------------------:|:------------------:|  \n|      0     |$0.0100 \\\\pm 0.1000$| $0.3900 \\\\pm 0.6497$| \n\n* **Experimental Setup:** We compare the empirical T-MEX value with/without noise. The noisy measurements are generated by adding Gaussian noise to the original latents. I.e., the measurement function writes $h(z) \\= z \\+ e$ with $e$ independent Gaussian noise. Results are evaluated over 100 datasets with 1000 samples each.  \n* **Results:** T-MEX remains largely consistent under noisy measurements. See **`Q3`** for a more detailed explanation.\n\n&nbsp;\n\n## Response to Weaknesses\n\n***W1: More explanations or visual aids***  \nGreat point—thank you\\! We will add a detailed explanation of the framework along with step-by-step usage guidelines in the appendix. Additionally, we will provide a modular and user-friendly implementation in the code library that can be used as a black-box.\n\n***W2: More complex or diverse causal environments***  \nThanks for your suggestion\\! We have provided additional experimental results on **higher-dimensional latents (`Table 1`),**  and for latents under **weak or nonlinear causal relations (`Table 5`).** We also demonstrate T-MEX’s effectiveness under noisy measurements, as shown in **`Table 6`**.\n\n&nbsp;\n\n## Response to Questions\n\n***Q1: Scale up the experiments and known limitations in CI tests***\n\n- **Scale up the experiments:** We have included additional T-MEX results in **`Table 1`** for different numbers of latents (up to 50) generated from a nonlinear structural causal model (SCM). We also report the runtime of T-MEX computation as a function of the number of latent variables, demonstrating its practical feasibility even in moderately high-dimensional settings.\n\n \n\n- **Known scaling limitations of CI tests:**  It is well known that the statistical power of conditional independence (CI) tests deteriorates as the dimensionality of the conditioning set increases—a limitation shared by most CI-based causal discovery methods **`[Zhang et al., 2012; Shah & Peters, 2020; Strobl et al., 2019]`**. Since our proposed framework is agnostic to CI test choices, T-MEX can be easily scaled to larger dimensions by adapting more powerful and scalable test methods as they are developed.\n\n\n*Zhang, K., Peters, J., Janzing, D., & Schölkopf, B. (2012). Kernel-based conditional independence test and application in causal discovery. UAI.*\n\n*Shah, R. D., & Peters, J. (2020). The hardness of conditional independence testing and the generalised covariance measure. Annals of Statistics.*\n\n*Strobl, E. V., Zhang, K., & Visweswaran, S. (2019). Approximate kernel-based conditional independence tests for fast non-parametric causal discovery. Journal of Causal Inference.*\n\n***Q2: Comparison to existing evaluation metrics in CRL under weak or nonlinear causal relations***\n\n* **`Table 5`** compares T-MEX with $R^2$ and MCC for latents generated from **weak and nonlinear** causal relations.  \n* Under identifiability assumptions that guarantee element-wise correspondence, T-MEX correctly detects the mixing effect in $\\\\widehat{\\\\mathbf{Z}}\\_1$ and gives a score near one (note that T-MEX \\= 0 indicates perfect alignment).  \n* In contrast, both mean $R^2$ and MCC fail to reflect this violation. Despite the entanglement in $\\\\widehat{\\\\mathbf{Z}}\\_1$​, they still assign the maximum score of 1, misleadingly suggesting perfect element-wise identification.\n\n\n  \n***Q3: T-MEX under noise variables***\n\nWe clarify below how T-MEX behaves under high noise levels and in the presence of missing data:\n\n* **High noise levels:** As discussed in **`Remark 2.2`**, noise in the measurement functions can be treated as **additional independent latent variables**. This means that if the noise is independent of the existing causal variables, the **conditional independencies required for T-MEX remain intact**. In such cases, one can safely compute T-MEX without explicitly modeling the noise; the score will remain valid, as shown in **`Table 6`.**\n\n* **Missing data:** Great suggestion\\! If we know which nodes block the paths between the variables we need to test, then T-MEX can also handle the missing labels of certain ground truth causal variables, as long as the path they appear in is otherwise blocked."}]

Author rebuttal:

Meta-review: The paper studies the role of causal representation learning and introduces a new evaluation metric, the Test-based Measurement EXclusivity (T-MEX) score. This score is designed to assess how well learned representations reflect the true causal structure. The proposal is validated through both theory and experiments with synthetic data and real-world video analysis.

The reviewers appreciated the exposition of the paper and choice of experiments. The T-MEX score is a clear and interpretable quantity that is of relevance to a large number of researchers in the field.

The discussion phase led to the authors conducting a number of new experiments, with noisy CI oracles, non-linear SCM's, increased number of latents, and with noisy data. The reviewers appreciated the new information, and we'd encourage including these findings in the final revision.
Decision: Accept (poster)