Measure-Theoretic Anti-Causal Representation Learning
Authors: Arman Behnam
Binghui Wang
Venue: NeurIPS 2025 poster ()
Keywords: Representation Learning
Causality
Measure Theory
Primary area: probabilistic_methods
TL;DR: We introduce ACIA, a measure-theoretic framework for anti-causal representation learning through two-level abstraction, supporting both perfect and imperfect interventions with theoretical guarantees.

Abstract: Causal representation learning in the anti-causal setting—labels cause features rather than the reverse—presents unique challenges requiring specialized approaches. We propose Anti-Causal Invariant Abstractions (ACIA), a novel measure-theoretic framework for anti-causal representation learning.  ACIA employs a two-level design: low-level representations capture how labels generate observations, while high-level representations learn stable causal patterns across environment-specific variations. ACIA addresses key limitations of existing approaches by: (1) accommodating prefect and imperfect interventions through interventional kernels, (2) eliminating dependency on explicit causal structures, (3) handling high-dimensional data effectively, and (4) providing theoretical guarantees for out-of-distribution generalization. Experiments on synthetic  and real-world medical datasets demonstrate that ACIA consistently outperforms state-of-the-art methods in both accuracy and invariance metrics. Furthermore, our theoretical results establish tight bounds on performance gaps between training and unseen environments, confirming the efficacy of our approach for robust anti-causal learning. {{Code is available at \url{https://github.com/ArmanBehnam/ACIA}}}.

Measure-Theoretic
Anti-Causal Representation Learning
Arman Behnam
Binghui Wang
Department of Computer Secience Illinois Institute of Technology Chicago, Illinois, USA abehnam@hawk. illinoistech. edu
Department of Computer Secience Illinois Institute of Technology
Chicago, Illinois, USA bxang77@@11inooiseeed edu
Abstract
Causal representation learning in the anti-causal setting- cause features rather than the liversc-pessem unique challenges requiring specialized ap- proaches. We propose Anti-Causal Invariant Abstractions (ACIA), novel measure- theoretic framework for anti-causal representation learning. ACIA employs a two- level design: low-level representations capture how labels generate observations, while high-level representations learn stable causal patterns across environment- specific variations. ACIA addresses key limitations of existing approaches by: (1) accommodating prefect and imperfect interventions through interventional kernels, (2) eliminating dependency explicit causal structures, (3) handling nigh-dimensional data effectively, and (4) providing theoretical guarantees for out- of-distribution generalization. Experiments on synthetic and real-world medical datasets demonstrate that ACIA consistently outperforms state-of-the- methods in both accuracy and invariance metrics. Furthermore, our theoretical results estab- lish tight bounds on performance gaps between training and unseen environments, confirming the efficacy of our approach for robust anti-causal learning. Code available at https: / thub..u...co com/ ArmanBehnam/AAnn ACCI
Introduction
Causal representation learning discovers causal relationships underlying data rather than statistical associations [47] At its core, causal representation learning seeks to identify high-level causal variables from low-level observations, bridging the gap between statistical pattern recognition and causal reasoning. Learning these causal variables offers transformative potential for artificial intelligence systems that can
about cause and effect.
particularly challenging yet promising domain is learning repre- sentations in the anti-causal setting, where the causal direction is reversed from traditional prediction tasks. Figure diagrams the anti-causal setting, where Y (target) is the causal variable, X (obser- vation) the observable variables, E (Environment) the environment variable introducing spurious correlations, U is confounder which affects both X and Y, and Z (latent variable) is an unmeasured in- termediary. The orange arrows represent direct paths to the observed variables X, and blue arrows represent confounding effects.
E
Figure Anti-Causal Dia- gram: causal (orange), spuri- ous (blue), and confounding (dashed blue) dependencies.
Consider disease diagnosis from chest X-rays across different hospitals 11]. A disease (Y) causes observable symptoms and measurements (X), with the relationship represented as Y X. The confounding factors U (e.g.. age and sex) affect both disease and symptoms. Environmental factors E (e.g., hospital-specific
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Input Raw Data
Theorem 3 Causal Dynamics
Theorem Causal Abstraction
Output (f Co dy (
Two-Level Optimization
ming,e dn me [C A2 R2 A Ag Ri: Env. Independence Rs: Causal Consistency
Φ7 X Dz, Lew-Level Representation 2z. = (X, Q Kz)
ou D2, + Dzn High-Level Representation Zμ (VB, RR.Ks)
X
Figure 2: ACIA: Anti-Causal Invariant Abstraction Framework.
protocols) introduce spurious correlations by creating hospital-specific variations, forming the anti- causal structure Y X + E. The orange arrows therefore depict the true disease-to-yy to-ssttt mechanism (Y + X), while the paths involving E' and U introduce noise. This anti-causal structure requires specialized methods disentangle causal mechanisms from environmental artifacts.
Early works 46 22 formalized anti-causal learning and showed that traditional methods fail in
this setting. Follow-up works can be categorized into three main approaches: 1) intervention-basee that models causal effects through interventions; 2) Structure-based explicitly models causal structures through Directed Acyclic Graphs (DAGs), requiring complete knowledge of the underlying Structural Causal Model (SCM); 3) Invariant learning methods (2
causal learning [59
10
35
causal methods 61
21
33
49
50
62 57 71 566 distributions. See more related work in Appendix A
63
18
that seeks representations invariant across
However, existing methods face several critical limitations. First, intervention-basee approaches assume perfect interventions- where intervened variables are completely disconnected from their causes a restrictive assumption rarely satisfied in real-world scenarios. Second, structure-based methods rely on explicit structural dependencies through SCM
[59 10
poses significant hurdles when the underlying SCM is unknown. Third, distribution-invariin approaches' assumptions on independent and identically distributed data or known test distributions limit their generalization capabilities [71 56). Fundamentally, these limitations arise because the single-level representations learned by these methods cannot simultancously capture the causal mechanism from Y to X while filtering spurious correlations from E to X in anti-causal structure [ 27).
49 50
We develop Anti-Causal Invariant Abstraction (ACIA), two-level representation learning method (Figure 2 to address above limitations. ACIA is inspired by recent causal representation learning 2 66) and measure theorreic causality [9). Specifically, ACIA addresses these limitations through three key innovations: Firs, we introduce generalized intervention model that accommodates both perfect and imperfect interventions via the designed interventional kernels. Second, ACIA learns directly from raw inpul without requiring explicit SCM specification. Third, we introduce environment -nivarrinnn regularizers that enable stable identification of invariant causal variables across environments. All together, ACIA involves:
studies 18 59
causal dynamies learn low-level representations directly from dath-without requiring explicit DAGs/SCMs-thar capture the anti-causal structure by encoding how labels generate observable features while preserving miviromment-speccne how diseases (Y) manifest as symptoms and measurements (X) in X-ray images, encompassing both disease-relatee patterns and hospital specificc factors. The learnt low-level representations support reasoning under both perfect and imperfect interventions, enabled by the interventional kernels.
variations. E.g.. in medical diagnosis, this reflects
causal abstraction to learn high-level representations that distill environment-invann invvrriinn causal features from the low-level representations. These abstractions generalize across environments by discarding spurious environmental correlations while preserving label-relevant mechanisms. For example, this involves identifying patterns that are consistently associated with specific discases regardless of the hospital environment.
theoretical guarantees that establish convergence rates, auttoofddisttrbbuuoo/ddmaain generalization bounds, and environmental robustness for the learned representations.
We extensively evaluate ACIA on multiple synthetic and real-worke datasets with perfect and imperfect interventions. For instance, our results demonstrate ACIA achieves almost perfect accuracy (e.g.. 99%) on stuely-studied CMNIST and RMNIST synthetic datasets with perfect environment independence and intervention robustness, significantly outperforming SOTA basclines. Most notably, on the real-

---

world Camelyon17 medical dataset, ACIA achieves 84.40% accuracy an 19% improvement over the best baseline (65.5% by LECI [18) while maintaining competitive environment independence and low-level invariance metrics. These results validate our theoretical framework's 8 ability to learn robust anti-causal representations across both synthetic and real-world settings, with particularly strong performance gains in scenarios with complex environmental variations.
2 Background on Measure Theory and Causality
measurable space $. consists of a sample space Q, a o-algebra & of measurable sets, and probability measure μ. Within the causal context, 52 represents possible states of the world, f represents events we can measure, and μ assigns probabilities to these events. The important notations in this paper are summarized in Table in Appendix.
2.1 Measure Theory
Definition (Environment Measurable Space). Given a finite set of environments E, each 6 E is associated with measurable input space (Ne, F... x, measurable output space (Je, (y...y.). and probability measure Pe on the product space (Xe X Ve,3xx Fx. 8 Fy.).
Definition 2 (Data Space). For each environment e E E, the data space is a tuple (De, F D.s Pe) where De [f}] is finite collection of input-output pairs from environment e, I.j are elements of the input space Ne, yr are elements of the output space Ve. T. is the index set that defines the component-wise sample space structure for environment e. Specifically, it indexes the components of the product space such that Qr = XtET, Et, where each Et represents measurable component space at index and Pe is probability measure on De defining the distribution of (f).
Definition 3 (Representation). A representation is measurable function & X - R mapping inputs to a latent space R, where (R, FR) is measurable space.
(R,FR)
A representation is causal if it captures the underlying causal mechanisms generating the data.
Definition (Kernel (25 ). A kernel K is a function K fQx x F - [0, 1 such that: For each fixed
St the mapping A K (w, A) a probability measure on (St F); 2. For each fixed A E F. the mapping w + K (w, A) is & -measurable.
Intuitively, K(w, A) represents the probability of A conditioned on the information encoded in w. Properties of kernels being used in this work are discussed in Appendix B.]
2.2 Causality
In the thasurc- framework, interventions modify kernels rather than structural equations 40 enabling unified treatment of both perfect and imperfect interventions.
Definition (Intervention (26)). An intervention a measurable mapping Q(-I-): H x S2 + (0, 1] that modifies causal kernels by modifying the underlying probability structure. There two types of intervention in causal representation learning:
1. A hard (or perfect) intervention sets Q(A(w) = Q(A), independent of w;
2. A soft (or imperfect) intervention allows Q(Alw) to depend on
The do-operator do(X X = xx) represents setting X to value v, breaking causal arrows into X. Note that P(Y]do{X = .r) differs from conditional probability P(Y = z.), which observes X while preserving all causal relationships.
Definition 6 (Causal Independence [40)). Variables X and Y are causally independent given Z. denoted 11, 2, if P(Y)do( X =x), 2) = P(Y|Z) for all x in the support of X, and P(X)do(Y v), 2) Z(X for all in the support of
Definition 7 (Causal Space (39). For environment
causal space is tuple (g., Hes Pey Ke),
where Ω2 = XITT, F,, is the sample space, P. is the probability measure on (Sa, He), and K is kernel function for environment e. For each t Te, s. is the o-algebra component space E,, and the overall o-algebra He iteT, A. is the tensor product of these component o-algebras.
This definition is the backbone of measure-theoretie causality in this paper, inspired from (3)].
3

---

3 ACIA: Mcasure-Theoretic Anti-Causal Representation Learning
This section presents our theoretical framework ACIA for anti-causal representation learning. Our theoretical analysis relies on the following assumptions: The number of training environments [Eminn] = K 0o. For any spurious feature s correlated with in e;, there exists ej E Emain where 8 IL Y. We have access to samples from interventional distributions P(X]do(Y = y)) via intervention Q and representations are L-Lipschitz continuous with respect to appropriate metrics.
3.1 Problem Formulation
We formalize the anti-causal representation learning problem as follows: given causal structure where label Y causes the observation X and environment E also influences X (Y - + E), our goal is learn representations that capture the causal generative invariant from Y to X Given observations from environments E {4:]== with corresponding datasets D = [De, }a learn two-level representations:
i== we aim to
a low-level representation Or : :D C X' + Zt that extracts features from raw data to uncover the anti-causa structure including both Y - X and E + X.
high-level representation 0H: z -> Zи that distills environmenttinvariaan causal features from L. i.c., enforcing ΦH (X)( 1 E Y.
The predictor C ZH + y then maps high-level representations to labels. With loss function e ) x y + R2 defined across all environments &, the full model f = C Φh o or can be trained in an end-to-end fashion.
More specifically, we introduce causal dynamic (Thm B) to facilitate learning low-level represen- tations 3r. by jointly optimizing the loss with causal structure consistency regularizer (R2 in Eqn 6. where minimizing it encourages the low-level representations to align with the true causal mechanisms underlying the data. On top of Z,,, we further introduce causal abstraction (Thm 4 to learn high-level representations, guided by another environment independence regularizer (R₂ in Eqn 7. This regularizer measures the discrepancy between the expected high-level representations across environments conditioned on the label Y. Minimizing it can remove environment-specifie information while retaining label-relevant causal features.
Two-level design rationale. This hierarchical structure enables us to do two procedures (1) Interventional effect calculation: du captures how the anti-causal setting responds to interventions, handling both perfect and imperfect intervention scenarios; (2) Information bottleneck: Φи retains only label-relevant invariants while discarding environment--peeciie noise.
3.2 The Theoretical Framework
Our framework establishes product causal space (Def. g) on measure-theoretic causality to handle anti-causal learning across multiple environments; causal kernel (0) and interventional kernel (Thm2) 2 to characterize anti-causal structures (Thm[]. Building on this foundation, we develop causal dynamics (Thm 3 to learn low-level representations that extract anti-causal relationships from the raw data. On top of it, we further develop causal abstractions (Thm 4 to learn environment- invariant high-level representations. Figure shows the detailed procedure of our framework. We first introduce necessary definitions below.
Definition 8 (Product Causal Space). Given causal spaces He,, Ke,) and (Reg, He,, IP, Pe,, Ke,) for environments e; and ej. product causal space is tuple (Ω, H. P. K) where Ω = Sle, x Qez is the sample space for combined environments e, and e,. H = He @ He, the product a-algebra, and TP = PP,, s9 Pay is the product measure. Kg = (Ks S (T)) is family of causal kernels with 3 the power set function and T Te, UTe, e, is the union of index sets.
This construction enables joint reasoning across environments while preserving individual causal structures. For analysis of environment subsets, we utilize sub-o-algebras:
Definition 9 (Sub-o allgbbaa....... Given a product causal space H, (/P,KK forr any subset C T, the sub-o-algebre Ht is generated by measurable rectangles A, x Aj where A; E He and A₁ E He, corresponding to the events in the time indices S.
Product causal spaces merge causal space of different environments by tensor products of oallgebras. Properties of causal atbbstruuttres are discussed in Appendix B.2 Next, we define the causal kernel to represent the causal structure.
A causal generative invariant is a stable function f y + X by which Y produces X, formally represented X f(Y,c) where represents noise, and f invariant across environments.

---

Definition 10 (Causal Kerne). A causal kernel Ks E Ks for index set S E (T) is a function Kş Ω x H [0, 1]. For fixed w E S2, (s. a probability measure on (Q, and for fixed A H, Ks Ks( A) is He-mmasurabll, where Hys is sub-o-algebra in S.
Intuitively, [s(w, A) is the conditional probability of event given causal information encoded in w, restricted to environments indexed by S. This enables characterization of anti-causal structures:
Theorem 1 (Anti- Causal Kernel Characterization). For an anti-causal structure with arbitrary feature space X, label space V, and environments &, the causal kernel satisfies:
Ks(w, -
EEE)
where jy is the marginal measure on y
This characterization captures how labels Y generate observations X across environment subsets S, integrating over all possible label values weighted by their marginal probabilities. Realization of anti-causal kernels is discussed in Appendix B.3. Moreover, the independence property of the anti-causal kernel is as follows:
Corollary (Independence Property of Anti-Causal Kernel). In anti-causal structure, for any w,w w E Q with identical Y -component and all A E .xx, B E Hy, S (TT):
Ks(w, (A(B)) = Ks(w,, {A[B))
This independence property reveals that conditional kernels depend only on the label Y, not on anvironment-specine information in w. In Appendix [B.4] we prove that causal events show kernel values varying with w, while anti-causal events maintain invariance under specific subset removals from the conditioning
We now characterize how interventions modify causal kernels, enabling unified treatment of both perfect and imperfect interventions.
Theorem 2 (Interventional Kernel), Let (Ω, H. P. K) be product causal space. For any subset S (T) and intervention Q H x Ω [0, 1.. there exists unique interventional kernel:
K d
2) (w. A)
Ks(w, du (QAlw")
(3)
provided that the integral is a Lebesgue integral w.r.t. the measure induced by Ks(w.-) on (H,,.). In addition, Ks(w,-) is o-finite for each w E Ω, and Q(A--) is H -f-masurrable for each A E H.
We emphasize that this construction encapsulates both intervention types: hard interventions where Q(AWw") Q(A) is constant across w', and soft interventions where Q(Alw") varies with w'. Soundness and causal explanation of interventional kernels are discussed in Appendix [C.5. The distinctive property of anti-causal structures is that interventions on X do not affect Y. while interventions on Y change the distribution of X (asymmetric response to interventions).
Corollary 2 (Interventional Kernel Invariance). In anti-causal structure, interventional kernels satisfy the following invariance criteria:
1. (gd(() {Y B}) Ks(w, (Y e BB) all measurable sets B C. meaning intervening on X does not change the distribution of
2. K KdolY (w. {X E A}) + Ks(w, {X E A}) for some measurable sets A C X, meaning intervening on Y changes the distribution of X, which is characteristic of an anti-causal relationship.
Building on the kernel framework, we now develop our approach to learning low-level representa- tions. We adopt the causal dynamics perspective 2 68), which identifies latent causal relationships from observed data under distribution shifts ppreosellyy the setting in anti-causal learning across environments. Our low-level representation mapping Φr implements causal dynamics by learning how labels Y generate observations X while preserving environment-specific information. Unlike traditional approaches that immediately pursue invariance, ér. intentionally captures both the causal pathway (Y + X) and environmental influences (E + X), providing rich features for subsequent abstraction by ÓH. We formally describe causal dynamic below:
Theorem 3 (Causal Dynamic and its Kernels). Given enviromments E, causul dynamic 2g = (X, Q K2) can be constructed under the following conditions:
1. The product causal space (Ωt H. K) is complete and separable.
5

---

2. The empirical measure Qn converges to the true measure Q i.e., su ((A)))((A)) و 0.
3. The causal kernel K (w,A)= = S K Ks(w', (wwww,' A) dQ(w') exists and is well-defined for all S C T. Further, the causal dynamic kernels are then given by: 1 = (K5" (w, :): (T), A E ). This theorem establishes the mathematical foundation for low-level representation learning. The conditions ensure: (1) the underlying probability spaces are well-behaved, (2) finite sample approxi- mations converge to the true distributions, and (3) the integration over empirical data produces valid kernels. The resulting causal dynamic kernels IK, capture how causal relationships manifest across all possible environment combinations. In addition, the integration over empirical distribution Q enables unified modeling of both perfect and imperfect interventions. Perfect interventions correspond to point masses in Q. while imperfect interventions use continuous distributions, providing flexibility for real-world scenarios where interventions are rarely perfect, inspired from (2).
We now define the low-level representation mapping ØL X' + 3 with 2 established in Thm. 3 We denote {Φr (w;)) as the collection of low-level representations for set of samples.
While causal dynamics operate on the raw input space to capture anti-causal relationships, causal abstraction further distills these representations into more abstract invariants. Previously, abstraction referred to the process of mapping complex, detailed representations to simpler ones that preserve only the relevant information 11 In our framework, causal abstraction specifically integrates over the domain of low-level representations to form high-level kernels that capture inviromment-invare relationships. This integration serves an information bottleneck, filtering out environmeett-specifee features while retaining label-relevant causal feature (also demonstrated by our empirical results).
Theorem 4 (Causal Abstraction and its Kernel). Let X' be the input space and H6x be its o-- algebra. Assume measure μ on the domain of low-level representations D 2,. Then, the high-level representation Zn = (V H: H)) can be constructed with kernel:
KS" (w. A)
Kg (w, A) dp(z)
(4)
DEL
The set of high-level causal kernels is then given by: 11 {K3 (w, A): SE ((T), Hxr The high-level representation mapping is defined as 0H Zz. + ZH with Zμ established in Thm. 4 We denote V =
as the resulting high-level representations for set of samples.
3.3 Objective Function of ACIA
ACIA's objective function bases on the theoretical results in Sec 3.2 Specifically, the kernel inddpen- dence property (Cor I) motivates the environment independence regularizer R1, while the intervention invariance criteria (Corl) guides the design of the causal structure consistency regularizer R2. The op- timization achieves the causal dynamics construction (Thm B) for Φ and causal abstraction (Thm for Φh, ensuring learned representations satisfy the anti-causal structure characterized in Thm
Let C be classifier and É be loss function. Our objective function of ACIA is defined as:
min max S/ ((Coor ob o (L) Y(W)) dIP (w) R1 + A2 CHLL CEEE
(5)
16 (x(oL(X(w))) di dF,, duy (v) / (H(OL(X(W)) (Pe,(wll) (uy duy (v)
R1 Σ
euiy Editt)
(6)
Ra E1l y wdd. [ 4,88
/ y d fest
(w, dy)
(7)
Remark 1: The minmax formulation in Eqn Eqn5aenntee eefforces worst-case robustness across environments.
This formulation is supported by out-of-distribution (OOD) generalization bound (Thm 7
Without it, the learned representations often fail to disentangle environmental factors effectively, as has been validated in prior work on invariant representation learning (IRM(5), Rex[27] VRex 27 )
Remark 2: Our two regularizers R, and R2 enforce key invariance properties essential for robust anti-causal representation l
2 Ri and Ha are also inspired by the invariant representation learning methods such as IRM connections are discussed in Appendix F3
5 Their

---

Ri enforces environment independence of representations. It measures the discrepancy between the expected high-level representttio across different environments, conditioned on the label Y. Minimizing Ri encourages: Jg dn (01.. ( (X(w))) dlP,, (wlw) & In th (X)()W dl og (wly) for all environment pairs (ei, ez) and labels promoting the invariance ΦH (x)( - E' Y.
R2 enforces causal structure consistency. It compares the expected value of Y given the high- ddK)) (w, dy).
level representation fy y dlP (ulow (X())) and that of Y under intervention: fy y/ d. (e:)
Minimizing R2 encourages the representation to be aligned with the true causal structure.
Theoretical Performance of ACIA: We also analyze the theoretical performance of ACIA, e.g.. convergence property (Thm 6), generalization bound in terms of sample complexity (Thm and interventional kernels (Thm in the anti-causal setting, and environmental robustness (Thm (which shows bounded distributional shifts between training and testing environments, providing robust anti-causal representations). All proofs are deferred to Appendix D
3.4 ACIA Algorithm Details
The complete ACIA algorithm include three components (see details of Alg Alg
i)
the low-level representation du by building causal spaces for each environment e E & and their product spaces, computing causal kernels Kg, g, and ultimately outputting the low-level causal dynamics Z,, = (X, Q, Kr.) as established in Thm
ïi) Alg 2 takes the set of low-level representations Oz L H outputted by Alg I and constructs
the high-level abstraction HH = (V H, KHH by integrating kernels across the low-level representation domain D2, as derived in Thm
iii Alg 3 integrates both algorithms by taking the outputs ΦL and ΦE as inputs, implementing the core optimization procedure defined in Eqn Particularly. it jointly optimizes both representations while enforcing environment independence through Ri and causal structure consistency through R2.
Practical Implementation: Ri is estimated via conditional distribution comparisons across en- vironments, while R₂ is approximated through alignment between predicted and interventional distributions. The verification of OOD guarantees (Thms. 7 and (
involves checking: (i) invariance of Φ across environments conditioned on Y. (ii) environment independence 6H (x)(X) 1 E|Y, and (iii) bounded distributional shifts between training and test environments.
Computational Complexity: The objective function in Eqn can be iteratively solved using stochastic gradient with time complexity of O ad log (t)) and space complexity of O(E d df), where is desired precision, 8 is failure probability, and d is dimension of the representation space.
4 Experiments
4.1
Experimental Setup
Datasets and Models: We test four datasets in anti-causal settings: Colored MNIST (CMNIST), Rotated MNIST (RMNIST), Ball Agent [9. and Camelyon17 17 [6]. In CMNIST and RMNIST, digit labels cause specific image features: colors and rotations (environment), respectively. Ball Agent is physical simulation environment where ball positions (continuous labels) cause pixel observations, with controlled interventions affecting object dynamies; Camelyon17 is a real medical dataset where tumor presence (label) causes tissue patterns in pathology images, with hospital-speciff staining protocols creating environmental variations. These datasets test various aspects of ACIA: discrete vs. continuous labels and perfect vs. imperfect interventions. Details of (building) these datasets are in Appendix E
The model architecture and hyperparameter settings of ACIA are in Appendix E.2
Evaluation Metrics: We use four metrics to measure predictive performance and causal properties.
1. Test Accuracy: Fraction of test samples correctly predicted by our predictor. 2. Environment Independence (EI): measures the degree to which high-level representations remain independent of environment-specife information while preserving label-relevant information. Specifically, we compute mutual information between high-level representations and environment labels, conditioned class labels. At the end, we weight them by class frequency and calculate their summation. Lower values indicate better environment independence.
3. Low-level Invariance (LLJ or R1): It quantifies stability of low-level representations across environments. We measure the variance of representations across different environments. At the end, calculate their average across feature dimensions. Lower values indicate greater invariance.
7

---

Table 1: Comparisons with basclines across four datasets and metrics.
CMNIST
RMNIST
Ball Agent
Camelyon17
Method
Acct Eq LLL
IR| Acct
EIL LLL
[R|
Acct EL LLL IR]
Acet
Eq
LLU
IR_
GDRO 44
92.00 1,85 0.80
0.91 63.00 16.03 4.10
1.53
66.00 1.04 0.69 0.75 58.00
1.32 0.87
0.83
MMD 29
94.00 1.22 1.73
1.13 92.00 6.88
15.62
0.69
68.50 1.13 0.87 1.82 60.00
4.43 2.16
1.12
CORAL 53
89.00 1.48 2.06
1.30 91.00 4.02 9.56
0.31
70.50
1.23 1.92 1.84
1.62 2.45
1.01
16
4.00 0.86
0.20 38.50 12.82 3.85
1.47
61.00
1.35 0.96 39.00.08
1.40
1.95
IRM
85.00 1.43 0.83 1.08 85.50 19.03
6.64
3.17
56.00
0.67 1.71
1.76
2.45
Rex 27
73.00 0.69 1.41 1.80 80.50 0.69 10.69
0.96
54,50 1.05 0.11 7.65 36.0990000.66
1.40
1.95
VREx 27
95,50 1.71 1.09 0.77 93.50 2.41 2.77
1.03
0.93 0.78 0.73 54.50 1.98 1.78
1.02
ACTIR 23
78.50 0.64 0.97 1.80 72.00 0.23 18.79
0.19
0.88 0.02 0.58 6.050 0.63
0.80
CausalDA 62
83,50 0.41 0.85 12.23 87.50 0.62
0.91
16.44
45.50
1.20 0.85 1.22
0.55 1.60 10.55
LEC
70.00
0.83 0.40 0.67 82.00 0.29
2.91
0.04
71..0 0.46 0.39 0.05 65.50
0.23 0.50 0.45
ACIA
99.20 0.00 0.01 0.02 99.10 0.00 0.03
0.01
99.98 0.52 0.03 0.03 84.40 0.28 0.42 0.43
4. Intervention Robustness (IR or R2): It evaluates model robustness under interventions by compar- ing the difference between observational and interventional distributions. Specifically, we first obtain probability confidence scores for original and intervened samples, and then calculate KL divergence between these distributions. Lower values indicate higher robustness.
Baselines: We compare AICA against 10 baseline methods spanning three main categories: (I)
Robust optimization methods: GDRO 44 optimizes the worst-group performance under distribution shifts. (2) Distribution/Dommii--nne
learning: MMD [29] minimizes distributional distances,
CORAL [53] aligns feature correlations, DANN [66 uses adversarial training, IRM 5 enforces invariant predictors, VREx [27] uses risk extrapolation with different variance penalties. (3) Causal representation learning methods: CausalDA [62] incorporates causal structure discovery for invariant representation learning. ACTIR (33 specifically targets anti-causal settings, and LECI [18 learns environment-wir causal independence through graph decomposition.
4.2
Experimental Results
4.2.1 Results under Perfect Intervention
Table [ comprchensively presents the comparison results of ACIA with existing baselines. These results validate our measure-theoreee framework's 's ability to capture and exploit anti-causal structures in synthetic and real-world settings. In particular, the results highlight several key findings:
Our ACIA performs the best and significantly outperforms baselines. For instance, on CMNIST and RMNIST, ACIA achieves accuracy of 99.00%+ perfect environment independence (0.00), almost perfect interventional robustness (0.02 and 0.01) and low-level invariance (0.01 and 0.03), significantly surpassing others baseline. On Ball Agent, our ACIA achieves 99.72% accuracy, and almost perfect levelevel invariance and interventional robustness. On the real-world Camelyon17, ACIA achieves the best test accuracy 87.00% and retains the underlying causal properties.
Causal dynamic construction (Theorem 3) is confirmed by the low-level invariance in our results. This matches the theoretical expectation of euvironment-indeeenddee feature learning.
Interventional kernel invariance (Corollary 2 is empirically validated via the intervention robustness score, implying the distinction between observational and interventional distributions.
Anti-Causal OOD generalization bound (Theorem 7 in Appendix) substantiated by the test accuracy improvements over the compared baselines across all datasets.
4.2.2 Results under Imperfect Intervention
Perfect/Hard intervention completely disconnects intervened variables from their causes, while imperfect/soft intervention modifies causal mechanisms and maintain partial original dependencies. This experiment aims to validate the effectiveness of our ACIA against imperfect intervention on the studied datasets, with details of constructing imperfect intervention discussed in Appendix E.33
Table 2 shows the results. We can see that imperfect intervention achieves similar results on the four datasets and metrics as perfect intervention. This supports theoretical claim that ACIA can effectively handle both perfect and imperfect interventions via its interventional kernel formulation.
4.2.3 Visualizing Learnt Representations
Results on CMNIST: F demonstrates ACIA's ability to learn representations that perfectly capture the anti-causal structure and predict the test data. (1) First panel: Low-level representations
8

---

Table 2: ACIA performance under imperfect intervention.
Dataset
CMNIST
RMNIST
Ball Agent IR
Camelyonl?
Metric
Acc
EI LLI IR Ace EI LLI IR
Ace
EI LLI
Acc
EI
LLI IR
Value 99.4 0.00 0.01 0.03 99.0 0.01 0.03 0.01 99.7 0.44 0.06 0.06 84.4 0.30 0.44 0.45
00
-27
Figure 3: t-SNE visualization of ACIA representations on CMNIST. From left to right: (1) Low-level representations show initial digit clustering with color influence; (2) High-level representations show improved digit separation; (3) Environment visualization demonstrate removal of environment- specific information; (4) Parity analysis reveals clear separation between even and odd digits.
show clear digit-based clustering while retaining certain environment information (note that some colored images are mapped to the digit cluster that they are not belonging to); 2) Second panel: High- level representations improve digit cluster separation with clearer boundaries; 3) Third panel: The environment visualization displays colored images from different environments are mixed, confirming information; and 4) Fourth panel: The parity visualization
the removal of environment-specd reveals how ACIA organizes digits based on their mathematical properties-the alternating pattern between even digits (orange) and odd digits (blue) confirms ACIA preserves meaningful numerical relationships while eliminating spurious color correlations. This organization aligns with findings from [24] showing that neural networks capture abstract number properties beyond visual features.
Figure 4: t-SNE visualization of ACIA representations on RMNIST. From left to right: (1) Low-level representations show digit clustering but with rotation influence; (2) High-level representations with better digit boundaries; (3) Rotation angle visualization shows uniform distribution across the representation space; (4) Digit complexity reveals semantic organization by structural properties.
See Figure 4 Similarly, (1) Low-level representations show clear digit- based clustering but keep certain rotation information; (2) High-level representations with more distinct boundaries; (3) The rotation angle visualization displays uniform coloring across the entire representation space, confirming successful abstraction of rotation-specific information; and (4) The digit complexity visualization reveals semantic organization where digits with similar structural properties cluster together. Simple (0,1.7): minimal stroke count (typically 1-2), more rotation- invariant features, and lower topological complexity; Medium (2,3,5): moderate stroke count (typically 2-3), mixed curves/lines, and intermediate visual density; and Complex (4, (4,6,899) most stroke count (3+), multiple curves/intersectinns, and higher topological complexity 65].
Results on RMNIST:
See Figure 5 (1) Low-level representations display position- based organiza- tion with considerable mixing between position values; (2) High-level representations show more pronounced position-based clustering with clearer boundaries, demonstrating improved abstraction of spatial information; (3) The intervention visualization displays a categorical distribution of interven- tion patterns (None, Single, Double, Multiple); their details are shown in Appendix E.1.3) revealing
Results on Ball Agent:

---

Figure 5: t-SNE visualization of ACIA representations on Ball Agent. From left to right: (1) Low- level representations display position-based organization with environmental mixing, (2) High-level representations show more pronounced position clustering; (3) Intervention visualization categorized by intervention patterns, and (4) Prediction error shows areas of high accuracy (green) versus areas requiring improvement (red).
how different intervention types affect the latent space structure; and (4) The prediction error visual- ization shows areas of high accuracy (green) versus areas requiring improvement (red), confirming that position-relevant information is preserved while achieving partial invariance to interventions.
ligh
-10
Figure 6: t-SNE visualization of ACIA representations Camelyon17. From left right: (1) Low- level representations show partial tumor/normal tissue separation, (2) High-level representations with improved class boundaries, (3) Hospital visualization demonstrates mixing of environment-specife features, and (4) Uncertainty analysis highlights regions of high confidence (green/yellow) versus regions requiring more evidence (red).
Results on Camelyon17: In Figure 6 (1) Low-level representations show separation between tumor and normal tissue samples, but with certain mixing; (2) High-level representations demonstrate more pronounced clustering with clearer boundaries between tissue types, particularly visible in the left- right separation; (3) The hospital visualization displays significant mixing between hospital sources despite their different staining protocols, confirming reduction of environment-speee and (4) The uncertainty visualization highlights regions where the model maintains high confidence (green/yellow) versus areas requiring evidence (red). More details in Appendix 1.1.4
information;
Conclusion
We presented ACIA, a theooree framework for anti-causal representation learning. ACIA provides: (I) a unified interventional kernel formulation that accommodates both perfect and imperfect interventions without requiring explicit causal structure knowledge; (2) novel causal dynamic that captures anti-causal structure from raw observations, together with causal abstraction that distills environment-invarte relationships; (3) a principled optimization framework based on min max objective with causal regularizers; and (4) provable out-of- dedssriiibutinn generalization guarantees that bound the performance gap between training and unseen environments. Overall, ACIA opens new directions for causal representation learning in settings where traditional assumptions do not hold. In future, we plan to generalize ACIA to handle more complex causal structures (such as minfounded-descendaee
mixed causal-anticausal scenarios (63]).
10

---

Acknowledgments
We thank the anonymous reviewers for their valuable and constructive feedback. This work was supported in part by the Cisco Research Award and by the National Science Foundation under Grant Nos. ECCS-2216926, CCF-2331302, CNS-2241713, and CNS-2339686.
References
[1] Kartik Ahuja, Ethan Caballero, Dinghuai Zhang, Jean -han-Chrittophe Gagnon-Audet Yoshua Bengio, Ioannis Mitliagkas, and Irina Rish. Invariance principle meets information bottleneck for out-of-distribution generalization. Advances in Neural Information Processing Systems, 2021.
[2] Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal repre- sentation learning. In International conference on machine learning, 2023.
[3] Kartik Ahuja, Amin Mansouri, and Yixin Wang. Multi-domain causal representation learning via weak distributional invariances. Artificial Intelligence and Statistics, 2024.
[4] Caridad Arroyo Arevalo, Sayedeh Leila Noorbakhsh, Yun Dong, Yuan Hong, and Binghui Wang. Task-agnostic crivacy-presenve representation learning for federated learning against attribute inference attacks. In AAAI Conference on Artificial Intelligence, 2024.
[5] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint wwtXiv:/9070028900 2019.
[6] Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi. Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Transactions on Medical Imaging, 2018
[7] Sander Beckers and Joseph Y Halpem. Abstracting causal models. In Proceedings of the aaai conference on artificial intelligence, pages
2019.
[8] Vidmantas Bentkus. On hoeffding's s inequalities. The Annals of Probability, 2004.
[9] Johann Brehmer, Pim De Haan, Phillip Lippe, and Taco S Cohen. Weakly supervised causal representation learning. Advances in Neural Information Processing Systems, 2022.
[10] Simon Buchholz, Goutham Rajendran, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, and Pradeep Ravikumar. Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems, 36, 2024.
[11] Daniel C Castro, Ian Walker, and Ben Glocker. Causality matters in medical imaging. Nature Communications, 1 11(1):1-0,, 2020.
[12] Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34:22405-22416 2021.
[13] Yongqiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian, Bo Han, and James Cheng. Under- standing and improving feature learning for out-of distrriiiot generalization. Advances in Neural Information Processing Systems, 36, 2024.
[14] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference machine learning, pages 1050-1059, PMLR, 2016.
[15] Juan L Gamella and Christina Heinze-Deml. Active invariant causal prediction: Experiment selection through stability. Advances in Neural Information Processing Systems, 2020.
[16] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. aomain-adversarial training of neural networks. Journal of machine learning research, 17(59):1-35, 2016.

---

[17] Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts, Causal abstractions of neural networks. Advances Neural Information Processing Systems, 34:9574-9558 2021.
[18] Shurui Gui, Meng Liu, Xiner Li, Youzhi Luo, and Shuiwang Ji. Joint learning of label and environment causal independence for graph out-of-distribution generalization. Advances in Neural Information Processing Systems, 36, 2024.
[19] Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
[20] Christina Heinze-Deml, Jonas Peters, and Nicolai Meinshausen. Invariant causal prediction for nonlinear models. Journal of Causal Inference, 6(2):20170016, 2018.
[21] Kasra Jalaldoust, Alexis Bellot, and Elias Bareinboim. Partial transportability for domain generalization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[22] Dominik Janzing and Bernhard Schölkopf. Semi-supervised interpolation in an anticausal learning scenario. The Journal of Machine Learning Research, 16(1):1923-198, 2015.
[23] Yibo Jiang and Victor Veitch. Invariant and transportable representations for anti-causi domain shifts. Advances in Neural Information Processing Systems, 211200722-0799. 2022.
[24] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9012-9020 2019.
[25] Achim Klenke. Probability Theory: A Comprehensive Course. Universitext. Springer, 2 edition, 2014. ISBN 978-1-4471-5360-3 doi: 10.1007/978-1144771 1-007997-114441-5311-00
[26] Murat Kocaoglu, Amin Jaber, Karthikeyan Shanmugam, and Elias Bareinboim. Characterization and learning of causal graphs with latent variables from soft interventions. Advances in Neural Information Processing Systems, 32, 2019.
[27] David Krueger, Ethan Caballero, Joern- -een--enrikk Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapo- lation (rex). In International Conference on Machine Learning, 2021.
[28] A Lemma. Mcasure theory details. Mathematical Notes, 2020.
[29] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5400-5409, 2018
[30] Chang Liu, Xinwei Sun, Jindong Wang, Haoyue Tang. Tao Li, Tao Qin, Wei Chen, and Tie- Yan Liu. Learning causal semantic representation for out-of-distribution prediction. Advances in Neural Information Processing Systems, 34:6155-6170, 2021.
[31] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018.
[32] Chaochao Lu, Yuhuai Wu, José Miguel Sernández-Lobate and Bernhard Schölkopf. Invariant causal representation learning for df-tsriibbttiin generalization. In International Conference on Learning Representations, 2021.
[33] Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, and Di Liu. Causality inspired representation learning for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022.
[34] Maggie Makar and Alexander D' Amour. Fairness and robustness in anti-causal prediction. arXiv preprint arXiv:2209.09422. 2022.
[35] Sergio Hernan Garrido Mejia, Patrick Blöbaum, Bernhard Schölkopf, and Dominik Janzing. Causal vs. anticausal merging of predictors. arXiv preprint arXiv:2501.08426 2025.
12

---

[36] Jovana Mitrovic, Dino Sejdinovic, and Yee Whye Teh. Causal inference via kernel deviance measures. Advances in neural information processing systems, 31, 2018.
[37] Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing domain gap by reducing style bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 8690-8699, 2021.
[38] Sayedeh Leila Noorbakhsh, Binghui Zhang, Yuan Hong, and Binghui Wang. (Inf2Guard): An (Information- / framework for learning (eracc--resrvving) representations against inference attacks. In USENIX Security Symposium, 2024.
[39] Junhyung Park, Simon Buchholz, Bernhard Schölkopf, and Krikamol Muandet. A measure- theoretic axiomatisation of causality. Advances in Neurul Information Processing Systems, 36: 28510-28540, 2023.
[40] Judea Pearl. Causality. Cambridge university press, 2009.
[41] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 1844-1247 2016.
[42] Mohammad Pezeshki, Diane Bouchacourt, Mark Ibrahim, Nicolas Ballas, Pascal Vincent, and David Lopez-Paz. Discovering environments with xrm. arXiv preprint arXiv:2309.1674 2023.
[43] Alexandre Ramé, Kartik Ahuja, Jianyu Zhang. Matthieu Cord, Léon Bottou, and David Lopez- Paz. Model ratatouille: Recycling diverse models for out-of-distribution generalization. In International Conference on Machine Learning. pages (1656-28669, PMLR, 2023.
[44] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint
2019.
[45] Nino Scherrer, Olexa Bilaniuk, Yashas Annadani, Anirudh Goyal, Patrick Schwab, Bernhard Schölkopf, Michael C Mozer, Yoshua Bengio, Stefan Bauer, and Nan Rosemary Ke. Learning neural causal models with active interventions. arXiv preprint arXiv:2109.02429, 2021.
[46] Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. arXiv preprint vrXivv1226..644 2012.
[47] Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612 634, 2021.
[48] Jia Shi, Gautam Gare, Jinjin Tian, Siqi Chai, Zhiqiu Lin, Arun Vasudevan, Di Feng, Francesco Ferroni, and Shu Kong. Lca-on-the-line: Benchmarking out-o--distribution generalization with class taxonomies. arXiv preprint 24X.v:2407.16667 2024.
[49] Chandler Squires and Caroline Uhler. Causal structure learning: A combinatorial perspective. Foundations of Computational Mathematics, 178 2023.
[50] Chandler Squires, Anna Seigal, Salil S Bhate, and Caroline Uhler. Linear causal disentanglement via interventions. In International Conference on Machine Learning, 2023.
[51] Bharath Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, and Gert RG Lanckrict. On integral probability metrics, \phi-divergennes and binary classification. arXiv preprint arXiv:090J.2698, 2009.
[52] Yongduo Sui, Qitian Wu, Jiancan Wu, Qing Cui, Longfei Li, Jun Zhou, Xiang Wang, and Xiangnan He. Unleashing the power of graph data augmentation on covariate distribution shift. Advances in Neural Information Processing Systems, 36, 2024.
[53] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer vision-ECCV 2016 workshops. 2016.
13

---

[54] Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoffrey J Gordon. Domain adaptation with conditional distribution matching and generalized label shift. Advances in Neural Information Processing Systems, 33:19276-19289 2020.
[55] David Tellez, Geert Litjens, Péter Bándi, Wouter Bulten, John-Melle Bokhorst, Francesco Ciompi, and Jeroen Van Der Laak. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Medical image analysis, 58:101544, 2019.
[56] Ramakrishna Vedantam, David Lopez-Paz, and David J Schwab. An empirical investigation of domain generalization with empirical risk minimizers. In Advances in Neural Information Processing Systems, 2021.
[57] Victor Veitch, Alexander D'Amour, Steve Yadlowsky, and Jacob Eisenstein. Counterfac- tual invariance spurious correlations: Why and how to pass stress tests. arXiv preprint 2XXv::106.00545. 2021.
[58] Edoardo Vesentini.
Variations on theme of carathéodory. Annali della Scuola Normale Superiore di Pisa-Classe di Scienze, 6(1):39-68, 1979.
[59] Julius von Kügelgen, Michel Besserve, Liang Wendong, Luigi Gresele, Armin Kekić, Elias Bareinboim, David Blei, and Bernhard Schölkopf. Nonparametric identifiability of causal representations from unknown interventions. Advances Neural Information Processing Systems, 2023.
[60] Binghui Wang, Jiayi Guo, Ang Li, Yiran Chen, and Hai Li. tion learning on graphs: A mutual information perspective. In ACM SIGKDD International Conference Knowledge Discovery and Data Mining, 2021.
representa-
[61] Xin Wang, Hong Chen, Zihao Wu, Wenwu Zhu, et al. Disentangled representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[62] Zihao Wang and Victor Veitch. The causal structure of domain invariant supervised representa- tion learning. arXiv preprint 228.:22088.06887, 2022.
[63] Zihao Wang and Victor Veitch. A unified causal view of domain invariant representation learning. arXiv preprint arXiv:2208. 21XX::2200800688, 2022.
[64] Liang Wendong, Armin Kekić, Julius von Kügelgen, Simon Buchholz, Michel Besserve, Luigi Gresele, and Bernhard Schölkopf. Causal component analysis. Advances in Neural Information Processing Systems, 36:32481-325222 2023.
[65] Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow. Har- monic networks: Deep translation and rotation equivariance. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5028-5037, 2017.
[66] Yuxin Xia, Peihao Yang, Zhen Xu, Yanbo Li, and Yong Xue. Neural causal representation learn- ing for high-dimensional time series. In International Conference on Learning Representations, 2024.
[67] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. A caurerr-basd framework for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14383-14392, 2021.
[68] Weiran Yao, Guangyi Chen, and Kun Zhang. Learning latent causal dynamics. arXiv preprint 220.02200..04888, 2022.
[69] Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. Towards a theoretical framework of out-of--dstributte generalization. Advances in Neural Information Processing Systems, 34:23519-233 23531, 2021.
[70] Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, and Zhiming Ma. Improved ood generalization via adversarial training and pretraing. International Conference on Machine Learning, pages 11987-11997. PMLR, 2021.
14

---

[71] Han Yu, Xingxuan Zhang, Renzhe Xu, Jiashuo Liu, Yue He, and Peng Cui. Rethinking the evaluation protocol of domain generalization. arXiv preprint arXiv:2305.15253, 2023.
[72] Binghui Zhang, Sayedeh Leila Noorbakhsh, Yun Dong, Yuan Hong, and Binghui Wang. Learn- ing robust and
representations via information theory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 22363 22631,, 2025.
[73] Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, and Zheyan Shen. Deep stable learning for out-of-distribution generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5372-5382, 2021.
[74] Dawei Zhou, Nannan Wang, Xinbo Gao, Bo Han, Xiaoyu Wang, Yibing Zhan, and Tongliang Liu. Improving adversarial robustness via mutual information estimation. In International conference on machine learning, 2022.
[75] Kaiyang Zhou, Yongxin Yang. Timothy Hospedales, and Tao Xiang, Deep Jomain--dversaan image generation for domain generalisation, In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 13025-13032, 2020.
[76] Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang, Learning to generate novel domains for domain generalization. ECCV, pages 561-578. Springer, 2020.
[77] Sicheng Zhu, Xiao Zhang, and David Evans. Learning adversarially robust representations via worst-case mutual information maximization. In International Conference Machine Learning, 2020.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state the paper's claims about developing two-level representation learning framework for anti-causal settings, addressing limitations existing methods, and demonstrating superior performance on multiple datasets. All claims are supported by theoretical results and experimental evidence in Sections 3 and 4.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: This paper discusses computational complexity (Section 3.4 and Table 9). Ap- pendix sections also discuss theoretical constraints and practical implementation challenges.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
This paper provides full assumptions and complete proof's for all theoretical re- sults (Theorems 3.1.3.5 and Corollaries 3.1-3 133.),, with detailed proofs available in Appendix B. All assumptions are clearly stated and cross-referenced in each theorem statement.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
This paper includes detailed information on dataset construction (Section 4.1 and Appendix D), model architectures, hyperparameters, and training procedures (Section 4.2 and Appendix D.5) sufficient reproduce the main experimental results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
17

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We have made code and data openly accessible through github repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.ccpputt wttpcccccppccccccee copollccggiieee gwwdggu/de/ooeoooo Cotp://ooeSSubmmmeee iobbbmmssiinnl ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Section 4.1 and Appendix D specify all training details including data splits, hyperparameters (learning rates, regularization parameters), optimization methods (Adam optimizer), model architectures, and batch sizes for all experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
The statistical significance of the experiments is discussed in the Appendix.
Guidelines:
18

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errоrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: We did not provide explicit information about the type of compute workers, memory requirements, execution time needed to reproduce the experiments. The nature of the algorithms' implementation is regardless of this information.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https http:: httpar/:///e....eeeee http://pww....pppp/cccccccccccaa
Answer: [Yes]
Justification: This research approach and methodologies presented in the paper align with the NeurIPS Code of Ethics, with no apparent violations terms research conduct or applications.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special cireumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
19

---

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed potential positive societal impacts and negative societal impacts of the work performed in Appendix H
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups). privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not release models or datasets that pose high risks for misuse, as it focuses on causal representation learning methods demonstrated on benchmark datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The baselines and models that have been used in this work cited.
Guidelines:
20

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thcode. com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: All the new code libraries, and models will be provided with complete docu- mentation in the final version.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
21

---

Justification: No IRB approvals were required as the paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
The core methods this research do not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hutpe://eeeupss.cco/oootrrrrrr/p for what should or should not be described.
22

---

Appendix
More Related Work
24
A.1
Distribution/Domale invariant Learning
24
A.2
Intervention baasttdooo--eddddd--e Causal Learning
24
A.3
Structure- Causal Representation Learning
25
A.4
Comparison of Prior Information Requirements
25
B
Properties
26
1
Properties of Kernel
................................
26
B.2
Properties of Product Causal Space Sub-o-algebii
................................
26
B.3
Properties of Anti-Causal Kernels
27
B.4
Properties of Causal and Anti-Causal Events
27
Proofs
28
C.1
Proof of Theorem
(Anti-Causal Kernel Characterization)
28
C.2
Proof of Corollary
(Independence Property of (Existence and Uniqueness of Interventional Kernel)
Kerne))
29
Proof of Theorem
29
C.4
Proof of Corollary
Interventional Kernel Invariance)
30
C.5
Proof of Corollary
Properties of Interventional Kernels)
C.6
Proof of Theorem
Causal Dynamic Construction)..
32
C.7
Proof of Theorem
Causal Abstraction Construction)
34
C.8
Proof of Theorem
(Existence and Uniqueness of Product Causal Kernel)
35
D
Theoretical Performance of ACIA
38
D.1
Theorem 6 (Convergence ACIA)
38
D.2
Theorem (Anti-Causal OOD Generalization Bound)
39
D.3
Theorem (Anti-Causal OOD Gap w.r.f. Kernels)
40
D.4
Theorem (Environmental Robustness)
41
E
Experimental Setup and Results
43
E.1
Datasets for Anti-Causse Causal Learning
43
E.1.1
Colored MNIST (CMNIST)
................................
43
E.1.2
Rotated MNIST (RMNIST)
45
E.1.3
Ball Agent
................................
45
E.1.4
Camelyon17
46
E.2
Model Architecture and Hyperparameter Setting
E.3
Imperfect Intervention Implementation]
48
E.3.1
Imperfect Intervention Datasets Construction
48
E.3.2
Results on Imperfect Intervention
49
Computational Efficiency
50
Discussion
50
F.1
Why Measure Theory"
50
Why Kernel Formulation?
50
F3
Connection to Existing Methods
50
F44
Anti-Causal Toy Example vs. Kerneis
F.5
Anti-Causal Representation Learning, Robustness, and Privacy
52
G
A
Concrete Example to Illustrate ACIA
53
G.1
Step 0 (Problem Setup): The Colored Digit Example
53
G.2
Step 1: Define Causal Kernels for Each Environment
53
G.3
Step Construct the Low- Representation
G.4
Step 3: Construct the High-Level Representation
54
G.5
Step Joint Optimize on Loss and Regularizations
G.6 Step 5 (Outcome): What ACIA Achieves?
55
H
Impact Statement
56
23

---

Table 3: Summarizing Causal and Invariant Representation Learning Methods
Requimments
Nongstametric
DieriaateeD-emmit-innee Loaring
Domain adapaation N
13
1.2A00T
10
FRM
DOMAINEED
StatteNe
12
X
SagNee
SWAD
FACT
Exaluationn Prrtecul
113
K
XRM
FH
ReAT
A1A
Rex
to Sputious
53
Information Buttlared
Camalla
Transoctuble Kep
Sirecture bazed Causal Representation Leorting
Causel Disemtagllmmet X3
ICP for sunlincae
Active
CSG
LECT
Seporation. & Risk
Intervantionhuseel Cansil Lomning
Nouperameiic ICR 10
Mixing
Weakly sipervised
CIRI
13
KKK
LCA
ICA
ACTA
A
More Related Work
In Table we broadly summarize causal and non non-causa. con-cluaal invariant representation learning methods.
A.1
DistributionDDomain--vvaaarr Learning
Early non-causal methods like (C-)ADA 31 and DDAIG 75 foouueed on domain adaptation
strategies, while more recent works such FeAT [33 and AIA have developed sophisticated
objective functions for distribution shift robustness. Domain adaptation methods [54]. L2A-OT 76
ERM 56 DOMAINBED 19,, Adversarial and Pre-training 70.. StableNet 733. SagNets [3]
SWAD 12 Theoretical Framework [69] FACT [67], Evaluation Protocol (7I). Ratatouille 4
XRM IRM [5]. Rex (27]. CI Spurious 577, Information Bottleneck [. CausalDA 62 and
Transportable Rep 23 also fall under this category as they aim to learn representations invariant
across different distributions or domains.
A.2 Intervention-basee Causal Learning
Foundational works in the causal domain include Nonparametric ICR (59 which jointly learns en- coders and intervention targets with SCM-based structures and General Nonlinear Mixing [00] which addresses non-linear relationships in latent spaces, which model causal effects through interventions.
Weakly supervised methods [9] iCaRL. (32] CIRI. 33 and I.CA 48 also leverage interventions for learning causal representations. ICRL
also falls under this category. Weak distributional
24

---

Name
Symbol
Name
Symbol
Environment
Product sample space
51
Se Re
Set of environments
Product o-algebra
Ma & Me.
Environment sample space
S2
Product probability measure
P.
r-algebra e2,,
He,
Product causal kemeel family
K {Ks s :: (T) X
Probahility measure
P.
Input space
Causal kernel
K.
Low-level latent space domain
Dza Dzy
Environment causal space
(94 Vu,,...
Rr. Fr, K.,
High-level space
Causal product space
(52, H I.K..)
Label space
Sub-o-algebra
Low-level representation
0L :. - D2,
Index set
Ve UTee
High-level representation
0y Dzs + Dzu
Interventional kemel
Kg K..... (ws,
Predictor
C:Dza
Intervention mcasure
Q(-1-)
Full predictive model
== 00 oz
Marginal measure on y
JyY
Loss function
{: y B
Causal dynamic
2g = (2C, (.)
Enviromment independence reg.
R1
Causal abstraction
2h = (V H. K.H)
Causal structure alignment reg.
R2
Set of low-level kernels
k1. =K (w. A]}
Regularization parameters
A Ag
Set of high-level kernels
= = (w, A)}
Conditional mutual information
1(X; = Y)
Table 4: Key Notations in Anti-Causal Representation Learning Framework
Table 5: Comparison of prior information requirements across causal representation learning methods
Method
Causal Structure
SCM Knowledge
Intervention Type
ACTIR 23
Anti-causal Y - X +- E
Variahle roles only
Perfect only
CausalDA 62
DAG structure
Variable types
Perfect only
L8CI
Partial connectivity
Variable relationships
Perfect only
ACIA
Anti-causal Y X < E
Variable roles only
Both perfect and imperfect
invariances 3 considers perfect interventions for single-node, and addresses multi-node imperfect interventions by identifying latent variables whose distributional properties remain stable Indepen- dent Component Analysis (ICA) [64] focus on unsupervised identification of latent causal variables through component analysis, and operate the discntanglement through taxonomic distance measures and graph- analysis. AIT [45] builds on SCMs with explicit DAG assumptions, and primarily focuses standard causal direction.
A.3 Structure-based Causal Representation Learning
Methods like DISRL 62 and Causal Disentanglement 50
explicitly model causal structures.
Foundational works like ICP [41] and its nonlinear extension 20 as well as CSG [30] and LECI [18]
which focuses identifying causal subgraphs while removing spurious correlations, also incorporate causal structure but often require explicit Directed Acyclic Graphs (DAGs) or focus on identifying
causal subgraphs. KCDC (36] employs kernel methods primarily for causal discovery and orientation, and focuses on statistical independence tests through kernel measures. Anti-causal separation and risk invariance (34] inputs are generated functions of target labels and protected attributes. They use conventional causal modeling with DAGs and do-calculus.
A.4 Comparison of Prior Information Requirements
Table 5 compares the prior knowledge requirements across state-of-the-art causal representation learning methods. Our results demonstrate that despite requiring less prior information, ACIA outperforms these methods.
"Variable roles only" refers knowing which variable the target (Y), observations (X), and environmental factors (E).
"Variable types" means knowing the data type of each variable, such as whether it is binary, categorical, or drawn from a specific noise distribution.
"Variable relationship" refers to knowing the causal structure (i.e., the directionality in the causal DAG) among the variables.
25

---

B Properties
B.1 Properties of Kernel
For each S (T), the kernel Ks 2x H + [0,1] extends from the component kernels as follows:
1. For measurable rectangles A, Aj with A, E He, and A, Hey and w (w), w,) E S2
Ks A4 x Aj) = Ke, A.) Key (W), Ay)
(8)
2. For general measurable sets A H. by the Carathéodory extension theorem:
Ks(w, A) E (w)
(9)
where Hyg is the sub-o-algebra corresponding to indices in S.
B.2 Properties of Product Causal Space Sub-o-algebra
Intuition: Sub-o-algebras capture partial information from subsets of environments. These proper- ties ensure our hierarchical structure well-behaved and consistent across different environment combinations. The following propositions formalize these characteristics.
Proposition (Properties of alg-br-ag Let P. K)) be product causal space and Hes be a sub-o-algebra for C T. Then: (i) Hs H for all S T (i) S S2 T, then Ms. S Hes, (ii) H6- H
Proof. We prove each statement separately:
(i) By construction, Hs is generated by measurable rectangles A, x A, where A; He and Aj E He corresponding to events in the time indices S. Since H = He @ He, is the product o-algebra that contains all measurable rectangles, we have Hgg H by definition.
(ii) Let S1 S S2 S T. Any measurable rectangle generating M corresponds to events in time indices from S1. Since S S S2, these same rectangles also in the generating set of .ys, By the minimality property of o-algebras, Ws. C Wss
(iii) When S = T, the generating rectangles of Hys include all possible measurable rectangles from Ho and Hey that can formed from the complete set of time indices. These rectangles generate H. He, 8 He,, so HT H..
Proposition (Probability Measure Restriction). For any S T, the restriction Play of the product probability measure to Hy is well-defined probability measure, and for S1 S S2 C T:
P Plaes (A) H3) for all A E Hs.
Proof. First, we establish that Plas is well-defined probability measure. Hs is o-algebra by construction, PP (A) = P(A) for all A Hs. Play (S2) P()) = 1, and Plas inherits countable additivity from IP.
To prove the consistency of the restrictions for S1 S S₂ S T, let E Hs. Since S S S2, by Proposition iii,, we have A E Xss.
For any measurable rectangle in time indices Sy:
A, A, where A, E H., and A, E He, corresponding events
Plae, (A) = P(A) Pe, (A.) P (Ag) (by definition of product measure) Plays, (A)
This equality extends to all sets in Hs, by the uniqueness of measure extension. Therefore:
Me, (A) Pl Plas, (A) for all A E Hs,
26

---

Proposition 3 (Monotonicity of Information). For S C S S T and any Ht -f-masuurblle random
variable X:
| |5 = [X
Proof. Let S1 C S2 S T and be any H measuraabel random variable. By Proposition 1 [i), have H's S Xy This nested relationship between the sub-o-algebras is crucial for applying the tower property of conditional expectation. By the tower property of conditional expectation, for nested o-algebras &1 92 C F, have:
E
E
Applying this to our sub-o allgebaa Ass C S2 S :t:
E Hos [s,]
This result has information theorrttc interpretation: conditioning on larger o-algebra (Hs,) provides more refined information than conditioning on a smaller one (Hss ). The tower property shows that the expected value of this refined information, when further conditioned the smaller o-algebra, equals the direct conditioning on the smaller o-algebra.
B.3 Properties of Anti-Causal Kernels
Based on Theorem | for a more precise characterization of the causal kernel that underlies these event properties, we provide the following remark relating kernels conditional probabilities in different settings.
Remark 1 (Characterizaatoo of Causal Kernels). For a causal space H, IP, K)) the relationship between causal kernels and conditional probabilities characterized as follows:
1. For any S E (T), the causal kernel induces conditional probabilities:
Ks(w, A) = EE(1 Hs)(w)
where 1 is the indicator function of set A.
Regular conditional probubilities (i.e., versions that are measurable in w) arise as a special cuse:
Ks(w, A) (PA HWs)(w)
3. In anti-causal structures:
Ks(w, A) = IP(P Y EES)
where y is the -compponett of
B.4 Properties of Causal and Anti-Causal Events
We can understand the causal and anti-causal events kernel differences as follows:
Proposition 4 (Causal and Anti-Causal Event Properties). For a product causal space with kernel Ks and any measurable event A M::
For a causal event A: K (w, A) + (P(A) for all w E Ω
For an anti-causal event A: Ks(w, A) = Ks (w, A) for all w e
where CSE (T)). Based on Def in our settings, an intervention a measurable mapping Q(-1-) Ht x Ω + 1]. Hard intervention is Q(Aww)) P(X e Aldo(Y = z/)), and soft intervention is Q(AWw)) = P(X E A|Y = y',E' E S) where y' denotes the -commonnent w'.
Proof. We establish the distinct properties of causal and anti-causal events through their behavior under the causal kernel.
Part 1: Causal events
27

---

Let A be causally dependent on variables in Vs. By definition of causal dependence, there exist w, w' E Ω such that Ks (w, A) [s(w', A). Since ((A) Jn Ks(w. A) dl(ww) is fixed constant, we cannot have (s A) P(A) for all Q. Therefore (s, A) P(A) for some w.
Therefore, K3 (w, A) # ((AA) for some w E S2 which is consistent with the causal structure A. Since ((AA) Jo [s( A) dl) is weighted average of (s(w, A) over all w, cannot have Ks(w, A) ((A) for all w.
Part Anti-causal events
Let A E H be an anti-causal event, and let U S S (T)). We need to show that Ks (w: A) = Kyyu (w, A) for all w E S2. By definition, anti-causal event one whose probability is invariant to certain interventions. Specifically, removing a subset U from the conditioning information S does not change the kernel's value if A is anti-causal with respect to U.
From the definition causal kernels:
K (w, A) E (s)( (syu A) = E Hsyul
For anti-causal event A, the information in My (corresponding to indices in U') has causal influence on when conditioning on Avs Formally, this means:
A IL Hi
Aey
By the properties of conditional expectation under conditional independence:
E E[1A
= E E
Therefore:
Ks(w, A) = K (s for all w Ω
This equality demonstrates that anti-causal events exhibit invariance with respect to certain subsets of the conditioning information, reflecting their position in the causal structure.
C Proofs
C.1 Proof of Theorem (Anti-Causa) Kernel Characterization)
Theorem 1 (Anti-Causal Kemel Characterization). For an anti-causal structure with arbitrary feature space X, label space Y, and environments E, the causal kernel satisfies:
Ks(w, A) / P(X
E P
= E S) (y)
(1)
where jy is the marginal measure on y.
Proof. In the anti-causal structure Y + + + E, the joint distribution factorizes as:
P(X, Y, E) PX))))))
By the local Markov property of causal graphs, node is conditionally independent of its non- descendants given its parents. In our case, E is not a descendant of X, and is the parent of X, so: X 1 E Y.
For any measurable set A E Axx, the causal kernel is defined as: (s A) = E(1A (w))
By the properties of conditional expectation and the definition of sub-o-algebras Ms. for any with Y -cmomponent y:
Ks(w, A) P(X AA y, E.E
By the conditioning formula and the conditional independence X IL E Y:
P(X y P A Y ==y)
28

---

Integrating over the label space with respect to the marginal measure jy:
Ks(w, A) / P(XEA = y E. E S) (uy(
=f P(Xe duy (v) (by conditional independence E Y)
C.2 Proof of Corollary (Independence Property of Anti-Causal Kernel)
Corollary (Independence Property of Anti-Causal Kernel). In an anti-causal structure, for any w, w' E Q with identical Y -component and for all A E Mx, B Hy, S (T)):
Ks(w, {AlB)) Ks(w,, (AAB))
Proof. We demonstrate the independence property of kernels anti-causal settings. Let us define the conditional kernel (s(w, {A(B)) as:
Ks(w, {A(B}) Ks(w, A (Y E BB), Ks(w, (Y E B})
when Ks(w, {Y B}) > 0
For any w,w' E Ω with the same Y-component y such that E B, and for any A E Hx and B E Hy:
Ksw (AIB)) P(X E A E B,Y Ey
By Bayes' rule and the fact that E B (otherwise the conditional kernel is undefined):
Ks(w, {A = F B Y = EE
P Y = E,
P(X Y = y, 1
1
= P(XE A Y = y, E. S)
By the anti-causal structure with d-separation X 11 E Y:
P(XE =, v, E
E A Y
Therefore, for any w, w E f2 with the same -comppnent:
Ks(w, {A B)) Ks(w', (A P(X
Y
This holds for all A E Nx. B E Myy and S E (T), forming the kernel independence property. D
C.3 Proof of Theorem 2 (Existence and Uniqueness of Interventional Kernel)
Theorem (Interventional Kernel), Let (Ω, H. K) be product causal space. For any subset Se (T) and intervention Q H x l2 + [0, 1]. there exists unique interventional kernel:
x K
(w. A)
( Ks(w, du du'))(A)(o))
provided that the integral is Lebesgue integral w.r.t. the measure induced by Ks(w,)) (2,H) H). In addition, Ks(w,)) .) is a-finite for each w E Q and ((A-)) is H -measurabll for each E H.
Proof. We proceed by establishing the existence of the interventional kernel and special case for hard interventions. For any fixed w E Ω and A E H. define:
K x (w. A) = Ks(w, de' ((AWw))
29

---

The integrand w/ + Q(Al')) is meeauuabbl for each fixed E E by our third assumption.
The integral exists Lehesgue integral with respect to the measure induced by (s(w,)) on (Ω, H) due to our first assumption. Since (s( is a probability measure and 0 I (AA(a)) S for all w' Ω we have:
0 R / (s(w,,dw))(4() dd'))((A))))) & J Ks(w, dw''
For hard interventions where Q(A(w")) = ((A) is constant with respect w':
Ks kdo(x.0) (w, A)
J, Ks(w, dw' du())((A)))) - -// Ks(w, dw' (Q(())
- Q(A) / Ks(w, dw') - Q(A) -144)-1-2 = ((A)
Thus, for hard interventions, the interventional kernel equals the intervention distribution, illustrating how our definition encapsulates both intervention types.
C.4 Proof of Corollary 2
(Interventional Kernel Invariance)
Corollary 2 (Interventional Kernel Invariance). In anti-causal structure, interventional kernels satisfy the following invariance criteria:
1. Kg
( {Y E B}) Ks(w, (Y E BB) all measurable sets B y, meaning intervening on X does not change the distribution of Y.
kgal (w,tX {X E A}) + Ks(w, {X E A}) for some measurable sets A C. X, meaning intervening on Y changes the distribution of X, which is characteristic of an anti-causal relationship.
Proof. We will prove each criterion separately, using the properties of interventional kernels and the structure of anti-causal relationships. We need to prove that for all measurable sets B S ):
KS
kdo(X) (ws (Y E BJ) = Ks(w, (Y E B})
In the anti-causal structure Y + X + E, intervening on X (through do(X)) breaks the incoming arrows to X, leaving Y unaffected. By the rules do-calculus, we have:
P(Y|do(X z)) P(Y)
This is because Y is not a descendant of X in the modified graph where incoming arrows to X are apply the interventional calculus:
removed. More formally,
P(Y|do = x)) = Σ PoY|do(X = x), E = e) = eldo(X= = x))
Σ -))(E= ()
(by independence after intervention)
P(Y)P(E = (since Y IL E in anti-causal structure)
= P(Y) Σ P(E = e)
Translating this our kernel notation, for any w E Ω with Y -coopponent and any measurable set
B y:
do() (w, (Y E B}) = P(Y E B F (for with Y Y-component y)
g
= P(Y E B) (by do-calculus as Y is not descendant of X)
= -KsW,, (Y E B))
30

---

The last step follows from the definition of the causal kernel in terms of conditional probabilities. This proves criterion I. We need prove that there exist measurable sets C X such that:
Kg K (w, {X E A)) + Ks(w, {X E A})
In the anti-causal structure Y X E, intervening Y (through do(Y)) breaks the causal link
Y + X. This fundamentally changes how X is distributed. Under the intervention do(Y y):
/ P(X E Aldo(Y = v). = e) = e|s)
P(X E AYdo(Y = 3).
P(X E = = (since Y no longer affects [
In contrast, without intervention:
P(X E A|Y v,E P(X E A|Y v, = elS)
These are not equal in general. because in the observational setting, X depends on both Y and E, and in the interventional setting, X depends only on E.
For this to be equal for all C X, we would need P(X = y, = P e) for all e, which contradicts the basic structure of an anti-causal relationship where Y is cause of X. Therefore, there must exist some measurable set C X' such that:
Kg Kdo(Y) (w, {X E A}) + Ks(w, {X E A))
This proves criterion Together, these criteria provide characterization of anti-causal structures: intervening on X doesn't change Y (the cause), while intervening on Y does change X (the effect).
C,5 Proof of Corollary (Properties Interventional Kernels)
Corollary (Properties of Interventional Kernel), Let kig do( (w, A) be the generalized interven- tional kernel in an anti-causal structure. Then:
Well-definedness: For Kg and QQ(l-)) measurable on (I):
f (s(w,,d') 1
and the integral exists as a Lebesgue integral.
Uniqueness: The kernel ks dolx.0) unique up to P-null sets, where IP is the product measure.
3. Consistency; For hard interventions where Q(Aww)) = Q(A):
dotx 3) (w, A) Q(A)
4. Structure Preservation: For the anti-causal structure Y + X + E: K (w, A) P(X E A|Y E.E
where y the Ycoomponent of and A E H.
The properties of interventional kernels provide the foundation for understanding how causal relation- ships manifest in anti-causal structures.
Proof. We prove each property separately:
Property 1: Well-definedness
For Ks and Q(+I.) measurable (2, HD,H), w' b> K Ks(w, dw') ) is measure for fixed w/ Q(A(w))
-meessurable for fixed A, and @(Aww)) E [0, since it's probability. Therefore:
/ Ks(w, du dw'dd/)/ (A(W)))) < Ks(w,dw')
31

---

The integral exists as Lebesgue integral by Tonelli's theorem, since both functions are non negative and measurable.
Property 2: Uniqueness
Let K1 and K2 be two versions of the interventional kernel. For any A H and B H:
A) A) / / Ks (ww,,dw)((A(a))) dP(w)
Ki B
K2(w, dlP(w)
By the Radon-Nikodym theorem, these must agree IP-almost everywhere since they define the same measure via integration.
Property 3: Consistency
For hard interventions where Q(AWw") = Q(A):
KS ( (w, A) / Ks(w, de' d(d))(AW)W))) = - Ks(w, du' (Q(A)
= ((A) / Ks(w, du') = (AA) = Q(A)
This confirms hard interventions correspond to setting the kernel equal to the intervention distribution.
Property 4: Structure Preservation
In the anti-causal structure Y - X + E, know by d-separation in the graph: X TT E. This implies for any w with Y -cooponeen and E-component in S:
Q(A[w')= P(X A|Y E
where y' is the Y componeent of w'. Using the properties of conditional expectation and d-separation:
K kdo((200) (w, (w, A)
Ks(w,dw))P(X E A|Y =
E AY (
Since Ks(w,.) gives highest weight to w/ values with Y -componnent equal to y (the Y-component of w) in anti-causal settings, and using the conditional independence X 11 E Y:
KS ddo( (w, A) P(X A|Y y, E S)
where is the Y -coomponent w. This reveals that the interventional kernel preserves the anti-causal
structure by maintaining the conditional independence relationships imposed by the causal graph. _
C.6 Proof of Theorem 3 (Causal Dynamic Construction)
Theorem 3 (Causal Dynamic and its Kernels). Given environments E, causal dynamic Zμ
(X' Q K2) can be constructed under the following conditions:
1. The product causal space H. IP, K) is complete and separable.
The empirical measure Qn converges to the true measure Q i.e., sup (A)-
3. The causal kernel KZ (w, A) = Jor Ks(w', A) ddQ(w)) exists and is well-defined for all S C T.
Further, the causal dynamic kernels then given by: IK,,, (K (w, A) S 9 (T), A E H).
Proof. We will prove the existence and well-definedness of causal dynamics by constructing the structure explicitly and verifying its properties.
By assumption (1), the probability space (, IP) is complete and separable. This means Ω is complete separable metric space, H is the Borel o -lagebra on Ω and IP is probability measure
32

---

(2, For a complete separable metric space, we can define a metric d on Ω. For any w. E S2 define:
d(w,w) w²)
||0
w/ 12
IET
where Wt and w/ represent the components of and w' corresponding to time index t E This metric induces the e-ball:
B((w) = {wl E Q d(w, <c
By assumption (2), the empirical measure Qn converges weakly to the true measure Q
sup [Q) - Q(A) و.6o AEX
This is stronger form of convergence than weak convergence (denoted Qn + Q. and implies that:
f - -f
for all bounded, continuous functions f. The empirical measure Qn typically defined as:
(A) Σ IA(v:)
where v, Vi are observed data points. By the Hlivenko-Cannten theorem, this uniform convergence holds almost surely when V, are i.ddd samples from Q By assumption (3), for each C T. we define the causal kernel:
KW (w, A) = [ Ks(w', A) A) dQ(w')
This integral exists and is well-defined since Ks(w', , A) is measurable w' for fixed A (by the properties of causal kernels), (s(w', A) E [0, 1] (since it's probability). and Q is probability measure.
We can verify that K (w, A) is indeed causal kernel:
1. For fixed w. A K (w, A) is probability measure.
K (w, 0) [ (s(w',0) d)d((")) = =/ odQ(ww) 0
/ Ks(w', (s(w),)) (Q) = 1 IdQ(ww)
K3 (w, (w,
For disjoint sets {
2
Ai
/ Ks
w' U. dQ(w))
i=1
i=1
|
Kslw', A²) (QW())))
1=1
8
Σ
Ks(w', A.) Σ
KT (w)
2. For fixed A, w + K3 (w, A) is Fss-mcsuraalc. This follows from the fact that constants are measurable, and K3
k.".w.. (w, A) does not depend on in its definition.
We need to show that for any c > 0, there exists 5 > such that if d(w, < then:
K Et (w,
K3 (w/ ,))l <e
33

---

where 1lr denotes the total variation norm. For our kernel - (w, A) for Ks(w", ,A) dQ(w").
By construction, K (w, A) Joy K Ks(w', A) dQ("") does not depend since are integrating over w' with respect to measure Q, and w does not appear in the integrand. Therefore, for any wi, w' E
KS (w, A) K5 (w', AA
for all w, w E Ω and A E H. This implies || | (w, -) - K (w', IV = <c..
for any 0 and any choice of i 0. Finally, define the causal dynamics
21 (X, (K)
where Ik, = [K5 A) : ((T), A X) is the family of causal dynamics kernels. This construction satisfies all requirements of the theorem:
It is built on a complete, separable probability space
It incorporates empirical measures that converge to the true measure
The kemee 2 is well-defined for all $ C
The family of kernels is uniformly equicontinuous in total variation norm
Therefore, the causal dynamic 2z (X, Q Kg) well-defined and properly constructed.
C.7 Proof of Theorem (Causal Abstraction Construction)
Theorem 4 (Causal Abstraction and its Kernel), Let X' be the input space and Hxx be its a- algebra. Assume measure μ on the domain of low-level representations D Dzy, Then, the high-level representation Zn = (V 1HH can be constructed with kernel:
Kg (w. A)
KS (w, A) dj((z)
(4)
D2L
The set high-level causal kernels is then given by: 1H [K 21 (w, A) S (T)), Ax).
Proof. We need to construct a high-level representation Zμ (V Kи) with appropriate kernel properties. We proceed step by step.
Given input space X, domain of low-level representations Dz,, and o-finite measure on Dz,, we define the high-level kernel K Zu
as:
I Dz,
KS Aw
K A)
where K-5 (w, A) is the causal dynamics kernel for specific low-level representation E Dz, S T is a subset the index set, and A E Hx a measurable set in the input space.
We first establish that this integral is well-defined:
(i) Measurability: For fixed w and A, the function = + K (w, A) is measurable with respect to the olaleerra on Dz,- This follows from the construction of K Z2 in Theorem 3 where we established that causal dynamics kernels are measurable functions.
(ii) Boundedness: Since K (w>, A) represents probability, we have K (w, A) < for all w,w, A. Therefore, the integrand is bounded.
(iii Measure space: (Dz,, . Dz, 11) is o-finite measure space by assumption, where FD. is the appropriate alaggebra on Dz,,-
By Lebesgue's 's dominated convergence theorem, the integral exists and is well-defined.
Next, verify that K3" (w, A) satisfies the properties of kernel:
34

---

1. For fixed w, the mapping A + K5 (w, A) is probability measure:
(a) Non-negativity: Since K (w, A) 2 0 for all 2,w, A, and is positive measure, the integral K (w, A) 2 0.
(b) Empty set:
K5 (w, dp(()) / 0du(z) =
K 2 (w,0) =
DEL
DEz
(c) Total measure:
K5 (w) f
Kgg (w, = J 1dp(z) p(DzL)
Dzz
y is probability measure, then (Dz,,) 1. If not, we normalize it by defining:
K (w, A)
K (w, A)
K (w,
assuming (Dzz)) < oc. For generality, we assume f is already normalized so that ((Zz,)) =.
(d) Countable additivity: Let {1 be sequence of disjoint measurable sets in Mx. Then:
K
w;
Ai
K
W,
A
dp(z))
DEL
K ZL (w, A,) dp(z)
i=1
The second equality follows from the countable additivity of KS (w. which is a probability measure for fixed w. By the monotone convergence theorem (since all terms are non-negative), we can exchange the sum and integral:
2t s
W,
A
=
K (w, du(z)
i=1
1=1
DEs
Σ KS" (w,Ai)
=
This establishes countable additivity.
For fixed E Hxx the function w K5' (w, A) Hy -easuuaaebble.
For each z E Dzy, the function w + K2 (wi A) is meameauuralle by the properties of causal dynamics kernels established in Theorem 3 By Fubini's theorem, the integral with respect to preserves measurability, so Kg (w, A) is also /esmmasurable.
Having verified all required properties, we define the high-level representation as:
Zh = (V KH)
where KH (K (w,A): (T),, A Xx} is the set of high-level causal kernels. This completes the construction of causal abstraction, which integrates all low-level representations. capturing their collective causal dynamics and abstracting them into higher- representation. E
C.8 Proof of Theorem (Existence and Uniqueness of Product Causal Kernel)
Pee,,Ko, Theorem 5 (Product Causal Kernel), Let (Le, Le,, Pe,, Ke.) and (Dey,Hey Peyyyee,Ke) Ke,) be complete,
separable causal spaces, and let (2, H. P. IK) be their product causal spuce. Then there exists unique product causal kernel K Q H + [0, 1] satisfying:
35

---

1. For each w E 22, ((w,-) is a probability measure on HE)
2. For each A H. ( A) is H -eeasurabbl
Proof. We will establish the existence and uniqueness of a product causal kernel by constructing explicitly and verifying all required properties. Let (Se, He, Ie,,, Ke,) and (Re,, He, e Pey. Key) be complete, separable causal spaces with causal kernels Ke, and Ke,, respectively, Let (2, H. P., IK) be their product causal space with Ω Sle, x Ωe X = He, @ He,, and IP Pe. @ Pey
Define K on measurable rectangles. For any measurable rectangle A, x A, E H with A, E Ha and A, E Ney, deñne:
((w, A, Aj) Ke,
(wi, A.) Key (wg Aj)
where W)) S. This definition is well-posed because both component spaces are complete and separable, so the kernels Ke, and Key exist as regular conditional probabilities by the Radon- Nikodym theorem.
Verify K ((w,)) 1) is a probability measure on measurable rectangles. Let {Ai x A, A Hep Aj E Hey be the collection of all measurable rectangles. R forms 7--yystm (closed under finite intersections) that generates H..
For fixed w E Q, the function A + K K(w, A) defined on R satisfies:
Non-negativity: K(w, A) Ke, (w;, Ke, (w).Aj) 2 0 for all A R, since both component kernels are non-negative.
Normalization: K ((w, .2) K(w, Qs: x Pey)) = Ke, (ws; Res)) Kez Key(wj, Pay)) ) 1.1 1-1
since both component kernels are probability measures.
Finite additivity: For disjoint A, B E R of the form A = A, x Aj and B = B, x B3 where A n B; = or A, n Bj = 0, we have:
K(w, A B) K(w, A) + K(w, B)
This follows from the additivity of the component kernels.
Here we extend to the full o-algebra by Carathéodory's 's Extension Theorem 558 By Carathéodory's Extension Theorem, there exists unique probability measure ((w,-) on (S. H) that extends definition from R to the full o-algebra M.
Then should verify the measurability of K K(w, A) for all A H. Define the class of sets M = {A H. w + K (w is H -eaasurabll). First, we show that R M: For any rectangle A = A; A R:
K(w, A) = Ke, (wi, A.) Ke, (wtj. Aj)
Since Wi + Ke, (wi, A,) ) is He, measurable by the kernel property of Ke and Wj ++ Ke, (w), Aj) Hz -measurable by the kernel property of Key- their product w K (w,, A) H Hes 8 Hee- measurable. Thus, A E M. Next, we prove that M is X-system:
Ω .M since K (w, 1 is constant and thus measurable.
If A E M, then A .M since K K(w, A9) K K(w,, A) is measurable as measurable
function of measurable function.
If A1, A2.... E M are pairwise disjoint, then _ An M. This follows because for any pairwise disjoint sequence {A, 1x0
8
K (w.. An)
K(w, A,)
Since each function w + K(w, A,) is measurable by assumption, and countable sums of measurable functions are measurable, the function + K(w, U A,) A,) is measurable. Therefore, U An E M.
36

---

We have established that R. C M and M is a A-system. By Dynkin's ' πーー theorem (28), M contains the olalgebra generated by R, which is precisely M. Therefore, M HY, meaning w + ((w, A) is -measurrale for all A E H.
To prove the uniqueness of the product causal kernel, suppose K' is another kernel satisfying conditions 1 and 2. For any measurable rectangle A, x A₁ E R, the condition that K'(w, -) is probability measure and w + (w,, A) measurable implies that K' must be regular conditional probability. By the properties of regular conditional probabilities in product spaces and the definition of K on rectangles, both and K' must agree on all measurable rectangles in R.
Let A = {A E H ((w, = (" A) for all w E Q}. We know R C A. and A forms a X-system by the properties of kernels. By Dynkin's 's π-À theorem, A contains all of H. so K K' on all of H. We should verify that (w:, A) E[1, (w:). For any subset 5 C T, the sub-o-algebra Ws is generated by measurable rectangles corresponding events in indices S. We define Kg as the restriction of to Hss.
We need to show Ks(w, AA expectation, for any B Hés:
B[1A| (s)(w) for all A E H. By the definition of conditional
J B[1A Hts (w) dP(w) =/ 1a(w) dP() P(AnB)
P(AN))
We will show that Ks satisfies the same property:
/ Ks(w, A) dP(w) = P(An
First, consider A, B R measurable rectangles, with A A, A₂ and B B, x Bj. By the definition of K and the product measure P.:
/ K(w, A) dP(ww)
/ Ke, As) Kez (wi, Aj) d(PPe 6 PPp))(w)
/ ( Ke, (er(wn))) Ke, (w). Aj) dIP (w,) dPe, (wi)
B, B,
S Ke, KerlWerA))
(f Ke, (Ke,(w,, dPp (wi) diP (wi)
By the defining property of regular conditional probabilities:
J/ n2 Ke, (We, As) diPe, (wi) Pe, (Aj BB)
Ke, (w, Ar) dP, dP = (A, 0 B,)
Bi
Therefore:
K(w, A) = Pe,(A Bi) Pe, (A, nB)))
=IP((A n B,) x (A, 0 B,)) =P((A, A,) n (B, B,)) = PPAn
This property extends from rectangles to all of H by the monotone class theorem. Since both Ks(w, A) and E[ Hs)(w) satisfy the same defining property of conditional expectation, by the almost-sure uniqueness of conditional expectation, we have:
(s A) = E (s)
for P-almost w E Ω2 for each H. Since (s( A) is proper kernel, provides a version of the conditional expectation that satisfics this equality everywhere, completing the proof. D
37

---

D Theoretical Performance of ACIA
D.1 Theorem (Convergence of ACIA)
Theorem (Convergence of ACIA). the loss function in E corvex and the regularization parameters in Eqn satisfy >1,22 0(1/vn) where is the sample size. Then the ACIA optimization problem solved via gradient descent converges with the distance to the optimum is bounded by
+
after T iterations.
Proof. We prove that the optimization problem is well-defined and converges under the given conditions. Let us denote the objective function in Equation 5aa:: as:
I e((C o on o (r)) (()) dP., AI + A2
F(O SH, C) = max e.E0
First, we verify that all components are well-defined. The composed function C o ΦH ØL is measurable, as each component is assumed to be measurable.
The expectation Jg (((C ΦF o (x)() (w)) (w) exists by the measurability properties and the assumption that & is integrable. The regularization terms Rı and R2 well-defined by Fubini's ' theorem and the definition of interventional kernels in Theorem 2
Let 0 (C, db.,, Φ1) denote the parameters of our model. Since l is convex, the objective function F(0) is convex in The optimization problem has the form:
min max Fe; (0) E,E8
where E., [[foo] R1 + A2 and fe C ΦØ ΦL.
=
For convex functions optimized using gradient descent with appropriate step sizes, the convergence rate in terms of objective value is:
F(OT) F(0*)= 1000 11
2T
where 0" is the optimal parameter. This implies that:
|0 - 0-
Additionally, the empirical objective function F(0) based on n samples approximates the true population objective Fpop(() (0) with error bounded by:
1
sup [(0)) Fpop (0)) =0 806
Vn
with high probability, by standard uniform convergence results in statistical learning theory.
The setting λ₃ A2 O(1/vm) ensures the regularization terms scale appropriately with the sample size. This balances between ensuring the regularizers enforce desired invariance properties, and allowing the strength of regularization to decrease sample size increases, preventing excessive bias. By the triangle inequality:
|| - - Pop |l & ||0, - 0- + 1|0-
0%
Popp
where opo is the minimizer of the population objective.
The first term (L//VT) from our gradient descent analysis. The second term is O(1/vn) due to the statistical error between empirical and population objectives. Therefore:
1 O 1 +0
|0r
Vn
This establishes the claimed convergence bound, with the first term representing optimization error and the second term representing statistical error.
D
38

---

D.2 Theorem 7 (Anti-Causs OOD Generalization Bound)
The ACIA framework provides theoretical guarantees for OOD generalization in anti-causal settings the performance on the unseen test environment cannot be arbitrarily worse than the worst-case performance on training environments, with the gap controlled by the sample size.
Theorem (Anti-Causal OOD Generalization Bound). For optimal representations 67 6i in anti-causal setting. i.e., 611 ( (X))
Y in all environments E. With probability at least 1
for any testing environment Etest with sample size Ress its expected empirical loss Eena [[([[) under the optimal predictor f C 0' o o4 is bounded below:
(V
(1 log(1/8) fleast
E [((f")] S max [e(f))
(10)
Proof. We establish generalization bound for optimal representations in anti-causal scttings by leveraging both concentration incqualities and the invariance properties of the learned representations.
Let o7 and Φ be the optimal representations obtained from solving the optimization problem in Equation 5 and let f = C o Φū o Φ7 be the optimal composed function.
First, establish concentration bounds for empirical true risks. For any environment E, let [e(")( be the empirical risk based ne samples, and EEg [é[]*) be the true expected risk. Assume the loss function L' is bounded in [0, MM for some constant M > 0.
By Hoeffding's inequality [8] for any 8 0, with probability at least dg:
log(2/0.)
- S M
2n
Next, we analyze the relationship between test and training environments under anti-causal structure. By the optimality of øi and Φh. the environment independence property is satisfied: Φi (oz (2)) LL I E Y. This implies that the distribution of high-level representations depends only on the label Y and not on the environment E when conditioned on Y.
This implies the optimal representation extracts the invariant causal mechanism Y - X while filtering out the spurious correlation E' + X. In other words, for any environments e1, 62 (including a test environment etest), we have:
Pog where ZH = o6 (0) (x)) is the high-level representation.
Pej (Zn (2))) P(Znl()
We now relate the expected loss in test environment to those in training environments. The expected loss any environment e can be written as:
E.(((( / [ ((C(zH).)) (P(c))))))P((Pl)))) dP(znlv)
dP.(y)
Due to the environment independence property, P(znly) ) is the same across all environments. There- fore, the differences in expected loss across environments arise only from differences in the label distribution Pz(y). Define the maximum expected loss across training environments:
Emax [ [[(f")]
max E, [((f")] e
Now, we bound the empirical risk in the test enviromment. For a test environment Crest we have:
[(()))] E, [e(f*]] (Bn..((((")) E [e()*)]
Eagga [(({)] B..n [e(]*) Eey [(([*)]
Due to the anti-causal structure and the invariance property of the optimal representations, the expected loss in the test environment is related to the losses in training environments. Specifically,
39

---

since the learned representations capture the invariant causal mechanism from Y to X while removing environment-sper specific effects, we have:
Bewn [(([")] S E [((x) street
where Cstruct is small enror arising from structural differences between test and training environments.
Finally, we apply union bound to combine the bounds. We set d, 8/|2 for each training environment e E & and Sjest = 6/2 for the test environment. By the union bound, with probability at least 1 6:
log(4/6)
E
Clod [e(f*]] S Emax [[()*)]
Estuuc + M
2nrsst
1og(2)81/0)
Estrect + M log(4/6)
< max E. [[[["]]]]]]]]]] + max M 3E8
EEE
2n,
2
For optimal representations that successfully enforce the invariance property, Garmet approaches as the regularization strength is appropriately tuned. Under the assumption that the test environment simplify to:
follows the same anti-causal structure as the training environments,
log(1/6) The
E [((f*)]
max Be [e(e(f*)]
.EE
D.3 Theorem 8 (Anti-Causal OOD Gap w.r.t. Kernels)
The below theorem implies that performance gaps camnot be smaller than the fundamental difference between the interventional distribution and observational distribution.
Theorem (Anti-Causal OOD Gap w.r.t. Kernels). The gap between test performance and training performance w.r.t kernels is bounded as:
Eevr [[[ - Ee (e("))) 2 min
cEE' || (w)))))(())>
do (w) K(e)(w)lH
(11)
where Kdo(()) X) is the interventional kernel and K(e) the observational kernel. [e}
Proof. We analyze the fundamental lower bound on the gap between test and training performance by relating it to the difference between interventional and observational distributions in causal settings.
First, express expected losses in terms of probability distributions. For any environment e, the expected loss under the optimal predictor f = C oj o 02 can be written as:
E. [.(((()) [ ((f"()((w)),, Y(w)) dPP
(12)
This expectation depends on joint distribution of (x, environment e, which can be repre- sented through the causal kernel Kte}-
Then, we relate performance gap between and training environments distributional differences.
[Eera[e(f")) EEem [(f / (X())) d(F Perm (w)
The measure (IPeren Pe Petramn) represents the difference between the joint probability distributions in test and training environments.
[ [(w) d(P - Perreenn (w)
[ [(w') d(K (eeee) dw') K [eerasn (w, dw')) dP(ww)
Now, we establish a dual representation using integral probability metrics. For a function class H containing functions of the form h(w) = E(f (X(w)), (w)), we can express the distributional difference an integral probability metric:
|| (ewx) K K(eam) ||
sup hEW
h hd(((ene)) K ((enn)))
40

---

Here, we relate environmental differences to causal structure using interventional kernels. In the anti-causal setting, the causal structure implies that the observational distribution K (c) differs from the interventional distribution K(l) due the spurious correlation introduced by E. The difference
[K())) (e) llx quantifies the strength of this spurious correlation in environment e. Intuitively,
if this difference is large, then E' has a strong influence on X in that environment.
Next, apply the variational characterization of integral probability metrics. For any function E H. we have:
hhd( - K ((erm))
/ hn d - K(t))
2 min 3EE
This inequality holds because the interventional distribution K (a)
k represents the distribution
when the spurious correlation E + X is removed. The minimum across all training environments
represents the smallest possible spurious effect that must be overcome in generalization.
Ultimately, we complete the proof using the specific loss function. Let h (w) = (()"()())) Y(w)) be the composed loss function. Then:
// h" d( (e(()
|eemi [e(])]] Eeuia [((S))]
-
K (emin)
})
min eEE'
d(k ((tt)
Since this holds for the specific function 4*, and the integral probability metric takes the supremum over all h E H. we have:
Bec. (e(S*) Fre. [e(S")] min eEE
|
K(e)
(c)
This establishes the lower bound on the performance gap in terms the minimum distance between interventional and observational kernels across training environments.
D.4 Theorem (Environmental Robustness)
Having established generalization bounds and performance gaps, examine how these translate to environmental robustness guarantees.
Theorem (Environmental Robustness). Denote the learnt two-level representations by ACIA as of and oy. Then for any new environment Cnew, the distrübutional distance du enow PP) between Pe Penee and I over the function class H containing all predictors is bounded by
dn PP)) < 51 82,
where Pg- TEi EeEs Pe the mixture distribution of training environments E: 8n and 82 respectively measure the degree of invariance violation in below conditions I and 2.
1. The high-level representation is murironment-independdee dh (0)((()) 1 E E
2. The low-level representation is invariant: Pr(02 (2) Y) is constant across environments
Proof. Let P. = TET EeEs Pe be the mixture distribution of training environments E, and IP Perce be the distribution of a new environment Enew- Let H be the function class containing all predictors of the form h = g o Φi o oं for some measurable function g. The distributional distance du (P (Pen.u Sanee:PP IP is defined as integral probability metric (IPM) (51]:
dr (IPP
P&) = sup Eenew [h] Eg WCH
Next, we express expectations in terms of representations. For any function h H. we express expectations terms of the learned representations:
[g[h] /
h(zH) dP (zulv) dlP
where zи = (X)) is the high-level representation.
41

---

By condition (I), the high-level representation is environment-indepeen given Y: o4 (02 1
E Y. which means for any environments e1, 62 (including new environment nnww):
Pej (zHly) = Peg (zHlu).
Assume certain level of violation on condition (1), we rewrite
Pe, (zH (v) (z + (=H 61,
where (z y/ C1, 62) is function quantifying the violation perfect environment independence. with | 56 for some small 8, 2 0 that depends on the degree to which property (1) is satisfied.
By condition (2), the low-level representation is invariant, Pr(Øī (X) Y) is constant across environments. This implies the distribution of z1 = (x') conditioned on Y is same across environments. Assume the degree of violation on condition (2) is 62.
We now bound the difference expectations. For any H. the difference in expectations between environments is:
11, h((H)) dl ane (zH (y) dlP ca (v) 1 J, l(zz)) dPe(=nlv) dPP())
EEe..w m [+] Ee [b]]]
Zn
1 { h(2H) d(P ener (=Hl0) Pe(zHlu)) dl (nnem(())
Zn
+ / [
h(zz) dP dF dnee (1) - P'e (v))
zn
By the triangle inequality:
I / h(ZH) d(P (e Ps(zH ((H))) dP aPenmn (v)
E EEn.m [h]
Eg [h]]
2n
h(=H) d(Pe(zHl)) d(Penew enew (v) (PE()))
+
Zu
We apply the bounds invariance violations. From condition (1), for each y V, have:
Crec (znly) (=( $ 51
where || || denotes the total variation distance.
Using condition (2) and the anti-causal structure, the difference label distributions is also bounded:
Peenn (v) - Pe (V) S 82
To bridge the theoretical and practical results gap, we apply Hölder's inequality to bound the expectation difference. Assuming bounded functions |[ S 1, and using Hölder's inequality:
E [h] - Es [h]]] /, |1Alloc [| (2H19) (PHEHHH)))lll lIv dP (v) + //llloo |PPene (v) PP(l)llTV
/ dTP enew (v) + 8g 51 55
< 81
By taking the supremum over function class H. we have:
dx C Pe)
sup
F [h] Ee [b]] S 81 + 8z.
kEW
This establishes that the distributional discrepancy between any new environment and the mixture of training environments is bounded by a term that depends only on how well these invariance properties satisfied w.r.t the learnt two-level representations by ACIA.
42

---

2
8
7
&
4
2
3
8
5
D
3
2
0
2
0
<
3
X
Figure 7: MNIST-based anti-causal datasets: (Left) CMNIST by color; (Righ) RMNIST by rotation.
E Experimental Setup and Results
E.1 Datasets for Anti-Causil Learning
E.1.1 Colored MNIST (CMNIST)
Let Y E {0, 1, .,.,, 9} be the digit label, Xo E 28x22 be the original grayscale MNIST image, E E (2) be the environment variable, and C (R, G} be the color variable (Red or Green). The generation process is as follows:
1. Sample Xo) from the original MNIST dataset.
2. Generate the color C based on Y and E:
0.75 if Y is even and E = E1
0.25
if Y is odd and E = e1
P(C = R) =
0.25
if Y is even and E 62
0.75 if Y is odd and e2
3. Generate the colored image X E 228x28x3,
If C = R: X ; Xo. [ 0, [[; = 0
C' = G: X : 0, X[: Xo. X [
Why is this an anti-causal problem?
1. The digit label Y causes the color C, which is part of the observed features X. This is opposite the typical prediction task where we try to predict from X.
The underlying causal graph is: Y + X <- E. This a classic anti-causal structure where the label Y is cause of the features X, and the environment E influences X.
Preparation of the Colored MNIST
According to Table 6 environment contains 60,000 images while e2 contains 10,000 images, maintaining roughly 6:1 ratio. A clear pattern emerges in the color distribution: even digits (0.2,4.6,8) in E1 are predominantly colored red 75%), while odd digits (1,3,5,7,9) are predominantly green 75%).. This pattern is inverted in 62, where even digits are mostly green (75%) and odd digits mostly red 75%). Despite these strong digit-color correlations within each environment, the overall color distribution remains relatively balanced both environments e, has 49.54% red and 50.46% green, while ea has 51.22% red and 48.78% green.
This design creates spurious correlation berween digits and colors that varies across environments, making it an ideal dataset for testing anti-causal representation learning methods.
43

---

Table 6: Distribution of digits and colors in Colored MNIST Environments él and e2
Environment e1
Environment eg
Digit
Count
Red
Green
Count
Red
Green
0
5923
4426 (74,73%)
1497 (25,27%)
980
284 (28,98%)
696 (71.02%)
1 6742
1757 (26.06%)
4985 (73.94%)
1135
881 (77.62%)
254 (22.38%%
2
5958
4490 (75.36%)
1468 (24.64%)
1032
280 (27.13%)
752 (72.87%)
3 6131
1548 (25.25%)
4583 (74.75%)
1010
753 (74.55%)
257 (25.45%)
4 5842
4365 (74.72%)
1477 (25.28%)
982
244 (24.85%)
738 (75.15%)
5 5421
1382 (25,49%)
4039 (74.51%)
892
658 (73.77%)
234 (26.23%)
6 5918
4439 (75.01%)
1479 (24.99%)
958
239 (24.95%)
719 (75.05%)
7
6265
1561 (24.92%)
4704 (75.08%)
1028
782 (76.07%)
246 (23.93%)
8 5851
4315 (73.75%)
1536 (26.25%)
974
222 (22.79%)
752 (77.21%)
9
5949
1438 (24.17%)
451 (75.83%)
1009
779 (77.21%)
230 (22. 79%)
Total
60000)
29721 (49.54%)
30279 (50.46%)
10000
5122 (51.22%)
4878 (48.78%)
How to create the causal space from CMNIST
1. We first define the sample space St:
Ω = ((Y, F, X) Y E (O...., L E 28x28 E E (E V E [
2. We then define the o-algebras. Hy o-algebra Vr. (the colored images). and My: o-algebra on Y (the digit labels).
3. Then define environments and datasets, causal spaces for each environment, and the product causal space based on Alg [
4. Next, we construct the causal kernel K: For the anti-causal CMNIST, construct KK to reflect the causal structure + X + E. For example, for just one environment e, (so not dependant on e; in the formula anymore):
P(C R Y = y) if A corresponds to red images
Ks(w, A)
y if A corresponds to green images
where P(C = R Y y) is given by the generation process:
0.75 if Y is even and E or Y is odd and E = 62
0.25 if Y is odd and E e1. Y is even and E 62
For S E (T), w E Ω and A E H::
Ks(w, A) = P(X E = E, E
where y is the component of w corresponding to Y, and E E means the enviromments indexed by S. So, in general.
PCCC = Y = =y ye x Dey) if A corresponds red images G = y. Rey x Qey) if A corresponds to green images
Ks(w, A)
5. For causal properties verification, we first check if Ks(w, (A[B)) = Ks(w', (A for all w,w' Ω2 A E .Wv. B Myy S (T) by comparing conditional probabilities across environments. This is verifying that the causal kernel Kş is independent of the specific outcome when we condition on Y. It's checking P(X) IY) the same for all instances, regardless of the environment, which reflects the anti-causal nature where Y causes X.
The we verify key properties:
(Vr) Y) is invariant across S E (T)
(VL)) 1 F y
To ensure the construction aligns with the anti-causal nature:
Ks(w, (Y E B X = =²) should be invariant across environments
kdol() (w: (Y E B)) = (s (Y E B))
1go0 (X e A}) * Ks(w, (X A})
44

---

E.1.2 Rotated MNIST (RMNIST)
Let Y E 10.,, 9) be the digit label, Xo E 28 be the original grayscale MNIST image, E E {09, 15" 30 45% 60", 75") be the environment variable, representing the rotation angle, and Rg 28282 282282 be rotation function that rotates image by 0 degrees. The generation
process is as follows:
1. Sample (Y, from the original MNIST dataset.
2. For each environment E' =, generate the rotated image: Xa Ro (Xo)
To define the causal kernel K: For RMNIST, construct to reflect the causal chain:
Ks(w, A) = P(Xa E A Y = y. E
where 0 is the rotation angle from environment E.
Why is this an anti-causal problem?
The original relationship Y + Xo anti-causal, as digit label causes the original image.
2. The underlying causal graph is: Y Xo Xo E, where rotation angle E influences the final image.
According to Table 7 distribution of digits and rotations in RMNIST Environments ej and eg is observable. The task predicting Y from X9 can be seen as an anti-causal problem (predicting cause from effect), but with additional causal intervention (E) applied the effect.
Table 7: Distribution of digits and rotations in Rotated MNIST Environments e and eg
Environment 61
Environment
Digit
Count
15
75
Count
15"
75
0
5923
4442 (75.00%)
1481 (25.00%)
980
245 (25.00 (5.00%%
735 (75.00%)
1
6742
1686 (25.00%)
5056 (75.00%)
1135
851 (75.00%)
284 (25.00%)
2
5958
4469 (75.00%)
1489 (25.00%)
1032
258 (25.00%)
774 (75.00%)
3
6131
1533 (25.00%)
4598 (75.00%)
1010
758 (75.00%)
252 (25.00%)
4
5842
4382 (75.00%)
1460 (25.00%)
246 (2.00%)
736 (75.00%)
5
5421
1355 (25.00%)
4066 (75.00%)
892
669 (75.00%)
223 (25.00%)
6
5918
4439 (75.00%)
1479 (25.00%)
958
240 (25.00%)
718 (75.00%)
7
6265
1566 (25,00%)
4699 (75.00%)
1028
771 (75.00%)
257 (25.00%)
8 5851
4388 (75.00%)
1463 (25.00%)
244 (25.00%)
(75.00%)
9 5949
1487 (25.00%)
4462 (75.00%)
1009
757 (75.00%)
252 (25.00%)
Total
60000
29747 (49.58%)
30253 (50.42%)
10000
5039 (50.39%)
4961 (49.61 1)
E.1.3 Ball Agent
Following 9.. let 2 be ball positions (where n the number balls), E [64x6433 be rendered image, and E be the set of sparse interventions. The generation process is:
Sample initial ball positions = [y....., U2n] where each yi ~ U...1.0.9).
2. Generate the base image X by rendering colored balls at positions Y.
For each environment e E E, apply sparse perturbations to specific ball coordinates.
Why is this an anti-causal problem?
Ball positions cause image appearance
2. Interventions E modify positions but not the underlying rendering process
3. The causal graph is: Y + X < E
How to create the causal space from Ball Environment
Define the sample space Ω:
Ω = (Y, Y E 10, E pp 64x6443
where P(I,...,,21 2n) represents possible intervention subsets.
45

---

Figure 8: Visualization of the Ball Agent environment with balls and 10,000 samples. (Left) Example configurations showing the colored balls spatial distribution. (Right) Analysis inter-ball distances for the environment spatial constraints
Define the causal kernel IK: Ks(w, A) P(X =
where e represents specific intervention coordinates.
For sparse interventions, we should verify that:
Changes in are localized intervened coordinates Non-intervened ball positions remain unchanged
K (do(Y)) (wr, {X E A}) + Ks(w, (X E A}) for intervened coordinates
3 shows that the Ball Agent dataset demonstrates a remarkably balanced intervention structure across 10,000 samples with 4 balls. Each ball receives interventions approximately 50% of the time (ranging from 49.7% to 50.8%), indicating uniform intervention probability. The distribution of simultaneous interventions is uniform, with each possible number of interventions (0 to 4 balls) occurring in roughly equal proportions (19.6% to 20.4%) This uniformity is reflected the high intervention pattern entropy of 1.609 bits, approaching the theoretical maximum for this scenario. Spatially, the balls maintain minimum distance 0.200 units, with a mean distance 0.478 units between pairs, demonstrating effective enforcement of proximity constraints while allowing significant positional variation. This structured randomness makes the dataset particularly suitable for studying anti-causal relationships between ball positions, rendered images, and interventions.
Intervention types
In the Ball Agent environment, intervention types are categorized according to the number of ball coordinates simultancously affected:
None: No intervention applied any ball coordinates ([4)
Single: Intervention affects exactly one ball coordinate ([/) 1)
Double: Interventions affect exactly two ball coordinates (!/
Multiple: Interventions affect three or more ball coordinates ((4) 3)
These intervention patterns follow the intervention distribution defined in [9] where the intervention
set I C (1,2.... ... 2n} represents the indices of coordinates receiving interventions. The visualization
reveals how ACIA'S learned representations are organized based on these intervention patterns, demonstrating ACIAI's ability to identify invariant features despite varying intervention complexity.
E.1.4 Camelyon17
Let Y e 0, 1 be the tumor label, X e 96x be the tissue image patch, and E E h1, he, hs, h4 his be the hospital identifier. The generation process follows the natural data collection:
1. Sample tumor tissue (Y 1) or normal tissue (Y = 0)
2. Process tissue according to hospital protocol F. h2
3. Generate patch X from the processed tissue slide
46

---

180
Figure 9: Camelyon17 shows hospital-specific tissue imaging variations. (Left) Examples of normal and tumor tissue images across five hospitals (h1-h5), illustrating distinct staining patterns. (Right) Distribution of normal and tumor samples across hospitals, showing the dataset's balanced nature.
Why is this an anti-causal problem?
1. The presence of tumor (Y) causes specific visual patterns in the tissue (X).
Hospital scanning and staining protocols (E) affect the image appearance but not the tumor status. The task of predicting tumor presence from images is inherently anti-causal. The causal graph is: Y X E
How to create the causal space from Camelyon17
1. Define the sample space Ωi
Ω = EY Y E 60,1,E hi,... hs, X E R 96x96x3
2. Define the causal kernel IK for each hospital:
Ks(w, A) P(X A y, E.E E= h4)
where h; represents the specific hospital protocol.
Table 8: Distribution of images across hospitals in Camelyon 17 dataset
Hospital
Total Images
Tumor
Normal
h1 17,934
7,786 (43.41%)
10,148 (56.59%)
h2 15,987
6,446 (40.32%)
9,541 (59.68%)
ha 16,828
7,212 (42.86%)
9,616 (57.14%)
h4 17,155
7,502 (43.73%)
9,653 (56.27%)
ha 16,960
7,089 (41.80%)
9,871 (58.20%)
Total 84,864
36,035 (42.46%)
48,829 (57.54%)
Visualization of Camelyon17 and its distribution in Fig which exhibits systematic variations in tissue imaging across 5 hospitals while maintaining consistent tumor detection challenges. Each hospital maintains a relatively balanced distribution between tumor and normal samples, with tumor prevalence ranging from 40.32% (h2) to 43.73% (h4). The dataset encompasses substantial total of 84,864 images, with individual hospital contributions ranging from 15,987 to 17,934 samples. These statistics are available in Table 8 The hospital-specint staining patterns are visually distinct, as evidenced by the color variations in the example images, yet the underlying tumor patterns remain consistent. This structure makes Camelyon17 an example of an anti-causal learning problem, where hospital-specins imaging protocols (E) affect the image appearance (X) but not the ground truth tumor status (Y).
For the Camelyon17 results (Figure the fourth panel shows confidence visualization that maps the model's certainty levels across the representation space. The confidence levels color-coded as:
47

---

Green/Yellow: Regions where the model maintains high confidence in tumor classification
Orange: Intermediate confidence regions, typically at class boundaries
Red: Low confidence regions requiring additional evidence for reliable classification
This uncertainty quantification follows 14] and extends the predictive uncertainty estimation. The visualization reveals how ACIA preserves diagnostically relevant tissue features while discarding hospital-specitn staining variations, which aligns with findings [55] on stain normalization effects in computational pathology. This confidence mapping is particularly important for medical applications where uncertainty awareness is critical for clinical decision support.
E.2
Model Architecture and Hyperparameter Setting
Table 9: Dataset configuration details and their properties.
Dataset
Type
Environments
Dimensionality
Label Space
Spurious Corr.
Train Size
Test Size
CMNIST
Synthetic
2 (c2, e2)
28x28x3
(0....9)
Color-digit
60,000
10,000
2 (15".75")
RMNIST
Synthetic
test (30", 45", 60")
28x28x3
(0....1
Rotation-digit
60,000
10,000
Ball
Agent
Synthetic
halls with interventions
64x64x3
[0, 112m
Coord-coupling
15,000
5,000
train (hospitals 0-2),
Camelyoa17
Real
test (hospitals 3-4)
96x96x3
(0.1)
Hospital-stan -tani
50,916
33,944
We provide detailed hyperparameter settings for ACIA across all datasets to ensure reproducibility. We discuss the datasets properties in Table
Model architectures: We design the architecture with the following principles: the low-level repre- sentation should be sufficiently rich to capture both label-relevant and environment-dependeen signals, while the high-level representation serves as an information bottleneck, filtering out environment- dependent variations, and thus is set to lower dimension than the low-level representation.
We used consistent network architectures across all experiments in our ACIA, For low-level represen- tation learner, used ConvNet (3 layers) FC layers 32/256-dim latent space. For high-level representation learner, we used MI.P (2 layers) - 128-dim latent space. For the on-top classifier, we used linear layer (output dimension varies by dataset).
Hyperparameter setting:
For datasets, the batch size is 32, the optimizer is Adam, the learning early stopping for Camelyon17 to avoid overfitting in the results. In our
rate is 1e-4, and regularier, we chose A1 as /ttc///tthhhgaeeeeeeeeeeeeee sizesszzeeeee (=0.0177) for CMNIST, RMNIST, and Ball Agent, and chose 0.5 for Camelyon17.
addition, we chose \2 as 0.5/ / bbtchh size (=00.0844) for CMNIST, RMNIST, and Ball Agent, and chose 0.1 for Camelyon17. These hyperparameters based on validation performance and thcoretical constraints from the ACIA framework requiring balanced regularization to achieve environment invariance without sacrificing predictive performance.
optimized
E.3 Imperfect Intervention Implementation
E.3.1 Imperfect Intervention Datasets Construction
In CMNIST, perfect interventions implement deterministic color assignment with probabilities restricted to be 1, creating a strict mapping between digits and colors. Imperfect interventions use continuous probabilities in [0, allowing for partial influence of the causal mechanism.
In RMNIST, perfect interventions apply fixed and deterministic rotation angles for each digit class. Imperfect interventions introduce variability through probabilistic angle distributions, creating blend between digit-specific and environment-specife rotations.
In Ball Agent, perfect interventions produce deterministic position shifts, completely overriding natural ball dynamics. Imperfect interventions implement probabilistic dynamics changes, where original positions are partially preserved while incorporating intervention effects.
In Camelyon17, perfect interventions standardize tumor appearance through complete staining changes. Imperfect interventions apply partial staining changes, preserving some hospital-specific characteristics while normalizing others.
48

---

E.3.2 Results on Imperfect Intervention
The perfect and imperfect interventions both achieve accuracy values that exceed 99%. The low values for environment independence and low-level invariance metrics indicate successful disentanglement of label information from environment factors. Intervention robustness remains low for both types (0.025-0.028), suggesting that the model maintains consistent predictions under small perturbations regardless of the intervention mechanism. This supports the theoretical assertion that ACIA's measure-theoretio framework can effectively handle both perfect and imperfect interventions through its interventional kemel formulation.
For CMNIST. perfect interventions completely break the probabilistic relationship between digits and colors by implementing deterministic coloring. For example:
P(Color Digit even, Env e1) 1.0
(13)
P(Color = red) Digit = odd, Env = e1) = 0.0
(14)
Imperfect interventions preserve partial dependencies by blending original probabilities with the target values (a = 0.5 in our experiment.):
P(Color = redi Enw) = 0.5 -
O.5--Priige Poriginat (Color Digit, Enw) + 0.5 Pper feet (Color Digit, Env)
(15)
Table Table[[show showws ACIA's effectiveness with rotation-based interventions. Perfect interven- tions yield test accuracy of 99.1%, marginally outperforming imperfect interventions (99.0%). The environment independence metric shows the model's ability to progressively remove rotation-specific information, with perfect interventions achieving better invariance (0.455 vs. 0.480). This confirms that deterministic angle assignments facilitate more complete abstraction of spurious correlations.
RMNIST:
In RMNIST, perfect interventions implement precise angle assignments for digit classes:
Obaso
if d is even
2 perffec (d)
(16)
Obuse + 45
if d is odd
Imperfect interventions implement probabilistic angle distributions:
immerteea (d) (1 a.) Oruse + Operteaa (d)
(17)
where a 0.5 represents the intervention strength.
The training dynamics reveal that RI values (environment independence) decrease from initial values around 10.0 to final values of 0.46-0.48, while R2 values (causal structure alignment) decrease from 0.21 to 0.02-0.03. This convergence pattern quantitatively demonstrates ACIA's ability to progressively disentangle digit identity from rotation angle through its measure-theoretic framework.
Ball Agent: Table demonstrates ACIA's exceptional performance a continuous spatial regression task. Extended training reveals near-perfect position accuracy of 99.97% (position error 0.0003) for both perfect and imperfect interventions. The environment independence metric shows perfect interventions achieve substantially better invariance (0.372 vs. 0.468), confirming that deterministic position shifts enable more complete abstraction of intervention-specife effects.
In Ball Agent, perfect interventions implement deterministic position transformations:
(ai, We) (x4 v/)
no intervention
P Pperfeer(1; 31)
(18)
if intervention on coordinate
Imperfect interventions implement probabilistic position shifts: Pimporteer (Z,, y:) = (1 (1-a): (z,, vi) + +a (4% v1)
(19)
where a 0.5 represents the intervention strength.
The RI values are steadily decreasing from initial values of 1.02-1.20 to final values 0.372-0.468, while R2 values decrease from 0.38-0.44 ...44..44 to 0.024-0.026. This progressive convergence pattern demon- strates ACIA's ability to effectively generalize from discrete classification to continuous regression tasks while achieving exceptional invariance properties and near-perfect positional accuracy.
49

---

Table 10: Runtime performance of ACIA across different datasets
Dataset
#Samples
Input Dimension
Time (seconds)
CMNIST
60K
2,352
580.7
RMNIST
60K
784
973.6
Ball Agent
15K
12,288
204.7
Camelyon17
84K
6,912
799.6
E.4
Computational Efficiency
Table 10 reports the runtime on a laptop with single GPU (3.3 GHz, GB RAM), using precision c 0.01 and failure probability 8 = 0.05.
ACIA completes training in approximately 3-16 minutes under these settings, demonstrating practical scalability for real-world applications.
Discussion
F.1
Why Measure Theory?
Traditional causal inference relies on Directed Acyclic Graphs (DAGs) and Structural Causal Models (SCMs), which require explicit specification of causal graph structure, parametric forms for structural equations, and knowledge of which variables are observed/latent.
These requirements prohibitive when learning representations from raw data. The measure- theoretic framework offers three key advantages:
(1) Direct learning from observations: Probability kernels K (ww, A) encode causal mechanisms without requiring explicit structural equations, enabling end-to-end learning from X latent causal variables.
(2) Unified intervention handling: Both perfect (Q(A(w) (()) and imperfect (Q(A)w) depends on w) interventions are expressed through kernel modifications, avoiding the restrictive perfect intervention assumption.
(3) Compositional structure: causal structures without specifying DAG edges, allowing flexible architecture design.
Tensor product o-lleebras Xh = SIeT, o4 naturally represent factorized
F.2 Why Kernel Formulation?
Interventional kernels play a pivotal role in distinguishing between causal and anti-causal relationships by capturing how distributions respond to interventions. In unti-causal structures. intervening on X does not affect Y (as shown in Corollary (2). while intervening on Y changes the distribution of X. Interventional kernels formalize the counterfactual question: "How would features X change we intervened on label Y?" This enables learning representations that capture the generative process from Y to X. In addition, the interventional kernel formulation enables both perfect and imperfect interventions, which critical since real-world interventions are rarely perfect. The kernel independence property also helps separate the mechanisms P(X YY) from the environmental influences P(X|E), This formalization through interventional kernels provides a rigorous foundation for causal representation learning that goes beyond mere statistical association.
F.3
Connection to Existing Methods
Our measure-theoretic approach connects to and extends several several existing frameworks. For instance, under linear representations and squared loss, the optimization problem in Eqn is an extended version Invariant Risk Minimization (IRM) 5 that includes two levels of invariance:
Φω Φ1 arg ming R oC Ø) We E
min Σ Rd (CoonO0L) s.t. &LiCH.C
ΦD E aargmnn, DRI (U
Ve,, ej E
(20)
50

---

where R is the risk in environment e and DKL is the Kullback-Leibler divergence.
Further, the min-max formulation in Eqn can be viewed as form of Distributionally Robust Optimization (DRO) with an uncertainty set defined by the causal structure:
min max E(xy))) &L C PEP
Φи (L)(X),
(21)
where P is the set of distributions consistent with the anti-causal structure. This connection implies that our approach inherits the worst-case performance guarantees of DRO while enforcing causal consistency through the regularization terms.
F.4 Anti-Causal Toy Example vs. Kernels
We illustrate how kernels capture conditional probabilities with simple structural causal model (SCM). See below:
Y Bernouli (.55)
Y
X
E ~ Bernoulli (0.5)
X f(Y, E, [x) where:
P(X = 1|Y = 0, E' = 0) = 0.2
P(X 1 0,
1) = =0.4
E
P(X = 1 1,E = 0) = 0..6
P(X 1 E, 1) = =0.8
Figure 10: Toy Anti-Causal SCM showing how Y and E determine X
In this model, and E both causally influence X. The environment E shifts the probabilities, but the causal effect of Y on X remains consistent: Y = 1 always increases the probability of X = 1 by 0.4 compared to =
This example illustrates a key property of anti-causal structures: intervening on Y changes the distribution of X, while intervening on E also changes X. Our kernels K and K capture these conditional probabilities. The role of ACIA is to extract the invariant relationship between Y and X (the consistent +0,4 effect) while removing the effect of environment E.
For the provided SCM where Y + X +- E, the causal kernel Ks (w. A) represents the conditional probability P(X E A|Y = y, E = e), where w contains components (y. e). For specific realizations:
0.2 if 0, 0
0.4 if 0, 1
Ks(w, (X 1]) P(X 1| E=
0.6 if y 1,e
0.8 if y 1,e =
The product causal space combines environments by defining = Peg x Ωer with S S e1) For S = {eo, e1 }, our kernel becomes:
K (eo.er) (w, fx = 1}) = P(X = 1 = y, E. (co.c1)) = P(X 1, E E feo, el e =
P(E E eco e = =y)
P(X 1, colY = y) P(X 1, = =y)
P(E = = = =y) + PPE =
P(X 1 y. E P = co) + x
P(E = co) + P(E (c==e)
For Y = 0 with equal environment probabilities, we get:
K Keea,e (w,, (X 1)) 0.2 0.5 + 0.4 0.5
0.3
0.5 + 0.5
The interventional kernel under perfect intervention do(Y = 1) would be:
KS do do (w, (X = 1}) = / Ks(w', (x = 1)) 0.8 if== 1
[06
if =
51

---

For imperfect interventions with strength a E [0, 1]:
K5 k (w: 1x = 13) = a)) Ks(w, (X =xx=10 =1] + a Ks KS (w, =1)
Our low-level representation kernel K integrates over the empirical distribution:
(w, (X = 13) = -1 Ks(w (X 1))dQ((>)
Ks(wi, (X (X=
The high-level kernel 2 integrates over low-level representations to abstract away environment- specific effects:
K (x)]]=
K (w, (X 1)) 2o P(X = 1 = y)
JDzz
= 0.5 P(X 1|Y E- + 0.5 P(X =ei)
In this way, ACIA captures the invariant relationship: Y consistently increases P(X 1) by 0.4 compared to Y = 0, regardless of environment.
F5 Anti-Causal Representation Learning, Robustness, and Privacy
Anti-causal representation learning focuses on modeling settings where labels generate features rather than the reverse. This perspective is particularly relevant in domains such as healthcare, biology, and natural sciences, where the underlying causal process is generative (e.g.. diseases causing observable symptoms or molecular mechanisms producing data patterns). By explicitly capturing these anti- causal dynamies, we conjecture that ACIA gains not only better generalization across environments but also stronger foundations for robustness and privacy.
Relationship to Adversarial Robustness: Adversarial vulnerabilities often arise because conven- tional predictive models latch onto spurious correlations or environment-speciff artifacts that are easily perturbed. Anti-causal representation learning mitigates this by forcing the model to encode how labels give rise to features. As a result, learned representations emphasize stable causal pathways instead of fragile correlations. For instance, in medical diagnoise, robust anti-causal representation prioritizes discasc-related structures rather than scanner-specific noise or nospital-speciff imaging protocols. This alignment reduces the attack surface for adversaries, since perturbations that exploit non-causal features become less influential. In this sense, adversarial robustness can be viewed as an emergent property of faithfully modeling anti-causal structure.
Relationship to Privacy Protection: Privacy risks (e-g., membership inference, data reconstruction) in machine learning models often arise when models overfit to environment- or individual-specine details. Anti-causal learning frameworks such as ACIA pursue high-level causal abstractions that discard environment-speciin signals. By abstracting away non-essential variations, anti-causal representations naturally suppress features tied to identity context, thereby reducing information leakage. For instance, anti-causal model for health prediction may learn "disease - symptom" relationships while ignoring hospital IDs demographic quirks, which protects patient privacy.
and adentity-revealing details while preserving task-relevant causal
Relationship to Adversarially Robust and versarially robust representation learning (77 74 72
Representation Learning: Ad- seeks to construct representation spaces that remain stable under small but adversarially chosen perturbations, ensuring that the predicted labels remain unchanged across adversarial variants. A key limitation, however, is that such models may still rely on spurious correlations or environment-speciine artifacts that are vulnerable to manipulation. In contrast, anti-causal representation learning explicitly models how labels generate features, thereby shifting learned representations toward invariant causal mechanisms rather than fragile correlations.
Privacy-pres preserving representation learning 60 38 aims to learn representations that prevent models from encoding or leaking sensitive, environment-seee specific, or identity-relate attributes. These methods often employ regularization or cottmmationnthhoreeee constraints to balance utility and privacy. Anti-causal learning provides a principled complement: rather than obfuscating sensitive features post hoc, it avoids encoding them in the first place by aligning representations with label-to-feature generative mechanisms rertaannmmm-rttiiinnnggggn task-relevant causal features while discarding nuisance signals tied to individuals or contexts.
52

---

Environment
Environment
Digit Label
Even Digits (0.2,4.6.8)
Even Digits (0,2,4.6,8)
causal
75% RED 25% green
75% GREEN 25% red
Colored X
Image
Odd Digits (1,3,5,7,9)
spurious
Odd Digits (1,3,5,7,9)
75 GREEN 25% red
75 % RED 25% green
Environment
Label (digit) causes the feature X (color)
(Different environments create spurious correlations between digits and colors)
Figure 11: Anti-causal structure in Colored MNIST. The digit label Y causally generates the observed colored image X, while environment E introduces spurious digit-color correlations.
G A Concrete Example to Illustrate ACIA
In this section, we present an example step-by step-by-step to illustrate how ACIA works in practice. We focus on simplified version of the Colored MNIST (CMNIST) dataset to make the concepts accessible.
G.1 Step (Problem Setup): The Colored Digit Example
Imagine scenario where have images of handwritten digits that are colored either red green. We have two environments with different digit-color correlations:
In causal learning, we predict causes from effects (e.g., predict disease from symptoms). But in anti-causal learning, we predict effects from causes (e.g.. predict symptoms from disease). The digit label Y (cause) generates the color feature in image X (effect). The task is to predict Y from X. The goal is to learn representations that capture the invariant causal relationship between Y and X while removing the spurious environment-specifee correlations.
In environment 61, even digits (0,2,4.6.8) are mostly colored red (75%), while odd digits (1,3,5.7,9) are mostly colored green (75%). ). In environment e2, the correlation is reversed even digits are mostly green (75%), while odd digits are mostly red (75%).
G.2 Step 1: Define Causal Kernels for Each Environment
Causal kernels mathematically capture "how likely is a colored image given digit label." " They formalize the data generation process: Y (digit) - color > X (image).
First, define causal kernels that capture the conditional probabilities in each environment. For environment e,:
0.75
if A has red images and Y is even if A has red images and odd
0.25
K (ws, A)
(22)
0.25
if A has green images and is even
0.75
if A has green images and Y is odd
For environment the probabilities are reversed:
0.25
if A has red images and even
0.75
if A has red images and Y is odd
Keo (w, A)
(23)
0.75
if A has green images and Y is even
0.25
if A has green images and is odd
53

---

These kernels formalize how the probabilities of observable features (colors) depend on the label (digit) in each environment.
G.3 Step 2: Construct the Low-Level Representation
While the causal kernels in Step define the theoretical probabilities, we now need a practical neural network Φr to learn these relationships from data. Our low-level representation Φr captures raw relationships from input data. For each input image I X, or (x) extracts features that include both digit-related patterns and color information.
In the product causal space combining both environments, we integrate over the empirical distribution:
/ Ks(w', A) dQ((")
Kg' (w. (w. A)
(24)
This integration averages the causal kernels across all observed samples, creating a low-level repre- sentation space that captures both digit shape information (invariant across environments), and color information (varies with environment). In practice, this integration is implemented as a weighted average over training samples:
D Σ
DDyl Σ j=1
1
1
K (5,
Ke (wy, A) +
Ke (ey A)
(25)
De.
Dezl
This gives us the low-level causal dynamics 2, Q K, (K3 A) SE (TT),
e
H)) that captures both digit-specific and environmentt-seecfic information.
Algorithm 1 Causal Dynamie: Construction of Low-Level Representations
1: Input: Dataset ((r Mis ea) with images, labels, environments
2: Output: Low-level representation function Φ
3: Initialize convolutional neural network Φ with parameters 0L
4: for each mini-batch (Ir, y. e) do
5: 2r. + Φr. (2; (z;0r, 0y.)
Forward pass through low-level network 6: Compute environmeentconddiionne distributions P(zzly,e)
7: Compute causal kernel K3² 21. using:
8: K&" (ws A) R s ΣjEDy P(=L E Alys,es)
9: Update Or. using gradient descent
10: end for
11: return d1
In practice, we extract low-level representation du (x) using a neural network as below: Φ1 (x)) = Linear (imear((latten(CNN()))))
(26)
G.4 Step 3: Construct the High-Level Representation
The high-level representation ΦH builds on the low-level representation to extract only the invariant causal features. For our digit-color example, this means distilling information that is consistently related to the digit across environments, while discarding the spurious color-digit correlation.
The high-level kernel is defined by integrating over the domain of low-level representations:
/
K (w, A)
(w, du(z)
(27)
D2L This creates a causal abstraction 2 (V Kn)) that focuses on the digit shape rather than its color. Φи processes the low-level features to extract environment-invarian features: In practice, du is implemented as multi-layer perceptron:
(HL) MLP(2z) (ReL + +b)) b2
(28)
The high-level kernel K 2 integrates over the domain of low-level representations create representation that is invariant environment-speccffe features.
54

---

Algorithm 2 Causal Abstraction: Construction of High-Level Representations
1: Input: Low-level representations {=r., Je=1 labels (16 Mi) environments {e:)
2: Output: High-level representation function dis
3: Initialize ML.P network ΦH with parameters 0н
4: for each mini-batch (ZL. y, e) do
5: 2H (0 (H))
Forward pass through high-level network
6: for each label value yk do
7: for environments c, Cj where + do Compute 2H,e:.24 = Avg( [= ( = yk, yk, el0) = c:
8:
Compute 2 Avg( (2H (0) (1) yk e() = =e,))
9:
10:
Compute distance dij,k 2H, 2H,eg.8all ll2
11:
end for
end for
13:
Update OHr to minimize distances dij,ı (environment independence)
end for
15:
return 0H
G.5 Step 4: Joint Optimize on Loss and Regularizations
The complete optimization objective is:
min max Ee, (e([(X)) A Rı + A2 R2 C, cEE
(29)
where f = C ΦH o 0r is the complete predictive model.
Ri Environment Independence Regularizer: For each label we compute the mean high- level representation for each environment, then measure the distance between these means across environments:
R1 = Σ Σ
|| e.]
[8[= Wik e2]12
(30)
Ve citt,
Minimizing Ri ensures that representations for the same digit are similar regardless of environment (color).
R2 Causal Structure Consistency Regularizer:
We measure how well the high-level representa-
tion preserves the causal structure:
Ro (( P(yldo(z=)..)))l
(31)
This can be approximated by adding noise to the input (simulating intervention) and ensuring that the predicted output distribution remains consistent with the theoretical interventional distribution.
Practical Implementation of Rí: For each label yk. we first compute the mean high-level represen- lation for each environment, and then measure the distance between these means across environments. Minimizing this distance ensures that representations for the same digit are similar regardless of the environment.
Practical Implementation of Ry: We measure how well the high-level representation preserves the causal structure. In practice, this can be approximated by adding noise to the input (simulating intervention) and ensuring that the predicted output distribution remains consistent with the theoretical interventional distribution.
G.6
Step (Outcome): What ACIA Achieves?
Figure 12 illustrates the outcome visually. Given a new colored digit image x from potentially unseen enviromment, the low-level representation ér. (a:) extracts features related to both digit shape and color. The high-level representation Φn (r)r)())) focuses only on features invariantly associated with the digit label, discarding the spurious color correlation. Finally, the classifier predicts the digit based only these invariant features.
55

---

Algorithm 3 ACIA: Anti-Causal Representation Learning
1: Input: Dataset {(rs; Vi, ex) initialized dr. and Φh
2: Output: Optimized Φr, Φи,, and classifier
3: Initialize classifier C with parameters Oc
4: for each epoch do
5: for each mini-batch (ar, y/ e) do
6: EL + 01(x)
Low-level representations High-level representations b Predictions
7:
2H (H(
8:
ÿ + C(=H
9:
Compute classification loss L(ū. y)
10:
Compute Rí regularizer:
Environment independence
11:
RI Ey Leste, [|[ 1u:, ex] Blar yy eglll
Compute R2 regularizer:
Causal structure consistency
13:
Estimate conditional distributions P(y[=H) and P(y]do(zz))
R₂ = Ee. PyylzH, e.) P(y)do(2H)) 212
15:
Update all parameters using Liotal = Lets + d1 R1 + R2 end for
17: end for
return Φ1., ΦH. C
(x)):
(x)):
Input: Red "2
C: Prediction =2
Digit: 10.9 088, ...
Digit: [0.9, 0.85. ...] Color: [filtere]] Only digit kept
Coler (0:95 red]
(from e2?)
Both preserved
Environment-deeendeen
Environmenntinvariaat
Figure 12: The outcome of ACIA with concrete values. The low-level representation captures both digit shape (0.9, 0.85) and color (0.95 for red), while the high-level representation filters out color and retains only digit features, enabling robust prediction regardless of environment.
H Impact Statement
Societal Benefits. Our work advances causality and machine learning by developing robust repre- sentations for anti-causal settings. Our experiments on real-world dataset Camelyon17 demonstrate potential for improving diagnostic tools that generalize across hospital systems. We recognize that in medical contexts, some hospital-specific variations may actually contain diagnostically relevant information. ACIA addresses this through its two-level representation architecture, where low-level features can preserve mnvironment-specinn details while high-level abstractions capture invariant causal relationships. The regularization parameters in our optimization problem in Equation can be adjusted to balance between robustness and retention of clinically significant variations.
Safety and Reliability. ACIA's ability to disentangle causal mechanisms from spurious correlations has applications in safety-critical domains: autonomous vehicles must distinguish genuine hazards from enviromment--pecife artifacts (lighting, weather), and climate models must separate fundamental physical processes from regional measurement biases. The regularization parameters in Eq. 5g.5lal allow practitioners to balance robustness with retention of contextually meaningful variations.
Fairness Considerations. While not our primary focus, ACIA's framework could help identify and mitigate dataset biases by distinguishing environment-speccficc associations from invariant causal relationships. For instance, certain demographic groups cluster specifie environments, ACIA can separate spurious demographic correlations from genuine causal factors. However, emphasize
56

---

that: (I) practitioners must validate that "spurious" features are truly can-causal in their domain, and (2) fairness requires additional constraints beyond environmental invariance.
Limitations and Responsible Use. ACIA assumes the anti-causal structure Y + X < E is
appropriate for the domain. Misapplication to causal settings (X + Y) may discard informative features. In high-stakes applications, human domain experts should validate learned representations and intervention effects. We provide code and documentation to facilitate responsible deployment and reproducibility.
57


Reviews:
[{"review_id": "UAkM54bwvG", "reviewer": "Reviewer_xrNC", "summary": "The authors propose ACIA, a measure-theoretic framework for anti-causal representation learning. ACIA aims to disentangle invariant causal features from environment-specific variations while making minimal assumptions about the underlying causal structure of the data-generating process. The main contributions of the paper include: A generalized intervention model that accommodates both perfect and imperfect interventions via interventional kernels; A construction of causal dynamics and causal abstractions to model invariant causal relations; A well-motivated optimization objective designed to identify stable causal features across domains/environments without requiring explicit DAG specifications.Theoretical guarantees for out-of-distribution generalization, including bounds on performance gaps between training and unseen environments. Experimental results on both synthetic and real-world datasets demonstrate ACIA’s ability to capture and leverage anti-causal structures, outperforming state-of-the-art methods.", "questions": "1. While the paper discusses theoretical complexity, could the authors also provide empirical training time or computational cost across different datasets to better assess practical scalability?\n\n2. Line 94: The reference to Appendix ?? appears to be a placeholder. Could the authors clarify or correct the appendix citation?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper is well written and logically structured, facilitating reader comprehension.\n\n2. The authors provide rigorous and in-depth theoretical analysis to support their proposed method.\n\n3. Comprehensive experimental evaluations validate the effectiveness of ACIA across diverse datasets.\n\nWeaknesses\n\nThe review currently does not list specific weaknesses, but a minor area for improvement might be in the empirical clarity regarding computational cost (see question below).", "was_revised": true, "final_justification": "I have read the authors' response and provided final feedback.", "rebuttal": "### Q1/W1: Empirical training time on the evaluated datasets\n\n**Response:** The table below reports the runtime on a laptop with a single GPU (3.3 GHz, 32 GB RAM), using a desired precision $\\epsilon=0.01$ and failure probability $\\delta=0.05$. We observe that ACIA completes in approximately 3–16 minutes under these settings.\n\n\n| Dataset | \\#Samples | Input Dimension| Time (seconds) |\n|---------|-------------|----------------|----------------|\n| CMNIST | 60K | 2352  | 580.7|\n| RMNIST | 60K | 784  | 973.6|\n| Ball Agent | 15K | 12288  | 204.7|\n| Camelyon17 | 84K | 6912 | 799.6|\n\n\n ### Q2: Appendix Placeholder in Line 194\n \n **Response:** Thanks for pointing out. It is Appendix C.5."}, {"review_id": "AaZKRVlVB0", "reviewer": "Reviewer_FVuS", "summary": "The paper “Measure-Theoretic Anti-Causal Representation Learning” addresses the challenging anti-causal representation learning problem, and anti-causal invariant abstractions (ACIA) that can accommodate both perfect and imperfect interventions through interventional kernels. It contains two-level of representation learning, where the low-level capture the causal dynamics and high-level learn invariant predictor. Theoretical results are given to guarantee the OOD generalization. The experiments on synthetic and real-world datasets demonstrate the effectiveness of ACIA, also validate the theoretical results.", "questions": "Please refer to the weaknesses.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The problem is well-defined and the theoretical results are promising\n\n2. A unified theoretical framework is given to model the anti-causal learning problem, the motivation of ACIA is clear and the objective of ACIA has a solid theoretical foundation\n\n3. The theoretical results (e.g., convergence of ACIA, OOD generalization bound, etc.) further demonstrate the effectiveness of ACIA\n\n3. The experiments validate the proposed theoretical results\n\n**Weaknesses**\n\n1. Both low-level and high-level learning processes encourage an environment-invariant representation, and its seems the goals of these two process overlap to some extent, what are the main differences between these two process? Are these two processes towards the same goal in different ways?\n\n2. The description of Figure 2 is insufficient, and Figures 1 and 2 overlaps, it is better to merge them into one figure\n\n3. Typos: Line 194 \"Appendix ??\"; Title of section 3.3, “objection” -> “objective”; Footnote in page 3 \"P(Y|do(X=x)\" -> \"P(Y|do(X=x))\"", "was_revised": true, "final_justification": "Concerns are well addressed, I have raised the significance score to 4", "rebuttal": "###  Q1: Clarify the roles of the two-level learning \n\n**Response:** The two-level representation learning process in our framework serves **distinct roles within the causal hierarchy**. We provide the following clarification:\n\n- **Low-level representation mapping** ($\\phi_L$) aims to extract features directly from raw observations (without requiring explicit DAGs), capturing the **anti-causal structure** of the data. This includes how both the label variable $Y$ and the environment variable $E$ influence the observation $X$. The learned features support reasoning under both **perfect and imperfect interventions**, enabled by the introduced **interventional kernels**.\n- **High-level representation mapping** ($\\phi_H$) is applied on top of $\\phi_L$, and its purpose is to extract **causally invariant features** — specifically, preserving the effect of $Y \\rightarrow X$ while **discarding spurious environment-related signals** from $E \\rightarrow X$. Formally, it enforces the condition: $\\phi_H(\\phi_L(\\mathcal{X})) \\perp E \\mid Y$, ensuring that the final representation is **invariant to environment-specific variations**.\n\nTogether, the two-level mappings cooperate to learn **robust anti-causal representations**, with $\\phi_L$ capturing rich generative mechanisms and $\\phi_H$ filtering out non-causal noise for improved generalization across environments.\n\n\n\n### Q2: Figure 1 and Figure 2 mergerd into one\n\n**Response:** Thanks for the suggestion! We will merge the two figures! \n\n### Q3: Typos: Line 194 \"Appendix ??\"; “objection” -> “objective”; \"P(Y|do(X=x)\" -> \"P(Y|do(X=x))\" \n \n **Response:** Thanks for pointing them out and correction!"}, {"review_id": "v1v49AEhc9", "reviewer": "Reviewer_WVhG", "summary": "This paper proposes a framework called ACIA for anti-causal representation, aiming at disentanglement between invariant causal features and environment variations. ACIA is a two-level representation framework, where the low-level representation captures the direct causal information from data and labels. And the high-level representation serves as an information bottleneck to filter environment-specific noise. The authors also prepare rather detailed experiments. Compared with different methods in four datasets, ACIA obtains several best results in several metrics including ACC, EI, LII, IR.", "questions": "1.As mentioned before, the experiment part is relatively sufficient, but the visualization only focus on the disentanglement of the representations using t-SNE, which is a very classical and common method to visualize representations. However, it can’t uncover the causal relationship between different features. Could you provide some visualization to uncover the correspondent relationship of the two-level representation learning? \n2.Explicit DAG is abandoned during the optimization, which may weaken the causal properties and required some prior to complement. Could you provide a comparison of the prior information used in different methods? Experiments should be conducted at the same prior level to be fair.\n3.ACIA captures intervention uncertainty through probability measures and avoid the assumption of perfect intervention. In addition to the experiment, can you provide some theoretical analysis of the intervention assumption?", "limitations": "The paper adopts an adversarial training approach to optimize the ACIA framework, but it does not explore alternative optimization strategies that could potentially achieve similar or better results. For instance, non-adversarial methods like variational inference or simpler gradient-based techniques might suffice, yet the paper lacks a comparative analysis to justify the exclusive reliance on adversarial training beyond its practical utility. This omission limits the understanding of the framework’s flexibility and robustness across different optimization paradigms.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1.The paper has thorough theoretical analysis of the performance of ACIA, e.g.,  convergence property, generalization bound in terms of sample complexity and interventional kernels in the anti-causal setting, and environmental robustness. These contribute to the scientific rigorousness of this paper and provide some reference meaning to peer work.\n2.There are relatively sufficient experiments in this work including several classical datasets such as CMNIST and RMINST. It also obtains competitive results in more challenging scenario. From an experimental perspective, this work is relatively qualified.\nWeakness:\n1.Though the paper has long discussion about the ACIA, the two-level representation algorithm is still vague especially in the usage of the causal dynamic and its kernels. In appendix G, the pseudo code only briefly describes this crucial part which could be more detailed properly.\n2.The paper proposes an adversarial method to train the model, it may be useful in practice. There are some theoretical analysis around the objective function in Section 3.2. But could you illustrate the necessity of the adversarial training method or it’s just a choice of methodology? Is there more analysis to bridge the gap between the theory and the training method?", "was_revised": true, "final_justification": "I would like to give the borderline acc as the final justification. This paper proposes some interesting ideas of anti-causal representation learning, although some aspects can be further improved. I would not be upset if the final recommendation is acceptance, either.", "rebuttal": "### Q1: Visualize causal relationship between different features from the two-level representation learning. \n\n**Response:** Thank you for the valuable suggestion!\n\nTo visualize the causal relationship between low-level and high-level features, we will employ **gradient-based attribution methods** (visualization via heatmap) to quantify how each dimension in the low-level representation contributes to each dimension in the high-level representation. For instance, on CMNIST, this approach will reveal how **low-level features**—such as digit **shape** and **color**—are transformed through the two-stage representation learning process. Specifically, we will generate **flow diagrams** illustrating:\n- Which low-level neurons are activated by shape vs. color,\n- And how only **shape-related dimensions** in low-level representations are used by high-level representations to form **causally invariant features** (i.e., digit identity).\n\nThis visualization will help confirm that the high-level representation successfully filters out environment-specific information while preserving task-relevant signals.\n\n\n\n### Q2: Explicit DAG is abandoned during the optimization. Comparison of the prior information used in different methods. \n\n**Response:** We highlight that **eliminating dependency on explicit DAG is actually one of the key advantages of ACIA**. This is due to the fact that obtaining the accurate and explicit DAG in real-world scenarios is often difficult and becomes our motivation.\n\nPlease see below table the prior information of the compared SOTA causal representation learning methods.  Our experimental results in the paper demonstrate that, though requiring less information, ACIA still outperforms the compared SOTAs. \n\n\n| Method | Causal Structure           | SCM Knowledge | Intervention Type |\n|--------|----------------------------|---------------|-------------------|\n|ACTIR (NeurIPS'22) | Anti-causal Y -> X <- E | Variable roles only | Perfect only |\n| CausalDA (ICLR'23) | DAG structure              | Variable types | Perfect only |\n| LECI (NeurIPS'24) | Partial connectivity       | Variable relationships | Perfect only |\n| **ACIA** | **anti-causal Y -> X <- E** | **Variable roles only** | **Both perfect and imperfect** |\n\n\n**Note:**  \n- **\"Variable roles only\"** refers to knowing the role of each variable in the causal model, i.e., identifying which variable is the **target** (Y), which are **observations** (X), and which represent **environmental factors** (E).  \n- **\"Variable types\"** means knowing the **data type** of each variable, such as whether it is **binary**, **categorical**, or drawn from a **specific noise distribution**.  \n- **\"Variable relationship\"** refers to knowing the **causal structure** (i.e., the **directionality** in the **causal DAG**) among the variables.\n\n\n### Q3: Provide the theoretical analysis of the intervention assumption.\n\n**Response:** Perfect and imperfect intervention types are unified in the the introdued **interventional kernel framework**, stated below: \n\n\nTheorem 2 (Interventional Kernel): For any intervention $\\mathbb{Q}: \\mathscr{H} \\times \\Omega \\rightarrow [0,1]$, there exists a unique interventional kernel:\n\n$K_S^{do(\\mathcal{X}, \\mathbb{Q})}(\\omega, A) = \\int_{\\Omega} K_S(\\omega, d\\omega') \\mathbb{Q}(A|\\omega')$\n\nIn *perfect interventions*, $\\mathbb{Q}(A|\\omega') = \\mathbb{Q}(A)$, while $\\mathbb{Q}(A|\\omega')$ varies with $\\omega'$ in *imperfect interventions*. \n\n\nFurther, **Corollary 2** shows that **interventional kernel invariance** properties $K_S^{do(X)}(\\omega, \\{Y \\in B\\}) = K_S(\\omega, \\{Y \\in B\\})$ hold regardless of intervention perfectness.\n\n\n### W1: More details on the two-level representation algorithm\n\n**Response:** Due to space constraints, we have provided the full details of the two-level representation algorithm in Algorithms 1–3 (Pages 44 and 55) in Appendix. Do these pseudocode descriptions help clarify the method?\n\n\n\n### W2/L1: Nessisity of the min-max adversarial training approach.\n\n**Response:** ACIA adopts a *min-max* optimization framework to ensure **worst-case** robustness across environments. This formulation is theoretically grounded in our OOD generalization bounds (Theorem 7), which require guarantees on worst-case performance across diverse environments.\n\n\nWe emphasize that existing (environment-)invariant representation learning methods—such as IRM, Rex, and VRex—are designed to optimize for **worst-case performance across environments**. This worst-case objective is critical: without it, the learned representations often fail to disentangle environmental factors effectively, as has been **extensively validated in prior literature**."}, {"review_id": "z7aHNb5HKe", "reviewer": "Reviewer_qtK8", "summary": "This paper presents a measure-theoretic framework for anti-causal representation learning. The proposed framework handles both perfect and imperfect interventions. The proposed framework builds two levels of latent representations: a low-level \"causal dynamic\" that disengages the environment-specific variations and a high-level \"causal abstraction\" that extracts environment-invariant features.", "questions": "- How do we know the causal direction between the label and the observations in general?\n- I'm a bit confused about the causal dynamic and causal abstraction part in the framework. What's the difference between \"invariant across environments\" and \"environment-independent\"? In l.137, if $P(\\phi_L(\\mathcal{X}|Y)$ is constant across environments $\\mathcal{E}$, isn't that also means it is independent of $\\mathcal{E}$ given $Y$?\n- Could you explain more about how the causal-dynamic learns the low-level representations and disentangles the influence of the environment? How is it different from the \"causal abstraction\"? Also, in practice, for example, when using a neural network, how do you decide the part of high-level and low-level features?\n\n\nbroken reference in line 194", "limitations": "I can't find an explicit discussion about the limitations of the proposed work. Including a discussion on the practical scenario of anti-causal assumption might be helpful.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n- The paper proposed a novel approach based on a theoretically rigorous framework. \n- The proposed framework, accommodating both soft and hard interventions, makes it more applicable and robust in real-world settings.\n- The proposed method is evaluated through both synthetic and real-world experiments.\n\n# Weaknesses\n\n- Some part of the writing seems unclear. For example, the motivation for having two-level representations.\n- It is not obvious when we can apply the proposed method by assuming the anti-causal relationship in the data.", "was_revised": true, "final_justification": "The paper proposed a novel approach with theoretically rigorous results. My concerns are adequately addressed during the rebuttal.", "rebuttal": "###  W1: Motivation for having two-level representations.\n\n**Response:** Thanks for the comment! \n\nWe first emphasize that **existing invariant or causal representation learning methods** typically focus on learning a **single-layer representation mapping** $\\phi: X \\rightarrow Z$. While effective in some settings, these approaches face several key challenges in the context of **anti-causal learning**, as outlined in our introduction:\n\n- They often assume **perfect interventions**, which are difficult or impractical to implement in real-world scenarios.\n- They typically require access to **explicit structural dependencies** (e.g., via Structural Causal Models), which are rarely available.\n- Many methods also assume **IID data** or require **knowledge of the test-time distribution**, limiting their applicability in realistic settings involving distribution shift.\n\n\nWe propose to address all the challegnes through the two-level causal hierarchy framework. \n- **Low-level representation mapping** ($\\phi_L: X \\rightarrow Z_L$) extracts features directly from raw observations (without requiring explicit DAGs), capturing the **anti-causal structure** of the data, including both causal ($Y \\rightarrow X$) and spurious ($E \\rightarrow X$) patterns. The learned low-level representations support reasoning under both **perfect and imperfect interventions**, enabled by the introduced **interventional kernels**.\n- **High-level representation mapping** ($\\phi_H: Z_L \\rightarrow Z_H$) is applied on top of $\\phi_L$. It serves as an information bottleneck that extracts **causally invariant features** — specifically, preserving the effect of $Y \\rightarrow X$ while **discarding spurious environment-related signals** from $E \\rightarrow X$. \n- Together, the two-level mappings cooperate to learn **robust anti-causal representations**, with $\\phi_L$ capturing rich generative mechanisms and $\\phi_H$ filtering out non-causal noise for improved generalization across environments.\n\n\n\n\n###  Q1/W2: When to apply the proposed method/How to know anti-causal relationship in general. \n\n**Response:** An anti-causal relationship occurs *when the task label causes the observational features*. This structure is common in many real-world applications. For instance, we show some examples below: \n\n- *Medical diagnosis*: The presence of a disease causes specific symptoms or test results.\n- *Image classification*: The identity of an object determines its visual appearance.\n- *Quality control*: Defects in a product lead to anomalies in sensor measurements.\n- *Recommendation system*: Users' preferences drive their behavior patterns.\n\n\nTechnically, we can identify anti-causal relationships through interventional tests: if intervening on features X does not change the label Y, but intervening on Y does affect X, then the relationship is anti-causal. Our **Corollary 2 (Interventional Kernel Invariance)** formalizes this intuition and provides a principled test using the interventional kernel we introduced.\n\n\n\n\n### Q2: Difference between \"Invariant Across Environments\" and \"Environment-Independent\"\n\n**Response:** Thank you for the comment, and we apologize for the confusion.\n\n- **Invariant across environments** refers to **consistency under distribution shifts**. Specifically, it means that the learned representation remains **stable** across different environments/distributions. \n- **Environment-independent**, on the other hand, implies that the representation is **statistically independent of the environment variable**. That is, the features **do not contain any information** about the environment.\n\nMapping this distinction to our two-layer representation learning setup:\n\n- The **low-level representation** $\\phi_L$ learns features such that the distribution $\\Pr(\\phi_L(\\mathcal{X}) \\mid Y)$ is **invariant across environments**, but it may still encode **residual information** about the environment $E$.\n- The **high-level representation** $\\phi_H$ then filters out this residual information to enforce **conditional environment-independence**, i.e., $\\phi_H(\\phi_L(\\mathcal{X})) \\perp E \\mid Y$.\n\n**Illustrative example:** Two environments may exhibit identical $\\Pr(\\phi_L(\\mathcal{X}) \\mid Y)$, meaning the representations appear invariant across environments. However, the features $\\phi_L(\\mathcal{X})$ may still contain **environment-specific markers** (e.g., background texture or domain-specific artifacts). $\\phi_H$ further removes these markers to achieve **true environment-independence**.\n\nWe will revise the wording in the paper to better reflect this distinction.\n\n\n### Q3: How does Causal Dynamic learn low-level representations? How is it different from Causal Abstraction? How are high-level and low-level feature parts determined in a neural network?\n\n**Response:** Thank you for the valuable question!\n\nRecall that the **low-level representation mapping** $\\phi_L$ aims to capture the **anti-causal structure**, i.e., both $Y \\rightarrow X$ and $E \\rightarrow X$. The **Causal Dynamic** module facilitates this learning by optimizing a **supervised loss** in conjunction with the **$R_2$ regularizer**. Minimizing $R_2$ encourages the low-level representation to align with the true causal mechanisms underlying the data, promoting better generalization under intervention.\n\nBuilding on top of $\\phi_L$, the **Causal Abstraction** module learns a **high-level representation mapping** $\\phi_H$, guided by the **$R_1$ regularizer**. The goal is to further enforce **statistical independence from the environment variable** $E$. Specifically, $R_1$ measures the **discrepancy between the expected high-level representations across environments**, conditioned on the label $Y$, and minimizing it removes environment-specific information while retaining causal features.\n\nIn practice, we design the architecture with the following principles:\n- The **low-level representation** should be **sufficiently rich** to capture both label-relevant and environment-dependent signals (i.e., $Y \\rightarrow X$ and $E \\rightarrow X$).\n- The **high-level representation** serves as an **information bottleneck**, filtering out environment-dependent variations, and thus is set to a **lower dimension** than the low-level representation.\n\nIn our experiments with image data:\n- $\\phi_L$ is implemented as a **CNN encoder**, producing a $d_L$-dimensional representation.\n- $\\phi_H$ is a **small MLP bottleneck**, outputting a $d_H$-dimensional high-level representation, where $d_H < d_L$.\n\n\n### L1: Discuss the limitations of the proposed work \n\n**Response:** Thanks for the suggestion! We outline the potential limitations below:\n\n- **Scalability**: Generalizing ACIA to large-scale, high-dimensional data may be computationally expensive and time-consuming.\n- **Optimal regularization selection**: The optimal hyperparameters $\\lambda_1$ and  $\\lambda_1$ in the objective function may dependent on the application domain. It requires careful tuning to obtain their optimal values.  \n- **Complex causal structures**: ACIA is primarily designed for anti-causal settings. Its applicability to more complex causal structures—such as confounded-descendant or confounded-outcome [59] scenarios—remains unclear and warrants further investigation."}]

Author rebuttal:

Meta-review: The authors study the problem of causal representation learning in the anti-causal setting where labels cause features. They propose a theoretically sound approach based on measure-theory that allows them to present a number of interesting results. Although there was some confusion about the problem and its relevance at first, these were alleviated during the rebuttal and discussion. All reviewers agree that the problem and solution are relevant and interesting. There remain minor concerns about the clarity and empirical evaluation, which I share. Nevertheless, I'm happy to recommend acceptance.
Decision: Accept (poster)