CARL: Preserving Causal Structure in Representation Learning
Authors: Yulong Li
Xiwei Liu
Feilong Tang
Zhixiang Lu
Ming Hu
Yichen Li
Haochen Xue
Peixin Guo
Jionglong Su
Yutong Xie
Eran Segal
Imran Razzak
Venue: ICLR 2026 Poster (2026)
Keywords: Structure-preserving Constraints
Representation Learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Cross-modal representation learning is fundamental for extracting structured information from multimodal data to enable semantic understanding and reasoning. However, current methods optimize statistical objectives without explicit causal constraints, where nonlinear mappings can introduce spurious dependencies or eliminate critical mediators, leading to representation-induced structural drift that undermines the reliability of causal inference. Therefore, establishing theoretical guarantees for causal invariance in cross-modal representation learning remains a foundational challenge. To this end, we propose Causal Alignment and Representation Learning (CARL), which explicitly embeds causal structure preservation constraints into cross-modal alignment objectives. Specifically, CARL introduces a multi-consistency loss architecture that jointly optimizes conditional independence preservation and information bottleneck regularization to balance cross-modal compression with critical variable retention, ensuring low-density modalities are not masked by high-density reconstruction demands. We further incorporate monotonic alignment consistency loss to establish correspondence between semantic similarity and representation distance through Spearman correlation, and Markov boundary preservation loss to maintain identifiability conditions including backdoor, frontdoor, and instrumental variable criteria in the shared representation space. In synthetic experiments with known causal ground truth, CARL achieves state-of-the-art performance in preserving conditional independence patterns and maintaining causal query identifiability under structural uncertainty. Real-world validation on Human Phenotype Project data reveals that CARL successfully preserves causal structures between fundus vascular representations and cardiovascular events, demonstrating its capacity for reliable cross-modal causal inference in complex biomedical applications.

Published as a conference paper at ICLR 2026
CARL: PRESERVING CAUSAL STRUCTURE IN REPRE- SENTATION LEARNING
Yulong Li!!,, Xiwei Liu'*, Feilong Tang', Zhixiang Lu², Ming Hu', Yichen Li', Haochen Xue Peixin Guo?, Jionglong Su², Yutong Xie!, Eran Segal Imran Razzak
'Mohamed bin Zayed University of Artificial Intelligence 2Xi' an Jiaotong-Liverpon University
Monash University
"Equal contribution
i "Corresponding authors
ABSTRACT
Cross-modal representation learning is fundamental for extracting structured in- formation from multimodal data enable semantic understanding and reasoning. However, current methods optimize statistical objectives without explicit causal constraints, where nonlinear mappings can introduce spurious dependencies or eliminate critical mediators, leading to representation-indueed structural drift that undermines the reliability of causal inference. Therefore, establishing theoretical guarantees for causal invariance in cross-modal representation learning remains a foundational challenge. To this end, we propose Causal Alignment and Represen- tation Learning (CARL), which explicitly embeds causal structure preservation constraints into cross-modal aligmment objectives. Specifically, CARL introduces multi-consistency loss architecture that jointly optimizes conditional indepen- dence preservation and information bottleneck regularization to balance cross- modal compression with critical variable retention, ensuring low-density modal- ities are not masked by high-density reconstruction demands. We further incor- porate monotonic alignment consistency loss establish correspondence between semantic similarity and representation distance through Spearman correlation, and Markov boundary preservation loss to maintain identifiability conditions including backdoor, frontdoor, and instrumental variable criteria in the shared representation space. synthetic experiments with known causal ground truth, CARL achieves state-of-the-art performance in preserving conditional independence patterns and maintaining causal query identifiability under structural uncertainty. Real-world validation on Human Phenotype Project data reveals that CARL successfully pre- serves causal structures between fundus vascular representations and cardiovascu- lar events, demonstrating its capacity for reliable cross-modal causal inference in complex biomedical applications.
INTRODUCTION
Representation learning is fundamental to modern deep learning |Bngii al. (2013), 3), yet prevalent methods risk distorting the very structure they aim to capture /Arjossyy et al. (2019). By optimiz- ing purely statistical criteria, objectives based reconstruction Goodffllow al. (2014); Kingma & Welling (2013). contrastive learning Radford et al. (2021). or large-scale pretraining Dosovitskiy al. (2021 i can inadvertently corrupt the underlying causal graph of the data. These nonlinear map- pings may introduce spurious dependencies or eliminate critical mediators [Giirhos et al. (2020). a failure we term pnarseetaaiion-iduuee structural drift. This drift critically undermines the promise of causal machine learning: reliable interventional estimation Shalit et al. (2017), counterfactual rea- soning Zuo et al. 2023), and distribuutn generalization Liu et al. (2021a). For instance, in the Human Phenotype Project (HPP such drift could obscure the true causal link between retinal
https://knnwweedgeeees www.lledeeeaseeeeee..eee pheno.al

---

Published as a conference paper at ICLR 2026
vascular features and cardiovascular events amidst confounders like age and blood pressure Shapira jet al. (2024). To alleviate this, we propose the Causal Structure Preservation (CSP) principle, which requires that representations preserve key conditional independences, Markov boundary informa- tion, and the identifiability of causal effects. Establishing theoretical guarantees for such causal invariance in cross-modal learning is foundational problem address herein.
A critical gap exists between the optimization objectives of current cross-modal representation learn- ing methods and the preservation of causal invariance. [Snn et al. (2025). For instance, classical approaches like DCCA Andrew et al. (2013) focus on maximizing correlation without structural verification mechanisms, failing guarantee Markov boundary preservation and identifiability con- sistency. More contemporary paradigms, while powerful, inherit similar blindness to causality. Contrastive models like CL.IP Radford et al. (202T) learn cross-modal alignment via an InfoNCE loss but fail to enforce monotonic relationship between semantic similarity and representation dis- tance, property crucial for structural consistency. Pivot-based alignment paradigms like AL et al. (2021) and ImageBind Girdhar et al. (2023) optimize geometric properties of representations rather than the explicit structural criteria required for causal inference. Consequently, none of these prominent approaches can guarantee that their learned representations faithfully maintain key causal properties like conditional independence, Markov boundaries, or identifiability conditions.
This abovementioned gap exposes three fundamental challenges that hinder reliable cross-modal causal inference: i): Cross-modal Information Bottleneck (CIB): asymmetric compression from in- formation density differences across modalities causes key variables from low-density modalities to be masked by reconstruction demands of high-density modalities, weakening identifiability of causal queries. ii:: Modal Alignment Consistency (MAC): after independent encoding, shared space dis- lances lack monotone correspondence with semantic similarity, preventing distance-based retrieval from guaranteeing semantic consistency. řij: Cross-modal Identifiability Consistency (CIC): iden- tifiability conditions including backdoor, frontdoor, and instrumental variables lack establishment criteria in shared representations Pearl] (2009). The core challenge, therefore, is to establish verifi- able, structure-preet pereetrr-preserving mechanisms that are robust to such modal heterogeneity.
To this end, we introduce Causal Alignment and Representation Learning (CARL), framework that establishes verifiable structure-preserving mechanisms while handling modal heterogeneity by embedding causal invariance constraints into cross-modal alignment objectives. CARL implements novel CSP principle by co-optimizing three synergistic losses: conditional independence loss to preserve the causal graph; a Markov boundary retention loss, implemented via conditional mutual information estimation, ensure key variables from low-density modalities
masked by high- density ones; and a monotonic alignment consistency loss guaranteeing semantic consistency by maximizing the Spearman correlation between semantic differences and representation distances. This joint optimization ensures crucial identifiability conditions, including backdoor, frontdoor, and instrumental variable criteria, are preserved in the shared representation space. Validation on HPP demonstrates CARL maintains causal structure among the representation of retinal fundus, sleep, anthropometrics and cardiovascular. Our main contributions are:
We formalize the Causal Structure Preservation principle and indentify three core theoretical challenges in causally-aware cross-modal representation learning (CIB, MAC, and CIC), establish- ing a new theoretical foundation for causal invariance guarantees in cross mooss scenarios.
We propose CARL, a framework that jointly optimizes three structure- prueture-preserving losses: condi- tional independence, Markov boundary retention, and monotonic alignment consistency, achieving verifiable causal structure preservation and cross-modal alignment under heterogeneity.
Empirical validation on HPP demonstrates that CARL preserves the causal relationships between retinal vascular features and cardiovaseular events established in existing research, validating the framework's 's effectiveness in maintaining cross-modal causal structure.
PROBLEM FORMULATION AND CAUSAL SETUP
We formalize the causal structure preservation problem by establishing the causal model and cross- modal setting (Section 2.I. encoder configurations (Section (.2),, and the CSP principle (Sec- tion 2.3). then instantiate these concepts in a biomedical application (Section 2.4
2

---

Published as a conference paper at ICLR 2026
2.1 CAUSAL MODEL AND TROSS-MODAL SETTING
We formalize the underlying causal mechanism with a Directed Acyelic Graph (DAG) 9 = (V, E), where V {I, Y ] consists of treatment T T. set of potential mediators M (M...... Ms a true outcome Y* E y and set of covariates X E A. These variables may be continuous, discrete, or mixed. The edges E represent direct causal relationships, where the parent set of any node V; E V is denoted by (V,( We assume the true graph G" satisfies the Causal Markov and Faithfulness assumptions Pearl (2009), enabling conditional independencies to be inferred from the graph. Appendix A.4 details these assumptions and Causal Sufficiency.
In the cross-modal setting, we observe tabular data (T, M, X) alongside image modalities: JM for the mediator and I as a proxy for the true outcome, Y*. For instance, in biomedical phenotyp- ing (Section 2.4). T" may represent blood pressure, M captures vascular features observed both as measurements and fundus images, and Y* denotes cardiovascular outcomes. The observed outcome Y equals Y when directly available; otherwise, Y = = serves as a feature-extracted proxy. CARL supports three configurations based on this: IM (using } IY (using I and DUAL (both).
We aim to learn encoder family & = {ET, EM, EJM, Eạr that map these heterogeneous in- puts shared representation space 2 C IRd, yielding representation Z7 (T) Zir (M), Ey )}, and Zy E {Ey(Y), Eyr }} Our core objective, structure preservation, dictates that if the original variables satisfy 1 Y M1 the learned representations must adhere to the constraint M1(2r; Zy ZM) S S", where M M-3--1 is the conditional mutual information and Ç* is the minimal approximation error for the encoder class.
Assumption 1 (Causal Markoy and Faithfulness): The data generating process is assumed to follow DAG that satisfies the Causal Markov and Faithfulness assumptions. We further assume that causal effects are identifiable, which requires that either set of covariates X blocks all backdoor paths, or the frontdoor/IV criteria are met (Appendix (..), and that both positivity (overlap) and consistency (SUTVA) hold (Appendix A.3).
Assumption (Separable Measurement and Encoders): Images modalities are assumed to fol- low a separable measurement structure aM gM buyle 7ar), IY gy gy(ay, bayle; (y)), where a.M ay represent semantic content variables (the causal signal we aim to preserve), bayle captures style factors, and TMs TJy denote measurement noise. We assume (MA 77x 1 (T, " and bayle 1 (T,Y*) XM (Appendix A.6.I)) All encoders mapping to the shared space Z are assumed be Lipschitz-boundee for stability (Appendix (...2).
With the causal model and structural assumptions in place, we now describe the encoder architec- tures that map heterogeneous inputs to shared representation space while respecting causal con- straints. We consider three configurations corresponding to different image modality availability.
2.2 CROSS-MODAL ENCODING CONFIGURATIONS
Building upon the causal assumptions defined previously, our framework learns an encoder family to map heterogeneous inputs into unified representation space Z C Rd. To support variable tracing and ensure stability, all encoders are deterministic, Lipschitz-constrainee mappings, with tabular inputs processed by multilayer perceptrons and images by convolutional architectures. The framework flexibly supports three configurations based on the availability of image data:
Scenario 1. (IM): image I M represents the mediator, where encoder EJι M + ZJM and tabular encoder EM : M + ZM learn shared representations through alignment losses. The constraint ensures no spurious path T + MM Y is introduced.
Scenario 2. (IY): image I' serves outcome proxy. Encoder E,y generates 2,r Efr (IY) Z. An observable outcome Y = ={I") E IR is extracted, assuming a monotonic and calibrat- able relationship with the true outcome Y*. The calibration function h : Z + R then map the representation back prediction Ý h(2,v), optimized to align with the observed value Y.
Scenario 3. (DUAL): both image modalities are used simultaneously, but conditioning on M and 7Y jointly is avoided in loss computation prevent collider bias.
3

---

Published as a conference paper at ICLR 2026
2.3 THE CAUSAL STRUCTURE PRESERVATION PRINCIPLE
Having defined the causal graph & and encoder configurations in the previous subsections, we now formalize what it means for learned representations preserve causal structure. We capture this through the Causal Structure Preservation (CSP) principle.
Definition 2.1 (s-Causal Structure Preservation). A representation learning system (E,2) sat- isfies E-CSP if and only if there exists ε 2 0 such that the following three conditions hold: (i) Conditional Independence Transfer: For any conditional independence T Y M implied by the original causal graph, the corresponding representations must satisfy MI(Zr; Zy ZM) S E; (ii) Markov Boundary Preservation: MI Zy) 2 MI
E: (iii) Monotonic Align- ment Consistency: In the IY configuration, the Spearman correlation ps between semantic differ- ences Aaij = [4 - az} and representation distances Σz; |||; - žحاl2 must be lower-bounded: Ps(Aa, Az) 2 Co E,, where co E (0, 1] is constant determined by the under-preserving margin sumption. These three conditions jointly ensure causal structure preservation: condition (i) prevents spurious dependencies by enforcing conditional independencies from the original graph; condition (ii) prevents information loss by requiring the mediator to retain predictive power; condition (iii) ensures geometric consistency between semantic similarity and representation distance. Together, they maintain the identifiability of causal effects in the representation space.
These three conditions jointly ensure causal structure preservation: condition i) prevents spurious direct pathways by enforcing conditional independence constraints from the original causal graph; condition üi) prevents trivial satisfaction of i) through information loss by requiring the mediator to retain predictive information about the outcome; and condition iii) ensures geometric consistency by aligning semantic similarity with representation distance. Together, they guarantee that causal relationships are neither distorted by spurious shortcuts nor weakened by information bottlenecks.
2.4 MOTIVATING APPLICATION: HUMAN PHENOTYPE PROJECT
Having formalized the framework, we now present our motivating application. We conduct our real- world evaluation on the HPP, large-scale, longitudinal study that collects deep phenotypic profiles from diverse population, offering holistic perspective on the factors influencing health and dis- ease Kohn et al. (2025). As previewed in Figure the dataset is distinctly multi-modal, containing a wide variety of participant information categorized into tabular records, time series, and images, covering multiple physiological systems. Specifically, the HPP encompasses health information spanning over 30 distinct modalities, including but not limited to gut microbiome, body composi- tion, genetics, and blood tests. This rich, multi modal structure, with its inherent complexity and potential for confounding, makes the HPP an ideal and challenging testhed for evaluating CARL's ability to preserve causal structures in real-world biomedical application.
This rich multi-modal structure presents a critical methodological challenge: standard representation learning methods that optimize purely statistical objectives may inadvertently distort the underlying causal relationships. For instance, a naive encoder could learn to predict cardiovascular events di- rectly from retinal images by exploiting spurious correlations, thereby bypassing the true mediating pathway through vascular features and blood pressure. Such oppresentationiindee induced structural drift would corrupt our ability to perform valid causal inference and estimate intervention effects. This motivates our focus on causal structure preservation in cross-modal representation learning.
3 METHODOLOGY
Our proposed CARL framework provides unified approach to preserving causal structure within cross-modal representation learning. CARL learns an encoder family & = {Er, EM E,M, E,y which can be extended with optional encoders Ey, Ex for outcome encoding and confounder con- trol. This is designed to map heterogeneous inputs, spanning tabular variables (T, M, Y*. X) and image observations I E {1 12 to a shared representation space while preserving causal relation- ships. The core of our framework is a multi-objective optimization strategy guided by three syner- gistic losses: conditional independencies preservation LC1, Markov boundary retention BR, and monotonie alignment consistency LMAC (Section 3.I The learned representations enable causal discovery via the PC algorithm with variable tracing back to original inputs (Section(3.2) (.22). We estab- lish theoretical properties proving these losses suffice for causal structure preservation (Section 3.3
4

---

Published as a conference paper at ICLR 2026
Appendicular lean mass
Enzymatic reation potential
Body fat percentage
Anthropometrics
Metabolic Pathway
Upid metabolism moduless
AG fat ratio Hoight/ Weight
Amino acid metabolism Gutmicrablome Metagenomics
Sex /
M
M
M
MF
M
90. 70
Oxygen desaturation
Lymphocytes
Sleep architecture
Monocytes (%
Sleep efficiency
X
Infiammation
Basophils (%)
Snoring
pAHl
Sleop
PLT
Pluse rate
X
HPP
Eosinophlls (16
Forritin
Sleep position
X1
Neutrophil (%6
Noctural actigraphy
bload
Pulbe rate variability
P2
Cardiovescular
Carotidultra sound
Fundus
Fovea
YOU
Arterial suifiness
Blood pressures
Maqula
Retina
Electrocardiogram (ECG)
Optica disc
retinal microvasstvassulluuueee
Figure Overview of the multi-modal Human Phenotype Project (HPP) dataset. This dia- gram illustrates the six key body-level domains central to our study. The indexed variable series {X Y M' represent the complex web of causal pathways. For clarity, this figure selectively illustrates subset of these causal relationship, focusing on significant pathways that are either dis- covered in prior work or newly validated by our model. Refer Appendix for detailed description.
3.1 VURUCTURE-PRESERRT
LOSSES
We implement the CSP principle through three complementary loss functions, These losses are defined using training target y (representing the outcome Y its proxy ((T)) and an auxiliary outcome embedding t/y R* -> IRd for InfoNCE computation, while encoder Ey produces Zy (Y) for downstream tasks. This joint optimization can yield causally coherent representations.
Conditional independence preservation. To enforce ZT 1 Y 7M. we minimize the Condi- tional Mutual Information (CMI), MI(ZT; Y ZM). Following the variational approach for CMI estimation (Poole et al. 2019), we estimate MI ZT; Y Zm) using the negative log- ligelikelihood difference between two independenll parameterized neural predictors, qo and q8:
Lcr E log (o ==m]]] R[- log qo 2t, 2m)]
Here, approximates the joint conditional distribution p(v Z4. &m) while qs learns ply Zm) with and 2m being the treatment and mediator representations. For continuous y E RA both heads daassnallGaussian
output diagonal- Gaussian parameters (H, o² E R*); for discrete y. they output class probabilities.
Markoy boundary retention. This component prevents the trivial satisfaction of conditional inde- pendence that can occur from discarding mediator information:
LMBR InfoNCE(zm. vy()))
where by : R + Rd is a deterministic outcome embedding. Following van den Oord et al. 2019 the InfoNCE objective provides lower bound the mutual information MI(ZM; Bhy (Y) (hence also regularizing MI Y)) and prevents the collapse of the mediator representation. When out- come embeddings are unavailable, an energy-style regularizer Xen || 11 can be used instead. the DUAL configuration, the two image modalities never jointly appear within the same conditioning set of a loss term.
Monotonic alignment consistency. When ground-truth semantic labels {a E R are available that quantify the underlying semantic content of each sample, we enforee monotonic correspondence between semantic ordering and representation distance, this loss maximizes their Spearman's 's rank correlation [Spearman (1904h (ps):
LMaC pps((sft Aa), rank
where Aajj = |a; ajl denotes semantic amplitude differences with a E R as semantic labels, Azij = ||= 2y]12 denotes 1.2 representation distances, and soft rank (Blondel et al. pro0ides
5

---

Published as a conference paper at ICLR 2026
differentiable rank approximation. The weight AMAC is set to 0 if semantic ordering {ar is not provided. This constraint ensures that semantically similar samples (small aa - ay)) are mapped to nearby representations (small ||2; 2yl12).
The overall optimization objective is:
((E) worCer + WMBRCMBR + WMACC + R(E)
(4)
where WC1, WMBR: AM > 0 are loss weights and R(E) = Aatign Latign + A Layle + AIB LIB includes cross-modal alignment loss Lalign ensuring representations of paired modalities are close in the shared space 2, style consistency regularization Layle preventing nuisance factors from dominating learned representations, and information bottleneck term Lin controlling representa- tion complexity. Configuration-speeee alignment pairs are (ZM, Z in IM and (Z,r, ZM) in (IY, DUAL), with constraint ZT 1 Zyas enforced in IM configuration (details in Appendix D).
3.2 DISCOVERY AND TRACING
Let Z = (Zz, EM, Zy, ZIM: Zry Zx) be the joint random vector of the learned representations with distribution Pz, and let Gz denote the true causal graph over Z. We define causal discovery operator V P(2)= - CPDAG that generates the corresponding completed partially directed acyelic graph (CPDAG) from the set of conditional independencies in the distribution.
For any pair of variables * j and any feasible conditioning set
S = { C Z 2 2,} | IM Zix) ni S {1},
the separation indicator is defined as oZz 25|5) = 1[2 1 Z, Zs] Under the assumption of multivariate Gaussian and faithfulness, conditional independence Z, Zz Zs is equivalent to zero partial canonical correlation, Pij|s = 0. For the univariate case where dim(Z,) = dim (2,) = 1, the test statistic is given by Fisher's s z- z-transformation
Tyys = v |S| 3 arctanh(2sy1s) Hy 10.
In the general multivariate case, we employ the Wilks-Bartler approximation. This test uses the product of the eigenvalues {Re} of the partial canonical correlations to form the statistic Anjs = [1,(1 RB) and tests the null hypothesis Ho Kr 0 O'e The significance of all tests is determined using the Benjamini- Hochberg procedure at level to control the false discovery rate.
From these tests, the graph skeleton Fiz is constructed by connecting any pair (74, Z,) not separated by any conditioning set in S, as defined by:
Ez = ( 2,} 1.S E Sij s.t. 8 02, 2,15) = 1 The set of separating conditionals for a pair is denoted by Sep( 2,) = 1S E S. 8(2,, 2515) 1}. The skeleton is then oriented by first idemtifying all v-structures according to
(6)
(7, 4 7,) = [,, ), ( 7 Fz and AS E (4, 72), 7 $
and subsequently applying the Meek rule closure to propagate all remaining edge orientations, yield- ing the final estimated graph CPDAGz ((2z). To render the latent graph interpretable, we de- fine a variable tracing mechanism using a surjective mapping म : 2 + {T, M, Y" and a calibration map h : range (Zy) range(Y). The surjection π links each representation to its source variable:
x <(zr) =T T (Z π (Zime M, T(y) Y
The calibration map h ensures that h(Zy) = Y + Ecal. where the calibration error satisfies B|ecai /y] = O. Finally, we use the pushforward operator to project the latent graph back to the variable space, defining:
(8) This operator maps each edge (Z, + Zj) in the latent graph to an edge (T(Z,) + π ((2)) in the variable-level graph, which is then simplified according to CPDAG rules after node merging.
CPDAGy = =. (CPDAGz).
The guarantee of consistency holds under several standard assumptions: (i) multivariate Gaussianity and faithfulness of the data distribution; (ii) the mutual exclusion and anti-bypass conditions en- forced during training; (iii) minimum non-zero partial correlation lower bound y > 0; and (iv) bounded maximum in-degree for the graph (A < oo). Given these conditions, we have
Pr[CPDAGz = CPD + 1 as n + 00, and n. (CPDAG(GZ)) is topologically equivalent to ground-truth variable graph CPDAG((V)
(9)

---

Published as a conference paper at ICLR 2026
3.3 THEORETICAL PROPERTIES
Under the assumptions established in Sections 2.1 and 2.3 lowing guarantees.
the CARL framework satisfies the fol-
Theorem (CSP Achievability). Under Assumptions A.5 and spectral norm constraints A.8.2
consider the loss function
wCs =wer Les WMBR LMBR + WMAC LMACs
(10)
where
Lcr = B[-log log (o zmm) El- log go (y z4, zm)l:
(11)
LMAR InfoNCE(zm (y)),
(12)
DM = Ps rank soft softt_ rank(
(13)
with qo, independently parameterized. Then the empirical risk minimization admits approximate minimizing sequences whose limit point (*) satisfies e-CSP with
max Op(n (2))n11)),) O0(K,pp(K)))) ) (0(00ppn3)))))
(14)
where K denotes the number of negative samples, n the sample size, and S is the essential approximation error for the encoder class.
Proof Sketch. Under A.5 (realizabilitty regularity with cross-fitting), our three surrogates are risk- consistent: Lcs calibrates MI(ZT; Zar), CMBR prevents mediator collapse via an MI lower bound for I (ZM; Y), and AM enforces monotone alignment through soft-rank approximation Spearman's 's ps (with approximation error (p (113)). Together with A.8.2 Sspectral/L.pschhiee control) ensuring compactness and stability, the sampling and approximation errors aggregate to Eq. (14) (dominated by Ç", Op(00((n///) (p(K-1/2), and Or(n ()(nn1/3)), yielding e-CSP.
Remark 1. The three loss components target the CSP conditions: LCl optimizes condition (i),
LMBR ensures condition (ii), and Cas AC guarantees condition (ii). When K = (2 the domi- nant error term becomes (n(n-1/2). The weights (WC1, WM BR. WMAC) determine which Pareto optimal point is reached.
Theorem 2 (Causal Query Preservation). Suppose representation system satisfies e-CSP and the causal query Q = (() is identifiable in the original data space via criterion C E (backdoor, frontdoor, instrumental variable). Let Q be the corresponding query computed in the representation space using the same criterion. Then
Q QI K-EE- 8cals where is is the sensitivity constant of the idenification formula, and
(15)
8eal sup
Y * t, x] ([y(Zyy) It, x]]
(16)
(txz)
the conditional calibration error satisfying 8 = (1) under cross-fitting and realizability.
Proof Sketch. We apply the same identification functional (c.g., backloor/frontdoor) in the repre- sentation space. By [A.8.2] the map from inputs representations is Lipschitz, the structural deviation propagates most linearly the causal query; giving IQ QI calibration term deal is Op(n-1/3) under A.5 via cross-fitting, (Full proofs in Appendix C.9
Seal- The
Remark 2. For backdoor: =0=Xx == (x)()y)) Er(t), P(z) with 1. This holds for IY and DUAL configurations where simultaneous conditioning on Zi and Z5r is avoided in both training and inference.
Corollary 3 (Overall Approximation Bound). Under the conditions of Theorems learned encoders (É, ?) satisfy E-CSP with
1 and the
S + Op(p Op(K Op(n-1/s))
(17)
where S" is the essential approximation error for the encoder class. Moreover, the PC algorithm consistent in the representation space, and if n compatibilitypreseeiii (all members in each fiber share identical adjacencies and orientations outside nodes), then
CPDAG(GV). )
(18)
Proof. The bound follows from Theorem [ Consistency and topological equivalence follow from Section 3.2 and the compatibiliyyypreervvin property of n.

---

Published as a conference paper at ICLR 2026
MBRI
MBRI
MBRI
0.03
MAC
MAC
MAG
0.55
0.48
0.68
0.56
0.62
0.64
0.50
0.20
0.76
1.00
CSI
CBI
0:61
0:41
0.20
0.34
Struckumi
Stuchiral
Struhire
0.40
0.68
(b)
0.43
(c)
D.09
0.01
Cance
ALIGN
Imageland
Figure 4: dtructre-preseevvatie evaluation. Panels report five metrics (CSI, MBRI, MAC, Struc- tural, RIC) under (a) sample sizes n E {500, 1000, 2000, 5000), (b) noise levels {0.1, 0.3, 0.5), and (c) comparision with baselines. Refer to Table 3or specific values.
EXPERIMENT RESULTS
We evaluate CARL on two complementary datasets: a synthetic benchmark with known causal ground truth for systematic evaluation, and the real-world HPP dataset for biomedical validation.
Synthetic dataset. We manually create a synthetic bench- mark dataset to encode a known ground-truth causal rela- tionships between different modalities, using MNIST Lecun et al.] (1998). The data generation is grounded in Struc- tural Causal Model (SCM) that defines latent causal chain T M Y where T is exogenous treatment, M is mediator, and Y* is an outcome. To create the cross-modat first mapped to normal-
6
0
6
6
Drightness
6
6
6
6
6
0
Contraat
6
6
6
link, the latent causal variables ized semantic amplitude a E [o, 1]. This amplitude then de- terministically controls sequence of visual transformations (e.g., rotation, brightness, contrast) applied to a base MNIST digit. This mechanism used instantiate three distinct ex- perimental configurations as described in Section 2.2 IM, IY, DUAL. In Figure 2 we illustrate the data synthesis process for the IM scenario, where the mediator M is encoded as the image modal- ity. This design ensures structured and quantifiable causal dependency that spans both tabular and visual modalities. Full details of the generation process are provided in Appendix
Figure Illustration of synthetic data generation in IM scenario.
HPP Dataset. The HPP dataset, intro- duced in Section 2.4 is used to validate CARL's ability to preserve known causal pathways in a real-world biomedical ap- plication. Implementation details are in Appendix
(8)
Arterbel aitheess
C-ractivve protein (CRPP)
(b)
Fundus
Blood
Cardiovasoulaa Disease
AgM
Cardanaenaterrr Discase
pressure
Renall Lundion
Pulte Rate Variability
(PRV)
Cantiowerr Dinnssn
Gue Mirrosooee abu aburdance
Candovascular
Baselines. CARL benchmarked
Apnea-lyppoeen Indax (AH))
Melaberiam
against the most representative cross- modal methods, including CL.PP
(8)
Figure 3: HPP mediation pathways.
al. (2021), ImageBind Girdhar et al. 20033 DCCA Andrew et al. (2013), Causal VAE Yang et al. (2021), ), ALIGN et al. (2021), IRM Arjovsky et al. (219). DEAR Shen et al. (2022) and a feature concatenation baseline.
Metrics. We use five core metrics designed to probe causal preservation: Causal Structure In- dex (CSI), Markov Boundary Retention Index (MBRI). Monotonic Alignment Consistency (MAC), Structural Accuracy (Structural), and Representation Information Content (RIC). Full list of results and experimental setup details are in the Appendix
8

---

Published as a conference paper at ICLR 2026
Table I: HPP results: Summary of total effect (TE), natural direct effect (NDE), natural indirect
effect (NIE), and mediation proportion of key pathways in latent space 2.
Mediation Proportion (%)
Causal Pathways (Latent Space 2)
TE
NDE
NIE
Blood pressure
+ ZCVD
0.486
0.271
0.215
44.24
Blood pressure
> 2 Arterial stiffness - ZCVD
0.097
19.96
Blood pressure Frundus + BCVD
0.074
15.23
Blood pressure + ZRenal function + ZCVD
0.044
9.05
Zage + Fundus + ZCVD
0.513
0.437
0,076
14.81
ZCRP + Fiundus + ZCVD
0.378
0.319
0.059
15.61
ZAHI ZPRV + ZCVD
0.352
0.281
0.071
20.17
ZBMI + MMetaholism - 2CVD
0.394
0.325
0.069
17.51
Got Mierohiome Mettabnlis ZCVD
0.316
0.248
0.068
21.52
4.1 CAUSAL PATHWAY DISCOVERY IN HPP DATASET
Based on CARL'S structure- preeeuringg learning and causal discovery as shown in Figure 3 we ob- tained the following stable effect decompositions on HPP data (See Table I for "blood pressure + cardiovasculat events (CVD)", total effect TE=0.486, NDE=0.271, NIE=0.215, with overall me- diation proportion of 44.24%; indirect effects are primarily transmitted through arterial stiffness (19.96%), retinal microvascular changes (15.23%), and renal function (9.05%); we simultaneously identified age retinal fundus CVD (14.81%), CRP - retinal fundus - CVD (15.61%). AHI - PRV/HRV - CVD (20. 17%). BMI - metabolism - CVD (17.51%), and gut microbiome metabolism CVD (21.52%). These pathways consistent with independent high-level search directions and have interpretable mechanisms: blood pressure reduction shows clear dose- response/risk reduction relationships with major adverse cardiovascular events G.I arterial stiffinss (PWV) can independently predict events and death G.2, retinal microvascular abnormalities pre- dict stroke and coronary heart disease G.3 renal function impairment (decreased eGFR, increased proteinuria) increases CVD and mortality G.4 retinal "biological age gap" correlates with all- cause/cardiovaseular mortality G.5) systemic inflammation (CRP) shows dose-response with mi- crovascular caliber changes and links to CVD G.6; OSA-related HRV reduction indicates higher CVD risk G.7, obesity mediates considerable proportion of CVD risk through metabolic factors such as blood pressure/lipids/glueee G.8) nicrobiome-TMAG and other metabolites are closely associated with atherosclerosis and adverse events G.9
4.2 SYNTHETIC DATA VALIDATION
Figure demonstrates CARL's structure-preserving performance across multiple evaluation dimen- sions. In sample size scaling experiments (n E (500, 1000, 2000, 5000)). CARL maintains CSI at 1.0 with structural accuracy stable within 0.61-0.75, validating that the E-CSP bound tightens at the theoretical (r(n - 1/2) rate. Under noise robustness testing (a E (0.1, 0.3, 0.5)), CARL'S MAC degrades from 0.89 to 0.42 while CSI remains at 1.0, and MBRI decreases from 0.77 to 0.63, in- dicating that monotonic alignment constraints maintain semantic-geometti distributional perturbation. Baseline comparisons reveal CLIP's CSI at 0.25, ImageBind's Structural at 0.33, and similar performance from DCCA, confirming that purely statistical objectives cannot guarantee conditional independence pattern preservation.
correspondence under
4.3 ABLATION STUDY
quantifgss the contribution patterns of each loss function. Removing Ler causes CSI to drop from to 0.25 while MAC increases to 0.83 but structural accuracy falls 0.40. indicating that se- mantic alignment and structural preservation decouple when conditional independence constraints are absent. Removing CMBR reduces MBRI from 0.63 to 0.46 and CSI to 0.75, confirming the necessity of Markov boundary information for complete structural preservation. Removing LM decreases MAC from 0.55 to 0.32 while CSI remains at 1.0, showing that semantic- -ecnettr- con- sistency and conditional independence preservation are mutually independent. The alignment-only

---

Published as a conference paper at ICLR 2026
configuration reproduces the fundamental problems of statistical methods: high MAC (0.89) but low CSI (0.25) and structural accuracy (0.32).
Table 3 validates validates the necessity of key design choices. Shared predictor heads reduce CSI from 1.0 to 0.75 with slight decreases in MBRI and MAC, confirming the importance of independent parame- terization for unbiased CMI estimation. Absence of cross-validation causes CSI plummet to 0.25, indicating severe risks of overfitting spurious conditional dependencies. With K = 32 negative sam- ples, all metrics decline and CSI drops to 0.75, validating the impact of InfoNCE estimation quality on structural preservation. Representation dimension d = 16 performs comparably to the full con- figuration, with all metric differences within 0.05, confirming the stability of constraint mechanisms across different representation scales.
Table 2: Loss function ablation: impact on structure preservation performance.
Table 3: Architecture design ablation study: impact of key design choices on performance.
CSI
MBRT
MAC
Structural
RIC
CST
MBRI
MAC
Structural
RIC
w/o Ces
0.25
0.62
0.83
0.40
0.44
Shared Predictor Head
0.75
0.61
0.53
0.54
0.41
w/o Cxrna
0.75
0.46
0.54
0.52
0.37
wio Cross Validation
0.25
0.66
0.63
0.37
0.43
weo Esrac
1,00
0.63
0.32
0.56
0.42
:232
0.75
0.55
0.49
0.48
0.41
only Calun
0.25
0.66
0.89
0.32
0.49
d:
0.58
0.56
0.60
0.43
CARI. (Full)
1.00
0.63
0.55
0.61
0.42
CARI. (Full)
0.63
0.55
0.61
0.42
RELATED WORK
Cross-modal representation learning has achieved great success in aligning heterogencous data like images and text into shared semantic space using contrastive objectives [Radford et al. (202T); Jia et al. (202T); ; Girdhar et al. (023). However, these methods optimize statistical associations and are agnostic to the underlying causal graph, risking the distortion of causal pathways Arjovsky et al. ) Geirhos et al. (2020). In parallel, Causal Representation Learning (CRL) aims to learn causally y robust representations that support generalization and counterfactual reasoning Schölkopf ct al. (2021); Bagi al. (2023); Zuo et al. 2023). Yet, CRL has predominantly focused on unimodal data and has not systematically addressed the unique challenges multi- modal settings, such as in- formation asymmetry across modalities ensuring consistent geometric-semantic alignment [Lang jet al. (2022); Zhang et al. (2024). While initial exploration into multi-modal causal representation learning exist Sun et al. (2025), the disconnect between the scaling power of modern cross-modal models and the structural requirements for reliable, verifiable causal inference creates a critical gap. Our work aims bridge this gap by introducing a framework that explicitly preserves key causal structures, including conditional independencies and effect identifiability conditions, during cross- modal representation learning. Please refer to Appendix E for detailed literature review.
CONCLUSION
We address the problem of representation-indueed structural drift in cross-modal representation learning through the introduction of the Causal Structure Preservation principle and the CARL framework. We identify and formalize three core challenges: Cross-modal Information Bottleneck, Modal Alignment Consistency, and Cross-modal Identifiability Consistency, proposing joint op- timization strategy combining conditional independence preservation, Markov boundary retention, and monotonic alignment consistency. Theoretical analysis establishes achievability guarantees for e- C-CSP and proves the preservation of causal query identifiability in representation space. Validation on the HPP dataset demonstrates that CARL preserves complex mediation pathways from blood pressure to cardiovascular discase through arterial stiffness, retinal microvascular changes, and renal function. Synthetic benchmark testing reveals the limitations of purely statistical optimization meth- ods in structural preservation while validating the theoretical necessity and practical effectiveness of our three loss function components. This framework provides verifiable theoretical foundations and methodological support for reliable causal inference multimodal data environments.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International conference on machine learning, pp. 1247-1255, PMLR, 2013.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint wwX.v:1977.029000 2019.
Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley. Generative causal representation learning for out-of- of-distribution motion forecasting. In International Conference on Machine Learming, pp.
PMLR, 2023.
F. Beghini, L. J. McIver, A. Mlaneo-Miguez, L. Dubois, F. Asnicar, S. Maharjan, A. Mailyan, P. Manghi, M. Scholz, A. M. Thomas, M. Valles-Colomer, G. Weingart, Y. Zhang, M. Zolfo, C. Huttenhower, E. A. Franzosa, and N. Segata. Integrating taxonomic, functional, and strain- level profiling of diverse microbial communities with biobakery 3. eLife, 10:e65088, may 2021. doi: 2..7554/eLiffe660888.
Y. Ben-Shlomo, M. Spears, C. Boustred, M. May. S. G. Anderson, E. J. Benjamin, P. Boutouyrie, Cameron, C. H. Chen, J. K. Cruickshank, S. Hwang, E. G. Lakatta, S. Laurent, J. Maldon- ado, G. F. Mitchell, S. S. Najjar, A. B. Newman, M. Ohishi, B. Pannier, T. Pereira, R. S. Vasan, T. Shokawa, K. Sutton- Tyrell, F. Verbeke, K. L. Wang. D. J. Webb, T. Willum Hansen, S. Zoun- gas, C. M. McEniery, J. R. Cockeroft, and I. B. Wilkinson. Aortic pulse wave velocity improves cardiovascular event prediction: an individual participant meta-analysis of prospective observa- tional data from 17,635 subjects. Journal of the American College of Cardiology, 63(())636-666 646, feb 2014. doi: 10. 10.1016/j, 1a.1066/..acc.2133.... jacc. Epub 2013 Nov 13.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 3588):1798-1822 2013.
Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference on Machine Learning, pp. 950-959, PMLR, 2020.
Blood Pressure Lowering Treatment Trialists' Collaboration. Pharmacological blood pressure lower- ing for primary and secondary prevention of cardiovascular disease across different levels of blood pressure: an individual participann- data meta-anan anallsis. The Lancet, 397(10285): 2021. doi: 10. (0.1016//0140 40- 6736(2) (773662))0090000 )00590-0.
Johann Brehmer, Pim De Haan, Phillip Lippe. and Taco S Cohen. Weakly supervised causal repre- sentation learning. Advances in Neural Information Processing Systems, 35:38319-38331 2022.
Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Car- los Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa De- hghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, Ceslee Montgomery, Paulina Pietrzyk, Marvin Ritter, AJ Piergiovanni, Matthias Minderer, Filip Pavetic, Austin Waters, Gang Li, Ibrahim Alabdulmohsin, Lucas Beyer, Julien Amelot, Kenton Lee, Andreas Peter Steiner, Yang Li, Daniel Keysers, Anurag Arnab, Yuanzhong Xu, Keran Rong, Alexander Kolesnikov, Mojtaba Scyedhosseini, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. Pali-x: On scaling up multilingual vision and language model. 2023. URL
wwwp:/wwwwwwrgaaa/aa/aaaaaaaae/a 2300. 11000.888555 8565
Victor Chermozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural pa- rameters. The Econometrics Journal, (1(1))C1-66,, 2018.
Carolyn Yim-Lui Cheung, Tien Yin Wong. Ecosse L. Lamoureux, Charumathi Sabanayagam, Jie Li, John Lee, and E. Shyong Tai. C-reactive protein and retinal microvaseular caliber in multiethnic asian population. American Journal of Epidemiology, 171(2):20622) 223, 2010. kwp357.
doi: 10. 10.30/9/aje/
11

---

Published as a conference paper at ICLR 2026
Chronic Kidney Disease Prognosis Consortium, Kunihiro Matsushita, Mare der Velde, Brad C. Astor, Mark Woodward, Andrew S. Levey, Paul E. de Jong, Josef Coresh, and Ron T. Gansevoort. Association of estimated glomerular filtration rate and albuminuria with all-cause and cardiovas- cular mortality in general population cohorts: a collaborative meta-analysis. The Lancet, 375 (0731))2073-2081 2010. doi: 4000166600400-6776)6))6666666
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley, 2nd edition, 2006.
Monroe D. Donsker and R. Srinivasa Varadhan. Asymptotic evaluation of certain markov process expectations for large time. Communications on Pure and Applied Mathemulics, 28(1):1-47, 1975.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. 2021. URL httph:tt///c arxiv. org/ abs//bb///// 2010.000000 11929
Alexey Fedotov, Peter Harremoës, and Flemming Topsøe. Refinements of Pinsker's inequality. IEEE Transactions on Information Theorr, 191-):4491-149, 2003.
Ronald A. Fisher. On the probable error of coefficient of correlation deduced from small sample. Metron, 1:3-32, 1921.
Ronald A. Fisher. The distribution of the partial correlation coefficient. Metron, 3:329-332, 1924.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 (111))665-6777 2020.
Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15180-15190, 2023.
Global Burden of Metabolic Risk Factors for Chronic Diseases Collaboration (BMI Mediated Ef- fects), Yuan Lu, Kaveh Hajifathalian, Majid Ezzati, Mark Woodward, Eric B. Rimm, and Goodarz Danaei. Metabolic mediators of the effects of body-mass index, overweight, and obesity on coro- nary heart discase and stroke: pooled analysis of 97 prospective cohorts with 1.8 million partic- ipants. The Lancet, 383(9921):970-90
2014. doi: 10.0016500400-6766(66666633333
Alan S. Go, Glenn M. Chertow, Dao Fan, Charles E. McCulloch, and Chi-yuan Hsu. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. The New England Journal of Medicine, 351(13) 12:661131333 2004. doi: 10. 10.16///JM00001131...
Ian J Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Stefanie Hillebrand, Kimberley B. Gast, Renée de Mutsert, Comnelis A. Swenne, J. Wouter Jukema, Saskia Middeldorp, Frits R. Rosendaal, and Olaf M. Dekkers. Heart rate variability and first cardiovascular event in populations without known cardiovascular disease: meta-analysis and dose-response meta-regression. Europace, 15(5):742-74, 2013. doi: 10. 10993//uroppacceeus3311
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference machine learning, pp. 4904-4916. PMLR, 2021.
Diederik Kingma and Max Welling. 1312v:1312.61 2013.
Auto-encoding variational bayes.
arXiv preprint
12

---

Published as a conference paper at ICLR 2026
Robert A. Koeth, Zeneng Wang, Bruce S. Levison, Jennifer A. Buffa, Elin Org, Brendan T. Sheehy, Earl B. Britt, Xiaoming Fu, Yuping Wu, Lin Li, Jonathan D. Smith, Joseph A. DiDonato, Jun Chen. Hongzhe Li. Gary D. Wu, James D. Lewis, Manya Warrier, J. Mark Brown, Ronald M. Krauss, W. H. Wilson Tang, Frederic D. Bushman, Aldons J. Lusis, and Stanley L. Hazen. Intestinal microbiota metaholism of l-carnitine, nutrient in red meat, promotes atherosclero- sis. Nature Medicine, 1965)576-585, 2013. ISSN 1546-170X. doi: 10. 10.1038/m..1445 URL 3145
atttss:/4oo..rrg/0.0.
Sarah Kohn, Alon Diament, Anastasia Godneva, Raja Dhir, Adina Weinberger, Yotam Reisner, Ha- gai Rossman, and Eran Segal. Phenome-wide associations of sleep characteristics in the human phenotype project. Nature Medicine, pp. 2025.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in neural information processing systems, 30, 2017.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient- based learning applied document recog- nition. Proceedings of the IEEE, 86(1 1):2278-22333 2324. 1998. doi: 10.1 109/5.72679..
Sarah Lewington, Rory Clarke, Nazim Qizilbash, Richard Peto, Rory Collins, and Prospective Stud- ies Collaboration. Age-specific relevance of usual blood pressure to vascular mortality: a meta- analysis of individual data for million adults in 61 prospective studies. The Lancet, 360 (9349):1903-1913 2002. doi: (0.1006//S1440-67760001111919999
Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Ad- vances in Neural Information Processing Systems, 35:17612-176653 2022.
Jiashuo Liu, Zheyan Shen, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: survey, arXiv preprint arXiv:2 (tti::2008.13244 13624, 2021a.
Min Liu, Clare Lovern, Kate Lycett, Mingguang Hc, Melissa Wake, Tien Yin Wong, and David P. Burgner. The association between markers of inflammation and retinal microvascular parameters: A systematic review and meta-analysis. Atherosclerosis, 336: 12-22, 2021b. doi: 10.1016/j.
2therooscerooss.222..99...
Yuejiang Liu, Riccardo Cadei, Jonas Schweizer, Sherwin Bahmani, and Alexandre Alahi. Towards robust and adaptive motion forecasting: A causal representation perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17081-17092 2022.
Chaochao Lu, Yuhuai Wu, José Miguel Hrrmández-Lobattee and Bernhard Schölkopf. Invariant causal representation learning for out-of-distribution generalization. In International Confer- ence on Learning Representations, 2022. URL https: http::ss///penreeee...... net. Forummii
-e4EXDWXnSm
Haiyi Mao, Hongfu Liu, Jason Xiaotian Dou, and Panayiotis V Benos. Towards cross-modal causal structure and representation learning. In Machine Learning for Health, pp. 120-140. PMLR, 2022.
Judea Pearl. Causality. Cambridge university press, 2009.
Ben Poole, Sherjil Ozair, Aaron van den Oord, Alexander A. Alemi, and George Tucker. On vari- ational bounds of mutual information. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceed- ings of Machine Learning Research, pp. 5171-5180. PMLR, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. 2021. URI.
httpp::/aaa
org/ alss//////a / 21...3.. 00020
Bernhard Schölkopf. Francesco Locatello, Stefan Bauer, Nan Rosemary Ke. Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 10915):612-634 2021.
13

---

Published as a conference paper at ICLR 2026
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: general- ization bounds and algorithms. In International conference on machine learning, pp. 3076-3085. PMLR. 2017.
Michal Shapira, Smadar Shilo, Yeela Talmor-Barkan, Yaron Aviv, Yotam Reisner, Anastasia God- neva, Adina Weinberger, Alon Skaat, Anat Loewenstein, Eran Segal, and Hagai Rossman. Un- veiling associations between retinal microvascular architecture and phenotypes across thousands of healthy subjects. medRxiv, 2024. doi: 10.10012204444445224466666666610
URL hetps:
04. 05 243051
Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, and Tong Zhang. Weakly super- vised disentangled generative causal representation learning. J. Mach. Learn. Res., 23(1), January 2022. ISSN 1532-4435.
Charles Spearman. The proof and measurement of association between two things. The American Journal of Psychology. 15(1):72-101, 1904.
Yuewen Sun, Lingjing Kong, Guangyi Chen, Loka Li, Gongxu Luo, Zijian Li, Yixuan Zhang, Yujia Zheng, Mengyue Yang. Petar Stojanov, Eran Segal, Eric P. Xing, and Kun Zhang. Causal repre- sentation learning from multimodal biomedical observations. 2025. URL htttps://arxiv.
org/ hrg/aba/2a 2411 06518
W. H. Wilson Tang, Zeneng Wang, Bruce S. Levison, Robert A. Koeth, Elizabeth B. Britt, Xiaoming Fu, Yuning Wu, and Stanley L. Hazen. Intestinal microbial metabolism of phosphatidylcholin and cardiovascular risk. The New England Journal of Medicine, 368(17) 2013. doi:
10.1056/NE/Moal 1094000.
Sevde Ucak, Harshi U. Dissanayake, Philip de Chazal, Yohan Bin, Kate Sutherland, Bryan Se- tionago, Brandon Tong, Brendon J. Yee. Konstantinos Kairaitis, John R. Wheatley, Andrew J. Piper, Peter A. Cistulli, and Sydney Sleep Biobank Investigators. Heart rate variability analysis in obstructive sleep apnea patients with daytime sleepiness. Sleep, 47(6)7555 2024. doi:
10.1099//leee//ssee05.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. 2019.
Charalambos Vlachopoulos, Kostas Aznaouridis, and Christodoulos Stefanadis. Prediction of cardiovascular events and all-cause mortality with arterial stiffness: a systematic review and meta-analysis. Journal of the American College of Cardiology, 55(13):1318-3321 2010. doi:
10.10166f.acc22200100000
Zeneng Wang. Erik Klipfell, Brian Bennett, Robert Koeth, Bruce Levison, Balkrishen Dugar, Ariel E. Feldstein, Eliiabeth B. Britt, Xiaoming Fu, Young-Mi Chung, Yuning Wu, Philip Schauer, Jonathan D. Smith, Hooman Allayee, W. H. Wilson Tang, Joseph DiDonato, Al- dons J. Lusis, and Stanley L. Hazen. Gut flora metabolism of phosphatidylcholine promotes cardiovascular disease. Nature, 472(7341):57-63, 2011. doi: 10. 1038/nauuee009922..
Larry Wasserman, All of Nonparametric Statistics. Springer, 2006.
Yake Wei, Siwei Li, Ruoxuan Feng, and Di Hu. Diagnosing and re-learning for balanced multimodal learning. In European Conference on Computer Vision, pp. 71-86. Springer, 2024.
Tien Yin Wong, Ronald Klein, David J. Couper, Lisa S. Cooper, Eyal Shahar, Larry D. Hubbard, Marion R. Wofford, and Alvaro R. Sharrett. Retinal microvascular abnormalities and incident stroke: the atherosclerosis risk in communities study. The Lancet, 358(9288): 2001,
doi: 10.1016//0144--66660))00
Tien Yin Wong, Ronald Klein, Alvaro R. Sharrett, Bruce B. Duncan, David J. Couper, James M. Tielsch, Barbara F. Klein, and Larry D. Hubbard. Retinal arteriolar narrowing and risk of coronary heart disease men and women. the atherosclerosis risk in communities study. JAMA, 287(9): 1153-1159, 2002. doi: 10.1001/jama.2877 287.9.1153.
Shaoxuan Xu, Menglu Cui, Chengxiang Huang, Hongfa Wang, and Di Hu. Balancebenchmark: A survey for multimodal imbalance learning. arXiv preprint arXiv:2502.10S 2025.
14

---

Published as a conference paper at ICLR 2026
Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. vaus Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp.
2021.
Xiaohui Zhang, Jachong Yoon, Mohit Bansal, and Huaxiu Yao. Multimodal representation learning by alternating unimodal adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 77456-27466, 2024.
Y. Zhou, K. Wagner, M. A. Chia, Zhao, Woodward-Court M. Xu, Struyven, C. Alexander, and P. A. Keane. Automorph: Automated retinal vascular morphology quantification via a deep learning pipeline. Translational Vision Science & Technology, 11(7):12, jul 2022. doi: 10.1 10.1167/tvst.11./7 1.7..12
Zhi Zhu, Dong Shi, Peng Guankai, Zhen Tan, Xiang Shang, Wei Hu, Hao Liao, Xin Zhang, Yan Huang, Hong Yu, Wei Meng, Wei Wang, Zheng Ge, Xue Yang, and Mingguang He. Retinal age gap predictive biomarker for mortality risk. The British Journal of Ophihalmology, 107(4): 547-554, 2023. doi:
Zhiqun Zuo, Mahdi Khalili, and Xueru Zhang. Counterfactually fair representation. Advances in neural information processing systems, 36:12124-12140, 2023.
15

---

Published as a conference paper at ICLR 2026
Supplement to "CARL: Preserving Causal Structure in Representa- tion Learning"
Notation and Assumptions
19
Notation
19
Identification Conditions
19
A.3
Positivity and Consistency
19
A.4
Causal Markov Property and Faithfulness
20
A.4.1
Formal Statement of Causal Markov Assumption
20
A.4.2
Faithfulness and Adjacency Faithfulness
20
A.4.3
Basic Identification Conditions]
20
A.5
Realizability and Regularity for Estimator Consistency
21
A.6
Measurement and Pairing Assumptions
A.6.1
Formalization of Separable Measurement Assumption
21
A.6.2
Verification Methods
A.6.3
Paired Data Prerequisites
22
A.7
Scenario-specific Conditions
22
A.7.1
Front-door Criterion and Instrumental Variable Conditions
22
A.7.2
Scenario-specific Necessary and Sufficient Conditions
22
A.8
Regularity and Implementation Conditions
22
A.8.1
Sample Exchangcability and Moment Conditions
22
A.8.2
Implementation Methods for Lipschitz Constraints
................................
23
A.9
Identifiability Conditions
23
Theoretical Properties and Technical Conditions
23
Theoretical Foundations of Loss Functions)
................................
23
B.2
NLL Calibration and Realizability
23
B.2.1
Theoretical Foundation of NLL Surrogates
23
B.2.2
Practical Verification Strategies
24
InfoNCE Lower Bound Conditions
24
B.3.1
Sufficient Conditions for Lower Bound Validity
24
B.3.2
Technical Requirements for Practical Implementation
25
B.4
4
Conditional Mutual Information Bias Control
25
B.4.1
Sources of CMI Estimation Bias
25
B.4.2
Sample Splitting and Cross-fitting
26
B.5
Boft-rank to Spearman Correlation Consistency
26
B.5.1
Theoretical Properties of Soft-rank Functions
26
16

---

Published as a conference paper at ICLR 2026
B.5.2
Gradient Boundedness and Optimization Stability
27
B.6
Structure Preservation in Representation Space
27
B.7
PC Algorithm Consistency Guarantees
27
Main Theoretical Proofs
27
Proof Strategy and Roadmap
27
Conditional Independence Preservation Theorem
28
Markov Boundary Retention Theorem
................................
29
Monotonic Alignment Consistency Theorem
30
C.5
Identifiability Consistency Theorem
................................
30
C.6
Outcome Calibration Assumption
Metrie Consistency Proof
................................
31
8
Metric Consistency
9
Proofs of Causal Structure Preservation Theorems
33
D
Implementation Details
34
D.I
Encoder Parameterization
34
D.2 Batch Construction for Loss Functions
34
D.3 Predictor Architecture for CMI Estimation
34
D.4 PC Algorithm Formalization
35
D.5
Optimization Algorithm
35
Related work
35
Experiment Details
36
F.1
Experimental Setup
36
F2
Evaluation Metrics
37
F3
Parameter Configuratioos)
38
Supporting Evidence for Causal Pathway Discovery
38
Blood Pressure CVD Evidence
38
G.2
Arterial Stiffness - CVD Evidence
38
................................
Retinal Microvascular - CVD Evidence
................................
38
G,4
Renal Function - CVD Evidence
38
G.5
Age Retinal Fundus - CVD Evidence
38
G.6
Inflammation Microvascular - CVD Evidence
39
Sleep Apnea HRV Evidence
39
G.8
BMI - Metabolism - CVD Evidence
39
Gut Microbiome - Metabolism - CVD Evidence
Real data
39
17

---

Published as a conference paper at ICLR 2026
H.1 Description of Human Phenotype Project Dataset
39
H.2 Body System- Derived Features
39
Synthetic Dataset Generation
47
Causal Grounding via Structural Causal Model
47
I.2
Numerical Instantiation and Auxiliary Variables
47
Semantic Parameter Mapping
47
14
Cross-Modal Image Generation
48
Dual-Scenario Framework
49
L6
Ground Truth Specification and Robustness Testing
49
1.7
Synthetic data sample
50
18

---

Published as a conference paper at ICLR 2026
LLM USAGE DISCLOSURE
In the preparation of this paper, we made limited use of large language model (LLM) tools such as ChatGPT and Gemini, After drafting the manuseript, we used these tools only for minor language polishing. All research ideas, analyses, and the substantive content of the paper were conceived, developed, and written by the authors.
A NOTATION AND ASSUMPTIONS
A.1 NOTATION
Throughout this paper, we adopt the following notation conventions. Let denote the treatment variable, M .M the mediator variable, Y E y the true but unobserved outcome, Y E IR the observed outcome (reduces to scalar when l = 1), and X E X the covariate set. Image observations IM 12 E R x W xC have height H. width W. and C' channels. The shared representation space is denoted as C IRd where d the representation dimension.
The encoder family & comprises the following mappings: treatment encoder Fy- T encoder EM .M + Z, image- mediator encoder E, IM + Z, outge- encoder Eạr I + Z. and optional outcome encoder Ey E - Z and covariate encoder Ex X Z. The corresponding representations are denoted as Zr = (T" for encoded treatment, Zas obtained from either Es (M) or M (IM), and Zy from either Ey E₁y (I)).
- Z, mediator
Key functions include the parameterized conditional distributions qo : Z x 2 2 P(R*) and qs
Z - P(RR)) modeling p(y| 2m) and p(y(em)))) respectively, outcome embedding function bh R Rdd image-to-outcome extraction function イY TR. posterior calibration function
h: + R and differentiable rank approximation soft_rank Tma (-) with temperature Trank-
Loss function parameters include primary loss weights WCI, WMB WMAC, regularization coef- ficients Aalign Aayle Åτn, temperature parameters Talign for InfoNCE alignment, Tmin for soft-rank approximation, balance coefficient Tinhr for MBR loss weighting, and significance level a for hy- pothesis testing (default 0.05).
A.2 IDENTIFICATION CONDITIONS
Definition A.2.1 (Backdoor Criterion) Given a DAG g EV, set of variables S C V (T,Y*) satisfies the backdoor criterion relative the ordered pair (T. Y*) if and only if S de(T) = @ where de(T) denotes the descendants of T, and (ii) S blocks all paths of the form T Y*.
Definition A.2.2 (Frontdoor Criterion) A set of variables M C V (T,Y") y*) satisfies the frontdoor criterion relative to (T, Y+) and only if (i) Vi E Ir--y- : M n æ * where II denotes all directed paths from T to Y*, (ii) (T 1 M 0)67_17 holds in the graph with edges T + M removed, and (iii) (M L Y (T) holds in the graph with edges M Y removed.
Definition A.2.3 (Instrumental Variable) A variable Z is an instrumental variable for (T. Y*) and only (i) Cov(Z, T) * (relevance), (ii) (Z Y")az ee holds in the graph with T and its outgoing edges removed (exclusion restriction), and (iii) Z U where U denotes unobserved confounders affecting (T, Y*).
A.3 POSITIVITY AND CONSISTENCY
Definition A.3.1 (Positivity) The treatment mechanism satisfies the positivity condition and only
if there exist constants c (0, 1) such that è P(T = t X x) I C2 for all (t, supp(T, X). For continuous treatments, we require bounded density c S frix S 62 almost everywhere for some C, C2 0.
Definition A.3.2 (SUTVA) Let (t(t) denote the potential outcome of unit under treatment t. The Stable Unit Treatment Value Assumption holds if and only if (1) Y.(t) Y:(4+) for all treatment
19

---

Published as a conference paper at ICLR 2026
vectors (f1,...,,ty tp) (no interference), and (ii) for all T, there exists unique potential outcome (t(t) (treatment uniqueness).
A.4 CAUSAL MARKOV PROPERTY AND FAITHFULNESS
A.4.1 FORMAL STATEMENT OF CAUSAL MARKOV ASSUMPTION
Definition A.1 (Causal Markov Property): Given a directed acyelic graph = EV, cach
variable Vi E V satísfies the conditional independence relation
V, (NonDesca \ Pac(Vi)) Pac(())
Where N VV)) (V, denotes the set of non-descendunt variables of V₁ in G excluding its parents, and Pag (Vi) denotes the parent set of V₁,
In the basic structure I' + M + Y of the CARL framework, the causal Markov property manifests as: for root node T, we have T' T1 NonDescc (T') (since Pac(T)) 0): for Y*, have Y 1 Pac(M)) Pac(MM) = (T).
TT Paa(("))= = {M]; for M, we have M IL (NonDesca(AN) When including covariates X and image observations IM I", the Markov property requires all variables to be conditionally independent of their non-descendant non-parent variables given their graph-structural parents.
A.4.2 FAITHFULNESS AND ADIACENCY FAITHFULNESS
Definition A.2 (Faithfulness): A distribution P is faithful DAG G if and only if the conditional independence relations in correspond one-to-one with the d-separation relations in G:
X XLY Z in P La
Z in G
For robustness considerations in practical applications, we adopt the adjaceney faithfulness condi- tion: if Vú and Vj are adjacent in graph G, then for conditioning sets S that exclude the endpoints and their descendants, we reject Ho V; TL Vj S level (up measure-zero exceptions).
Faithfulness fails under parameter cancellation, deterministic relationships, and special parameter configurations. Parameter cancellation occurs when causal effects cancel each other through differ- ent paths, and deterministic relationships exist in cases of strict functional dependencies. In con- tinuous parameter spaces, parameter configurations that violate faithfulness constitute measure-zero sets.
A.4.3 BASIC IDENTIFICATION CONDITIONS
Positivity Condition: The treatment variable must have positive probability of being observed under all covariate conditions:
0 < P(T =t x= =)) <
for all (f,z) in support
For mediator variables, we similarly require
0 < m t, = 2) <
For continuous treatments/mediatoos, we use positive density conditions 0, farpr, z) > 0 to replace discrete probabilities.
frix(t)))
Consistency Condition (SUTVA): This includes the no-interference assumption and treatment ver- sion consistency. The no-interference assumption requires that individual i's potential outcomes are not affected by other individuals' treatment status, while treatment version consistency requires that there are no different versions of treatment for the same treatment level.
Positivity violations manifest treatment probabilities of 0 or 1 under certain covariate combina- tions, detectable through propensity score distributions. Consistency violations involve treatment spillover effects or ambiguous treatment definitions, requiring domain knowledge for assessment.
20

---

Published as a conference paper at ICLR 2026
A.5 REALIZABILITY AND REGULARITY FOR ESTIMATOR CONSISTENCY
Assumption 1 (Causal Structure). The data generating process follows directed acyclic graph G = (V. F)) with core causal chain T + M Y This structure satisfies the Markov prop- erty (Definition A.4.1) and faithfulness (Definition A.4.2) with identification conditions detailed in Section
3.4.3 A.4.3
Assumption (Anti-bypass). The outcome representation Zy satisfies no direct information flow from (T, M) bypassing the outcome pathway. When Zy = Ev-YY, we require Zy 1. (T, M) When Zy Efr (I*) ) in IY configurations, we require Zy 1 (T, M where 1 captures all outcome-relevant information. These encoders are mutually exclusive and share no feature extrac- tion layers or gradient paths from (T, M), ensuring complete branch isolation in the representation space.
Assumption 3 (Configuration-Specfic Conditions). In the IM configuration, no spurious path T + 1 + Y* exists, and / observes M with independent noise. In the IY configuration, the proxy = o(I²) satisfies monotonic calibratability with Y*, meaning there exists a monotonic h(d((*)) )). The DUAL configuration satisfies both IM and IY
function à such that ( conditions, with I and I' never simultancously conditioned in both training loss computation and sescovery-phase conditioning sets.
Assumption 4 (Technical Regularity). Encoders satisfy Lipschitz continuity with ||
E(x22]l < L| 2:2l for Lipschitz constant L 0. All representations have bounded second moments E[] 2 < co. Observed samples are independently and identically distributed or satisfy mixing conditions. Representations maintain non-degeneracy with Var(Z) o 0 preventing representation collapse.
Assumption 5 (Estimator Consistency). The following conditions ensure consistency of key timators. For NLL-based CMI approximation, the predictor achieves realizability or bounded calibration error with respect p(y|z). For InfoNCE lower bounds, in-batch negative samples satisfy approximate independence. Theoretically, K a(n) achieves optimal rate Op(n-1/2) Practically, K x log n provides sufficient statistical power with error Op(log (I)og))-1/3 -1/2), balancing computational cost and estimation accuracy. For soft-rank consistency, the temperature decays as Trank On ") for (0, 1/2) ensuring both rank consistency and bounded gradients.
A.6 MEASUREMENT AND PAIRING ASSUMPTIONS
A.6.1 FORMALIZATION OF SEPARABLE MEASUREMENT ASSUMPTION
Definition A.3 (Separable Measurement Structure): Image observations satisfy separable mea- surement structure
MM = 99as (aM, Dayle; TM)) 7x gy (ay, bbyye: ?)Y)
where as; ay represent semantic content variables related to causal variables, bayle denotes the style variable affecting visual appearance but not causal relationships, and TJM, vy measurement noises independent of all causally relevant variables.
Key independence conditions for separable measurement include noise independence (Ma,, m/y) JL (T, M,M",, YY, XX) and style conditional independence bayle IL (T,Y*) ) XM.
A.6.2 VERIFICATION METHODS
At significance level we fail to reject Ho Corr(ñ, = 0 (HSIC/permutation- based indepen- dence tests; multiple testing controlled via Benjamini-Hochhe FDR). For style independence, we test Hu T(baykk T, Y M, X) = (conditional mutual information permutation test).
Practical verification strategies include using style disentanglement techniques to verify the sep- arability of style variables, verifying the invariance of causal relationships through multiple style transformations, and cross-validating the consistency of causal effects under different style condi- tions.
21

---

Published as a conference paper at ICLR 2026
A.6.3 PAIRED DATA PREREQUISITES
Strong pairing conditions require entity-level exact pairing (I, ,M, Y,,,, ,T, X;), where the subscript indicates the same individual or entity. When exact pairing is infeasible, allow tem- poral window pairing (observations paired within reasonable time windows), matched pairing (near- est neighbor matching based on key covariates), and distributional alignment (ensuring marginal distribution matching P(TM ~ Punrge ))-
Paired data is used construct positive sample pairs in InfoNCE loss. When paired data is unavail- able, we need to ensure positive sample pairs come from the same semantic category or continu- ous semantic neighborhood, filtering alternative positive sample pairs through semantic similarity thresholds, Under weak pairing or distributional alignment (e.g.. MMD), if the sampling and cross- fitting conditions B.3 and [.4 are satisfied, and positive pairs/negatives are sampled indepen- dently, the validity of lower bounds and bias bounds can be maintained.
A.7 SCENARIO-SPECIHG CONDITIONS
A.7.1 FRONT-DOOR CRITERION AND INSTRUMENTAL VARIABLE CONDITIONS
Front-door Criterion (Three Conditions): For path T M + Y*. M as front-door variable must satisfy: M intercepts all causal paths from T to y", back-door blocking from to M, and back-door blocking from M Y*. Specifically, all directed paths from T Y pass through M, there are no back-door paths from T to M or all back-door paths are blocked by X, and given T and X, all back-door paths from M to Y* are blocked.
Instrumental Variable (Three Conditions): Let Z be an instrumental variable for T, satisfying Relevance: Cov (2, T') # 0 (excluding weak instruments; first-stage F-statistic exceeds conventional thresholds, e.g., > 10); Exclusion restriction: Z ..L Y* (T; Exogeneity: Z 1L U X, where represents unobserved confounders affecting Y*
Positivity conditions require 0 < P(M
x) < 1 in the front-door case and T t, X P(T = t Z X x) < the IV case.
A.7.2 SCENARIO- NECESSARY AND SUFFICIENT CONDITIONS
IM Scenario Conditions: The necessary condition is the absence of directed path T I Y*. The sufficient condition is satisfied when JM gas (M, T))) with т)м 11 Y* and bayle IL Y M. Violation detection fails to reject Ho I[I M) = 0 at level a (conditional mutual information test); Ho is rejected, we deem a violation.
IY Scenario Conditions: Proxy sufficiency requires Y* IL X o(I"),T, where s(I") contains all information about Y given X. Strict monotonicity requires the existence of strietly monotonic function h such that
BY = where h is strietly monotonic
Equivalently, monotonicity manifests as Pr(Y > Y2" = Pr(o(1))) > )(1))). The order- consistent equivalent form is used for binary comparisons and does not require linear linkage.
DUAL Scenario Conditions: Simultancously satisfies both IM and IY conditions. Due to the existence of collider structure I +- M Y + I", simultaneously conditioning on M and I' introduces spurious associations. Implementation strategy requires avoiding simultancous use of both image modalities in the same batch or same loss term when estimating conditional mutual information or training discriminators.
A.8 REGULARITY AND IMPLEMENTATION CONDITIONS
A.8.1 SAMPLE EXCHANGEABILITY AND MOMENT CONDITIONS
Observed samples (Z,, Zz.... Zn) satisfy either independent and identically distributed Z, it Pz B-mixing conditions. For time series data, B-mixing requires B(k) O(k-*), where 2.
22

---

Published as a conference paper at ICLR 2026
Moment conditions include bounded second moments 2 for all and optional light- tail assumptions where Z; satisfies sub Gaussian conditions
E(,x)(tZ;)] & exp(o /2) for all t R
Non-degenerate encoder conditions require Var(Z.) 2 og > 0 to prevent representation collapse to constants.
A.8.2 IMPLEMENTATION METHODS FOR LIPSCHITZ CONSTRAINTS
Encoder functions E' X Z must satisfy Lipschitz continuity
[[())) E(x2)]l2 L|[21 x222
where is the Lipschitz constant.
Practical implementation constrains each layer's weight matrix W through spectral normalization W
WsN o(W). where o (W) is the largest singular value of W. Gradient clipping limits gradient
clip min (1, C | CGP λ [max((, [())l) -)
V, where C' is the clipping threshold. Optional gradient penalty adds the loss function.
Verification methods numerically estimate the Lipschitz constant I, 2 max21
[ E(wx) =2
monitor spectral norm changes of encoders during training, and verify the boundedness of represen- tation changes under perturbed inputs. A computable upper bound I IIe (max(W) (tracked via power iteration).
A.9 IDENTIFIBBLLTY CONDITIONS
Causal effect identification requires backdoor, front-door (Section A.7.1 or instrumental variable criteria. Under Assumption AA.5aand the structure- structure-preserving losses, these criteria apply in the repre- sentation space using (Zr, ZM, Zx, Zy) with directional constraints maintaining 7 as exogenous.
These assumptions provide the foundation for theoretical guarantees in Appendix and Ap-
pendix C and guide implementation choices in Appendix D
B THEORETICAL PROPERTIES AND TECHNICAL CONDITIONS
B. THEORETICAL FOUNDATIONS OF LOSs FUNCTIONS
The alignment loss Latign provides a lower bound on mutual information via noise contrastive es- timation. Let (a+ (=+.y*) be a positive sample pair and {vp H be independently sampled negative samples. The InfoNCE loss is defined as:
exp(f (2 1 T/Talian)
LInfNNCE = -E log
(19)
exp(S(=t, ,orst,n+/ 1/ Talign)) + Ee exp(S(=+, TTalign))
where f x y - the scoring function. By the Donsker- Varadhan variational representation, T(Z; Y) 2 log K LInfANCE. This lower bound improves with increasing K, and its validity relies
on the negative sampling independence assumption Y 1 (Z- .Y+) (see Assumption A.5
Conditional mutual information estimated via the density ratio method. Define ANLL B[- log 4a [v] - E[- log (y) 2. where qo and 9o are two independently parameter- ized predictors to avoid bias CMI estimation. Under the realizability assumption (Assump- tion A.5). i.e., there exists 0* such that qo- (z/l) p(y] 2) almost surely, we have lim ANLL
[22;; ZM). Under model misspecificatioo, the bias C* = infa (L(pl)ge). and consistency
requires C* + 0.
B.2 NLL CALIBRATION AND REALIZABILITY
B.2.1
THEORETICAL FOUNDATION OF NLL SURROGATES
Theorem B.1 (Calibration of ANLL). Assume (i) ET. EM are Lipschitz and predictors qa- 4s are well-specified (realizable), and (ii) the regularity conditions in Appendix A.8dhold. The conditional
23

---

Published as a conference paper at ICLR 2026
independence loss employs two independently parameterized predictors qo and do avoid bias in CMI estimation. Specifically:
qo : 2T 2M P(Y) models p(y) St, 2m.
q4 : ZM P(Y) models pmy)=m)
Then
ANLL E. I(ZTiY) ZM).
Without realizability, letting
5" = :iinf B2z KL (p(- Zr, 2M) 20(. Zr, ZMn)) BZz [L ZM) ll 4.(-1)
}
we have
-
ANLL, = I(ZT;Y| Zm) + 5* + Exi En
Proof: Let the true conditional densities be ply Zei 2m) and p(z)(zm), and the parameterized densities be go go(1le 21. 2m) and (3)22m) Define
NLL
log (yi Zr,i, Zm,i),
NLL
log go (yi (mmi)
ANLL, NLL NLL I(; 2m) + + 0p
Under realizability, by the law of large numbers and continuous mapping theorem, convergence true entropies follows. Under misspecification, the additional fixed bias (* quantifies the degree of
model mismatch.
Lemma B.1 (Realizability Condition): If there exist parameters 6*, o0 such that qa- (ylze, 2m) = p(ylas Zt, 2m) and Qø- (y) Fm) = p(y) zm almost surely, then
lim E A[NNL, = H(Y ZM) HYY Zm) ((ZT;
100
Lemma B.2 (Approximation Error Bound): Under model misspecificatioon let 0n, dn empiri- cal risk minimizers. Then
ANLL, - YIZm) -5"1 s E,. + 8n
where E. = Op((logn/n))/3 is the finite sample error and 6, Op(n00p11//100000 is the optimization error.
B.2.2 PRACTICAL VERIFICATION STRATEGIES
Corollary B.1 (Calibration Testing): NLL calibration verified through the following procedure. First, training convergence is monitored by checking |NLLL, ANLL., for sufficiently large k. Second, cross-validatio consistency requires computing ANLLval on independent vali- dation sets and ensuring ANNLeraiin ANLLval < 8. Third, conditional independence testing directly examines the null hypothesis Zr JL Y ZM, where rejection at < 0.05 provides statisti- cal evidence for I(Zr; ZM) > 0.
B.3 INFONCE LOWER BOUND CONDITIONS
B.3.1 SUFFICENTT CONDITIONS FOR LOWER BOUND VALIDITY
Theorem B.2 (InfoNCE Lower Bound Theorem): Let (Z+ y+) be positive sample pair and
be negative samples drawn independently from marginal distribution Py. Define the InfoNCE as
{Y, k
exp(f(zt,Xt))
LINfANCE = -E log
exp(f(Zt, pop(f(Zttt)//tm + Σk_ exp(f(zt (YT)/T)
24

---

Published as a conference paper at ICLR 2026
where 2 x - R is scoring function and > 0 is the temperature parameter. Under
the following conditions: (i) Negative independence: Y i.d. Py and yx TT (Z+, ++); (ii) Lipschitz scoring: (f f22, y) < 2 22]1: (iii) Sufficient negatives: Theoretically K (2(n) minimizes bias Op(n 1/2). Practically, K log n (validation selects constant C such that K 2 n) achieves error εк Op(K
Op( n) -1/2,, which suffi- cient for most applications while maintaining computational tractability, We have the lower bound relation (van den Oord al. 2019 Poole al., 2019):
I(ZY) 2 log K LInIONCE EK
where cx Op(K-1/2).
Proof: According to the Donsker- Donsker-Varadhan Varadhan variational representation (Donsker & Varadhan 1975).
IZ; Y) sup Epsy (f, log E [exp(f
Lemma B.3 (Negative Sample Approximation): Under negative sample independence,
Seaery (exp([(()))
Lemma B.4 (Temperature Scaling): For any t > 0, replacing by fr :- f/T yields the same functional family up to a scale; the InfoNCE lower bound retains the form I(ZYY) > log K CintoNCE (fr). Hence 7 acts as a numerical conditioning parameter rather than tightening the bound per se; it should be selected via validation.
B.3.2 TECHNICAL REQUIREMENTS FOR PRACTICAL IMPLEMENTATION
Corollary B.2 (Negative Sampling Strategy): To ensure the conditions of Theorem B.2 are sat- isfied, practical implementation requires the following. Global batch sampling ensures that in dis- tributed training, negative samples are drawn from the global batch rather than local batches. Tem- perature adjustment selects т [0.01, 0.1] through validation sets for optimal performance. Nega- tive sample quantity follows K log ensure statistical power.
Theorem B.3 (Distributed Consistency): In distributed settings with D devices and batch size B per device, if negative samples drawn from the global batch DB, InfoNCE lower bound validity maintained (van den Oord et al., 2019):
I(Z 2 (DB - 1) InfoNCE - (P((DB)-113)
globbal
Proof: Extension of Theorem B.2 analysis to multi-device settings, with the key requirement being maintenance of global negative sample independence.
B.4
CONDITIONAL MUTUAL INFORMATION BIAS CONTROL
B.4.1
SOURCES OF CMI ESTIMATION BIAS
Theorem B.4 (CMI Bias Decompasition Theorem): Let i(Zr; Zl be some estimator of CMI. Under the density estimation framework, total bias decomposes as:
E[i Z1)) - I(Zr; Y = Baeusity + Bauite Baiscretization
where Bdensity represents conditional density estimation bias, Būuite represents finite sample bias, and Bdiseretization represents discretization bias when applicable.
Lemma B.5 (Conditional Density Bias Bound): Using kernel density estimation with bandwidth h 11((441)) (Silverman's rule plug-in bandwidth in practice),
Baensity O(n (ooo444/)))))
where d is the dimension of the conditioning variable Za
(Wasserman 2006).
25

---

Published as a conference paper at ICLR 2026
B.4.2 SAMPLE SPLITTING AND CROSS-FITTING
Theorem B.5 (Cross-fitting Bias Control); When estimating CMI using K-fold cross-fitting, let
D == U Dp be the data partition and Pー be density estimators trained on D \ Dj. Define
D~*(84 Re, 2m.i)
Icv
log
K
D&
p p_x(3s(2m,))
iED
Under regularity conditions (Chernozhukov et al.
2018):
[R[fev] (Zr; Y [Mm) = O(n 1/2))
Proof: The key advantage of cross-fitting lies in avoiding overfitting bias. For each fold
EiED, log p ZT, ZM,,)
= [(2;; ZM) + o1))
since p is independent of (Y,, Zr, 3,.) Summing and averaging yields the desired result.
Corollary B.3 (Practical Bias Control Strategy): Effective bias control employs 5-fold or 10-fold cross-fitting. Conditional density estimation utilizes regularized neural networks or Gaussian pro- cesses. Monitoring variance across different folds guides regularization strength adjustment when excessive variance is observed.
B.5 SOFT-RANK TO SPEARMAN CORRELATION CONSISTENCY
B.5.1 THEORETICAL PROPERTIES OF SOFT-RANK FUNCTIONS
Definition B.1 (Soft-rank Function): Given vector æ "ei...,yy and temperature parameter the soft-rank function defined as:
Ronk
T
Ij
j=1
where (2) (1 + exp(-=))-1 is the sigmoid function.
Theorem B.6 (Soft-rank Consistency Theorem): Let (X, Y') be bivariate random vector and Rank(z;) be the true rank of x, in the sample. As r + 0,
lim SoftRank, (z;) = Rank(z,) -1 7-0
with uniform convergence over compact domains (Blondel et al. 2020).
Proof: As 7 + 0, o(Iz - kT)/T) + 12,2,, hence
lim SoftRank, (zt)
Σ 1nisy = Rank(æ;) - 1.
J=1
Uniformity follows from the monotonicity of the sigmoid function.
Theorem B.7 (Spearman Correlation Convergence Theorem): Let ps(X, Y) be the true Spear- man correlation coefficient and p, be the soft-rank based estimator:
P Corr(SoftRank, (X), SoftRank,
Under conditions: (i) (X,, Y)) independent and identically distributed for 1,..., n. (ii) joint distribution has continuous marginals; (iii) O(n for some E (0, 1/2), have:
p ps X, Y) = (p 1/2 + T log
Proof: Decompose error into bias and variance components:
P ps (X, Y) = [[ (s(X, Y)] + j0- - Elp-1].
Lemma B.6 (Bias Term Analysis): -s Y) = 0(T vz)
Lemma B.7 (Variance Term Analysis): Var[ør] 11
26

---

Published as a conference paper at ICLR 2026
B.5.2 GRADIENT BOUNDEDNESS AND OPTIMIZATION STABILITY
Theorem B.8 (Soft-rank Gradient Bound): The MAC loss LMAC = -Ø, has gradient with respect to representation satisfying:
aLMAC
Oz
where C' is constant depending only the data range (Blondel et al. 2020).
Proof:
@SoftRank, (x::)
dr;
j#i
using d'(z) 1/4. The result follows by chain rule and correlation coefficient gradient formulas.
Corollary B.4 (Training Stability): Choosing = (15) balances bias and gradient stability: Bias: (n/ logn); n); Gradient norm:
Overall rate: Op(n-1/4
B.6 STRUCTURE PRESERVATION IN REPRESENTATION SPACE
The conditional data processing inequality guarantees information monotonicity. For deterministic mappings E, we have ( E Z)) s [[;; Z) (Cover & Thomas 2006). Under stan- dard regularity conditions in Assumption A.5 (Lipschitz continuity and bounded second moments), standard estimators achieve Pp(n 1/2) convergence rates.
The anti-bypass condition (Assumption A.5) ensures correct causal pathways. Defining the Markov chain T > (M,Y) > Zy. the conditional data processing inequality yields I(T; ; Zy [M) S [[T; MJ Combined with the original conditional independence T I Y*[M and the IY bridging assumption E|y "
h(o(1") (Assumption A.5). the mutual information [(T; Zy (M) is controlled by the calibration
h(((T*)) ))12 Under bounded support sub-Gaussian conditions, standard f-divergence inequalities (e.g., Pinsker's inequality) convert the mean squared error EY mutual information upper bound (Fedotov et al. 2003).
EΛ = |[
B.7 PC ALGORITIM CONSISTENCY GUARANTEES
Partial correlation testing achieves asymptotic normality via Fisher transformation (Fisher 1921 1924j. Given conditioning set S, the Fisher-transformed sample partial correlation T
Vn [] - 3aretanh(pxy-s) ) asymptotically (00, 1) under Ho PXY.s 0.
Multiple testing correction employs the Hinjamini-HHohber procedure. For hypotheses with ordered p-values P(1) <
P(m), define k* max(í P(t) S ia/m} and reject (Ha)..... H(a-))- Under independence or positive regression dependence on subsets (PRDS), this procedure controls the false discovery rate at FDR < (c.
Under the faithfulness assumption (Assumption A.5 consistent conditional independence testing, and the existence of a minimum effect size (minimum non-zero partial correlation y > 0) with bounded degree conditions, ability approaching 1. The error probability decays exponentially polynomially in n, with the specific rate depending on y, maximum degree, and test power.
PC algorithm recovers the true Markov equivalence class with prob-
C MAIN THEORETICAL PROOFS
C.1 PROOF STRATEGY AND ROADMAP
To improve readability and make dependencies explicit, we organize the proofs so that the reader can proceed linearly: from the statistical consistency of each surrogate loss (Appendix B), to structural preservation the joint objective (Theorem [], and finally to query-level identifiability guarantees (Theorem
27

---

Published as a conference paper at ICLR 2026
1. Step 1: Statistical Foundations (Appendix [B. We establish rons-consystency of our vari- ational/contrastive surrogates under Assumption A.5 (Realizability and Regularity with cross-fittings fosss-fitiin):
Lemmas B.1-B.2: the calibrated surrogate for conditional mutual information makes Ce1 estimate (I; Y ZM) with enror Op(n-1/2), and the InfoNCE term in LMBR lower-bounds MI( Y) with finite-K deviation Op(K))/2).
Lemmas B.6-B.8: the soft-rank operator used in LMAC approximates Spearman's Ps while keeping gradients bounded.
with enror
2. Step 2: Structural Preservation (Appendix
Using Step 1, we prove Theo- rem (CSP Achievability): minimizing the joint loss Ltotal enforces the e-CSP conditions (see equation 14). Here Assumption A.8.2 (Lipschitzsspectral control) provides compact- ness and stability. yielding uniform convergence of ERM and controlling the generalization gap at rate (p -1/3).
3. Step 3: Identifiabilit Preservation (Appendix C.4
Theorem translates e-CSP into a the representation error e propagates where the calibration term satisfies
query-level bound. By the Lipschitz stability in A.8.2
at most linearly to the causal query: [Q-Q) < K KE-00
deal= Op(n 1/2 under
A.5 (see equation 16P.
C.2 CONDITIONAL INDEPENDENCE PRESERVATION THEOREM
Consider CARL framework's three sce- narios: IM/DUAL scenarios with Y*, and IY scenario with Y = x(T)) as proxy variable. Under the assumptions in Appendix A.5 suppose the original variables satisfy T IL Y M. Representations trained by CARL and converged to approximate empirical risk minimizers satisfy:
Theorem C.1 (Conditional Independence Preservation):
I(2r;2 Zy 2m) S (Er, EM) + o1)
with high probability, where (",T,, EM) is the model misspecification bias for given learned en- coder classes (see Appendix B.2
Premise Conditions: (1) Representation Zy EY depends solely on Y without additional dependence on (Zr, ZM) or information bypassing Y (anti-bypass, see Appendix A.5); (2) Markov boundary retention loss LMBR is enabled with weight Wmbrr > 0; (3) Encoders are deterministic functions during evaluation contain randomness exogenous to (T) M, Y); (4) IY Bridging: For IY scenarios, there exists measurement model Y = h(Y", v) where IL. M... such that T IL Y M, 11 XX) T IL Y M.
Proof: Under IY scenarios, the bridging assumption ensures that original conditional independence T IL Y* M implies conditional independence on proxy variables T IL Y M.
Let (ÉT, EM, Ev) be approximate empirical risk minimizers of the conditional independence loss - Y)). According to Theorem B.1 in Appendix B.2 upon training convergence we have LC1 = I[Zr; Y + EM) (p(1), where C* (Er, EM) depends the learned encoder classes.
Lei = E (Zr, Za + Y)] -
The key observation is that since Zy = (Y) depends solely on Y (premise condition 1), there exists Markov chain Zr - (Y, 7M) + Zy. Applying the conditional data processing inequality yields 7 T(Z;; Zy M < T(Zr;; 7 Notably, even when evaluation involves exogenous noise (random encoding Markov kernels), conditional DPI remains valid since the Markov chain property is preserved.
The Markov boundary retention loss LMBR ensures Za does not collapse to zero information by maximizing an InfoNCE lower bound of I(Z Y). Specifically, there exists a constant c > 0 determined by temperature parameters and batch size such that I(Zar; Y)
Ex. where εк Or(KO (see Appendix B.3). This excludes trivial solutions that achieve conditional independence through ZM degradation.
When original variables satisfy T 1 Y M, theoretically the minimum value of I(Zr; Y ZN) is EM) (limited only by model misspecification for given encoder classes). Minimization of Tor drives the system toward this theoretical lower bound. Combining conditional data processing
28

---

Published as a conference paper at ICLR 2026
inequality and NLL calibration results, obtain:
I(r; Zy Z T(Zr;Y Zm) S (Er, EM) + or()).
Here (* (",,,, depends on the learned encoder classes; under Lipschitz constraints and sufficient capacity, (* can be controlled to be small (see Appendix [8.2].
Remark: In all scenarios, we employ Zy = (Y) ensuring it depends solely on Y (see Ap- pendi and enable LMBR to prevent ZM degradation. The IY scenario bridging assumption ensures conditional independence transfer from Y* to proxy Y, making the conclusion applicable the original target variable y* well.
C.3 MARKOV BOUNDARY RETENTION THEOREM
Theorem C.2 (Markov Boundary Retention): Under the assumptions in Appendix AA.3 and the premises of Theorem C.I, with ZM g(M) as a deterministic encoder function and Zy depending solely on Y (anti-bypass condition), representations trained by CARL preserve Markov boundary properties:
I(Zm: Zy) 2 I(MYY* 5MB
T(Z Zy Zaa) S min {I(T;Y* Y M) H(M Zas); (* (Er, + or(1)}
where 8MB (M ZM) + H(Y Zy) represents encoder information loss (interpreted as conditional cross-entropy w.r.t. a common base measure using discretization approximation; both provide upper bounds for continuous variables), and (Fr, EM) is the model misspecification bias from Theorem C.I.
Proof: For first inequality, since Zm = g(M) is deterministic, 1(2M; Y | M) = 0, hence the chain rule gives (M; = I(ZM; Y*) I(M; Y ZM). Since [(M; Y ZM) S H
Zи). we obtain:
I(ZM;Y") I(M;Y") H(M ZM).
For the information -iformationn-hhorrtiic inequality 1 1(2Mi Zx) 2 I(ZM: Y*) H[Y* Zr), we use the standard derivation:
[(2M; Zy) I(ZM; = H(Z Y") - (2M [xx) 2 -Y |Zy)
where the inequality follows from:
H (Zas Zy) S H(M,, Y (x) H(M Y HH Zy) S H(ZM |Y*+H(Y"* Zy).
Combining yields:
1(ZM; Zv) > I(M;Y*) H(M 2M) - H(Y Zr). Defining δMB = HM 2m) + HY | Zy) yields the first inequality.
For the second inequality, we derive two independent upper bounds. First, by data processing in- equality with T > Zr and Y Y + Zy:
1(2r Zy MM) & 1(2r;) Y ZMs) s ((T:Y ZMM)
Since conditioning coarsening gives I ( ZM) & IT;YY MM + H(M ZM), and using IY bridging T(T;Y Y M) < T(T:Y* M):
1(2r: Zv ZM) < IT;YY* M) + H(M ZM)
Second, Theorem C.1 provides an independent training error bound: T(Zy; Zy 7M) < (FT, FM) op(1).
Taking the minimum of these two independent bounds yields the second inequality.
Estimable Certificate Version: In IY scenarios, using observable proxy Y:
I(ZM; Zv) Z I(M;Y) HMM Znr) H(Y Zv)
where ((M) | ZM) and H(Y Zy) are estimated through validation set cross-entropy. Since Y* - Y is post-processing. T(M:Y)
("M."). The CARL framework optimizes lower bound of I(Za:: Y) through the InfoNCE term LMBR, with practical estimation requiring consideration of statistical error Ex = 0p (11)). The estimation version incorporates cross-fitee cross-fitting bias control from Appendix B.4
29

---

Published as a conference paper at ICLR 2026
C.4 MONOTONIC ALIGNMENT CONSISTENCY THEOREM
Theorem C.3 (Monotonic Alignment Consistency Preservation): Under Assumptions A.2 (sep- arable measurement & measurable encoders), A.4.2 (Lipschitz constraints), and B.4 (soft-rank con- sistency), let Aas |a; azl and Azj = || 23112. If the additional order-preserving margin assumption holds: there exists y > 0 such that
Pr[(Aau (Aas) Azue) > 0]
Under copula density upper and lower bound regularity conditions, there exists k E (0, 1] such that Ps( Aa, Az) 2 KTK (Aa, Az). Since the order-preserving margin assumption gives Tк 2 27, have e(r)) = K 2/y.
Then there exists a constant e c(n) [0,1] such that upon convergence of AMAC training,
ps(Aa, A2) c c c-cc Egen
where
ÓMAC Op(T log N + neel 12)
Ner is the number of subsampled pairwise distances, Egen Op(n-1/2) with constants depending on the Rademacher complexity the representation class and IB/Lipschitz regularization; details
-1/3 achieves ôMAC in Appendix B.5 In practice, choosing neff gradient vanishing.
Op(n -1/3) while avoiding
Proof: By Theorem B.6, have sup, |SoftRank, (x) Rank(x)] S C'7 log N, where N is the number elements participating in ranking, equal to Natt-
By Theorem B.7, we have f 1---ps( -spss((a Ae)l = (0 + log N). Here the pairwise distances (Day,. 2 are controlled to have size Neir through subsampling strategies, avoiding excessive computational complexity while maintaining statistical effectiveness.
The order-preserving margin assumption combined with Lipschitz stability provides a positive lower bound for the Spearman correlation through Kendall's т. This assumption can be verified through pairwise comparison tests on validation sets, equivalent to permutation tests for Kendall's т.
ERM convergence maximizes the training objective to neighborhood of the population target, producing generalization eπror Egen Op(m0pm1p 1/2).
Regarding the choice r, theoretically without gradient lower bound constraints. the optimal choice would be -* x (log -N- -1.-,,,11111 1/2 giving ÓMAC ( (12)) However, in practice to avoid soft- rank gradient vanishing, we take r x Par 1/3 balance approximation bias and optimization stabil- ity, achieving the practical convergence rate dMAC Op (m -1/3).
C.5 IDENTIFIABILTYY CONSISTENCY THEOREM
Theorem C.4 (Cross-modal Identifiability Preservation): Under Assumptions A.4 (causal struc- ture and faithfulness), A.7.1 (identification conditions). A.7 (scenario-specine requirements), and the outcome calibration assumption C.6 suppose the original causal query [t] Q(t: (TP,P(T,M.. M, " xX) is identifiable via backdoor adjustment, front-door criterion, or instrumen- tal variables. Let (Zr, ZM, Zv, Zx) be the learned representations from CARL training satisfying Theorems C-I.C.3.
Define the corresponding query representation space E(Ý(()) Qt; P(Zr, Zas, Y, Zx)), where := h((y). Then
Bl E(y < SMnn + 1r SMAC + deat + Eident
30

---

Published as a conference paper at ICLR 2026
where 8MB is from Theorem C.2, ÓMAC is from Theorem C.3 (activated in IY scenarios, otherwise
0), deat |||] - [[k(Zy)]] || is the outcome calibration error, and Eidett represents identification- specific statistical error (Eident Op(n (1(2) under parametric/caliirratabe settings; Op(n75 P) under nonparametric settings depending on smoothness and dimensionalityy)
Proof: We establish identifiability preservation for each identification strategy.
Backdoor Adjustment Case: By Theorem C. I.I's conditional independence preservation and A.4.2's Lipschitz conditions, the d-separation of X is maintained in (Zr, Xx, Zy) with op(1) error, yield- ing
([) Ezx 2 = 2t, 2x] + op(1)
which equivalent to the original backdoor formula, with discrepancies dominated by beal and Eident-
Front-door Criterion Case: Theorem C.2 ensures Markov boundary information retention of ZM for Y combined with C.1's CI preservation, yielding the representation space front-door formula
BIÝ (1) Ezy
E(yIZAr) Ezx (P(ZM) = 2t, Zx)] dMn-
Due to the deterministic mapping ZM = EM (M) and Lipschitz property of h, the mediation path- way identification formula remains valid in the calibrated representation space, with errors contain- ing 8MB and 6cal-
Instrumental Variable Case: representation space, relevance Cov(Zz, Zz) * and exclu- sion/exogeneity 7z 1. (Zr, 7x) approximately hold (by C.I and A.4.2), yielding IV estimation based on h(Zy) equivalent the original formula, with error Seal + Eident-
Scenario-unified Treatment: IM scenarios directly apply the above results with ÓMAC = 0 since no image-to-outcome monotonic alignment is involved. IY scenarios introduce the 8MAG term through IY bridging and Theorem C.3's monotonic consistency, ensuring order consistency from ((T²) to Y*. DUAL scenarios avoid simultaneous conditioning on 1M and 1Y A.7 applying the corresponding identification strategies by pathway.
The total error comprises four components. δMB reflects Markov boundary information loss during encoding, primarily from H (M)) ZM) H((*)ZZ)) δm AC is activated only semaanic-representttion distance mono- tonic alignment errors. deal captures calibration error from representation space outcome space, empirically assessable through | Y ElA Zy )]] appropriate calibration losses validation sets. ident encompasses statistical fluctuations of identification formulas under finite samples and generalization errors from representation learning.
Error Decomposition:
IY scenarios, arising from soft-rank approximation
all error terms converge at con- trollable rates, ensuring asymptotic preservation mross-modal identifiability.
Under regularity conditions A.8 and technical conditions B.2 B.5
C.6 OUTCOME CALIBRATION ASSUMPTION
Let h" (2) B| Zy 2] be the true conditional expectation function. There exists learned estimator h Zy R that is Lipschitz continuous and approximates h". The calibration error defined as deat = Y ([ which can be empirically assessed through validation set performance the calibration head. In practice, h is implemented through regression heads or calibration layers, inheriting Lipschitz properties from the overall network constraints A.8.2 A.8.2
C.7 METRIC CONSISTENCY PROOF
This section establishes formal connections between the four evaluation metrics and the main theo- retical theorems, proving that these metrics can effectively detect the theoretical guarantees.
Theorem (Metric Consistency): Suppose the representations from converged CARL training sat- isfy the conditions of Theorems C-1.C.4. Then the four evaluation metrics have the following consis- tency relationships with the corresponding theoretical results, where 1 1 indicates the IY scenario:
31

---

Published as a conference paper at ICLR 2026
CIP and Conditional Independence Preservation (Theorem C.1): Under Gaussian conditions,
by the data processing inequality T(Y; Zr 7M) v T(Zy: 7 (M) S Ç" we have
[Y:2T|ZM 2 log(1 prrc) - lppll S g((*) = V e-
Let (p, a)) be the power function of the partial correlation test (significance level a, sample size n). Then under the local alternative !ppel - g(5*),
Pr(CIP = 1) = Pr(reject Ho) 2 1 - -3(((("),
CSI and Identifiability Preservation (Theorem C.4): Let Sw := (M; SM := SM - dMB (Markov boundary signal retention). Under mnaar-Gaussian approximation and calibration models, there exist constants K1, K2 > such that
Ro 2 K2SM, Ra S xig((")²
Using the definition CSI (R R2 / / + 8), we obtain
CSI
En
N2SM +
MBRI and Markov Boundary Retention (Theorem C.2): Define
Ý(M; MBRI I(M:Y) i(T:Ý)+ +E
Let (n, B) denote the histogram MI estimation bias with B bins. Then by C.2 and C.I,
I(M;Y*) 8MB EMT MIBRII IN I(M;Y-)- dMB - EMI + 5 + EMI +
Op(n0pn111 11/2)
MAC and Monotonic Alignment Consistency (Theorem C.3):
E[MAC| 2 c(7) SMAC Egen Erank (Nett)
where Srank (ne#) = (n relation, and Nett is the non-overlapping pair subsampling size.
1/2 comes from the U-statistic convergence of sample Spearman cor-
C.8 METRIC CONSISTENCY
Theorem C.6 (Metric-Theor Correspondence) Suppose the encoder family & satisfies Assump- tions 1-4 and (Zr, ZM,, Zy) are the converged representations. Then the evaluation metrics have the following correspondence with theoretical guarantees.
Proposition C.6.1 (CIP Metric) [(2r; Zy Zm) S (", then the partial correlation coefficient satisfies Ippc Zy ZM) S /1
Proof: Under Gaussian assumptions, the relationship between conditional mutual information and partial correlation is [Zr; Zy ZM) - log(1 Pp.c). From I [Zr; Zy ZM) S 5*, we obtain
2 < 1 exp(-2(*).
Proposition C.6.2 (CSI Metric) Let : - M - Y be the correct causal path and L : T + Y the direct path. If [(M; Y*) 8Mn > then R2(C)- R((C)e
O0C "2
T(M:Y*)
Proof: By Markov boundary retention (Theorem C.2), R²(L) I(ZM; Zy) 2 I(M;Y") 8MB-
By conditional independence preservation, (C')') x I(Zr; Zy) $ S5" under linear approximation.
Proposition C.6.3 (MBRI Metric) Define MBRI = i(M:Y)/(((M)) + i(T:Y) + c.. Then --
E|MBRI| 2 TOMY"))6MB-C" 1/22.
32

---

Published as a conference paper at ICLR 2026
Proof: By Theorem C.2, the numerator ((M _ 1(2M: Zy) ( By Theorem
C.I. in the denominator I(T;Y) [ I(ZT; Zy) 1 5* The result follows from the continuous
mapping theorem.
Proposition C.6.4 (MAC Metric) Let A. = {|a - 0g v, be semantic distances and A, {1lz Zj Eزllet be representation distances. Then ps (A,, A:) 2 c(7) - O log n +n where c(7)
is determined by the order-preserving margin and is the soft-rank temperature.
Proof: By Theorems C.3 and B.7, soft-rank converges to true ranks at rate O(T logn) and sample Spearman correlation converges at rate O/2-1/2).
C.9 PROOFS OF CAUSAL STRUCTURE PRESERVATION THEOREMS
Lemma C.8.1 (Conditional Independence Transfer). Under the anti- assumption (Zy de-
pends only on Y or I"), there exists Markov chain ZT + (Y, ZM) + Zy. By the condi- tional data processing inequality, I[Zr; Zy 2m) S I(Zr;Y
Zar). Combined with Theo- rem B.2. s NLL calibration consistency, when q is realizable or under cross-fitting, I(Zr;) ZM)
LCI - 5" - o1(,), where 5* is the model misspecification bias.
Lemma C.8.2 (Markov Boundary Necessity). Let ZM = (MM be a deterministic encoding. By mutual information decomposition, 1(2M; Zx) 1(2M; Y*) + [[ZM: Zy (ZM;;
Zy). Since (Z Zy
Y*) = 0 (Markov property) and (2 Y Zx) S H Zy), we
have a(Z Zy) I( H(Y Zy). Furthermore, by chain decomposition (()
I[ ) + I(M;Y* ZM) and I(M;Y" ZM) I H(M ZMz), we obtain [[Zs:: Y*)
I(M;Y*) (M Z.). Combining yields I(ZM; Zx) Z I(M;Y*) Y*) H(M Zas) H(Y*
Zy). Defining information loss daB H H(M Zm) + H(Y" Zy), we have T [(2M; Zr) 2
[MM; -MM. InfoNCE provides a consistent lower bound for [(Mi: Zy) as K 00 with
estimation error Op(K-1/2)
Proof of Theorem 1 Under the parameter space θ = (0- [| < ce, (0) S
k with spectral norm constraints, the loss function L is lower semicontinuous on compact sets. By variational analysis, the empirical risk minimization problem minøge C(0; Pr)) admits solution sequences.
For condition (i), by Lemma C.9 and Theorem B.2. Ler converges to a consistent estimate of I[2T; Y ZI) with bias ( Z <
). For condition (ii), LMBR = -InfoNCT( (zm; 2y) directly optimizes I(ZM; Zy). By Theorem B.3.2 InfoNCE provides lower
-1/2). 11/2/)
bound with bias Op(K-1/2) and combined with Lemma C.9 I (Z Zy) ((M:)))) δMB Op(K-1/2) For condition (iii), by Theorem B.5.1 choosing temperature = (n-11//))) yields Jø, - Ps On(n 1/3), and combined with the order-preserving margin assumption, ps 2 co Op(n-113)
Combining estimation errors from all three conditions, there exists e (Frr), (FM))
such that the limit point satisfies e- CSP.. When K 2 en, K-1/2 and n -1/2 are of the same order; if K = w()) (e.g.,
K = n log n),, then K-1/2 = o(n 1/2). Therefore, the dominant order is Op(n-1/2) Different choices of weights (wer, WMR WM AMAA) A correspond to different points on the Pareto frontier.
Proof of Theorem We prove for three identification criteria.
Backdoor criterion: The original query is Q = =x, E, P(x). In representation space, Q = Σ. E[b(Zy) Er (t), Ex (()))) Define intermediate quantity Q = Σ. I(x
Error decomposes as IQ - Q1 < Q Q + Q Q). The first term is controlled by Lipschitz properties: [ETh( (Zy) [t), Ex() E(h(Zr))t, < Lip(h). The second term doal sup(..x) E x] )t, is the conditional calibration error. Thus IQ QI s E 8eals
i.c., K 1.
Frontdoor criterion:
The original query is Q = =m E[ (mm)) In representation
space, Q EE [E(h(Zy) Fmmy znd P((m For (zz) Error decomposes into three terms: (a)
33

---

Published as a conference paper at ICLR 2026
mediator distribution enror [P Et)) - P(m[t)] E Lip(Prrr)) (b) outcome model error
[E [zm, zx] - [m, - + (c) marginal distribution error P((x) P(x) s
Lix( Since the frontdoor formula involves nested expectations, total enror 1Q Q S
(Lip(PMIr) + Lip(h) Lip(Px))
+
Seal- Define =
Liph)
Instrumental variable criterion: Let Z be an instrument. The original query via two-
stage least squares is Q
Cov(Y", ()/Cov(T.22)
In representation space, Q
Cov(h(Zy). (íz)/CCo(([n) [T. Zz) where Zz = Ez(Z). Numerator error [Cov(h(Zy)) Ez)
Cov(Y", Z)
Lip(h) h) deal ||Z||2; denominator error Cov(Zr:
Cov(T, 2) I
e. (Lip(Er) + Lip(Ez)) By Lipschitz properties ratios, when Cov(T, 2) 2 6 > 0 (rel-
evance lower bound), total error IQ - QI S 2 [(Liphh) + 1 E + Seat]. Thus & =
" max(Lip(h), |1
Liz(E))
Under realizability assumption Zy = (Zy) and cross-fitting, deal = Op(n-1/2) by The- orem B.3.
D IMPLEMENTATION DETAILS
D.1 ENCODER PARAMETERIZATION
For representation mappings E X' 2 C TRd specify the functional forms. Throughout this section, || ll2 denotes the operator spectral norm. Tabular encoders Fr and E employ multilayer perceptron architectures defined as the composition E' ΦL o ... Φ1, where each layer or Rd+-1 + IRd has the form (x)x) = (Waz - bạ) with a being a I-Lipschitz activation function. To satisfy the Lipschitz constraint (Assumption (A.5). weight matrices We satisfy spectral norm constraints ||Well2 S ce, yielding total Lipschitz constant L S II.
Image encoders Eyss and Esr adopt convolutional architectures F OCNN .CCNN, Each convolutional operator CNN; satisfies ||CNN&ll2-2 I Kg (estimated via spectral normalization). The final projection ıhx Rhxwxe I is linear mapping satisfying || 1.
D.2 BATCH CONSTRUCTION FOR Loss FUNCTIONS
The InfoNCE loss constructs positive and negative samples following this mathematical procedure. Given batch B = {I,, y:)) positive pairs are defined as P = {(2,, thy (3t)) : E [N]}- The negative sample set AG (3,) * i} serves approximately independent samples (see Assumption A.5)
The scoring function f 2 depend on the specific form; in practice, normalized inner products or other bilinear forms may be employed.
R is bounded Lipschitz function. Theoretical results do
The two terms in conditional mutual information estimation utilize the same predictor family to ensure consistency (theoretical justification in Appendix B.I
D.3 PREDICTOR ARCHITECTURE FOR CMI ESTIMATION
We adopt a two-tower design to avoid information leakage:
Encr Xx + Zr, EnCM XM + ZM.
The joint predictor and the marginal predictor are parameterized separately as
go ZZ,, Zm] MLP, v,
go: Zm MLP,, y.
No layer in qs receives any function of Zr. We compute
Lei = El- log ( ZM)] - E log ( Zr. 2M)
on held-out mini-batches (or via K-fold cross-fitting) reduce overfitting bias.
34

---

Published as a conference paper at ICLR 2026
D.4 PC ALGORITHM FORMALIZATION
Conditional independence testing employs hypothesis testing based on partial correlation coeffi- cients. For node pair (2s; 2z) and conditioning set S, the test statistic Tajss v [S] - 3 arctanh(risis). where rajis the sample partial correlation coefficient. Actual testing uses Fisher transformation (see Appendix B.7p. The recursive relation
(k) = Ti J
(20)
v(L ss s)s))
serves define partial correlations.
Edge orientation follows the PC algorithm skeleton with Meek orientation rules (Spirtes-Glymond
Scheines): v-structures Zi + Zu f- Zj are identified if and only if Z; Zz Zz forms triple and Zz & Sep(Z;: Zz).
Variable tracing is achieved through deterministic mapping Vz Vong. arising from the single- valuedness of encoders.
D.5 OPTIMIZATION ALGORITIIM
CARL employs first-order optimization of the joint objective:
min (EE; D) wcrCer + WMBRRMBR + WMAC + XR(E) BEE
(21)
where the regularization term REE) incorporates weight norm and spectral norm penalties to control capacity and maintain Lipschitz bounds.
Parameter updates are implemented via projected gradient descent:
g++l II 7p Vo(()))
(22)
where Hg is the projection operator onto the feasible parameter space:
+ {0: IW, S cz, v4; ||CNN, S Kg, vk}
(23)
encompassing spectral norm constraints for multilayer perceptrons and convolutional operators.
E
RELATED WORK
Cross-Modal Representation Learning. Cross-modal representation learning has become a cor- nerstone of modern AI et al.] (2025). enabling machines to understand and integrate information from disparate sources, such as vision, language, and tabular data Mao et al. (2022). The primary goal is to learn shared semantic embedding space where data from different modalities are aligned and comparable. Early pioneering work, such as Deep Canonical Correlation Analysis (DCCA) An- drew et al. (2013), focused maximizing the statistical correlation between modality-specifee rep- resentations. In recent years, contrastive learning has emerged as the dominant paradigm, fueling the success of large-scale vision-language models like CLIP Radford al. (2021) and ALIGN Jia al. (2021), and more recent architectures that unify even more modalities, such as ImageBind Girdhaa (2023) and models from the PaLI family Chen remarkable zero-shot performance on various tasks by aligning vast unpaired datasets.
(2023). These models have achieved
Despite their empirical power, the training objectives of these methods fundamentally statisti- cal and remain agnostic to the underlying causal mechanisms that generate the data Pearl (2009); Arjovsky al. (2019). By optimizing for reconstruction accuracy, statistical correlation, or con- trastive similarity these powerful nonlinear mappings can inadvertently distort the data's 's intrinsic causal graph. critical failure we identify as representation-indueed structural drift. This distor- tion can manifest the introduction of spurious dependencies not present in the original system or, conversely, the elimination critical mediators essential for correct causal reasoning. This phenomenon is closely related to the problem of shortcut learning, where models exploit statistical artifacts rather than the intended causal features Geirhos et al. (2020). Consequently, while current cross-modal models excel at pattern recognition and statistical association, they lack the structural guarantees necessary for robust causal inference, motivating the urgent need for a new class of -awee representation learning frameworks.
35

---

Published as a conference paper at ICLR 2026
Causal Representation Learning. In response to the limitations of traditional methods, the field of Causal Representation Learning (CRL) has emerged with the ambitious goal of learning repre- sentations that encode not just statistical patterns, but the underlying causal structure of the data- generating process Schölkopf et al. (202T). Such causally-informed representations are believed to be more robust, generalize better out-of-distribution scenarios [iiu et al. (2021a] 2022); ; BBagi al.. (202)) and provide a principled basis for fair and explainable models Kusner et al. (2017) A signifi- cant body of work in CRL has focused on learning disentangled representations, where latent factors correspond to independent causal mechanisms, often by leveraging assumptions of invariance across different environments or interventions Lu et al. (0022) ; Brehmer et al. (2022). Other lines of re- search have focused on enabling counterfactual reasoning within the learned latent space, which is critical for tasks like algorithmic fairness and personalized decision-making Zuo et al. (2023).
However, the vast majority of existing CRI has been developed and validated on unimodal, often single-domain, datasets (e.g., images). Consequently, these methods are often ill-equipped to handle the unique challenges posed by multi-modal data, such as information asymmetry between modalities or the need to ensure consistent geometric and semantic alignment, which we identified earlier. Furthermore, while some recent studies have begun to explore causal learning from multi- modal biomedical observations Sun et al. (2025). a general framework that explicitly guarantees the preservation of causal effect identifiability conditions, such the backdoor, frontdoor, and instru- mental variable criteria Pearl (2009), in a shared representation space remains an open challenge. This gap is particularly critical in high-stakes domains like healthcare, where preserving the integrity of causal pathways for reliable decision-makin is paramount.
Preserving Causal Structure in Multi-Modal Settings. A natural and critical frontieris is therefore to bridge the gap between these two lines of research: to enable the powerful alignment capabili- ties of cross-modal models with the robust causal principles from CRL. However, this synthesis is profoundly challenging, as the unique characteristies of multi-modal data introduce specific obsta- cles that are not addressed by conventional unimodal CRL methods Schölkopf et al. (2021). One major obstacle is the Cross-modal Information Bottleneck (CIB), where information-dense modal- ities (e.g., images or text) can dominate the learning objective, potentially masking or discarding causally salient variables from sparser modalities (c.g., tabular biomarkers) Zhang et al. (2024); Weil eet al. (2024); Xu et al. (2025). Another is ensuring Modal Alignment Consistency (MAC); standard contrastive objectives guarantee coarse-grained alignment but do not enforce a monotonic relation- ship between semantic similarity and latent space distance, a property crucial for preserving ordered relationships and the relative strength of causal effects Liang et al. (002)).
Most importantly, the complex, nonlinear transformations inherent in deep encoders offer no guaran- tee of Cross-modal Identifiability Consistency (CIC). The very conditions required for causal effect identification, such as the backdoor, frontdoor, and instrumental variable criteria, can be invalidated during the encoding process, rendering downstream causal queries unreliable Pearl (2009). While initial and important steps have been taken explore causal learning from multi-modal biomedical data (un et al. (2025), these pioneering efforts often focus on specific applications or do not provide a general mechanism for verifiably preserving these identifiability conditions. As of yet, a general framework designed to explicitly tackle CIB, MAC, and CIC simultancously, thereby ensuring that learned cross-modal representations are "causally sufficient" for downstream inference, remains an open and pressing challenge.
F EXPERIMENT DETAILS
F.1 EXPERIMENTAL SETUP
Network architectures and optimization protocols are fixed a priori and selected by cross-validated grid search to ensure comparability across methods. Tabular encoders adopt a three-hidden-layer MLP with 128 units per layer, each block ordered as Baaear--BBhNNoon--RRR use ResNet-18 88 trained from scratch with the same data partitions as the tabular streams. The conditionallindependeen independence objective are two parameter-independeen MLPs with two hidden layers of 64 units and ReLU activations. All models optimized with Adam at learning rate 1 x 10-4 81 = 0.9, B2 = 0.999, and L2 regularization 1 x 10-5, Mini- batches contain 256 samples. Training runs for at most 200 epochs with early stopping monitored on
Image encoders
predictors q and ds used in the conditional
36

---

Published as a conference paper at ICLR 2026
the validation objective and patience of 20 epochs. CARL employs fixed loss weights wos 1.0, WMBR = 0.5, WMA = 0.1, representation dimension d = 64, the number InfoNCE negatives Kneg 128, and soft-rank temperature Trank 0.1. Model selection uses five-fold cross- volss-aaii::ion;; hyperparameters are chosen by inner validation on the training folds, and all baselines follow the same protocol, budgets, and representation dimension.
F.2 EVALUATION METRICS
All metric values lie [0, 1]. Let denote the batch size. The number of InfoNCE negatives is Kneg = 128 and the total number of candidates in the denominator is Ktot K Kneg + 1. The significance level is a 0.05. The kernel-ridge regularization parameter is À = 10-3, Numerical stability uses
10-3 The centering matrix H - 11 Cosine similarity is s(u, v) =
2 v
Causal Structure Index (CSI).
CSI Σ
(p((k) > a)
where the four null hypotheses are Zr Ly ZM, T Zy ZM, Zr LY ZM. and TLY M. Each p-value is computed by a kernel-based conditional-independenne test with Gaussian kernels. Bandwidths follow the median heuristic. Residual kernels obtained by kernel ridge regression with X and the statistic
1 (KC V W) N'
HHKKLWHKKi
calibrated by wild bootstrap with B 500 resamples.
Markoy Boundary Retention Index (MBRI).
exp v)
TN; V) = E log EveN( exp v²). + log Kiot-
T(U:V) min [TNCE (U; log Keot> 1}.
The final score
i(ZM: Zx) T(ZM:; Zx) + T(Zr: Zy) +
MBRI
Monotonic Alignment Consistency (MAC). From semantic amplitudes {as} and representations
{=:}. uniformly subsampled of pairs P of size IP| min(50,000, NNN-11) forms the vectors Aa = {la; -
-aylten) and A [||= ;; - (k)))e. Spearman's 's rank correlation is mapped to
[o, 1] by
MAC _/4( 11 ps (a, A=)).
Structural Accuracy. Let Askel and Askel denote the predicted and true skeleton adjacency matrices on the evaluation node set. The skeleton Structural Hamming Distance is SHD (Aske) |[Upper( pred Athhl)llo. Let my = [[ppper (Apeed V Askel with guard my 2 The score
(Askel
SHD
Structural =
mu
Representation Information Content (RIC-avg).
RIC-avg = (T(Zr;T)+ + i(ZMr i(Zr:
with 7 identical the definition used in MBRI and the same K Knegs Krot, and similarity function.
37

---

Published as a conference paper at ICLR 2026
Table 4: Parameter configurations for the synthetic datasets (full-factorial grid).
0.1
0.3
0.5 quadratic
76
lincar
quadratic
neural
linear
quadratic
neural
linear
neural
500
1000
v
r
2000
V
r
v
r
v
5000
v
F.3 PARAMETER CONHIGURATIONS
The synthetic datasets follow a full-factorial design over sample size n E {500, 1000, 2000, 5000), noise level {0.1, 0.3, 0.5), and nonlinearity type (linear, quadratic, neural]. The direct effeet fixed to 5 = 0, implying the ground-truth condition T Y* M. For each (n, a, nonlinearity) combination, instances generated under the IM, IY, and DUAL scenarios using identical random seed and train-validation-test split.
G SUPPORTING EVIDENCE FOR CAUSAL PATHWAY DISCOVERY
G.1 BLOOD PRESSURE CVD EVIDENCE
The Prospective Studies Collaboration meta-analysis of individual data for one million adults demonstrates age-specific relevance of usual blood pressure to vascular mortality, showing robust relationships across age groups Lewington et al. (2002). The Blood Pressure Lowering Treatment Trialists' Collaboration individual participant-leve data meta-analysis shows pharmacological blood pressure lowering reduces cardiovascular disease across different blood pressure levels, with each reduction associated with proportional risk reduction in major adverse cardiovascular events Blood
Pressure Lowering Treatment Trialists' Collaboration (2021).
G.2 ARTERIAL STIFFNESS + CVD EVIDENCE
Systematic review and meta-analysis demonstrates that arterial stiffness predicts cardiovascular events and all-cause mortality Vlachopoulos al. (2010). Individual participant meta-analysis of 17,635 subjects shows aortic pulse wave velocity improves cardiovascular event prediction Ben- Shlomo al. (2014).
G.3 RETINAL MICROVASCULAR CVD EVIDENCE
The Atherosclerosis Risk in Communities Study demonstrates retinal microvascular abnormalities predict incident stroke Wong et al. (200I). The same cohort study shows retinal arteriolar narrowing increases risk of coronary heart discase in men and women Wong al. (2002).
G.4 RENAL FUNCTION CVD EVIDENCE
Collaborative meta-analysis shows association of estimated glomerular filtration rate and albumin- uria with all-cause and cardiovascular mortality in general population cohorts Chronic Kidney Dis- ease Prognosis Consortium et al. (2010). Chronic kidney disease independently associates with risks of death, cardiovascular events, and hospitalization Go et al. (2004).
G.5 AGE RETINAL FUNDUS > CVD EVIDENCE
Retinal age gap predictive biomarker for mortality risk, with biological age differences between chronological and retinal predice-ee age associating with increased mortality Zhu et al. 2023)
38

---

Published as a conference paper at ICLR 2026
G.6 INFLAMMATION -> MICROVASCULAR CVD EVIDENCE
C-reactive protein associates with retinal microvascular caliber in multiethnic populations, demon- strating
microvascular connections Cheung et al. (2010). Systematic review and meta- analysis confirms associations between markers of inflammation and retinal microvascular parame-
ters Liu et al. 2021b)
G.7 SLEEP APNEA - HRV - CVD EVIDENCE
Heart rate variability analysis shows differences obstructive sleep apnea patients with and without excessive daytime sleepiness [Ucak et al. (2024). Meta-analysis demonstrates heart rate variability predicts first cardiovascular event in populations without known cardiovascular disease Hillebrand
et al. (0133).
G.8 BMI + METABOLISM CVD EVIDENCE
Pooled analysis of 97 prospective cohorts with 1.8 million participants identifies metabolic media- tors of body-mass index effects on coronary heart disease and stroke Global Burden of Metabolic Risk Factors for Chronic Diseases Collaboration (BMI Mediated Effects) al. (2014).
G.9 GUT MICROBIOME - METABOLISM - CVD EVIDENCE
Landmark studies demonstrate gut bacterial metabolism of choline and phosphatidylcholine pro- motes cardiovascular disease through production of proatherogenic metabolite TMAO Wang et al. 201T NEJM research confirms intestinal microbial metabolism of phosphatidylcholine associates with cardiovascular risk in humans Tang et al. (2013). Nature Medicine studies reveal intestinal microbiota metabolism of L-carnitine, abundant red meat, also produces TMAO and promotes pathway Koeth et al.
atherosclerosis, establishing the hataliommmmmmabollmmaatteeeeeeee
2013
H
REAL DATA
H.1 DESCRIPTION OF HUMAN PHENOTYPE PROJECT DATASET
Comprehensive biomedical datasets that integrate multiple modalities are essential for understand- ing the complex interplay between different physiological systems. The Human Phenotype Project (HPP) provides such a resource, representing a large-scale, longitudinal collection of deep pheno- typic profiles. Our study utilizes data from 6,366 adult participants (3,043 male and 3.323 female) within this cohort, with a mean age of 52.4+7.7 years and a mean BMI of 24.1 1411///²² The project collected wide array of clinical, physiological, behavioral, and multi-omic profiling data, categorized into 17 major body systems Kohn et al. (2025).
key strength of the HPP cohort its rich, multi-modal structure, encompassing tabular data, time- series signals, and medical imaging. For instance, the sleep data alone consists 448 characteristics collected over 16,812 nights of home sleep аpnеa testing, providing high-resolution time-series in- formation. This is complemented by other modalities such as tabular data from anthropometrics and body composition (120 features), blood tests for inflammation and immune system markers (12 features), and medical imaging from retinal fundus scans, which is part of the cardiovascular assess- ment (162 features). In this work, we focus on data spanning six key systems: Anthropometrics, Sleep, Cardiovascular, Fundus, Metabolic Pathway, and Inflammation. The integration of these dis- tinct yet causally related modalities, each with a high-dimensional feature set, provides a unique opportunity to apply cross-modality causal inference methods, aiming uncover the underlying mechanisms connecting these physiological domains.
H.2 BODY SYSTEM-DERIVE FEATURES
The sleep monitoring data is a high-resolution. time-series dataset designed to capture the complex dynamics of sleep. Each participant underwent at least series of home sleep
Sleep Monitoring
39

---

Published as a conference paper at ICLR 2026
Table 5: Complete experimental results across varying sample sizes, noise levels, and nonlinearity functions. Performance metrics include Causal Structure Index (CSI), Markov Boundary Retention Index (MBRI,, Monotonic Alignment Consistency (MAC), Structural Accuracy, and averaged Rep- resentation Information Content (RIC-avg).
Sample Size
Noise
Nonlinearity
CSI
MBRI
MAC
Structural
RIC-avg
500
0.1
linear
1.0000
0.5284
0,5312
0.4506
0.4483
500
0.1
quadratic
1.0000
0.5152
0.5484
0.4994
0.4539
500
0.1
neural
1.0000
0.4877
0.5292
0.6013
0.4388
500
0.3
linear
1.0000
0.5350
0.5419
0.4741
0,4534
500
0.3
quadratic 1.0000
0.5380
0.5696
0.5514
0,4526
500
0.3
neural
1.0000
0.5089
0.5358
0.6001
0.4313
500
0.5
linear 1.0000
0.5159
0.5243
0,4526
0.4293
500
0.5
quadratic
1.0000
0.5267
0.5490
0.5242
0.4273
500
0.5
neural
1.0000
0.4840
0,5327
0.5821
0.4323
1000
0.1
linear
1. 1.0000
0.5088
0.5847
0.6693
0.4408
1000
0.1
quadratic
1. 1.0000
0.5405
0.5894
0.6584
0.4388
1000
0.1
neural
1.0000
0.4692
0.5665
0.7075
0.4518
1000
0.3
linear
1.0000
0.5149
0,5953
0.6762
0.4419
1000
0.3
quadratic
1. 1.0000
0.5463
0.6068
0.6884
0.4404
1000
0.3
neural
1.0000
0.4980
0.5702
0.7102
0.4545
1000
0.5
linear
1.0000
0.5097
0.5804
0.6565
0.4284
1000
0.5
quadratic
1.0000
0.5337
0.5938
0.6651
0.4196
1000
0.5
neural
1. 1.0000
0.4819
0.5658
0.6954
0.4364
2000
0.1
linear
1. 1.0000
0.5660
0.6301
0.6893
0.4176
2000
0.1
quadratic
1.0000
0.6027
0.6484
0.6784
0.4212
2000
0.1
neural
1.0000
0.5102
0.6285
0.7275
0.4298
2000
0.3
linear
1.0000
0.5893
0.6352
0.6962
0.4248
2000
0.3
quadratic
1.0000
0.6500
0.6536
0.7084
0.4346
2000
0.3
neural
1.0000
0.5354
0.6320
0.7300
0.4308
2000
0.5
linear
1.0000
0.5808
0.6174
0.6765
0.4133
2000
0.5
quadratic
1.0000
0.6421
0.6362
0.6851
0.4257
2000
0.5
neural
1. 1.0000
0.5013
0.6128
0.7154
0.4256
5000
0.1
linear
1.0000
0.6141
0.6432
0.7093
0.3948
5000
0.1
quadratic
1.0000
0.6344
0.6507
0.6984
0.3954
5000
0.1
neural
1.0000
0.6193
0.6384
0.7300
0.4231
5000
0.3
linear
1.0000
0.6437
0.6515
0.7162
0.4024
5000
0.3
quadratic
1.0000
0.6600
0.6609
0.7284
0.4070
5000
0.3
neural
1.0000
0.6209
0.6457
0.7300
0.4240
5000
0.5
linear 1.0000
0.6238
0.6397
0.6965
0.3949
5000
0.5
quadratic
1.0000
0.6500
0.6441
0.7051
0.4112
5000
0.5
neural
1.0000
0.6062
0.6252
0.7300
0.4221
monitoring tests, with each series consisting three nights of continuous recording within two- week timeframe. This multi-night protocol allows for a more reliable assessment by accounting for night-to-night variability. The collected data encompasses wide range of metrics, including sleep architecture architecture (sleep stages), body position, respiratory events, pulse rate, peripheral blood oxygen saturation (SpO2), and snoring. Figure pgrovidds an example raw time-series records on selected sleep channels, illustrating the continuous nature of the data collected, such as respiratory move- ment, SpOz. and body position. A primary focus this dataset is the study of Obstructive Sleep Apnea (OSA), prevalent sleep disorder characterized by repeated interruptions in breathing due to upper airway obstruction. These interruptions often lead to significant physiological responses, such loud snoring, reductions in blood oxygen levels, awakenings, and fragmented sleep.
40

---

Published as a conference paper at ICLR 2026
5002
-5000
23.02
00:32
02:59
22:02
13:00
94:50
02:00
06:00
Sleep Stage
10M
2201
28/24
20108
00
23.03
00:22
01:00
32.00
13:00
54:00
05:00
06:00
Snore (di)
2/208
2/003
20115
20005
250B8
20:08
25008
8p02
100
22:00
23303
20:32
01:50
27:00
E3000
94:00
05/00
(6:00
Body Posttion
-2:
72304
7284
DAXTA
PAyRA
Paynt
PAyCR
Figure 5: Example of Raw Time-Series Records Selected Sleep Channels. The figure displays 8 hours continuous data from a single participant, showing five different channels: Respiratory Movement (top), Sleep Stage, Snore (dB), Peripheral Oxygen Saturation (SpO2), and Body Position (bottom). This illustrates the high-resolution, multi-channel nature of the raw sleep monitoring data.
The raw signals collected by devices are processed by clinically-validated algorithms to extract a rich set of 448 features for each participant. These features include key diagnostic indices for OSA, such as the peripheral Apmea-Hyen Hyropnea Index (pAHI), the Respiratory Disturbance Index (RDI), and the Oxygen Desaturation Index (ODI). Figure (alshoss the distribution of pAHI for male and female participants at the baseline visit. The histograms and empirical cumulative distribution functions (ECDF) clearly indicate that the male population exhibits a tendency towards higher PAHI values compared to the female population. In addition respiratory events, the algorithms provide detailed statistics on sleep architecture, including the percentage time spent in different sleep stages and overall sleep efficiency. F further illustrates the derived events during sleep, such as sleep stages, various respiratory events, and pulse rate changes, highlighting the granular insights extracted from the raw channels. Furthermore, to capture autonomic nervous system activity, 348 Pulse Rate Variability (PRV) features are computed from the PAT channel, spanning time-domain, frequency-done domain,, and non-linear metries. Table 6 provides statistical summary of the key sleep- derived measurements for the cohort at their first visit (N=6,336).
Fundus imaging The fundus data modality provides non-invasive window into both ocular and systemic health through high-resolution imaging the interior surface of the eye. In the HPP cohort, center-view 45 retinal images were collected for both eyes of each participant without pupil dilation, using the iCare DRSplus confocal fundus imaging system. This technique allows for the detailed visualization of critical structures within the fundus, including the retina, optic disc, macula, and the retinal microvasculature. The resulting images are essential for diagnosing and monitoring variety of eye conditions, as the appearance of these structures, particularly the network of small blood vessels, offers valuable information about the health of the eye and the central nervous system.
To transform the raw images into quantitative dataset suitable for analysis, the HPP employs an open-source deep learning pipeline known as AutoMorph Zhou et al. (002). This automated pipeline processes the fundus images through several stages, including pre-processing, image qual- ity grading. and anatomical segmentation of the optic disc and retinal vessels into arteries and veins. Following segmentation, a comprehensive set of morphological features is extracted, such as vessel caliber, tortuosity, density, and fractal dimension. Figure 7b provides example of one such tracted feature, showing the distribution left eye vessel density for male and female participants the baseline visit. By quantifying the intricate geometry of the retinal microvasculature, this data
41

---

Published as a conference paper at ICLR 2026
actigraph
general
heart rate
heart rate rew
pat Infra
respiratory movament
spo2
pat ampliade
SSeep
Dees sleep
Light Sleep
REM
REM Period
Vake
A/t obstructive fiedd
Time
Hesp: Event
HR changed
Excluded
Premnatur Beal
Pot Amplitude
Artifact
Trunc
Desaliraation EMi-tecllde
1111
28/08
28/08
28/08
28/08
28:08
22:00
23:00
03:00
05:00
Figure 6: Derived Events During Sleep from Raw Channels. This figure presents a timeline of clinically relevant events and metrics derived from the raw sleep monitoring data. It includes sleep stages (Ap. V.Sleep, Deep Sleep, Light Sleep. REM Period, Wake), various respiratory events (e.g., A/H obstructive, Resp. Event), and other physiological events (e.g., HR changed, Desaturation), providing summary of key sleep patterns and disturbances.
dwrk, .67
012
050
Viiial denaly (100))
(a) Apnea- Index (AHI).
(b) Left Eye Vessel Density.
Figure 7: (a) Distribution of AHI for male (N=2,777) and female (N=2.785) participants Baseline Visit. The left panel is histogram, and the right panel is an ECDF. The data indicate that the AHI distribution for males is generally shifted towards higher values compared to females. (b) Distribution of Left Eye Vessel Density for male (N=2,369) and female (N=2,458) participants at Baseline Visit. The left panel is a histogram, and the right panel an ECDF
modality provides a rich set of features for investigating the links between microvascular health and broader systemic disease processes. Figure 8 displays the colored fundus images and corresponding segmentation results in HPP cohort.
Anthropometrics The systematic measurement of the size, weight, and proportions of the human body, serves as a fundamental modality in the HPP cohort for assessing health and nutritional status. This dataset is primarily composed of tabular data collected through standardized protocols. To en- sure precision and consistency, these measurements were taken using specific equipment, including a Shekel Stadiometer for height and weight. This set of measurements provides essential baseline characteristics for evaluating body composition and identifying potential health risks like obesity.
From these basic anthropometric measurements, important indices such as the Body Mass Index (BMI) can be derived. BMI is a key covariate used in a wide range of phenome-wide association studies. Figure 9a illustrates the distribution of BMI across the cohort at the baseline visit, stratified by sex. The histograms and ECDF clearly show that male participants (N=4.943) in this cohort tend to have higher BMI compared to female participants (N=5,439). This quantitative, tabular data provides a foundational layer for our cross- cross-modality causal analysis, allowing us to control and investigate the effects of body composition on other physiological systems.
Cardiovascular System The cardiovascular system assessment within the HPP is comprehen- sive and modal evaluation, designed to provide a holistic view of cardiac and vascular health. This modality integrates data from several distinct, non-invasive measurement techniques, including
42

---

Published as a conference paper at ICLR 2026
Table 6: Summary of Key Sleep-Derived Measurements.
First visit (n=6,333)
Second visit (n:574)
Parameters
Min / Max
Moan (s.d.)
P10/ PSD
Min / Max
Meand (s.d.)
P10 PS0
Snore events above 40 dB (h)
0.01 10.77
1.63 (1.54)
0.22/ 3.93 3.93
0.02/ / 8.33
1.80 (1.59)
0.27 4.16
Total sleep time [h]
2,00/ 11.50
6.07 (1.21)
7.51
2.00 11.58
6.08 (1.21)
4.51/7.52
Sleep efficiency (94)
43.0/97.9
88.4
81.1 94.3
49.1/98.0
88.5 (5.4)
81.3/ 94.2
Number wakes (events)
1/39
(3.9)
2/12
1/
6.5
2/12
Percenlage of deep sleep time (%)
38.8
17.6
10.8/ 24.3
37.7
17.3
10.5/ 24.2
Percentage of REM sleep time (%)
1.8/45. 45.2
23.9 (7.3)
14.2/
2.4 44.8
23.9 (7.4)
13.8/ 33
Aprea--typopnea
per
0/90.8
(10.5)
1.5 24.4
0/915
(10.4)
1.5 23.7
Oxygen Dessturation Index (avants
8686
4.5 (6.4)
0.4/ 11.3
0/96.3
4.5 4.6
0.3/ 11.2
Mean heart rate during sleep (SPM)
35 /98
59.7 (8.0)
50
35 /98
60.2 (8.1)
50 / 70
Maximum heart rate during sleep (BPM)
55 / 180
95.3 95.3 (11.7)
82 / 109
55 /80
94.8 (11.6)
81 / 108
Minimum heart during sleep (BPM)
30 10/79
46.3 (6.3
38 55
30/
46.9 (7.1)
38 / 56
during NREM sleep (BPM)
/7.8
(8.0)
49.8 70.2
31.9/92.5
50.1/ 70.5
Mean heart rate during REM sleep (BPM)
33.1 107.9
61.4 (8.2)
51.2/72.11
33.8/95.4
61.9 (8.2)
51.5 72.6
Mean heart rate during wakefulness (8 (MP
31.9 97.6
63.7 (8.3)
$2.4 74.3
34.3/100.0
64.0 (8.3)
52.6 74.8
Mean nadir of desaturetions 196)
67 98
92.1
90 99
69 /97
91.8
89.6/ 93.7
sleep derived rrtaurenens
are
deviatinn
percenti
decorion 1od jo hours. Siilary, y 501 the row indicates that Jel, Appoppss indiiu APM heors merueee
deep
hour the 90th percentile (100) was 7.51
Fundus image
Vessel segmentation
Vein segmentation
Artery/Vein classification
Left
Right
Figure 8: Example of fundus images in HPP. From left to right: (1) The original color fundus image; (2) Vessel segmentation; (3) Vein segmentation; and (4) Artery/Vein classification, distinguishing vessels into arteries (red) and veins (blue).
assessments of peripheral vascular health, multi colti-positional blood pressure readings, carotid ultra- sound imaging, and electrocardiography (ECG). The combination of these methods yields rich dataset that includes tabular data (e.g.. blood pressure, PWV/ABI values, derived ECG features), medical imaging (e.g., carotid ultrasound), and raw time-series signals (e.g., ECG waveforms), mak- ing it ideal modality for cross-modal analysis.
Vascular health is quantified through multiple techniques. Arterial stiffness, a key predictor of fu- ture cardiovascular events, is measured via Pulse Wave Velocity (PWV), specifically the leg PWV (faPWV). Peripheral Arterial Disease (PAD) is assessed using the Bikkhe- Index (ABI), which is the ratio of ankle to arm systolic blood pressure. Figure 96 shows the distribution of the right Bracchal Index (r abi) across different sexes, where the values are concentrated around 1.1, consistent with the normal range for healthy population. Both PWV and ABI are measured
43

---

Published as a conference paper at ICLR 2026
C4-
644
13 22
10. 03.
70
1.0
1.0 150
40
1.0 137
(a) Body Mass Index (BMI).
(b) Right Ankle Maaee-rrachial Index (ABI)
Figure 9: (a) Distribution of BMI for male (N=4,943) and female (N=5,439) at Baseline Visit. The left panel is histogram, and the right panel is an ECDF. The data indicate that the BMI distribution for males is generally shifted towards higher values compared to females. (b) Distribution of Right ABI for male (N=4,290) and female (N=4,560) participants at Baseline Visit. The left panel is histogram, and the right panel is an ECDF.
using a Falcon (Viasonix) device. Table 7 details the descriptive statistics for these key peripheral vascular health measurements. Furthermore, early signs of atherosclerosis suring the Carotid Intima-Medie Thickness (CIMT) from ultrasound images of the carotid arteries, performed with a Supersonic Aixplorer MACH 30 system. Figure T0a provides distribution exam- ple of the left Carotid Intima-Media Thickness (imt_left), showing the specific numerical range of this feature within the cohort, with males showing slight tendency towards higher values. Blood pressure is also systematically recorded with multiple readings taken while the participant is seated, lying down, and standing capture comprehensive hemodynamic profile. Table summarizee the detailed statistical data of these key blood pressure measurements across different positions.
evaluated by mea-
Table 7: Summary of key peripheral vascular health measurements
First visit (n-7,632)
Second visit (n-1,878)
Parameters
Min / Max
Mean (s.d.)
P10 P90
Min Max
Meand (s.d.)
P10 / P90
Right brachial pressure (mmHHg)
25 240
114.8 (19.4)
93/
61 220
114.5 (19.5)
91 / 139
Left brachial pressure (mmHH)
24 /
115.6 (19.4)
94/ 139
217
(18.7)
94 139
Right (ratin)
0.3
1.1 (0.1)
1.3/13
0.45 / 2.09
1.14 (0.14)
/ 1.30
Left (ratio)
0.46/ 2.19
(0.14)
1.29
0.49 / 2.23
1:14 (0.14)
1.30
PWV from right thigh right anile (m/s)
38.3
8.1
5.9/
3.5/
7.4 (1.6)
9.5
PWV from left thigh left ankle (m/s)
3.3 17.2
8.1 (1.8)
5.9/ 10.5
3.5/
7.5 (1.6)
5.5/
Summary koy perphera wascaler health messurement the fist (:83) and the socand viel (px2578) Data presented minimum and mas valuss While (Maci, the mean
ebwncadd dedeer on (Moon (s di and the 10th and sith perteenile (P10) (Ma) for eorrpee, for the High bowttill areounet during the fins wiet, the wiiue 11.4 119-1 repaosents mean preesue 114. mirhhg with ssandard deviatin jod bf merit, S millarl, 52/ 320' in the samerce indicatis that 100h percantlic (P20f was 52 mimigg and the 900n possontli (P50) was mority for the tome group. All, Acice-trnctis indec PWa, Palse Wrese Veincity.
The electrical activity of the heart is captured using resting 12-lead electrocardiogram (ECG), as shown in Figure [T] significant feature of the HPP's ECG data is its dual format. For each participant, the dataset includes not only set of automatically extracted tabular features (e.g.. P- wave duration, QT interval) but also the complete raw 10-second waveform signal from all 12 leads, recorded at high sampling rate of 1000Hz. For example, Figure [0b] shows the distributional difference of one of the key derived features, P wave duration (p ma), between male and female par- ticipants, highlighting sex-specific variations in cardiac electrical activity. This dual output provides an exceptionally detailed characterization of cardiac function, suitable for both traditional feature- based analysis and advanced signal processing techniques.
Inflammation The inflammation modality in the HPP is characterized through panel of hemato- logical and biochemical measurements. The data for this modality primarily originates from blood tests conducted during participants' regular medical care at their respective Health Maintenance Or- ganizations (HMOs), which are then uploaded by the participants themselves. This dataset is entirely tabular, providing a quantitative snapshot of systemic immune and inflammatory status.
The core of this modality consists of the complete blood count (CBC), which includes the total white blood cell count and a detailed differential of its major subtypes: the absolute counts and relative
44

---

Published as a conference paper at ICLR 2026
Table 8: Summary of key blood pressure measurements
First visit (m=8,090)
Second visit (e=2,020)
Parameters
Min MMax
Mean (s.c.)
P10/P90
Min/ / Max
Meand (s.d.)
P10/
lying diactoiic blood pressure (mmHig)
/4
76.5
67/87
61
76.7 (7.4)
67/87
Lying blood pressure pulse rate (8MP)
35 110
62.4 (9.7)
51/75
35 105
61.9 (9.5)
50/74
lying systolic blood pressure (mmiiig)
74/ 74/199
119.2 (14.9)
101 /138
72/189
(15.0)
101 /138
Sitting diastolic blood pressure (mmHtg)
47/ 140
78.4 (3.413
66 66/91
41/119
77.6 (9.5)
66/90
Sitting blood pressure pulse rate (8MP)
35 15/11 /13
66.6 (10.6)
54
36/116
66.3 (10.4)
53/80
Sitting systolic blood pressure (mmHgg)
73/ 215
120.1 (25.6)
/142
62
119.8 (15.8)
100/142
Sitting second diastolic blood pressure (mmHHg)
47/ 117
77.7 (10.0)
65 90
47 /004
77.2 (9.4)
66 /88
Sitting second blood pressure pulse rate (BMF)
40/ 108
66.3 (30.6)
54/
37/110
65.2 (10.2)
53/78
Sitting second systalic bload St Standing one minute diestalic blood pressure (mmmHtg
(mmHgl
77/
116.6 (15.8)
76/173
116.5
98.7 136
38 138
82.1 (9.6)
70 99
55/127
82.2 82.2 (9.3)
71 994
Standing one minute blood pressure pulse rate (BMP)
36/ 122
73.6
59/60 88
35/119
73.4 (12.0)
58
Standing minute systolic blood pressure [mmHgl
66/ 214
116.9 (36.0)
98/ 137
79/1
116.8 (16.1)
97 138
Standing three minute diastolic blood pressure
66 / 101
82.4 (7.9)
72 93
66 /100
82.5 (7.8)
72 /93
Standing minute
pressure pulse rate (BMP)
37/
72.9
59
/27
(11.2)
Standing three minute systolic blood pressure (mmHH))
69 214
117.7 (35.5)
99/ 137
76/ 217
118.1 (15.8)
89 139
Summony craveiin of bey tanne precture mesuur monrs fine secoet vith jaur pe09 now presented
mosimam Balllieseeiie Noaa The manns Coonatee decirtoon (Seenn 2010
norrple, Indicone 10th pertennile
reporsevee
minig Sinilariy 32/47
the (PG)) for
Teees Cal
111E Jeft
Jmi_jort
pyms
(a) Left Carotid Intima-Meddi Thickness (IMT).
(b) ECG P-wave Duration
Figure 10: (a) Distribution of Left Carotid IMT for male (N=3,089) and female (N=3,202) partici- pants at Baseline Visit. The left panel is a histogram, and the right panel is an ECDF (b)Distribution of ECG P-wave Duration (p_ms) for male (N=3,606) and female (N=3,949) participants Baseline Visit. The left panel is a histogram, and the right panel is an ECDF, with the data showing that males generally have longer P-wave duration than females.
percentages of neutrophils, lymphocytes, monocytes, eosinophils, and basophils. Furthermore, this modality is enriched with other key inflammatory markers, such as C-reactive protein (CRP) and ferritin, which are widely used clinical indicators for assessing systemic inflammation levels. This comprehensive panel of tests provides a solid foundation for studying bascline inflammatory states and their associations with other physiological systems.
Metabolic Pathways The metabolic pathway data offers functional perspective on the gut mi- crobiome by characterizing its metabolic potential. This modality derived from metagenomic sequencing of stool samples collected from the participants. Specifically, the data consists of mi- crobial pathway abundances which are functionally profiled from the metagenomic data using the HUMAnN3 BBehhini et al. (202T) software package. This approach allows for the quantification of the relative abundance of various metabolic pathways present in an individual's gut microbial community. The resulting dataset enables the investigation of associations between the collective metabolic functions of the gut microbiota and other physiological systems, providing insights into how microbial activities, such as amino acid biosynthesis or energy metabolism, may influence overall health and disease states.
45

---

Published as a conference paper at ICLR 2026
5309837561
FODG. ORS
Sgrater
M
aVR
V1
V4
aVL
V2
V5
11
aVF
V3
V6
V5
Figure 11: Processed ECG time series
Table 9: Summary of physiological systems, features and medical explanations
System
Features
Medical Explonation
body pescentags
Quamifies
celative bods weight, elevated lovss indieste obesity, metabolic dywegu- intressed cardovasuula ris.
Body Fut Composition
fat miioo
Bepresent sudnidl-t-gganii lieritutin; a higher ratio sigrab contral lbeuity and greater risk for insilin socstans and vaculer discasc.
Visceni fut volume
Meastres fai accumuazion aroud inserall orgass; stmuggy associated with inflammation, impuirdd
glucose metbblisn, and higher cardomeubolic risk. Befast muscl
Appenuiisllr
the lisbs; soduced lovels indiaae sucoperiia fniily, and diminishad fanc-
capacity.
CKA seuriity
Indicates the degroe of ohanactive slepp spucar higher valses comrllate with elovated cordommbbllic harkkn and deytime fanetional inpaimment
OSA-Related Phenotypes
Saowing seserity Sworing ralio
Captanss stariig iniersity fisquccc: a proay marker for airway obstnuction and OSA risk.
Prepertion of seep time svering higher raties reflect diespted shopp and elessted OSA probobiliyy
Sleep duration
Objective total sleep tine; both shott and fragmented sleep lisked mesiboli dysfusction, cog.
decline, cudionascular marbiiiy.
Sleep Structure Quality
Sleep efhiiency
Rano actoal sleep to time bed: reduced ethenney indicates impaine steep maintenance and higher nak foigger and pour menvery
Sleep comminiity
Messtres the stabiity of sleep across the night. lower contiturty reflects fraginenned sleep, oftes associsted with intummation and stress.
Restoeative sleep eatio
Fraction and REM seunospatiive
relatve total sleep; lower satios suggest soduced secovery
Night mon HR
Avenge nammmal hean nac: chooral volues sggast ruluual satoomnc bolance and higher strees ur matsh-lic kad.
Slesp HRV
HRV index
Global of heutt vaiabiliny lower
litked autooonic disfuuation, yyscaii
influmatiion and neuubllic
HRV stage variation
Aecssox varibillyy HRV acses sleap siages aedusee varistion indisatos inquired astoramic adap- lation and poseer soep andritsture
HRV wake delta
Difference in HRV betwern wake and sleep sates, blutted differences signal impuired recovery and autonomicc disregilatio..
Nuctorral Hypusin Burden
Osygen dooturations
Degnse bload usygen deop daring sleep. frapunn or sevene drops indicate leponiseccltedd uedin- vsaculae und nestonogsiiive stres.
Orygen bunker
Camobeine mezsre of msygin deficit armoes the night mollets the physiologiral livd of intermient bypusia and lins to syscnic riek.
Lying dissolic pressure Lying pulse rate Lying systolic pressure
Diasolic blood peesoure Ising: helps assess beseline vascolar so0e and resting blood pressure.
Blood Prosure- - Lsing
Palse sate messured while lving serves a maker beaa sutoomiic function.
Systolic Nood pressure
while lyleg; provides baseline anecil load evalaation.
Sating dissiolic precane Siming pole Sanng syaelic prossre
Dissilic grosuuc in the siting proiton: uscful for dasesing posisoral biosd prosure changes.
Blood Pressure- Siiting
Palse mie while sming: reflects antonomic bolance in resting seneed
Systolic prosure while siting infiones sysiemic procure load necatrl postsrc
Standing diastolic presiure Sunding pule rate
Dissolic pressure aherr stading
detects early onhostate changes.
Palse zate stadins for minue sscfil fo usssiin ausononic respotacs posture clange.
Blood Pressur- - Standiag
Suading dissteic pecsars
Stading liia sysollic pressure
Syssolic presure after standing for mintee evalustes rujld hemodguanic adjusinetr.
Diseulic pressuc
ninuess of sunding, deaca delasal untaoeatic hypstmmius.
Stowlng pole cate Staddnn Sain systuii prosure
Palse mit aher miiutees carding prolinged clextion may vuggrai sutmmonic dysiunciiu.
Systolic ppogue afler 3 muintto standng inaicatos statiina kemolyuunic adqpatton
Kesting systoli pressure
Boreline systii blood pressnee; high valses indiore elevned candevaaculor risk.
Blood Presuur- - Reating
Resting dianelic pressure Resting puise
Busclin dianolic pressure reflees resting vascular reisttnce.
Keeting heurt ose. higher rates may indicate poor cordiovaculur titness incoessed stres.
Octhntutic SBP doop Luin
Statalic presure drop within minut standing, slows uutomomnic volune conuistion inpaineett

---

Published as a conference paper at ICLR 2026
I SYNTHETIC DATASET GENERATION
To facilitate principled and reproducible evaluation of cross-modal causal inference methods, we developed a comprehensive data generation pipeline to create a synthetic dataset with fully speci- fied, ground-truth causal structure. Our procedure is grounded in a Structural Causal Model (SCM) that defines the latent causal relationships between variables. The numerical outputs of this model are then mapped to a set of semantic and style parameters, which in turn deterministically guide sequence of transformations applied to base images from the MNIST Lecun et al.) (1998) dataset. This multi-step process allows for precise control over the underlying causal graph, the manifesta- tion of variables across tabular and visual modalities, and the introduction of realistic complexities such nonlinearities and nuisance variables. The technical details are outlined in Algorithm
1.1 CAUSAL GROUNDING VIA STRUCTURAL CAUSAL MODEL.
To enable rigorous evaluation of cross-modal causal inference methods, we construct a synthetic dataset grounded in fully specified SCM. This approach ensures that all causal relationships are known by design, providing clear ground truth for method validation. Our SCM implements three-variable mediation chain with the structure T + M + Y*, where represents the treatment variable, M the mediator, and Y' the continuous outcome. The data generating process is defined through the following system of structural equations:
T ET M = a1T + ph1 (T) ECM
(24)
Y a.M + ph2 + +
where Cr, Cr.CM,C ~ o2) are independent Gaussian noise terms with configurable variances o2 oz. The treatment 7'is exogenous, while both the mediator M and outcome Y incorporate linear and nonlinear components to model realistic causal complexity.
The linear effects governed by coefficients on and 022, representing the primary causal pathways T + M and M + Y respectively. Nonlinear relationships are introduced through functions h1 (-) and he(-)
{square, sin, tanh). scaled by parameter p [0. 1]. Crucially, the direct effect from treatment to outcome is controlled by parameter 8. When 5 O, all causal influence from T to Y flows through the mediator M, ensuring that the conditional independence relationship T IL Y M holds by construction. This parameter allows systematic evaluation of both full
mediation scenarios (6 = 0) and partial mediation with confounding (8 > 0).
1.2 NUMERICAL INSTANTIATION AND AUXILIARY VARIABLES
Following the SCM specification, we numerically instantiate the model by sampling N 5000 independent observations. This process yields concrete realization of the core causal variables (T, Y") for each sample. To enhance the dataset's complexity and support the subsequent cross- modal synthesis, these core variables are augmented with two distinct types of auxiliary variables. First, we introduce set q 30 high-dimensional nuisance variables, denoted by the vector W E Nx for each sample i. These are generated by drawing each component independently from a standard normal distribution, such that W, ~ (O,0, L,). By design, these variables are causally independent of the primary T M + Y chain, serving as realistic distractors that robust inference method must learn to ignore. Second, we sample a scalar style variable Sayle ~ (0 1), which is causally orthogonal to the main SCM structure. This is specifically designated to control stylistic variations in the image generation phase, thereby ensuring a clean separation between the semantic content driven by the causal variables, and the visual style driven by Sayle-
1.3 SEMANTIC PARAMETER MAPPING
To bridge the numerical latent space of the SCM with the perceptual domain of the images, we conducted the semantic parameter mapping. This stage transforms the raw, unbounded variables generated by the SCM into normalized and interpretable parameters that directly govern the visual attributes during image synthesis. This transformation is achieved using the standard normal Cumu- lative Distribution Function (CDF), denoted as (-)., which maps input the interval (0, 1].
47

---

Published as a conference paper at ICLR 2026
For the core causal variables, the mediator vector M and outcome vector Y converted into mantic amplitude vectors, aM and ay, respectively. This is accomplished by first standardizing the variables using their empirical mean (MM--My My My) and standard deviation (a ar oy) calculated across all N samples, and then applying the CDF. The specific mappings are defined as:
M-
PM
ky
+
+
(25)
MM
dy*
An analogous procedure is used to derive the style parameter vector bayle from the independent style variable vector Suyl:
bayle +b Suyle
Ms
(26)
0g
This normalization ensures the resulting parameters provide stable and consistent control over the subsequent image transformations. Critically, this design enforces a clean separation of generative factors: the semantic parameters aM and ay are exclusively derived from their corresponding causal variables, while the style parameter bayle originates from the causally orthogonal variable Sayle-
1.4 CROSS-MODAL IMAGE GENERATION
With the semantic and style parameters established, we synthesize the cross-modal data by applying a sequence of deterministic transformations to a set of base images. The base images, denoted as collection of matrices { [0, 1 1| Ji=1 are sourced from the MNIST dataset. To prevent the introduction of spurious correlations, these images are sampled using class-uniford distribution, ensuring balanced representation across all digit classes, Celass E {0.... 9) This sampling performed independently of the SCM variables, thereby guaranteeing statistical independence between the semantic amplitudes and the class labels by construction (as 1 Celass and ay Celass).
For each of the samples, fixed sequence of spatial transformations is applied. The transforma- tions are parameterized by the corresponding semantic amplitude a; (aM,i.ay; and the style parameter bayle,;- The steps are as follows:
Rotation An angular transformation is applied to orient the base image. The rotation angle is linearly determined by the semantic amplitude a; according to the formula:
f = Odeg (2a, - 1),
(27)
This function maps the full range of semantic amplitudes [0, 1] to a continuous spectrum of rotation angles spanning from [-Oagg +
Brightness Adjustment The overall brightness of the image is modified through an additive shift A. This shift is calculated as:
A = 3 (a; 0.5),
(28)
where the hyperparameter 8 controls the strength of the effect. This ensures that the image becomes progressively brighter as the semantic amplitude a increases from 0 to 1.
The contrast of image is adjusted by scaling pixel values around the mean brightness μG. The transformation is controlled by multiplicative gain factor:
Contrast Enhancement
g = 1 + 72a; -1), 1),
(29)
where 7 is hyperparameter for the effect strength. A value of a, > 0.5 results in increased contrast,
while a, < 0.5 reduces it.
Style-Dependent Noise we add Gaussian noise to the image. The standard deviation of noise a is controlled by the style parameter bayle, via the relation Opix bayle,s- where pii is the configurable base noise level.
To introduce stylistic variation that is independent of the causal semantics,
Each step in this transformation pipeline is followed by a clipping operation to maintain all pixel values within the valid range of [0. 1]. This complete sequence ensures that the final visual features of the generated images systematically and monotonically vary with their corresponding semantic style parameters.
48

---

Published as a conference paper at ICLR 2026
1.5 DUAL-SCENARIO FRAMEWORK
Our data generation framework is designed to produce two distinct yet causally equivalent cross- modal scenarios from a single underlying SCM. This unique design allows for comprehensive and controlled evaluation of causal inference methods by systematically altering which variable in the causal chain - M + Y* is manifested in the visual modality. This approach tests model's ability to reason flexibly across different modal configurations while the ground-truth causal graph remains fixed. The two scenarios are detailed below:
7M Scenario (Image as Mediator). resulting in are synthesized using the semantic amplitudes aM, which are themselves a direct function of the la- tent tabular mediator variable M. This configuration is designed to test model's ability to correctly identify a mediated causal pathway (T + M + Y-) where the intermediate step is only observ- able through high-dimensional visual data. To succeed, model must effectively learn extract the causally salient information encoded in the visual features of I and use it to reason about the relationship between the tabular treatment T and outcome Y This setup mimics real-world scien- tific and industrial problems, such as medical diagnostics where a treatment's 's effect on a patient's outcome mediated by changes observable only through medical imaging.
Irrlaarr-mmme---- this scenario, the mediator represented by image data, ar causal chain. The mediator images, denoted as 7
7Y Scenario (Image as Outcome), This scenario creates a Tabu aapularr-ae -aabuarrrraaaarr-aaaaaaaaa causal chain. The outcome images 1 are generated using the semantic amplitudes ay, which are derived from the latent tabular outcome variable Y*. This configuration evaluates model's capacity to discover causal relationships where www-iiensioonl tabular mediator (M) directly influences a high-dimensional, complex visual outcome (L'). This scenario presents a different but equally important challenge, requiring the model to understand how changes in an abstract tabular variable manifest as structured visual changes. It reflects problems such as predicting the visual outcomes of manufacturing process based on machine settings or determining the effect of drug treatments on cellular morphology.
1.6 GROUND TRUTH SPECIFICATION AND ROBUSTNESS TESTING
To facilitate a systematic and rigorous evaluation of inference methods, we explicitly define a set of testable ground truth hypotheses, denoted as H. for each scenario. These hypotheses serve as the benchmark against which model performance is measured. For the 7M scenario, the corresponding set of hypotheses, H.M. requires model correctly identify: (i) the causal dependence between treatment and mediator (T' 16 M); (ii) the dependence between the mediator and the final outcome (M 14 Y*); and (iii) the crucial conditional independence of the treatment and outcome given the mediator (T' IL Y* M) when the direct causal path is absent (ō =).
Beyond the core causal chain, we specify two critical sanity checks. The first semantic-class independence (am 1 Celoss, which verifies that no spurious correlations exist between the seman- tic amplitude and the underlying MNIST digit class, condition guaranteed by our independent sampling procedure. The second is a test of Monotonic Ampiitude-Corrspoodene (MAC):
MAC(aM, ((I >T.
(30)
This metric quantifies the Spearman correlation between the vector of semantic amplitudes and the feature distances of their corresponding images, where Φ(-) is a predefined feature extraction function. A high MAC score confirms that the semantic information is faithfully encoded in the visual modality, fundamental prerequisite for cross-modal reasoning. An analogous set of hy- potheses, HY defined for the I' scenario with the appropriate variable substitutions (e.g., testing M (1)))) )).
To assess model robustness against corrupted data, we introduce an optional permutation step. By randomly shufiling fraction of the generated images, controlled by permutation ratios PM and py, we deliberately break the remantic-visual correspondence for a subset of the data. This serves a critical negative control, enabling the evaluation of both model's sensitivity (ability to discover true causal links in unpermuted data) and its specificity (ability to avoid discovering spurious relationships in permuted data).
49

---

Published as a conference paper at ICLR 2026
The final generated dataset D (Dib.. Ding. S, {M M H²), is comprehensive package. It in- cludes the complete tabular data Dub. the cross cross-modal images Ding. stratified train/validation/test splits S, and the full ground truth specification H. This principled construction provides robust and transparent benchmark for the evaluation of cross-modal causal inference methods.
Algorithm Cross-Modal Causal Synthetic Dataset Generation
Require: Sample size N, SCM parameters {an, 02, p, 6), noise variances {oř, o J nonlinear func- tions {fi, /z2, imaging parameters (Oag: 8, pix cross-modal flags {useyar use ). permutation ra-
tios {pM. py muisance dimension q
Ensure: Synthetic dataset D with tabular data, images, ground-truth truth causal relationships. and evaluation splits
// Step I: Structural Causal Model Sampling
1: for
to do
2: T: + er,s, en (0,
b T is exogenous treatment variable Mediator with linear nonlinear effects Outcome with mediation direct effect
3:
M; <- a; + phu (T:) eM,GeM, ((0,
4:
Y. +- a2 + (M:) 8T; eyseers (V
W
q-dimensional nuisance variables > Independent style variable for imaging
6:
~N0,,)
7: end for
// Step 2: Semantic Parameter Mapping
8: Let & denote the standard normal cumulative distribution function
aM < ((M (M)/aM)) PM < mean(M), аи < std(M)
Semantic amplitudes mediator
10: ay f- + ((Y" my- 3/oy" jay &- mean Jy + std Y")) > Semantic amplitudes for outcome
11: bayk t 4 ((S
(ss)/as)) J: μa mean(S as <- std (Sayle)
> Style parameters in [0, 1]
// Step 3: Base Image Sampling with Class Independence
[Iuse: Ceinss idhase f- Sample N MNIST images with class-uniform distribution
// /Step 4: Cross-Modal Generation
13: for scenario E {IA } do
b Image as mediator or outcome
Select semantic amplitudes: a <- am IM else ay 15: for to N do
16:
r + Rotate(Tnur d, Oze. (2a, -
b Rotation Brightness
17:
I clippo.y (I, + B (a, - 0.5))
18:
I." + clipjo. (Ju: + (1 ((a: 1T! ji:))
D Contrast
19:
Icceanio clippo.y (I." NO0 (opabuyte.))))
b Noisc
20: end for
21: end for
Optional: Apply random permutation fraction {pMs pV) } of images // Step 5: Data Assembly and Stratified Split Generation
22: Dnu f- MT Y", W, a., ay, Dayle; C
23: Ding (I",
Create stratified train/validationneess splits using Cetassi S {Temin Loats Iness}
// Step 6: Ground Truth Specification
25:
For AM scenario (image as mediator):
26:
HM: T Y M
Treatment causally affects mediator
27:
HM M Y
Mediator causally affects outcome
28:
:.:T L" + =
Conditional independence test
29:
HM aM Celoss
Semantic-class independence
30:
H. MAC(aM: ((M))) >
Monotonic correspondence
31: For 1 scenario (image as outcome):
32:
H. L. M
Treatment causally affects mediator D Mediator causally affects image features
33:
H. L )
34:
HY:T T ((TT) M +> 0
> Conditional independence test
35:
H.: ay Gelns
> Semantic-class independence
36: H. MAC(ay, ((I))) >T
b Monotonic correspondence
37:
return (Dub, Dimg: S, M H²)
1.7 SYNTHETIC DATA SAMPLE
To provide intuitive understanding of the generated data's properties, present series of visualizations. Figure 12 illustrate the statistical relationships within the Image as Mediator (I Image causal chain. The plots show
scenario, which corresponds the Tabulars Tmaulllar-mmaa---eaa-
50

---

Published as a conference paper at ICLR 2026
the relationships between the latent treatment variable (T), the final outcome (Y*), and four distinct visual features extracted from the synthesized mediator images (I*). These visualizations serve empirical confirmation of the dependencies and conditional independencies specified by our ground-truth SCM.
Rocation"
Brightness
Contrast
MiNoise
0.35
0.25
Rocooon
M Brightneet
0.30
MANace
0.20
-30
0.20
0.15
3.5
0.0
8.5
0.5
0.5
03
05
0.5
0.10
0.
05
M-Rotationn
BiBrightness-..
Mr-onntast
KcNiisa
1.5
1.3
1.3
1.3-
1.0
1.0
1.0
1.0
0.5
-05
-1.0
-10
-10
-1.
30
0.5
0.2
0.5
0.2
U.S
0A
0.30
0.15
0,20
0.2
Brightness
Contrast
Notz
M-furation" - Y
Hsbrightness 1
McContrast Y
M-Noise
0.5
0.5
(resid))
(resson
(reetto
(iwidd)
0.0
0.0
0.0
0.0
-100
-30
-0.3
0.0
633
-0.1
0.0
0.3
-0.00
0.00
0.05
630
PiPatation Desid)
Brightnnes (resid)
N.Contrast (resid)
tresid)
Figure 12: Visualization of the causal relationships in the Image as Mediator (I scenario. Top: The causal link between the treatment T and the visual features of the mediator image. Middle: The marginal correlation between the mediator's visual features and the outcome Y*. Bottom: The conditional relationship between the mediator's 's features and the outcome, controlling for T. "Resid" denotes the residuals of each variable after being regressed T, confirming the M Y link.
Figure 13 provides an empirical analysis of the Image as Outcome (IY scenario), which instantiates Sabular-TTabus -mage causal chain. The visualizations confirm that the relationships defined in our SCM are successfully propagated through to the final generated images. The scatter plots illustrate the foundational causal links, starting with the correlation between the tabular treat- ment 7 and mediator M. They then show how M directly influences various visual features (e.g., brightness, contrast) extracted from the outcome images, 1X The residual plots the middle rows confirm that this influence from M to the visual features persists even after controlling for T. The image panels at the bottom provide compelling qualitative evidence by showing a clear, monotonic progression in visual characteristics the value of the mediator M increases across its quantiles, visually demonstrating that the causal information is successfully encoded in the image domain.
51

---

Published as a conference paper at ICLR 2026
Rotation
H YBrightness
y Contrast
0.45
1.0
50
0.3
Y-lotation
0.33
0.30
0.2
-05
2.2
-6.5
0.0
6.5
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
1.0
- Rozetion" T
Brrigheness LT
Y-Contrast
50-
0.2
0.25
Valatatton (raat))
*Contraat (waad)
0.3
YAU55
0.20
0.0
-10
3.0
0.33
-120
-05
0.0
0.5
1.0
-0.5
0.0 (resdd)
63
-0.5
0.0
6.5
-6.5
0.0
0.5
(resdd)
(mesic)
Notoo T
Y immgge IM luant Seaa
Vimage ('eriyhmees quancies
Yimages (CContrast ouantles
0.30
Yeloee (reatt
0.05
-2.5
0.0 (revd))
0.5
Figure 13: Visualization of the causal relationships in the Image as Outcome (7Y) scenario. Top
Scatter plots showing the tabular causal link T + M, the marginal relationships between the mediator M and visual features extracted from the outcome images I2 and the condi- tional relationships (1 T) between M and the image features after controlling for T. Bottom Row: Example images from the dataset, sorted into columns by quantiles of the mediator M (left) and by quantiles of their extracted brightness and contrast. The visible progression in rotation, brightness. and contrast across the M quantiles provides qualitative confirmation of the M - 1x causal link.
& Middle Rows:
complexity-tigh Inoninnarriy-iiiek
M-Rotat
MrBriggtteesss
McContrast
M-Norse
0.40
0.35
0.25
0:30
Mchooe
0,20
-50
0.15
0.0
0.0
M-Rotation" YY
M.Srightmess
H. C.notrastt
N.Noise YY
2.0
2.0
2.5
2.0
3.5
0.0
0.0
0.0
0.0
-0.
-25
-055
-03
-1.0
-1.0
-10
-10
CA
0.3
0.15
0.55
M-Rotation"
iggttness
M&Cantrast
+TIT
M-Drightnees-
M-Contrast Y
1.0
1.0
3.0
1.0
(rrsid)
0.5
0.5-
0.5
0.5
-0.5
-1.0
11.0
-1.0
-10
-100
-50
-0.1
00
022
00
0.
-0.5
0.00
005
633
MiRotation
MiBrighoness (resio)
McContrast
M-Nols (resld)
Figure 14: Synthetic data sample in setting (n 5000, a 0.1 and high linear), 7 + M +
52

---

Published as a conference paper at ICLR 2026
n=5000 [-0.1 complnitt-lighh Inodinearit--inne
M
- Kilaariin
Y-Contrast
1.0
Viocatoon
vonghoness
V-Compast
0.20
0.30
0.0
0.23
-0.5
-30
0.20
0.15
-3.5
6.5
05
-6.5
0.0
0.5
0.0
0.5
1.0
M. ViNozse
- V-Rctation" TT
H YBrightress IT
Y-Contrast
0.15
0.25
Y-traaion" (resid)
(sazal)
0.10-
0.1
b.ts
0.20
-00
0.00
-0.05
-0.3
-0.10
M (reskd)
14 (resko)
M (resis)
qumniees
14ttr ghttness querkies
Y-Contrast quant
4 2
(1o50)
0.05
1 Of
0.00
M fresidi
Figure 15: Synthetic data sample in setting (n 5000, 0.1 and high linear), T + M Y.
n-3000 -=0.1| complosiy=tighi nentincaritynneurll
- M- M-Ratation"
W-Brightress
1- M- Contrrtttt
- 34 NoNos
0.35
6.25
IStumatton"
0.30
0.20
0.25
6:35
-0.5
0.0
-3.5
0.0
6.5
0.30
-0.5
0.0
0.5
H Rotat on" +
M Brightnesss +
M C-nottastt Y
M:Nose -
0'3
34
0.00
015
0.2
Rotstion
W-Brightness
M-Contrast
M4-Nose
M. M:Rotation YY
14 Briggteeess YIT
Concratt =
M-hoise -
1.5)
1.5
3.5
1.0
1.0
3.0
1.0
0.5
0.5
Fressti
0.5
Fresien
0.5
0.0
1t. ".RotaCon" (resdd
MiBrighnnes (resid)
M-Contrast (reskd)
MiNolse (resi))
Figure 16: Synthetic data sample in setting 3 (n 5000, = 0.1 and high neural), T - M - Y*.
53

---

Published as a conference paper at ICLR 2026
n-3000 [=0.1| complexit-tigh honnnnarry-nnuurl
T- M
M - Y-Roration"
M V-Brghéness
0.45
V-Contrast
0.40
1.0
Yiootatoon
0.33
0.5
0:30
0.0
0.25
0.20
0.15
.YNolse
14. - 'K-ottatio""
--Brighhness T
N. Y-Cortrast T
0.15
0.25
Xilotation (mwal))
Brigghtmeee (resid)
0.10-
0.03
0.20
-50
0.00
0.15
00.05
-100
-0.10
((wsk))
(resdd)
(resid)
Y-Moie
quantees
Y-brightness quentles
Vimages |>Contrast quartles
p000
01100000110000
0.00
-0.05
Figure 17: Synthetic data sample in setting 4 (n 5000, 0.1 and high neural), T M Y.
complexity-high nennincarty-pallyoniii
M-Rotaation
W-Brighttwess
McContrant
T M-Niiise
0.40
inerghtness
0.35
0.25
M-ftrtatton
Mchooee
0,20
-50
0.13
MMRotationn
M-Brightness
Contrast
MiNbise
CA
0.35
M-Rotation"
M-Contrast
-YIT
M-Grightness
1.5
1.0
1.0
1.0
1.0
(resid)
((ooid)
-0.5
-8.5
-0.5
-0.5
-100
00
02
0'
-0.05
0.00
0.0
030
K-Brighoness (resi))
M-Contrast
Figure 18: Synthetic data sample setting (n 5000, a 0.1 and high polynomial). - M - y*
54

---

Published as a conference paper at ICLR 2026
n-3000 a-0.1 compiexity-high pantiseartty-palyynmni
T+M
M V-Rotat on"
- V-Bigttnees
- V-Cottrsst
0.4
Viotadion
0.7
0.2
6.5
1.0
K-Nolse
M. + Y-Rotatiio"
+YBrightnass T
YY-Contrast
0.25
T80bation" (resit)
0.2
VContrest
0.05
0.20
-50
0.00
0.15
00.05
-0.1
-0.10
((weid)
(rwwi))
(resid)
Vilolse
Brigghnness
Contrast
DOOK 5000000
SOYA SORSE
0.00
(nesid)
Figure 19: Synthetic data sample in setting 6 (n 5000, 0.1 and high polynomial), T M + Y.
n-5000 a-0.1 complesityyloo nondinearit-iinor
N-Brightness
4.4
Mutunation
IM giiggseess
Mn-Contrast
8.30
0.20
0.25
0.20
0.15
0.3
-0.5
6,5
0.5
-3.5
6.5
-05
0
05
0.00
-25
0.5
M-Rotation" =
14 Brigghhnnss
M. CoConntast =Y
M:Noie =Y
1.5
1.5
1.5
3.5
1.0
1.0
1.0
3.0
0.5
0.5
0.5-
0.5
0.0
0.0
0.0
0.0
-0.5
-03
-0.5
-1.
-1.
-10
63
0A
52
C4
010
0.15
0.25
M-Rlstation Sotationn Y
N-Brigheness
M+-Contrast
N Brightness +Y TT
M Contrastt Y T|T
M. Moio Y
1.0
1.0
1.0
1.0-
(rresd)
(resitt
rresidt
(resid)
0.0
0.0
0.0
0.0
-1.0
-1.0
-1.0
-1.0
"ARotation" (nesid)
MiBrightness (resid)
M.Contrast (1oci)
N-Molse (nesd)
Figure 20: Synthetic data sample in setting 7 (n 5000, 0.1 and low linear), T + M Y
55

---

Published as a conference paper at ICLR 2026
e-5000 1ow0.1 complexity-low nenlincarity-iinn
M
Kilonarion
M y-Contrast
1.0
0.40
Viocatoon
vonghoness
V-Commsst
0.35
0.30
0.0
0.23
-30
0.20
0:15
-0.5
0.5
-0.5
0.0
-0.5
0.0
-05
5.0
0.5
M - YiNolse
- Y-Rctationn LT
H YBrightress LT
Y-Contrast
0.25
Y-ttaaion" (resid)
(vaza))
Conmaat (razd)
0.1
0.20
0.0
-0.1
M (reskd)
14 (resko)
M (resks)
images quiniles Leoyonooon
14tt ghtnness quarties
Y-Contrast quent
0.10
(1ood)
0.05
-0.25
0.
M
(residi
Figure 21: Synthetic data sample in setting 8 (n 5000, 0.1 and low linear), T - M - Y.
o-5000 a=0.1 complexity=low maniiscarki-neuurl
- M- M-Ratation"
T W-Brightrees
1 M- Contrattt
- M-Nose
0.35
6.25
0.30
0.20
0.25
6:35
-3.0
-0.5
0.5
-10
-0.3
-3.5
0.0
0.30
-0.5
0.0
0.5
H Rotat on" +
M. Brightteees +
M M-Contastt Y
M:Nose -
0'3
34
0.0
015
0.20
0.2
Rotstion"
M-Brightness
C-Contrast
M4-Nose
M. M:Rotation YY T
14 Briggteess YIT
Concratt +
M-hoise -
1.5
1.5
3.5
15
1.0
3.0
3.0
1.0
Fressti
Fresien
0.0
1t ".RotaCon" (resdd
MiBrighnness (resi))
M-Contrast (resid)
MiNolse (resi))
Figure 22: Synthetic data sample in setting 9 (n 5000, -0.1 and low neural), T - M + Y*.
56

---

Published as a conference paper at ICLR 2026
p=5000 aw0.1 complexity=ion nnntinarityyssuuall
T M
M - Y-Rotariion
M V-lrggtneess
Y-Cntrast
Vilotatoon
0.30
0.22
-0.5
0.20
-1.0
0.0
YYMalse
H. 'K-otattion"
YYBr ggtnnss T
M. Y-Contrstt
0.13
Xilotation (mwal))
Brigghteeee (resid)
0.25
0.10
0.03
0.20
-50
0.00
0.15
-0
-0.16
(rwakd)
(swild)
(resud]
Y-Noie
quanties
V-brighoness quentles
CoCttrass gaamlees
FoOk 7OOL Mincca
We Doonss
0.00
TE400 MANARE Hour
-0.05
Figure 23: Synthetic data sample in setting 10 (n 5000, 0.1 and low neural), + M Y.
complexity=low poodinnarity-polymoodd
M-Rotatiion oon"
M-Brrightnwes
T McContrad
T M-Ninise
0.40
0.25
inerghtness
0.35
M-ftrtatton
Mcooee
0,20
-50
0.15
MMRotationn
M-Brightness
Contrast
N:Noise
0.3
0.35
M-Rutation"
McContrast
-Y
M-Drrightnees
T
1.0
1.0
1.0
1.0
(reoi))
((ooid)
0.0
0.0
0.0
0.0
-1.0
-1.0
-100
-01
00
02
0'
-005
0.20
0.05
630
K-drighoness (resi))
M-Contrast
Figure 24:
Synthetic data sample in setting (n 5000, 0.1 and low polynomial),
M + Y*.
57

---

Published as a conference paper at ICLR 2026
complexity=lov nentinearity-polnneee
T M
Yilorarion"
M t-brightmes
&-ontrest
1.0
0.4
Viucation
50-
venghneess
V-Comaast
0.5
0.0
0.25
-00
0.20
0.15
Villose
'FRotattion'
Vibrightress
Y-Contrast
0.2
0.15
0.25
Niptanton (reatd)
0.10
1000010
0.05
0.70
-50
0.00
0.15
-100
-0.1
0.00
M (resso)
M (resid)
YiNuise
quanitls 2 wwn neeeae
Y-Contrast
ENE Horober
(resc)
0.00
-0.05
Figure 25: M + Y.
Synthetic data sample in setting 12 (n 5000, 0.1 and low polynomial),
D=3000 amo.1 complesit--mmduu moniineartt--llteer 34 Brigntnees
H Rotation"
T B--oontaat
T M-Noise
Meltoration
K-argghnes
Contrust
0.30
0.35
0,20
0.13
Mollotation" =
Helrightness
M-Cantrast Y
M-Nlcise
1.5
1.5
3.5
1.5
1.0
1.0
3.0
1.0
0.5
0.5
0.5
0.0
0.0
9.0
0.0
-05
-0.5
-05
-1.0
-1.0
-1.0
-10
Rotation"
M-Grightness
M-Contrast
M-Nose
Rottttion"
Brightnees
M-Contrast
1.0
1.0
1.0
1.0
-1.0
-1.0
-1.0
-10
-100
-50
03
0.0
-0.05
6.00
0.05
650
K-Brightmess (resid)
Figure 26: Synthetic data sample in setting 13 (n 5000, 0.1 and medium linear), T M y'
58

---

Published as a conference paper at ICLR 2026
toemplesit-mmedium nonfinearityyiin
T M
X-lonarion"
M tebrggtness
&-ontrest
1.0
0.4
Viucation
50-
Trnnghness
VCComaatt
0.25
0.50
0.0
0.25
-0.5
-30
0.20
0.15
ViNolse
Y-Rotation"
Varightnees
YcContrast
0.25
Vilotation (resid)
(vuci))
0.2
0.10-
0.15
0.03
0.20
-50
0.0-
0.00
0.15
-100
(1e3i))
0.00
-0.55
DOS DE 000000
-05
Figure 27: M + Y.
Synthetic data sample in setting 14 (n 5000, 0.1 and medium linear),
Icmmplesty-mmdllu ssollinaaity-neurall
M. M-Ratation"
Biggighteesss
M-Contrast
Nonos
0.40
50
0.35
6.25
0.20
-00
6.35
0.15
Rotst
M-Grightness
Contrastt
M:Nose
Rotation"
M-Brightness
M-Contrast
M-Rotation
H-Corrast
1.5
1.5
3.5
15
1.0
3.0
1.0
Freestn
Fresidi
-2.5
-1.0
-1.0
-1.0
-100
-0
0.00
Merighmass (resi))
M.Contrast (nesid)
MiNoise
Figure 28: Synthetic data sample in setting 15 (n 5000, o 0.1 and medium neural), T M Y*
59

---

Published as a conference paper at ICLR 2026
complexity-mdiin ponninearty-nneural 1-Br Kerightnss
15
T M
H Silstation"
H M-and rast
2.40
1.0
Vilcsttnn
50
Yerghonss
VCCenrest
0.15
0.5
0.30
0.0
0.25
30
0.20
0.35
M4
Y-Nolse
Viloration'
Yarrghtnees
M. V-Contrast
0.25
Viluaatten (rudid
0.2
X-ontrstt (wwaad)
0.15
0.30
1000010
0.05
0.70
Sttrigttmes
-50
0.0
9.50
0.15
-100
-0.3
-0.10
10.0
(resid)
(restd)
(resc))
SISAPT T MONOOTE
0.00
-0.05
Figure 29: M + Y.
Synthetic data sample in setting 16 (n 5000, o 0.1 and medium neural), T +
n-3000 a=-0.1 | complexit-needium memlineantgeeplyyymmall
H Rotation"
34 Brightnees
T Bi-oontratt
T M McNoss
Moltoration
K-arghnees
Contrust
0.30
0.35
0.50
0.23
Mollstation -
Helliggtness
M-Cantrast -
M-Nlcse
Rotation"
M-Grightness
M-Contrast
M-Mose
M-Rotation"
Brigghnness
M-Contrast
1.3
1.5
1.5
1.:
1.0
0.5
0.5
0.5
0
-0.5
-2.
-0.5
-0.%
-100
-50
03
0o
0.00
0.05
K-trrighmees (read)
Figure 30: Synthetic data sample in setting
5000, o 0.1 and medium polynomial),
T - M - Y
60

---

Published as a conference paper at ICLR 2026
p-5000 @-0.1 complextt-mmeluu menlisearrt-ppplyymmelll
T M
- Y-Rotation"
M X-Brightness
M I-Contrest
0.43
1.0
0.5
0.50
0.15
M. YiNoie
'-Pottttian" [
H. .SBrightness T
+XContrass
0.25
1-2bation" (resid)
(vaii))
(Cnntrent (reud)
0.1
0.05
0.20
-50
0.00
0.15
0..05
-0.3
-0.10
M (seeid)
14 (tewic)
14 (swwis)
Yiools
Ymmapes] quantles
IKBr ghiness ouantles Pere
Y-Contrest quantles
Ho good
FIC110000 1300
0.00
21 BI
fresidi
Figure 31: Synthetic data sample in setting 18 5000, o 0.1 and medium polynomial), M - Y.
5=0000 [n0.33 complecity-high |snnnnnartt-lnnrr
T- Retation"
Brighnnes
T- M-Contrast
T- N-Nocce
M-Rotation
0.50
1AN000
0.20
0.20
0.15
-10
-05
0.
1.0
-10
110
-05
0.10
10
M-Ratationn
16 BrBrighneess a
Conttrastt =
H Noios Y
0.3
030
B M-Nocis
0.25
Millstation
N-Brigheness
M-Contrast
1 Mi-Rotatoon Y
drdrgttness + Y
IS-Contrast - YIT
N N-Noore Y
(nacdd)
(reend)
-100
-01
66
3:0
Rotaelinn (resid)
Mr-rightness Iresid)
W- Contrest (resid)
14-Nose Iresidi
Figure 32: Synthetic data sample in setting 19 (n = 5000, = 0.3 and high linear), T M - V
61


Reviews:
[{"review_id": "SZUUSLtvS2", "reviewer": "Reviewer_DeFu", "summary": "This paper introduces a cross-modal representation learning framework named CARL, designed to address the issue that optimizing purely statistical objectives can disrupt underlying causal structures. CARL jointly optimizes three structure-preserving losses—Conditional Independence Preservation, Markov Boundary Preservation, and Monotonic Alignment Consistency—to ensure that the learned representation space retains the causal structure of the original data. The authors validate the approach on synthetic datasets and a real-world Human Phenotype Project (HPP) dataset, and provide theoretical guarantees showing that causal queries remain identifiable in the representation space.", "questions": "1 Have the authors considered evaluating CARL on larger-scale cross-modal tasks (e.g., CLIP-style vision-language alignment) to assess generalization?\n\n2 CARL trains multiple independent predictors with cross-validation. Has its scalability on large-scale data been evaluated?\n\n3 For highly imbalanced modality-information densities (e.g., image vs. text), can CARL still effectively prevent the lower-density modality from being overshadowed?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1 First systematic treatment of cross-modal causal structure preservation, formalizing the CSP principle and the three core challenges.\n\n2 Introduces the ε-CSP definition, a attainability/consistency theorem, and a theorem for preserving identifiability of causal queries, providing rigorous guarantees.\n\n3 The three losses are well motivated and complementary, balancing conditional independence with information retention.\n\n4 Synthetic and real data jointly verify effectiveness, robustness, and interpretability.\n\n5 Successfully recovers known medical causal pathways on the HPP dataset, showcasing potential in complex biomedical scenarios.", "weaknesses": "1 Although the appendix contains detailed proofs, the main text could better explain some theoretical results (e.g., an intuitive reading of the error bounds) to improve readability.\n\n2 While experiments span synthetic and real data, they do not include larger-scale cross-modal benchmarks (e.g., vision-language tasks), limiting the demonstration of generalization.\n\n3 In the DUAL configuration the method avoids using both image modalities simultaneously, which may limit information utilization in certain practical settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ACj5uob61B", "reviewer": "Reviewer_8ef8", "summary": "This paper presents a set of causal structure preservation principles and practical approaches to address the causal invariance problem in cross-modal scenarios. The approach has been validated on multimodal biomedical dataset HPP and synthetic hand-digit datasets, showing promising performance gains.", "questions": "- About the CSP principle. What are the advantages of the proposed CSP principle in Sec. 3.1 in the field of cross-modal causal invariance? And how does each principle motivate/guide the development of the proposed approaches?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The problem of causal structure preservation is practically important especially in the cross-modal scenarios. This paper has made a fundamental exploration and proposed a principled approach to this problem.\n+ This paper presents both theoretical analysis and empirical study on real or synthetic data. Especially, experimental results could show clear performance gains through ablation study and parameter sensitive analysis. \n+ The paper is well structured and has clearly stated the research challenges, proposed approaches, and core contributions.", "weaknesses": "- Unclear claim. For the contemporary cross-modal representation approaches like CLIP, ALIGN, and ImageBind, it is encouraged to provide  literature or empirical results to support the claim that they cannot guarantee the three mentioned causal properties (conditional independence, Markov boundaries, identifiability conditions). Besides, for each of these properties, it’s suggested to explain with an example to correlate with CIB, MAC, or CIC, as they closely correspond to the key technical contributions in this paper.\n\n- Experimental Discussion. Structure-preservation evaluation showed that CARL method could keep a CSI metric of 1.0 under varying sample size and noise level, and baselines only achieved 0.25 CSI. It could be better to discuss this metric more to help understand why these superior performances come from the monotonic alignment constraint that maintains semantic-geometric correspondance (as claimed in Sec. 5.2).\n\n- Missed Ablation Study. Given that the overall loss consists of a regularizer R that has cross-modal alignment loss, style consistency loss, and IB loss, why are the latter two terms not considered in ablation study in Table 2? This is important as the paper's Abstract explicitly mentioned the IB regularization as a key component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yp3ROcdSLy", "reviewer": "Reviewer_i8uF", "summary": "This paper presents CARL, a framework for learning cross-modal representations that are intended to preserve causal structure. The paper introduces the notion of Causal Structure Preservation (CSP) consisting of three conditions, which are enforced during training via three distinct loss terms. The authors show that under consistency and other assumptions, joint optimisation of these loses ensures CSP. CARL demonstrates applications to real-world cross modal data. The presented theoretical framework along with experiments is promising. However, the presentation of the paper needs to be improved.", "questions": "See above.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- The problem of preserving latent causal structure is very interesting.\n- Cross modal data is a challenging scenario with significant real-world applications. The paper demonstrates applications to medical data.\n- The method is compared with other baselines on synthetic data and it shows superiority in preserving causal structure.", "weaknesses": "**Clarity concerns**\n\nThe main concern I find in this paper is presentation and clarity. In its current form I needed to jump back and forth between sections and between main text and appendix to understand even the basic setup. This makes the work very difficult to evaluate in terms of correctness and novelty. Please find below some recommendations.\n  - The contents of the paper could be organised to ease the flow of the paper and improve clarity.\n    - The paper starts with a presentation of the datasets used. It is very interesting to mention applications at the start, but I believe an improvement would be to first introduce the problem setup, and then connect it to the real-world application.\n      - In this line, I found it confusing that Figure 1 already introduces notation {X, M, Y}, as covariates, mediators, and outcomes. However, the concrete formalisation comes into Section 3.\n      - Also here, in Figure 2, the 3 setups (IM, IY, and DUAL) are mentioned, which are briefly described in section 3, and are explained in detail in section 4.1.\n      - Similarly, Theorem 2.1 in Section 3.1 depends on loss terms that are only introduced in Section 4.2. This forces the reader to jump forwards and backwards to understand the theorem.\n    - Consider the following organisation: \n      - Have a section with problem setting where (i) some initial  motivation is presented (cross-model CRL for HPP applications), (ii) the problem setup is presented with all its elements (that would be section 3 and 4.1),  (iii) when introducing concepts such as mediators, treatments, link them to the HPP example, and (iv) present the formalisation of CSP with high-level intuitions linking to HPP as well.\n      - Present the methodology section (loss functions and strucutre discovery).\n      - Present the theoretical guarantees.\n  - The notation introduced in this paper is very heavy, with some elements not discussed clearly, making it difficult to follow when reading. The main text should be self-contained, with details in the appendix. However, I believe the paper introduces notation in the main text with explanations in Appendix, and this impacts presentation significantly.\n      - In section 3, I is used for images, but I(.,. |.) also denotes conditional mutual information.\n      - Assumption 2 (line 176) introduces new notation (a, b, \\eta) which is not explained.\n      - Definition 2.1 introduces the Spearman correlation without citation.\n      - Theorem 2.1 introduces notations O_P(\\cdot) which are not explained, and loss functions which are discussed in later sections.\n      - The monotonic alignment consistency (Equation (3)), introduces semantic labels a, which have not been introduced before. Please provide intuition.\n      - Equation (4) groups discussed loss functions, along with 3 additional terms “align”, “style”, “IB”, which are not discussed in the main text.\n- Section 4 needs revision in terms of citations and clarity:\n  - For example, since the loss functions are not novel themselves, citations should be accompanied for clarity. This also helps the reader what is novel in the presented methodology.\n  - Section 4.3 is very hard to follow, and it is not clear whether the idea is novel or it is derivative from previous work. I believe the idea is to present some analysis in which the causal structure can be recovered from latents. However, there are no notations to understand where the metrics come from, or intuitions for the steps to show consistency guarantees.\n    - Lines 327-333: Is this a known result? Can you provide a citation, or formalise a statement in a theorem if it is novel?\n  - Section 4.4 lists theoretical properties. For clarity, the corollary should be read as a formal standalone statement with proof, and not as a paragraph.\n- The Appendix section contains repeating labels from line 1718 (Label ordering goes back to A).\n\n**Theoretical correctness**\n\nThe theoretical results of this paper are very difficult to follow. This does not mean that the results are incorrect. However, the current presentation does not allow the reader to follow the logic intuitively. Please find details below:\n  - Section 3.1 contains incorrect labels for definitions and theorems.\n  - Some intuitions for the CSP principle should be explained after Definition 2.1. Intuitively from my understanding is to ensure that learned latents preserve structure. Now, why are conditions (i-iii) sufficient/necessary for this idea to hold? I believe this would significantly improve the presentation of \\epsilon-CSP.\n  - The Definitions and Theorems lack clear structure, and don’t include standalone statements or results.\n    - Theorem 2.1 introduces the result in lines 201-204, and continues with additional remarks. The remarks should be separate from the theoretical statement.\n    - Theorem 2.2: Lines 218-219 are additional remarks that should be outside of the theoretical statement.\n  - Equations in Theorems 2.1, 2.2, are inline, making them hard to follow. Some of these equations are introduced with an explanation in Section 4. Consider moving the presentation of these equations before the theoretical statements.\n  - I briefly checked the proofs in the Appendix, and I find them very hard to follow.\n    - At least a proof sketch should accompany the main theorems and corollary presented in the main paper to intuitively explain how CSP is achieved. How are Assumptions A.5, and A.8.2 utilised to show CSP by minimising the loss function?\n      - For example, to verify this I first read A.5 and A.8.2, then I jumped to C.8, then to theorem B.2.1, which refers to a list of assumptions from A.8. The proof requires to jump back and forth across the document several times, which makes it hard to follow.\n    - Assumption A.5 assumes consistency. This sounds like a very strong assumption. Is this standard? I believe this assumption is very important for CSP to be achieved, and therefore it should be discussed.\n    - There are some Lemmas with missing proofs in the Appendix. If the results are known, please provide concrete citations.\n    - I consider the appendix should be organised so that the theoretical framework can be followed more intuitively. For example, consider introducing a proof strategy first, and then explain the intuition behind each of the Lemmas.\n\n**Comments on experiments**\n\n- The experiments on synthetic data seem to have been reported on one seed. Please provide additional random seeds (3-5) with ranges in the Appendix tables to ensure robustness of your claims.\n- The interpretability of real-world data seems interesting. However, it does not show CARL’s superiority. An additional baseline (e.g. CausalVAE), with a results comparison would improve the stance of the paper for interpretability gains of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PJS8HRk2Nm", "reviewer": "Reviewer_enbA", "summary": "This paper proposes a method for preserving known causal structures (e.g., mediation) in cross-modal representation learning. The approach leverages the independence relation implied by mediation, $T⊥Y^{*}| M$, to design a mutual-information-based loss that enforces the mediation causal structure. The authors validate the effectiveness of the method on a synthetic dataset and further estimate causal effects on the Human Phenotype Project (HPP) data, which align with causal modeling under mediation assumptions.", "questions": "⦁\tThere seems to be an issue with some \\ref{} in the appendix, as they do not correctly link to Section ABC. Section ABC in appendix appeared twice.\n⦁\tIn the $I^M$ and $I^Y$ settings, the causal relationship visualizations in the appendix show weak correlations in some cases, with small $R^2$ values. In particular, for the Rotation transformation, the results hardly reflect the conclusions stated by the authors. Could the authors clarify this discrepancy? Moreover, the statement “These visualizations serve as an empirical confirmation of the dependencies and conditional independencies specified by our ground-truth SCM” is somewhat misleading, since the visualizations only verify dependencies. Why not empirically test $T⊥Y | M$ on the synthetic data?\n⦁\tRegarding the real-world HPP dataset, it is unclear what the causal effect of the learned latent representations signifies. Could these causal effects not be directly estimated from the observed data? How large is the discrepancy between the causal effects estimated in the latent space and those from the true causal effects? Please provide quantitative results.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "⦁\tThe experiments are comprehensive. In particular, the design of the synthetic dataset is well-documented in the appendix.\n⦁\tThe model architecture and implementation details are clearly described, which enhances reproducibility.", "weaknesses": "⦁\tThe claimed contribution regarding the preservation of causal structure in cross-modal representation learning appears somewhat overstated. First, the modalities are limited to tabular and image data. Second, the assumed causal structure is restricted to mediation. The core loss functions conditional independencies preservation $\\mathcal{L}{CI}$ and Markov boundary retention $\\mathcal{L}{MBR}$ are heavily dependent on a predefined causal structure.\n⦁\tThe authors seem to assume a causal graph under mediation. If such prior knowledge is unavailable, can the assertion of “preservation of causal structure” be generalized to unknown causal graphs? Without knowledge of the causal graph, how would one ensure preservation of the causal structure?\n⦁\tThe method design is rather straightforward. Employing mutual information to enforce conditional independence has been widely explored in causal representation learning, and Modal Alignment Consistency is also a common idea in cross-modal representation learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)