ICYM$^2$I: The illusion of multimodal informativeness under missingness
Authors: Young Sang Choi
Vincent Jeanselme
Pierre Elias
Shalmali Joshi
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal learning
Distribution shifts
Missingness
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: Multimodal learning is of continued interest in artificial intelligence-based applications, motivated by the potential information gain from combining different data modalities. However, modalities observed in the source environment may differ from the modalities observed in the target environment due to multiple factors, including cost, hardware failure, or the perceived *informativeness* of a given modality. This change in missingness patterns between the source and target environment has not been carefully studied. Naïve estimation of the information gain associated with including an additional modality without accounting for missingness may result in improper estimates of that modality's value in the target environment. We formalize the problem of missingness, demonstrate its ubiquity, and show that the subsequent distribution shift induces bias when the missingness process is not explicitly accounted for. To address this issue, we introduce $\text{ICYM}^2\text{I}$ (In Case You Multimodal Missed It), a framework for the evaluation of predictive performance and information gain under missingness through inverse probability weighting-based correction. We demonstrate the importance of the proposed adjustment to estimate information gain under missingness on synthetic, semi-synthetic, and real-world datasets.

Published as a conference paper at ICLR 2026
ICYM'I: THE ILLUSION OF MULTIMODAL INFORMA- TIVENESS UNDER MISSINGNESS
Young Sang Choi'; Vincent Jeanselme'; Pierre Elias!.?, Shalmali Joshi" *Department of Biomedical Informatics, Columbia University, New York, NY, USA Seymour, Paul, and Gloria Milstein Division of Cardiology, Department of Medicine, Columbia University Irving Medical Center, New York, NY, USA
ABSTRACT
Multimodal learning is of continued interest in artificial intelligence-basse ap- plications, motivated by the potential information gain from combining different data modalities. However, modalities observed in the source environment may differ from the modalities observed in the target environment due to multiple fac- tors, including cost, hardware failure, or the perceived informativeness of a given modality. This change in missingness patterns between the source and target envi- ronment has not been carefully studied. Naïve estimation of the information gain associated with including an additional modality without accounting for missing- ness may result in improper estimates of that modality's 's value in the target envi- ronment. We formalize the problem of missingness, demonstrate its ubiquity, and show that the subsequent distribution shift induces bias when the missingness pro- cess is not explicitly accounted for. To address this issue, introduce ICY M (In Case You Multimodal Missed It), a framework' for the evaluation of predictive performance and information gain under missingness through inverse probabil- ity weighting-basee correction. We demonstrate the importance of the proposed adjustment estimate information gain under missingness on synthetic, semi- synthetic, and real-world datasets.
INTRODUCTION
Multimodal learning is ubiquitous in machine learning as practitioners combine multiple data types improve predictive performance in applications healthcare (Perochon et al., 2023; Tu et al., 2024), robotics (Gao et al., 2024; Shah et al., 2023), and recommender systems (Chen et al., 2019). However, factors such as privacy concerns (Jaiswal Provost, 2020: Zhang et al., 2021), cost- benefit tradeoffs of data-acquisition (Buck et al., 2010), and user preferences (K (Kossinets, 2006) imprint multimodal data with missingness. Additionally, even modality complete data is available or curated at training, data noise (Cohen et al., 2004; Ma et al., 2023) and sensor failures (Inceoglu et al., 2021; 2023) may result in missing modalities in the target environment.
Although the missingness of modalities is recurring challenge in real-world settings, current multi- modal machine learning methods often assume that modalities are fully observed, both in source and target environments. When missingness is considered, the literature has focused engineering ef- forts (Le et al., 2025; Wu et al., 2024) such as data selection (Hosseini et al., 2022), imputation (Tran al., 2017; Cohen Kalafut al., 2023; Malitesta et al., 2024), and architecture design (Chen al., 2022; Zeng et al., 2022), which implicitly assume a stable missingness process between source and target environments. When this assumption is violated, the missingness mechanism induces a dis- tribution shift (Zhang et al., 2023; Liu et al., 2023b) that biases the estimated informativeness of a given modality. Missingness pervasive and impacts a broad range of application domains en- countered in the multimodal literature: in breast cancer screening, biopsies are only performed if there are abnormal findings in a mammogram; in autonomous vehicles, LiDAR sensor dropout can occur due to weather and lighting conditions; and in online recommender systems, reviews are only
"Equal contribution. Corresponding author: young. sang. saog..cooiii wwwlllmiia..m Code available on Github: https: /ttps:///itttt thub. ccm/ce////ee//eeeeeee rea IM- L aaaaa///// CYM22

---

Published as a conference paper at ICLR 2026
Collected data
Source dataset [Reamee)
Training
P(Y P
Evaluation
Deployment
Target data (Sucert)
Suboptimal data collection and modeling decisions
P(Y
Blased performance informativeness estimates
Corrected performance and informationess estimates
P
Reliable performance and informativeness estimates
P(Y
IPW-
P(Y
IPW-
Deployment
corrected
Curation
training
evaluation
Figure Overview of the proposed framework. Curation often discards missing data, resulting in a discrepancy between the collected Ω and source datasets Qgomre used for training. Current practice is denoted in blue: naive training and evaluating on Queure leads to biased estimates of performance and informativeness on target data. The orange path illustrates the proposed ICYM'I: double in- verse probability weighting (IPW) mechanism that yields accurate performance and informativeness estimates under the larget distribution.
collected after certain consumer behaviors. Across these settings, ignoring the distribution shift be- tween source and target due to missingness when quantifying modality informativeness may conflate missingness with signal, leading to flawed data collection and modeling decisions.
In this work, we propose framework to overcome the (mis)estimatiin of both inherent informative- ness and predictive utility under missingness in multimodal learning. Our contributions, summarized in Figure 1, are as follows:
Framework for multimodal learning with missingness. We formalize the impact of missingness as distribution shift intrinsic multimodal learning, where the observed source distribution differs from the target distribution due to missingness. We show that not accounting for missingness, common practice, may bias the estimate of a modality's predictive and information-theerrtie utility.
I. Under the bassingness-at-randdee (MAR) assumption, a much more realistic assump- tion than the common and often implicit assumption of hatprnggnnsssoooopleee---------e made by statc-of-the-ar multimodal strategies, we propose ICYM"I (In Case You Multimodal Missed It), a double iiverse-rrppensee weighting correction overcome missingness-induee induced dis- tribution shifts. The proposed method does not aim to improve predictive performance; rather, enables unbiased estimation of model performance and information gain across modalities, even un- der missingness. Specifically, we demonstrate that ICYM"I improves correlation in predictive and information-then -hhooretic utility of modalities.
(MCAR)
Experiments on diverse data. We demonstrate the broad applicability and utility of our methods in synthetic, semi-synthetic, and real-world benchmark datasets, including case study in multi- modal learning in health.
2 RELATED WORK
Multimodal benchmarks suppress missingness encountered in real-world environments. Prior work multimodal models often assumes fully observed modalities (Ngiam al., 2011; Zadeh et al., 2017; Hou et al., 2019). Missingness has largely been an overlooked problem (Le et al., 2025; Wu et al., 2024), to the extent that current benchmarks rarely contain samples with missing modal- ities. Curation often involves discarding incomplete samples, filtering them based on data-quality criteria, such as text length or file size (Sharma et al., 2018; Schuhmann et al., 2022), or imputing them using automatic tools (Miech et al., 2019). This curation implicitly assumes that rejected sam- ples follow the same distribution as the observed ones. This assumption may not hold. For example, in autonomous driving datasets, samples with sensor failures often caused by extreme weather or lighting conditions may be filtered out. Models trained on complete data may consequently not generalize to these settings, posing real-world risk at deployment. When missingness is considered, previous works focused on robustness through imputation (Tran et al., 2017: Cohen Kalafut et al., 2023: Malitesta al., 2024), representation learning (Wu et al., 2024; Liu et al., 2023a), knowledge distillation (Li et al., 2024; Wang et al., 2020a), and model ensembling (Chen et al., 2022; Zeng al., 2022) all ignoring the potential shift resulting from the missingness process.
2

---

Published as a conference paper at ICLR 2026
Multimodal missingness in the target environment. Prior work has explored missingness in the target environment (Lin & Hu, 2023: Zeng et al., 2022), e.g., when a captor fails at deployment (Ma et al., 2022). Broadly, two strategies have been proposed (Wu al., 2024): (i) data preprocess- ing through cross-modal imputation (Cohen Kalafut et al., 2023; Malitesta et al., 2024; Tran et al., 2017), where one replaces the missing modality (Ma et al., 2021; Zhou et al., 2022), as well (ii) model training strategies such as architecture design (Lee et al., 2023; Ge et al., 2023), distillation- based methods (Li al., 2024; Wang al., 2020a), and ensembling (Chen et al., 2022; Zeng al., 2022). Through the proposed formalization, our work distinguishes between different missingness assumptions, demonstrating that the previously studied framework is only one among various plau- sible mechanisms for which current strategies are not well-designed.
Distribution shifts in multimodal learning. Addressing multimodal shifts has been studied in vision-language models (Zhou et al., 2024; Verma et al., 2024) or using information-theorete no- tions to understand multimodal behavior under distribution shifts (Oh et al., 2025). Augmentation and regularization strategies have been leveraged to address temporal shifts for conversation under- standing (Woo et al.. 2023; Lian et al., 2023). Advances learning, such as in-context learning, have been studied to characterize adaptation to multimodal distribution shifts (Zhou et al., 2024; Xue et al., 2024). However, existing strategies aim to improve robustness under domain shifts only, while ignoring the potential shift in missingness between source and target environments. Instead, our work aims to correct estimates of performance and modality informativeness under missingness inform modality collection at deployment.
value of a modality. Existing works often implicitly assume that additional modalities improve performance, ignoring the prohibitive cost, complexity, and po- tential noise added by these additional dimensions. When limited resources or constraints limit availability in the target environment, a central challenge is quantify the cetemmation-hhhoreeee value of a modality (Liang et al., 2024c). Liang et al. (2024a) proposed method for recovering partial information decomposition measures of the redundancy, uniqueness, and synergy of the in- formation provided by the different modalities (Bertschinger et al., 2014; Williams & Beer, 2010). However, these decompositions ignore the impact of missingness, a gap that our work aims to fill.
Quantifying
Correcting for missingness bias. The impact of missingness in multimodal learning is understud- ied, in part due to the lack of formalization of missingness in this area. Ignoring this process can bias estimates of interest (Phelan et al., 2017), because the observed distribution differs from the under- lying distribution that practitioners aim to model. The statistical literature has introduced strategies such matching (Stuart, 2010) and reweighting (Jethani et al., 2022) address missingness. We leverage these statistical tools to address the overlooked missingness in multimodal settings.
3 MULTIMODALITY AND MISSINGNESS
X1
X1
X Mi
Y
My
Y
Y
C
X
X2 X
2,hs M2
Figure 2. Directed Acyclic Graphs of the assumed data-generating processes. On the left the com- monly assumed graph with no missingness. On the right is the proposed missingness formalism. X1 and X2 are two modalities of interest, Y is the label of interest. The missingness process depends on C. Filled point nodes are observed variables, while unfilled nodes are unobserved. Gray edges indicate MAR missingness for given modality.
Consider two modalities, X, Xi and X2 Xr and the state of interest Y E V. We denote the joint underlying distribution = Xi x X x Y.. Without loss of generality, we assume an anti- causal setting for the data-genert gentraeenn process, in which the modalities are dependent 2 (left). We use the binary indicators of missingness M,. M2, and My. which equal to 1 if the associated variable is missing, 0 if observed, following the convention of Mohan
the states
Y. as shown in Figure
3

---

Published as a conference paper at ICLR 2026
& Pearl (2021). Observed variables are subscripted by "obs', which corresponds to the underlying
modality if observed, and unobserved otherwise (denoted by Ø). Formally, the observed variable Yohs and observed modalities oooo and X2,obs can be defined as follows:
Ø
if My =
Yabs
Y
otherwise.
In this setting, we denote the observed joint distribution Dohs == (M -1) x (M X22) (M )) where M = M1 M2 My. This complete modalities analysis has been the focus of multimodal learning.
Missingness in multimodal learning, We distinguish three mechanisms that cover potential miss- ing modality in multimodal settings (Rubin, 1976):
Missing Completely At Random (MCAR): A modality is missing completely at random the missingness process is independent of any other variable.
Missing At Random (MAR): The missingness mechanism depends on observed variables only. Missing Not At Random (MNAR): Missingness depends on unobserved variables.
In Figure 2 (right), we describe the missingness mechanisms as dependent on C, a set of covariates that determine the missingness mechanism. Note that C' may include of the modalities of inter- est, e.g., whether X₂ is observed may depend on the realization of X1, In general, the set C may differ for each modality depending on the data-generating process.
Missingness-induced distribution shifts. Missing modality X; and/or missing label Y can induce distribution shifts between the source and the target distributions. For example, if modality is observed in the target environment only if another one meets some criterion, then this distribution may not match the source distribution. Theoretically, we know that a non-MCAR missingness mechanism induces distribution shifts (Liu et al., 2023b; Zhang et al., 2023), i.e., the observed distribution differs from the underlying distribution. Critically, models trained and evaluated on the observed distribution are statistically biased estimates under any other missingness process.
For instance, consider an autonomous vehicle setting in which video and LiDAR are two modalities of interest. If the LiDAR sensor randomly fails, the missingness is MCAR. However, as previously mentioned, LiDAR may be more likely to malfunction under extreme weather conditions. If these conditions can be extracted from the video modality, the video cannot capture the variable that explains the LiDAR dysfunction (e.g., temperature), then unobserved variable. In the last two scenarios,
may assume MAR patterns. However, if
LiDAR would be MNAR, it is dependent focusing solely on complete samples would exclude all cotriie-condition settings with critical real- world repercussions.
A common and often implicit assumption in the multimodal literature is the absence of missing- ness, which corresponds to either MCAR mechanism (S ②2) stable missingness process between the source and target environments, i.c., the source and target distributions are the same (Noou ) In other words, not adjusting for missingness assumes that the missingness process is uninformative or will remain the same in the target environment.
Prior works focus on improving the robustness of multimodal models when performance may de- grade due to modality missing in the target distribution, i.e, the source distribution reflects the true distribution while the target may present missingness Quouree Ω f Ω Barget
Our work questions the applicability of these assumptions where modality collection costly. While missingness may result in distributional shift (Zhou et al., 2023), emphasize that demonstrating the value of a modality in the source environment may lead to increased collection of this modality in the target environment, inducing a distribution shift akin Assumption A.
Assumption A (Multimodal analysis informs data collection). Demonstrated multimodal perfor- mance gain induces shift in the missingness process the larget, i.e. Qzounce + Qiarget Ω.
We focus on settings in which historical data used to train a model are marked by missingness. Under such settings, aim to: (i) identify which modalities are informative and may consequently be collected in the target environment, and (ii) train models using the observed source data that generalize to the target environment where the modalities are fully observed.
4

---

Published as a conference paper at ICLR 2026
4 IS THIS MODALITY INFORMATIVE?
We aim to assess whether partially missing modality would be informative if fully observed. To this end, we introduce ICYM²1 (In Case You Multimodal Missed It), a framework for correcting model performance trained modality complete samples where all modalities and labels are ob- served (to to estimate the predictive utility of the partially missing modality if it were observed for the whole population (S2). Additionally, we propose correction to derive unbiased estimates of the ceermation-theorete utility of a modality, using Oohe- We rely on Partial Information Decompo- sition (PID) (Williams Beer, 2010) bounds introduced by Bertschinger et al. (2014) for this task, which quantifies the information value of a target of interest captured by two input variables.
Correction. We propose an Inverse Probability Weighting (IPW) approach (Robins et al., 1994), which reweights samples according to their probability of being observed. Under Assumption B that relaxes the common MCAR assumption made in the multimodal literature, IPW recovers unbiased estimates of the true distribution, enabling learning and evaluation on the true distribution from observed samples. IPW-adjustment critical for both training and evaluation multimodal models under missingness. IPW-adjusted training results in a model trained to infer the underlying distribution S2, while correction of the evaluation allows for measuring performance on Q, despite evaluating the model only samples from the observed distribution Ω obbs
Assumption B (MAR and Positivity). The missingness mechanism is MAR, and pn (M1 M2 0, My = 0|C)= O.
4.1 MOTIVATING EXAMPLE
We consider the common multimodal example of learning bit-wise logic operators (Bertschinger al., 2014; Harder et al., 2013; Liang al., 2024a). We generate 10, 000 points with two modal- ities drawn from Bernoulli distributions (p 0.5). The output state Y is defined using the binary operators AND, OR, and XOR of input bits X1 and X2. In this setting, we induce missingness M2 in X2 and Y as function of X1 (MAR): M2 ~ Bern(0.6X, + 0.2), resulting in 50% missingness in X2. We investigate the impact of missingness on current strategies for evaluating the predictive and information-theorrtic utility of a given modality.
Estimating performance for informativeness. A common practice to measure the predictive value of adding a modality is through ablation studies where practitioners train models using different modalities on the subset of samples where all modalities are observed (22m.) First, unimodal models f(;) are trained to approximate pa., (y I.), Vi E {1,2} and multimodal model f to approximate PR,, (y) x2) on the same observed dataset. The performance is then compared in a hold-out set, sampled from Pyhc- The informativeness of a modality is measured by the relative performance gain of the multimodal model over the unimodal model. However, multimodal models can underperform their unimodal counterparts due to data characteristics (Zhang et al., 2024) and learning dynamics (Wang al., 2020b; Zhai et al., 2024). Thus, relying solely on performance proxy for informativeness, particularly under distribution shifts, can be misleading.
Partial Information Decomposition (Bertschinger et al., 2014). As an alternative estimating performance, existing works have decomposed the informativeness associated with each modal- ity (Liang et al., 2024a). Bertschinger al. (2014) formalized this decomposition by analyzing the total (three-way) mutual information [(Y (X))X2) (McGill, 1954; Te Sun, 1980), a measure of dependency between the target variable Y and the modalities (X,, X2). This quantity can be decomposes into four components: shared information (information both X1, X2 share about Y), unique information (information only X, has about Y), unique information (information only X₂ has about Y), and complementary information (information about Y that requires both X; and X₂) as follows:
T(Y: : (X1, X2)) = ST(Y: Xx +UI(Y : x
UICY: x2\x)+ CI(Y: X2)
shared information
unique information
unique information 2
complementary information
5

---

Published as a conference paper at ICLR 2026
Bertschinger et al. (2014) specifies how to estimate these quantities (see Appendix B); for instance, the unique information between Y and X1 can be estimated using the following:
UI(Y : x2) = min (Y (X1, X2)) - (Y X2)), gEdn
where An is the set qX, = Zz, Y v) = Pg (X, = Ia Y y) Vz, E N, E ViLE {1,2), that is, the set of joint distributions over X,, Y) that match the true two-way data distributions. Note that the objective function requires minimization with respect to the three-way mutual information. Importantly, Bertschinger et al. (2014) demonstrates that the solution to any one objective specifies an optimum solution for all other quantities.
Prior work that relies on this Partial Information Decomposition (PID) to attribute information- theoretic value implicitly assumes that Puource fuoorce
Q Ω. Instead, we evidence the limitations of these strategies performed on Dogs * f when the target decomposition is Qurget Q2 i.e., the true generating mechanism.
Table 1. Impact of missingness on multimodality information for bitwise logic operators. Parentheses
denote standard deviation across batches.
AUROC
Information Decomposition
X1
X₂
X1 + X2
Unique
Unique
Shared
Complementary
XOR OR AND D
Oracle
0.83 (0.01) 0.84 (0.01) 1.00 (0.00)
0.05 (0.00)
0.03 (0.00)
0.26 (0.00)
0.47 (0.00)
Observed
0.66 (0.01) 0.93 (0.01)
1.00 (0.00)
0.44 (0.00)
0.00 (0.00)
0.15 (0.00)
0.36 (0.00)
ICYM'I
0.83 (0.01) 0.85 (0.02)
(0.00)
0.03 (0.00)
0.03 (0.00)
0.27 (0.00)
0.45 (0.00)
Oracle
0.84 (0.01) 0.83 (0.01) (0.00)
0.04 (0.00) 0.05 (0.00)
0.27 (0.00)
0.46 (0.00)
Observed
0.95 (0.01) 0.77 (0.01) (0.00)
0.01 (0.00)
0.15 (0.00)
0.10 (0.00)
0.23 (0.00)
ICYMºI 0.85 (0.02) 0.82 (0.01)
1.00 (0.00)
0.03 (0.00)
0.02 (0.00)
0.27 (0.00)
0.50 (0.00)
Oracle 0.51 (0.02) 0.49 (0.01) 1.00 (0.00)
0.00 (0.00) 0.00 (0.00) 0.00 (0.00)
0.99 (0.00)
Observed 0.52 (0.02) 0.80 (0.02)
1.00 (0.00)
0.34 (0.00) 0.07 (0.00) -0.07 (0.00)
0.62 (0.00)
ICYM'1 0.53 (0.03) 0.49 (0.03) 1.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.01 (0.00)
0.96 (0.00)
As a motivating example, we analyze the impact of missingness on estimating PID) for unidimen- sional modalities with a bitwise-logic outcome (AND, OR, and XOR). Table (left) presents the discriminative performance associated with neural networks trained on each individual modality and their combination under three scenarios: (i) access to all data (Oracle), (ii) focusing only on datapoints with all covariates observed (Observed), and (iii) adequately accounting for missingness (ICYM2I using IPW to adjust 22os + 12, by modeling the missingness mechanism), as proposed in Section 4.2. Table I (right) presents PID, discussed in Section 4.3 under the same scenarios, demonstrating how information decomposition is also biased due to missingness.
Specifically, relying on Pats overestimates the performance of X; for OR but underestimates it for AND. Similarly, biased decomposition overestimates the informativeness of X, ("Unique I" compared to "Unique 2"") for OR. As X1 informs the missingness process, it indirectly informs the outcome of interest, despite the true underlying generative process being dependent on both. The use of IPW can correct for such bias under positivity, provided the IPW propensities can be estimated (i.e., the MAR assumption). We study sensitivity to this assumption in Appendix D, where we further evaluate the robustness of method under MCAR and MNAR, demonstrating robustness under MCAR.
We now formally describe two methods for reliably inferring the informativeness of modalities using (i) unbiased estimation of unimodal versus multimodal model performance using supervised learn- ing (ICYM21-learn) and (ii) high-dimensional autodifferentiable partial information decomposition II-PID- In addition, we demonstrate the need for IPW-adjuster evaluation as key element to determine modality informativeness using supervised learning.
4.2 ICYM'I-LEARN: ESTIMATING PREDICTIVE PERFORMANCE UNDER MISSINGNESS
Training. Under the MAR assumption, i.e., the missingness is fully explained by observed covari- ates C; that is, the probability of a data point being missing depends only on C. we propose to train the model with weighted loss using samples from Sats- The proposed IPW idj-seeested loss accounts for the distributional shift (1) + ?) by up-weighting undervobserved samples, as described in the following lemma.

---

Published as a conference paper at ICLR 2026
Lemma (IPW Training). The loss function computed on the observed data li (21, y) can be reweighted to approximate the target loss (1 F2, follows:
1
12(21,
p(m1, m2, My C) In (21, (21,,22,3 v)
where p(m1, m2, my C) is the probability of missingness, given the covariates C.
Evaluation. Existing work suffers from analogous bias in model evaluation, by relying hold- out set from the observed distribution (Rebs) To estimate a given metric under the true underlying distribution, must correct this metric using similar correction previously described. For instance, Li et al. describes how to correct for both AUC and Brier score using IPW.
Corollary (ICaM)I-eaa)), Consider a model f trained and evaluated on data drawn from Dobs To correct the model and estimate its performance on D, one must correct both its training and evaluation following the previous corrections.
4.3 ICYM'I-PID: PARTIAL INFORMATION DECOMPOSITION FOR MULTIMODAL INFORMATIVENESS
Obtaining the partial information decomposition bounds corresponds to estimating the distribution q An that minimizes three-way mutual information (see Bertschinger et al. (2014), Lemma 4), where Ag is the set of all distributions over (X1, X2, Y) such that the two-way joints between X; focus solving:
and Y match the true data-generating distribution. Formally,
min qeôp
(Y X2))
To operationalize this minimization, Liang et al. (2024a) proposes to minimize this quantity using the observed samples:
An ~ {g exp(f( f2(x2)) q(x;,y) pr (II, v) Va: E X, y E Vvii E (1,2))
where o is ce-parametrization of Pots using neural networks. To enforce the marginals of to match po..* Liang et al. (2024a) proposes an iterative process using a Sinkhom-Knopp proce- dure (Knight, 2008).
Under missingness, we observe samples from 2,bs rather than Q. To unbias the PID measures in this setting requires adjusting for the Potss + Ω shift. Effectively, approach corrects the previous optimization to account for the distribution shift induced by MAR missingness.
First, introduce corrected mutual information computation ensure that optimize biased estimate of the three-way mutual information under Ω using samples from 2oo. (see Ap- pendix A for the complete proof).
Lemma 2 (Corrected mutual information).
p(m1, m2)
(1 (#1, 22, v) pa 2 (1)
1P (Y X2), Ex Ex (21,22) y~PPa (1)))))2)))
log
p(m1, m2 1 .2.
Then, ICYM uses the following projection set to operationalize PID-bound estimation while accounting for missingness:
An ICYMMM1 z bg x exp(fi(z1) fe(22)) ((2,,)) y) (.)
(1,2)} =(q (xp(f((x1) fe(#2)) q(z.. y) IPW pa. (Po... (y. 2)) E N.y E V,i e {1,2]}
where (p) is an IPW correction for the shift + q using samples from and Ps is a re- parametrization of Ω using neural networks and learned using samples from 2,ps via the weighted loss introduced in Lemma 1. Importantly, the proposed correction is agnostic to the parametrisation of q. We note that these mutual information-based quantities can be equivalently estimated using entropy-based measures (see derivations in Appendix B).
Estimating PID under missingness therefore consists in optimizing:
min JPW (Y X2),
ge ge
7

---

Published as a conference paper at ICLR 2026
Similarly Liang al. (2024a), the unbiased PID estimation framework consists of the following steps (detailed in Appendix C):
1. Model the missingness mechanism. Train a model estimate the probability of missing- ness given C' to obtain IPW weights for correcting the distribution shift.
2. Train corrected unimodal and multimodal models. Train each model (fi(z)), f2 (122, and fx11 using the IPW-corrected loss introduced in Lemma These models can be trained with flexible inductive biases tailored to the application at hand.
3. Solve PID optimization. Estimate q A ICYMPI minimizing (Y (x)), where is parameterized by the product of two unimodal networks. To match the marginal of to the estimated probabilities of the corrected unimodal neural networks, we apply modified Sinkhorn Knopp procedure (Knight, 2008) using IPW-correeted unimodal distributions" Our framework is agnostic to the choice of parametrization of q. provided that calibrated probabilistic scores learned.
4. Estimate the PID components. Given q, compute PID quantities using the bounds Bertschinger et al. (2014), corrected via the IPW-correction introduced in Appendix A.
5
EXPERIMENTS
To better understand the connection between performance, information decomposition, and miss- ingness, we propose simulation (detailed in Appendix E), semi-synthetic studies that reflect real-world missingness mechanisms (see Appendix F), and a real-world case study.
5.1 SIMULATION AND SEMI-SYNTHETE EXPERIMENTS
Table 2. Comparison between estimated AUC performance under the diffcrent training and evaluation corrections and oracle performance on Q. e denotes the RMSE between estimated and oracle AUCs.
Evaluation
Observed 2y
Underlying
0.7
0.7
Dracle
Model
d= 0.058 (0.009)
0.050 (0.011)
0.045 (0.003)
0.025
Xs
Training
0.037 (0.003)
0.036 (0.002)
0.5
0.5
0.8
Oracle AUC
0.7
0.7
^
Oracle
c 0.035 (0.002)
0.017 (0.001)
0.0 0.041 (0.002)
0.015 (0.002)
(0.002)
0.015 (0.001)
0.5
0.5
In Table 2, each point represents the estimated performance value for simulation under the training and evaluation IPW-corrections and the oracle performance, i.e., a model trained and tested
Prior work typically matches g(v v:) to Pette (1) æ:), which yields biased PID estimates under missing-
8

---

Published as a conference paper at ICLR 2026
on S2 Specifically, columns reflect evaluation conrection, while rows reflect training correction. These results underline the importance of correcting both training and evaluation, as proposed in ICYM²I, to best align with the performance one would obtain on ft, as shown by the smallest Root Mean Squared Error (RMSH) observed when both corrections are applied. This observation shows that the proposed ICYM²I best estimates the performance associated with each modality, despite relying on Puts Appendix E echoes the same observation when estimating PID.
The semi-synthetic experiments examine the effect of enforcing increasing missingness on the performance and information decomposition of UR-FUNNY (Hasan et al., 2019) and hateful memes (Kiela et al., 2020), two foundational real-world datasets used in the multimodal litera- ture for affective computing and content moderation. Table 3 summarizes the effect of enforcing 70% missingness on estimating multimodality informativeness across these datasets, demonstrating the generalizability of our proposed strategy across real-world datasets. Appendix F further illus- trates the robustness of the methodology under different levels of missingness in these datasets and explores MNAR patterns.
Table 3. Impact 70% missingness on multimodality information for UR- UR-FUNNY (Hasan et al., 2019) and Hateful Memes (Kiela et al.. 2020). Parentheses denote standard deviation across batches.
AUROC
Information Decomposition
Text
Image/Video
Image Text
Uniquews
Uniqueimage
Shared
Complementary
Memes UR-FU.
Oracle 0.68 (0.01)
0.60 (0.02)
0.69 (0.02)
0.10 (0.00) 0.02 (0.00)
0.00 (0.00)
0.00 (0.00)
Observed
0.61 (0.03)
0.54 (0.04)
0.63 (0.03)
0.05 (0.00)
0.00 (0.00)
0.03 (0.00)
0.00 (0.00)
ICYM'I
0.66 (0.03)
0.57 (0.04)
0.62 (0.04)
0.07 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
Oracle 0.71 (0.01)
0.57 (0.01)
0.72 (0.01)
0.09 (0.01) 0.00 (0.00) 0.04 (0.00)
0.05 (0.01)
Observed 0.68 (0.02)
0.61 (0.02)
0.71 (0.02)
0.13 (0.00) 0.04 (0.00)
0.01 (0.00)
0.00 (0.00)
ICYMºI 0.67 (0.02)
0.61 (0.02)
0.71 (0.02)
0.10 (0.00)
0.01 (0.00)
0.02 (0.03)
0.03 (0.01)
5.2 CHEST RADIOGRAPHS ARE UNINFORMATIVE OVER ELECTROCARDIOGRAM FOR STRUCTURAL HEART DISEASE DETECTION.
While our core contribution is methodological, this section illustrates how ignoring missingness can lead to biased estimates of the informativenesss of a given modality in real-world setting where modalities are commonly missing. Specifically, we study structural heart disease (SHD), a set of conditions that affect the heart's physiology, which is typically diagnosed using transthoracic echocardiograms (TTEs) (Writing Committee Members et al., 2021). However, TTEs are often un- derutilized in the United States due to diagnostic stewardship and competing financial incentives (Pa- polos et al., 2016). Prior work using unimodal models with common modalities in electrocardio- grams (ECGs) (Elias et al., 2022; Ulloa-Cerna et al., 2022) and chest radiographs (CXRs) (Bhave et al., 2024) has demonstrated that non-TTE modalities can detect structural heart disease labels. However, CXRs are not systematically collected in conjunction with ECGs, leading to systematic missingness patterns. We therefore evaluate ICYM2 on this clinical task to assess the informative- ness of CXRs for diagnosing SHD, despite their missingness.
Dataset. Our study population comprises a retrospective cohort of 98,397 adult patients who re- ceived an ECG and a TTE within year of each other. The population has 20.56% SHD prevalence. In this cohort, 12.587 patients (12.79%) have recorded CXRs. For subjects with multiple echocar- diograms, we select the first TTE to model opportunistic screening with non-TTE modalities. All data were collected from Columbia University Medical Center, large academic medical system in New York City, between 2008 and 2022. Data are split temporally, with subjects with TTES collected on or after 2018 (n = 40, 734) allocated to the test set. All data were de-identified, retro- spective, and collected for clinical purposes within an academic hospital system, with Institutional Review Board approval. Appendix G contains further details regarding preprocessing, embedding generation, and the ICYM ICYM'Iimplene Iipllmmmnatin..
Results. Table 4 presents the performance of each uni- and multimodal model, along with the associated information decomposition. While both the observed and corrected analyses demonstrate the importance of ECG in modeling SHD, the corrected results raise questions about the information gain associated with CXR. Naive decomposition suggests the unique information in CXRs accounts

---

Published as a conference paper at ICLR 2026
for about 5% of the total information. However, ICYM reduces this unique contribution to 1.8% while increasing estimates of shared information between ECG and CXRs for SHD detection. In contrast to domain knowledge. where ECGs capture electrophysiology while CXRs capture structure and anatomy, two distinct aspects of cardiac health, the corrected complementary and shared results, and low unique information of CXRs suggest that CXRs are not independently useful for SHI) diagnosis. Note that our results indicate that the multimodal model performs slightly worse than the unimodal ECG model, reflecting the potential risk of overfitting associated with a large number of features.
Table 4. Informativeness of ECG and CXR modalities on model-based structural heart disease detec- tion. Parentheses denote standard deviation across batches (n 1024).
AUROC
Information Decomposition
ECG
CXR
ECG + CXR
Uniqueaco
UniquecxR
Shared
Complementary
Observed 0.83 (0.01) 0.72 (0.02)
0.82 (0.01)
0.11 (0.00) 0.01 (0.00) 0.10 (0.00)
0.00 (0.00)
ICYM'T 0.82 (0.01) 0.73 (0.02)
0.83 (0.01)
0.07 (0.00) 0.01 (0.00) 0.48 (0.00)
0.01 (0.00)
6 DISCUSSION
This work formalizes the issue of missing modalities in multimodal settings. We emphasize that existing work commonly overlooks missingness by discarding samples with any missing modal- ity during curation, or implicitly assumes that the missingness mechanism remains stable when a model is deployed in the target environment. Our work formalizes this problem and demonstrates its ubiquity in the multimodality literature. Most critically, prior work ignores that any perceived informativeness of a modality may result in increased rates of data collection, inducing different missingness patterns at deployment. Our work, therefore, introduces ICYM'I, correction to timate the information gain associated with a partially observed modality. Our results demonstrate the methodology's capacity correct for biases introduced by missingness across synthetic, semi- synthetic, and real-world multimodal datasets. Finally, we demonstrate the practical utility of this methodology on a healthcare dataset, showing the divergent conclusions that one would reach if missingness were ignored. Our work highlights the critical importance of missingness in multi- modal research and urges practitioners to pay particular attention to this issue by systematically collecting data with incomplete modalities and carefully modeling and accounting for missingness to enhance robustness.
Limitations. The key assumption in our work is that missingness is MAR. No theoretical guaran- tees exist under MNAR patterns. While distinguishing these assumptions is empirically untestable, practitioners should ensure that this assumption is appropriate for their data. Importantly, MAR is less restrictive than the implicit MCAR assumption made in the multimodal literature, and does not require distributional assumptions that one must assume to tackle MNAR patterns. Additionally, our work is based on Partial Information Decomposition (PID) (Bertschinger et al., 2014), which focuses on two input modalities. In practice, practitioners could consider one-vs-all approach to inform modality informativeness using our method. However, extending the decomposition to more than two modalities remains an open challenge (Griffith & Koch, 2014; Kolchinsky, 2022), where notions of mutual information are not well-defined beyond threc-way mutual information. As in prior work on PID-based measures of information gain on high-dimensiooal data (Liang et al., 2023; 2024a), the quality of the representations used may impact the measures returned by (CYMPL. We ensure that our probabilistic estimates are calibrated mitigate such challenges. Additionally, PID is designed for datasets with paired modalities, where the input modalities are associated with an instance (e.g., a patient and their associated chest X-ray and electrocardiogram) consequently, our method does not extend to unpaired settings. Finally, our proposed method relies on notion of instance for which all modalities are observable. Extending this method when there is no notion of an instance, i.e., unaligned modalities, could be considered but would require different inductive biases to model the underlying unimodal and multimodal probabilities.
10

---

Published as a conference paper at ICLR 2026
Ethics statement. Our work demonstrates the impact of missingness performance estimates in multimodal learning. We demonstrate the utility of our method in crucial healthcare use case. However, the methodology remains proof of concept that would require additional testing to be deployed in a real-world context. Our study is approved by the Columbia University Medical Cen- ter Institutional Review Board (IRB-AA (AR-974. We have extracted all data in HIPAA-complian servers, and our experiments are also conducted on HIPAA-compliant compute despite being dei- dentified for extra caution. While beyond the scope of this work, modality completeness is not uniform across demographic subgroups and can manifest in data collection policies, such as differ- ential access to care based on insurance status. Our method could provide important insights into the utility of multimodal predictions in such settings.
Reproducibility statement. Theoretical proofs are provided in Appendix All code for apply- ing the proposed ICYMII and reproducing all synthetic and semi-synthetic results presented in this work publicly available on Github'. A summary of the computational resources required to reproduce our results is given in Appendix H.
ACKNOWLEDGMENTS
VJ and SJ acknowledge partial support from NIH RO1 MHL37779. YSC and acknowledge partial support from Google Research Scholar Awards. SJ acknowledges partial support from the SNF Cen- ter for Precision Psychiatry and Mental Health at Columbia. Any opinions, findings, conclusions, or recommendations in this manuscript those of the authors and do not reflect the views, policies, endorsements, expressed or implied, of any aforementioned funding agencies/institttinn..
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jía, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens. Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing heterogeneous systems, 2015. URL https: /www tensorflow org/. Software available from tensorftow.org.
Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. Openface: open source facial behavior analysis tooliit. In 2016 IEEE winter conference on applications of computer vision (WACV), pp. 1-10. IEEE, 2016.
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay. Quantifying unique information. Entropy, 16(4):261121133 2014.
Shreyas Bhave, Victor Rodriguez, Timothy Poterucha, Simukayi Mutasa, Dwight Aberle, Kath- leen M Capaccione, Yibo Chen, Belinda Dsouza, Shifali Dumeer, Jonathan Goldstein, al. Deep learning to detect left ventricular structural abnormalities in chest x-rays. European Heart Jour- nal, pp. chad782, 2024.
Andreas K Buck, Ken Herrmann, Tom Stargardt, Tobias Dechow, Bernd Joachim Krause, and Jonas Schreyögg. Economic evaluation of pet and pet/ct in oncology: evidence and methodologic ap- proaches. Journal of nuclear medicine technology,
Lei Chen and Chungmin Lee. Predicting audience's 's laughter during presentations using convo- lutional neural network. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 86-90, 2017.
Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, and Binqiang Zhao. Pog: personalized outfit generation for fashion recommenda- tion at alibaba ifashion. In Proceedings of the 25th ACM SIGKDD international conference 2019.
knowledge discovery & data mining, pp.
hittp::/githhi com/ reAl M- lab/ 1CYM2

---

Published as a conference paper at ICLR 2026
Yi-Ting Chen, Jinghao Shi, Zelin Ye. Christoph Mertz, Deva Ramanan, and Shu Kong. Multimodal object detection via probabilistic ensembling. In European Conference on Computer Vision, pp- 139-158. Springer, 2022.
Ira Cohen, Fabio Gagliardi Cozman, Nicu Sebe, Marcelo Cesar Cirelo, and Thomas S Huang- Semisupervised learning of classifiers: Theory, algorithms, and their application to human- computer interaction. IEEE Transactions Pattern Analysis and Machine Intelligence, 26(12): 1553-1566, 2004.
Noah Cohen Kalafut, Xiang Huang, and Daifeng Wang. Joint variational autoencoders for multi- modal imputation and embedding. Nature machine intelligence, 5(6):631-642, 2023.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (eds.), Ad- vances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. URL https: www.inngg..uuurrsee neurips oc cc/pappr__iiie wtw.ffieespapr/ee/// Paperapppreee pdf.
Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. Covarep a collaborative voice analysis repository for speech technologies. In 2014 ieee international con- ference on acoustics, speech and signal processing (icassp), pp. 960-964. IEFF, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: A large-scals hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. leee, 2009.
Jacob Devlin, Ming- Ming-Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Pierre Elias, Timothy J Poterucha, Vijay Rajaram, Luca Matos Moller, Victor Rodriguez, Shreyas Bhave, Rebecca TT' Hahn, Geoffrey Tison, Sean A Abreau, Joshua Barrios, et al. Deep learning electrocardiographiic analysis for detection of left-sided valvular heart disease. Journal of the American College of Cardiology, 80(6))613-626, 2022.
Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, and Dorsa Sadigh. Physically grounded vision- languag models for robotic manipulation. 2024 IEEE,
IEEE International Conference Robotics and Automation (ICRA), pp. 2024.
Chongjian Ge, Junsong Chen, Enze Xie, Zhongdao Wang, Lanqing Hong, Huchuan Lu, Zhenguo Li, and Ping Luo. Metabev: Solving sensor failures for 3d detection and map segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8721-8731, 2023.
Ary L Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph Mietus, George Moody, Chung-Kan Peng, and Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 101(23):e21-=e220, 2000.
Virgil Griffith and Christof Koch. Quantifying synergistic mutual information. In Guided self- organization: inception, pp. 159-190. Springer, 2014.
Malte Harder, Christoph Salge, and Daniel Polani. Bivariate measure of redundant information.
Physical Review E B-Saitistical,, Nonlinear, and Soft Malter Physics, 87(1):012130, 2013.
Md Kamrul Hasan, Wasifur Rahman, AmirAli Bagher Zadeh, Jianyuan Zhong, Md Iftekhar Tanveer, Phiil-P Morency, and Mohammed (Ehsan) Hoque. UR-FUNNY: A multimodal language dataset for understanding humor. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCNLP), pp- 2046-2056, 2056, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.19653/1/D11-1121
12

---

Published as a conference paper at ICLR 2026
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Seyedmajid Hosseini, Raju Gottumukkala, Satya Katragadda, Ravi Teja Bhupatiraju, Ziad Ashkar, Christoph W Borst, and Kenneth Cochran. A multimodal sensor dataset for continuous stress detection of nurses in a hospital. Scientific Data, 9()):55,, 2022.
Ming Hou, Jiajia Tang, Jianhai Zhang, Wanzeng Kong, and Qibin Zhao. Deep multimodal multi- linear fusion with high-order polynomial pooling. Advances Neural Information Processing Systems, 32, 2019.
Arda Inceoglu, Eren Erdal Aksoy, Abdullah Cihan AK, and Sanem Sariel. Fino-net: A deep mul- timodal sensor fusion framework for manipulation failure detection. In 2021 IEEE/RSJ interna- tional conference intelligent robots and systems (IROS), pp. 6841-6847. IEEE, 2021.
Arda Inceoglu, Eren Erdal Aksoy, and Sanem Sariel. Multimodal detection and classification of robot manipulation failures. IEEE Robotics and Automation Letters, (12):1396-144) 2023.
Mimansa Jaiswal and Emily Mower Provost. Privacy enhanced multimodal neural representations for emotion recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pp. 7985-7993, 2020.
Neil Jethani, Aahlad Puli, Hao Zhang, Leonid Garber, Lior Jankelson, Yindalon Aphinyanaphongs, and Rajesh Ranganath. New-onset diabetes assessment using artificial intellggence-enhanced elec- trocardiography, arXiv preprint 2200002005.02000, 2022.
Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard. Sicheng Hao, Benjamin Moody, Brian Gow, et Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10(1):1, 2023.
Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ring- shia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech multimodal memes. Advances in neural information processing systems, 33:2611-2624, 2020.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Philip A Knight. The sinkhom- algorithm: convergence and applications. SIAM Journal Matrix Analysis and Applications, 30( (0(11:261-275, 2008.
Artemy Kolchinsky. A novel approach to the partial information decomposition. Entropy, 24(3): 403, 2022.
Gueorgi Kossinets. Effects of missing data in social networks. Social networks, 28(3):247-268, 2006.
Lien Le. Thu Nguyen, Michael
Michael A Riegler, Pal Halvorsen, and Binh T Nguyen. Multimodal missing data in healthcare: A comprehensive review and future directions. Computer Science Review, 56: 100720, 2025.
Kw Kwanhyung Lee, Soojeong Lee, Sangchul Hahn, Heejung Hyun, Edward Choi, Byungeun Ahn, and Joohyung Lee. Learning missing modal electronic health records with unified multi-modal data embedding and modality-aware attention. arXiv preprint arXiv:2305.025500 2023.
Mingcheng Li, Dingkang Yang, Xiao Zhao, Shuaibing Wang, Yan Wang, Kun Yang. Mingyang Sun, Dongliang Kou, Ziyun Qian, and Lihua Zhang. Correlation-decoupeee knowledge distillation for multimodal sentiment analysis with incomplete modalities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22458-12468, 2024.
Pin Li. Jeremy MG Taylor, Daniel E Spratt, R Jeffery Kamees, and Matthew J Schipper. Evaluation of predictive model performance of an existing model in the presence of missing data. Statistics in medicine, 40(15):3477-3498 2021.
13

---

Published as a conference paper at ICLR 2026
Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. Genet: Graph completion network for incomplete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intelligence, 45:8419- 84392, 2023.
Paul Pu Liang, Yiwei Lyu, Xiang Fan, Zetian Wu, Yun Cheng, Jason Wu, Leslie Chen, Peter Wu, Michelle A Lee, Yuke Zhu, et al. Multibench: Multiscale benchmarks for multimodal represen- tation learning. Advances in neural information processing systems, 2021 2021(DB1): (DB1):1, 2021.
Paul Pu Liang, Yun Cheng, Ruslan Salakhutdinov, and Louis-Philippe Morency. Multimodal fu- sion interactions: A study human and automatic quantification. Proceedings of the 25th International Conference Multimodal Interaction, pp. 425 435, 2023.
Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy Auerbach, Faisal Mahmood, al. Quantifying & modeling multimodal interactions: An information decomposition framework. Advances in Neural Information Pro- cessing Systems, 36, 2024a.
Paul Pu Liang, Akshay Goindani, Talha Chafekar, Leena Mathur, Haofei Yu, Ruslan Salakhutdi- nov, and Louis-Philippe Morency. Hemm: Holistic evaluation of multimodal foundation models. Advances in Neural Information Processing Systems, 37:42899-429400, 2024b.
Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. Foundations trends in multimodal machine learning: Principles, challenges, and open questions. ACM Compuling Surveys, 56(10): 1-42, 2024c.
Ronghao Lin and Haifeng Hu. Missmodal: Increasing robustness missing modality multimodal sentiment analysis. Transactions of the Association for Computational Linguistics, 11:1686- 1702, 2023.
Hong Liu, Dong Wei, Donghuan Lu, Jinghan Sun, Liansheng Wang, and Yefeng Zheng. M3ae: Multimodal representation learning for brain tumor segmentation with missing modalities. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 1657-1665, 2023a.
Jiashuo Liu, Tianyu Wang, Peng Cui, and Hongseok Namkoong. On the need for a language de- scribing distribution shifts: Illustrations on tabular datasets. Advances in Neural Information Processing Systems, 36:51371-511 2023b.
Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. Smil: Multimodal learning with severely missing modality. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 2302-2310, 2021.
Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal transform- ers robust to missing modality? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18177-18186 2022.
Yingbo Ma. Mehmet Celepkolu, Kristy Elizabeth Boyer, Collin Lynch, Bric Wiebe, and Maya Israel. How noisy is too noisy? the impact of data noise on multimodal recognition of confusion and conflict during collaborative learning. In Proceedings of the 25th International Conference on Multimodal Interaction, pp. 326-335, 2023.
Daniele Malitesta, Emanuele Rossi, Claudio Pomo, Tommaso Di Noia, and Fragkiskos D Malliaros. Do we really need to drop items with missing modalities multimodal recommendation? Pro- ceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 3943 0948,,998..11 2024.
William McGill. Multivariate information transmission. Transactions of the IRE Professional Group on Information Theory, 4(4):93-111, 1954.
Kaden McKeen, Laura Oliva, Sameer Masood, Augustin Toma, Barry Rubin, and Bo Wang. Ecg-fm: An open electrocardiogram foundation model. arXiv preprint arXiv:2408.05/77 2024.
14

---

Published as a conference paper at ICLR 2026
Antoine Miech, Dimitri Zhukov, Bean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic. Howtol00m: Learning a text-video embedding by watching hundred million narrated video clips. In Proceedings the IEEE/CVF international conference on computer vision, pp- 2630-2640, 2019.
Karthika Mohan and Judea Pearl. Graphical models for processing missing data. Journal of the American Statistical Association, 166653)):003--133, 2021.
Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, Andrew Y Ng, et al. Multi- modal deep learning. In ICML, volume 11, pp. 689-696, 2011.
Changdae Oh, Zhen Fang, Shawn Im, Xuefeng Du, and Yixuan Li. Understanding multimodal Ilms under distribution shifts: An iiformation-theoretiie approach. arXiv preprint arXiv22502.00577, 2025.
Alexander Papolos, Jagat Narula, Chirag Bavishi, Farooq A Chaudhry, and Partho P Sengupta. Us hospital use of echocardiography: insights from the nationwide inpatient sample. Journal of the American College of Cardiology. 67(5):502-51) 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Jeffrey Pennington, Richard Socher, and Christopher D) Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532-1543, 2014.
Sam Perochon, Matias Di Martino, Kimberly LHH Carpenter, Scott Compton, Naomi Davis, Brian Eichner, Steven Espinosa, Lauren Franz, Pradeep Raj Krishnappa Babu, Guillermo Sapiro, et al. Early detection of autism using digital behavioral phenotyping. Nature Medicine, 29(10)22489- 2497, 2023.
Matthew Phelan, Nrupen A Bhavsar, and Benjamin A Goldstein. Illustrating informed presence bias in electronic health records data: how patient interactions with health system can impact inference. EGEMs, 5(I), 2017.
Stephen M Pizer, Philip Amburn, John Austin, Robert Cromartie, Geselowitz, Trey Greer, Bart ter Haar Romeny, John B Zimmerman, and Karel Zuiderveld. Adaptive histogram equaliza- tion and its variations. Computer vision, graphics, and image processing, 39(3):355-363 1987.
M. Reyna, N. Sadr, A. Gu, E. A. Perez Alday, C. Liu, S. Seyedi. A. Shah, and G. Clifford. Will two do? varying dimensions in electrocardiography: The physionee/commputig in cardiology challenge 2021 (version 1.0.3). PhysioNet, 2022. doi: 10. 10.13026/34va-7q144
Matthew A Reyna, Nadi Sadr. Erick A Perez Alday, Annie Gu, Amit J Shah, Chad Robichaux, Ali Bahrami Rad, Andoni Elola, Salman Seyedi, Sardar Ansari, et al. Will two do? varying dimensions in electrocardiography:: the physionet/computing in cardiology challenge 2021. In 2021 computing in cardiology (CinC), volume 48, pp. 1-4. IEEE, 2021.
James M Robins, Andrea Rotnitzky, and Lue Ping Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89 (427):846 866, 1994.
Donald B Rubin. Inference and missing data. Biometrika, 63(3):581-592, 1976.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- formation processing systems, 22122577-22292 2022.
Dhruv Shah, Blažej Osiński, Sergey Levine, et al. Lm-nav: Robotic navigation with large pre- trained models of language, vision, and action. In Conference on robot learning, pp. 492-504. PMLR, 2023.
15

---

Published as a conference paper at ICLR 2026
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp- 2556-2565, 2018.
Richard Sinkhorn and Paul Knopp. Concerning nonnegative matrices and doubly stochastic matri- ces. Pacific Journal of Mathematics, 21(2):343-345 1967.
Elizabeth A Stuart. Matching methods for causal inference: A review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics, 25(1), 2010.
Han Te Sun. Multiple mutual informations and multiple interactions in frequency data. Inf. Control, 46:26-45, 1980.
Luan Tran, Xiaoming Liu. Jiayu Zhou, and Rong Jin. Missing modalities imputation via cascaded residual autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1405-1414, 2017.
Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ira Ktena, al. Towards generalist biomedical ai. NEJM AI, 1(3):Aloa2300138, 2024.
Alvaro E Ulloa-Cerna, Linyuan Jing, John M Pfeifer, Sushravya Raghunath, Jeffrey A Ruhl, Daniel B Rocha, Joseph B Leader, Noah Zimmerman, Greg Lee, Steven Steinhubl, et Re- chommend: an ocg-based machine learning approach for identifying patients at increased risk of undiagnosed structural heart disease detectable by echocardiography. Circulation, 14661):36-47, 2022.
Aayush Atul Verma, Amir Sacidi, Shamanthak Hegde, Ajay Therala, Fenil Denish Bardoliya, Na- garaju Machavarapu, Shri Ajay Kumar Ravindhiran, Srija Malyala, Agneet Chatterjee, Yezhou Yang, et al. Evaluating multimodal large language models across distribution shifts and augmenta- tions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5314-5324, 2024.
Qi Wang, Liang Zhan, Paul Thompson, and Jiayu Zhou. Multimodal learning with incomplete modalities by knowledge distillation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1828-1838 2020a.
Weiyao Wang, Du Tran, and Matt Feiszli. What makes training multi-modal classification networks hard? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12695- 12705, 2020b.
Paul L Williams and Randall D Beer. Nonnegative decomposition of multivariate information. arXiv preprint arXiv: 2515, 2010.
Sangmin Woo, Sumin Lee, Yeonju Park, Muhammad Adi Nugroho, and Changick Kim. Towards good practices for missing modality robust action recognition. In Proceedings of the AAAI Con- ference on Artificial Intelligence. volume 37. pp. 2776-2784, 2023.
Catherine M Writing Committee Members, Otto, Rick A Nishimura, Robert O Bonow, Blase A Carabello, John P Erwin III, Federico Gentile, Hani Jneid, Eric V Krieger, Michael Mack, et al. 2020 acc/aha guideline for the management of patients with valvular heart disease: a report of the american college of cardiology/american heart association joint committee on clinical practice guidelines. Journal of the American College of Cardiology, 77(4) e197, 2021.
Renjie Wu, Hu Wang, Hsiang-Ting Chen, and Gustavo Cameiro. Deep multimodal learning with missing modality: A survey. arXiv preprint arXiv:2409.07825 2024.
Aaron D Wyner. A definition conditional mutual information for arbitrary ensembles. Information and Control, 38(1):51-59, 1978.
16

---

Published as a conference paper at ICLR 2026
Shawn Xu, Lin Yang. Christopher Kelly, Marcin Sieniek, Timo Kohlberger, Martin Ma, Wei-Hung Weng, Atilla Kiraly, Sahar Kazemzadeh, Zakkai Melamed, et al. Elixr: Towards general purpose x-ray artificial intelligence system through alignment of large language models and radiology vision encoders. arXiv preprint 0rXiv:2308.013/7 2023.
Yihao Xue, Siddharth Joshi, Dang Nguyen, and Baharan Mirzasoleiman. Understanding the robust- ness of moulti-modal contrastive learning to distribution shift. In The Twelfth International Confer- ence on Learning Representations, 2024. URL settss:/openneeviee net www.:/ertttrttmneoeee forum?id- rt 4
Jiahong Yuan, Mark Liberman, et al. Speaker identification on the scotus corpus. Journal of the Acoustical Society of America, 123(5):8878, 2008.
Amir Zadeh, Minghai Chen, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. Tensor fusion network for multimodal sentiment analysis. arXiv preprint arXiv:1707.07250 2017.
Jiandian Zeng, Jiantao Zhou, and Tianyi Liu. Mitigating inconsistencies in multimodal sentiment analysis under uncertain missing modalities. In Proceedings of the 2022 conference on empirical methods in natural language processing, pp. 2924-2934, 2022.
Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, and Yi Ma. Investigating the catastrophic forgetting in multimodal large language model fine-tuning, In Conference on Parsimony and Learning, pp. 202-227 PMLR, 2024.
Haoran Zhang, Harvineet Singh, Marzyeh Ghassemi, and Shalmali Joshi. "Why did the model fail?": Attributing model performance changes to distribution shifts. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Pro- ceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 41550 41578. PMLR, 23-29 Jul 2023.
Peng-Fei Zhang, Yang Li, Zi Huang, and Hongzhi Yin. Privacy protection in deep multi-modal retrieval. In Proceedings of the 44th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 634-643, 2021.
Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Screna Yeung-Levy. Why are visually-grounde language models bad at image classification? In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: openreviewweeeeeeeeeee net / forum? id M-ddm-MWBBBl11 VYg.
Guanglin Zhou, Zhongyi Han, Shiming Chen, Biwei Huang, Liming Zhu, Salman Khan, Xin Gao, and Lina Yao. Adapting large multimodal models to distribution shifts: The role of in-context learning. arXiv preprint arXiv:2405.122117 2024.
Helen Zhou, Sivaraman Balakrishnan, and Zachary Lipton. Domain adaptation under missingness shift. In International Conference Artificial Intelligence and Statistics, pp. 9577-9606. PMLR, 2023.
Tongxue Zhou, Pierre Vera, Stéphane Canu, and Su Ruan. Missing data imputation via conditional generator and correlation learning for multimodal brain tumor segmentation. Pattern Recognition Letters, 158: 18:1125-33. 2022.
17

---

Published as a conference paper at ICLR 2026
A PROOFS
This section provides the proofs for Lemma and Lemma 2.
Lemma The separable loss function computed on the observed data 1r (01, 22, 2) can be reweighted to approximate the target loss ( 32. y) as follows:
1
11 v) po (m1, m2, my C) D (2,,4)
where Pn (m1, m2, my C*) is the probability of missingness, given the covariates C.
Proof. The proof is analogous to that of Lemma 2, which show in detail, for any separable loss function ( I2, x2.
Lemma 2.
p(m1, m2) p(m,, m2/ar1 22, v)
pa (Z) 22, 21 Pa (31, )ra (v)
In X2)) = 2arpa (21,22)
log
y~Pa (1)
Proof. Let (m),
ln (2))
(1 x2 20 1)
En log
P 22 Pa
(111 v) log Po(211 (22(21,22)12(1)
Pal 22 v)
22) )
E E (111 go-pa (2)l
Pn (21,
Pa(T1, x2. 3) Pn F2 F2,
Pa (1. y) Pr (r x2 x2)
log
m =
Pq Po T2,
(1 Pn (2 32 x2)p (3))
=E2 (m) a2)) ax)
log
p
.r .12.1 (21.22200
p(m=0)
p(m log
pn (1 (2 (2)p
E2 (22)
p(m = 1111, v)
g~pa (y)l
That is, to estimate the mutual information under the true data distribution, we adjust for the shift in Pa (21, + Pa (Z1, 222 and sample y from the IPW-adjusted (parametrized approximations) of Pa (y) x2).
18

---

Published as a conference paper at ICLR 2026
B PARTIAL INFORMATION DECOMPOSITION (PID)
Partial information decomposition (PID Williams & Beer (2010)) consists in decomposing the total mutual information (MeGill, 1954; Te Sun, 1980) between target variable and two input vari- ables into information about the target variable that both input variables share ("Shared" informa- tion), only one input variable has ("Unique" information) and emerges from the interactions of both ("Complementary" information). Bertschinger et al. (2014) introduces bounds for these, reiterated below. In this Appendix, we express these bounds as entropy. First, Table 5 summarizes the nota- tions used in the literature and those used in our work.
Table 5. Quantities and associated variables. Note that the four information measures are approxima- tions.
Quantity
Bertschinger
ICYM
Input Variable
Y
X1
Input Variable
Z
X2
Target Variable
X
Y
Redundant / Shared Information
ST(X: Y 22
ST(Y: X2)
Unique Information (Input Variable
UI(Y X1\X2)
Unique Information (Input Variable 2)
UI(X:
(I(Y:X2AM
Synergistic/ / Complementary Information
CI(X:Y;2)
CI(Y: x2:22)
PID decomposition of the three-way mutual information T(Y : (X1,xx)) results in the quantities of interest follows:
1(Y : (Xi, = SI(Y
x1XX2) + UUIY x2)X)) + CC(Y x,;
Shared
Unique
Unique
Complementary
Let A be the space of all distributions over (X1, X2, Y) and let Q denote the true data distribution (without missingness) and define AQ := {q E A q(X, Y,,Y y) = Px( = x,, Y y) Va; E X,y y E V E [1,2)] That is, An is the set of all distributions over (X1, X2, x2, Y') such that the two-way joints between X; and match the true data-generating distribution. Equipped with this set, Bertschinger et al. (2014) provides the following bounds S1, UI, and Cl on the analogous quantities:
SI(Y X1: max CY X1: gEAn
max [Y,Y : X1) I,YY x2
qEán
max [[4(Y: (X1, - 1(Y: X2/X1)] 4(Y: x11X))]
gEAn
max [T,Y: (X1, - -I, x2(x)) + L(Y Xx
qEAn
: min T,(: :x1)22)
gEDn
min (Y (X1, X2)) - LY X2)]
qEAn
TI(: :2Xx)) = min (Y:: :2()7)
veDn
min
(Y (X1, X2)) - 1, : X1)]
geAn
CI(Y : Xx:X,) = (Y (X1, min (Y:: (X,, X2))
qeAp
In this context, Bertschinger et al. (2014) demonstrates that solving the optimization for q E An that satisfies one of the four conditions above is sufficient to obtain all the quantities of interest.
19

---

Published as a conference paper at ICLR 2026
Importantly. these bounds tight if there exists 90 An such that Cl Bertschinger et al. (2014) further shows that under common (but unverifiable) assumptions on the data-generating process, the inequalities
X1: 0.
tight for all q An- This results in a compelling argument for relying on these entities, as it suggests that it is not possible to decide whether com- plementary information exists when only the marginals (Y, X1) and (Y, X2) known.
Formulating PID quantities in terms of entropy. For stability, we propose to formalize the pre- vious bound terms of entropy, H.), defined for general distributions of X and Y as follows:
H(X) == Σ
p(z) logp(z)
DEA
H(Y,)) =
Σ
plu, x) log (p(y,
xE KEy
(p(y,x) p(xx)
H(Y
Σ
ply, x) log
NY
= x) - H(X)
Using these notations, the mutual information I(-) can be defined as:
I(Y: X) H(X) H(
= (X)) + H() X))
I(Y: x2 = (Y,x,, + -
H(x,)
The previous quantities of interest can then be derived as:
T(Y: (2), = T(Y Xi) + I(Y X1)
=H(Y) + -LXTT-((GG-))) HIG + X2) H(Y, H(XT
1(Y:X))
I(Y:X.Ix.)
H(Y + H(X,, X2) H(Y, X2)
where the first equation comes from the chain rule of mutual information (Wyner, 1978).
Similarly, we can get the expression for co-information Col(Y; X1;
C =I(Y : X1) + I(Y X2) I(Y X2))
= =(H(Y) - H + H(Y) X2)
((X:)))
T(V:X2)
(Y + H(G, X2) H(Y, X1, X²)]
=[H(Y) - [H(Y - H(X,)]] HHY [H(Y,X,) - H(x2)l
- -IHAY) + H(XI,
-H(Y) + H(X)) HXX2)
(XH,X,, X2), + x2) H(Y,,,) H(Y,
+ (T,X,X1,X,
20

---

Published as a conference paper at ICLR 2026
The PID bounds can then be expressed in terms of entropy:
SI(Y: X2) max (x;;((;x;;Xx)) gEAn
max (H2 + H (X,, + (x()2)
(X1, + HY, + +,
UI(Y: X1) min I,(Y: xx
nEAn
min [X, X2) + gEAn
-H(((
UI(Y X1 min Ig(Y: x22(11)
gEAR
min [Y, X1) + (X,,,, X2) HY, X1, X2) - Hx() rEAn
CI(Y: x1) =Y (X1, - min LY X2), gEAn
=HH + Hx( - Ho (1,YY,,, X1, X2)] min (Y) (X1, Hq (Y,
gEAp
where H.)() and (-) is the entropy of variable under probability distributions q and Q, respec- tively.
21

---

Published as a conference paper at ICLR 2026
C ICYM'I
C.I ICYMPI-LEARN
Table 6 summarizes the proposed approach estimate performance using 2obs Critically, one must correct both training and evaluation to obtain the performance on Ω.
Table 6. ICYM'I: Inverse probability weighting adjusted multimodal training and evaluation under missingness shift.
Evaluation distribution
Observed
Underlying
Pobs
Current practice
IPW-adjusted evaluation alone
Training
IPW-adjusted training alone
ICYM (IPW-adjusted training and evaluation)
C.2 ICYMºI-PID
The following describes the algorithm estimate PID under missingness.
Algorithm ICYM*1-PID
Require: X1, X2, Y ~ Pa
1: Step 1: Adjust Jons + Ω.
2: Estimate missingness mechanisms pn (M1, M2, My C") for IPW.
3: Step 2: Train corrected unimodal and multimodal models.
4: Training each model with weighting IPW-loss: f(y x²) poly 21), Vi {1, and
fly (t,)))) RS po (y #2)
5: # Step 3: PID optimization.
6: Initialize parameterizations for g: flly 2.), Vi {1,2}.
7: go(y) x2) + cxp (fily x1).f(l x)
8: while not converged do
9: for samples in batch do
10: # Ensure E ICYM²t by projection.
go æ1, F22 5
æ1, x2, (pp (y, xx.) P
12:
Estimate the loss (Y (X,, X2)) as batch sample mean.
13:
0 0+00- Vol,(Y (X1,X)))
14:
end for
15: end while
# Step 4: Estimate mutual information under Pa-
16: Estimate (Y (X,, X2)), and TY X2), Vi E (1,2) using adjustment in Appendix A.
17: PID(R) <- (CI(Y: X1; SI(Y Xi; X2), UI(Y XiXX),, X2)X
18: return PID(Ω.)
22

---

Published as a conference paper at ICLR 2026
The traditional SINKHORN-KNOP algorithm updates matrix to enforce its marginals to be unit vectors. In our work, we adapt the algorithm to enforce the marginals to match Pp -mrainniasss en- suring that (o(-) Åq. To ensure proper gradient propagation and reduce memory use, unrolled $ WARKHORN-KOPP (Sinkhorn & Knopp, 1967; Cuturi, 2013) algorithm. In the follow- ing, subscripts (ay to denote (0, (1,22) and Pz; to denote pe (y, The algorithm is detailed below:
the
Algorithm 2 Unrolled SINKHORN SIIIHORRN-KOOP2 KNOPP update
Require: q2122 Per Paz tolerance atol
1: 4x1
22 Uxim2
2: Qu 2x, 9xyæ2
3: while do
4: # Avoid update both exit conditions have been met.
5: if
8e PEi
S atol and
4ay Piz
atol then
PE1
Ps2
6: return (2102
7: end if
8:
# Update marginal.
9:
G2122 91122 P22
10:
92, f -Ep 12,02
# the other marginal still matches, done.
12:
if
2E PPL
S atol then
Pe1
13:
return (F102
14:
end if
15:
# Repeat for the other marginal.
41102 Pr1
G=1=2
4=1
17:
982
Σn, 21 22,
18: if
202 Pan
S atol then
Prg
return 12100 20: end if
19:
21:
end while
23

---

Published as a conference paper at ICLR 2026
D BIT-WISE LOGITS
In this section, we perform a sensitivity analysis of the logit setting presented in Section 4.1 under two additional missingness patterns: MCAR (Missing Completely at Random) and MNAR (Missing Not at Random). In this setting, we fit a logistic regression to estimate the probability of missingness on the observed modality, which is then used estimate the IPW. The performance estimates and PID for these two missingness processes are illustrated in Tables 7 and 8.
Since MCAR does not result in a distribution shift, one expects the same performance estimates for both the full and observed populations. Furthermore, in this setting, the IPW correction corre- sponds to a constant value, as any point has the same probability of observing both modalities. This correction also results in no change in performance estimates.
On the contrary, MNAR patterns do not guarantee similar behavior. Particularly, this missingness process may result in distribution shift that cannot be assessed or accounted for without assump- tions about the data distribution, as one does not observe the covariates that impact the missingness process. The results demonstrate that both the observed and corrected strategies result in biased estimates.
Table 7. Impact of missingness on multimodality information for bitwise logic operators under MCAR. Parentheses denote standard deviation across hatches.
AUROC
Information Decomposition
X1
X2
X1 + X2
Unique
Unique 2
Shared
Complementary
Oracle
0.83 (0.01) 0.84 (0.01) 1.00 (0.00) 0.05 (0.00)
0.03 (0.00) 0.26 (0.00)
0.47 (0.00)
XOR OR AND
Observed
0.83 (0.01) 0.83 (0.01) 1.00 (0.00)
0.05 (0.00)
0.03 (0.00)
0.23 (0.00)
0.52 (0.00)
ICYM'T
0.83 (0.01) 0.85 (0.01) 1.00 (0.00) 0.03 (0.00)
0.06 (0.00) 0.27 (0.00)
0.44 (0.00)
Oracle
0.84 (0.01) 0.83 (0.01) 1.00 (0.00) 0.04 (0.00)
0.05 (0.00) 0.27 (0.00)
0.46 (0.00)
Observed
0.84 (0.01) 0.84 (0.01) 1.00 (0.00)
0.06 (0.00)
0.03 (0.00) 0,25 (0.00)
0.51 (0.00)
ICYM'1
0.85 (0.01) 0.83 (0.01) 1.00 (0.00) 0.06 (0.00)
0.02 (0.00) 0.25 (0.00)
0.51 (0.00)
Oracle
0.51 (0.02) 0.49 (0.01) 1.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00)
0.99 (0.00)
Observed
0.51 (0.02) (0.02)
1.00 (0.00) 0.00 (0.00)
0.00 (0.00) (0.00)
0.95 (0.00)
ICYM'I 0.51 (0.02) 0.51 (0.02) 1.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00)
0.95 (0.00)
Table 8. Impact missingness on multimodality information for bitwise logic operators under MNAR. Parentheses denote standard deviation across batches.
AUROC
Information Decomposition
X1
X₂
X1 + X₂ Unique
Unique 2
Shared
Complementary
XOR OR AND
Oracle
0.83 (0:01) 0.84 (0.01) 1.00 (0.00)
0.05 (0.00) 0.03 (0.00) 0.26 (0.00)
0.47 (0.00)
Observed
0.93 (0.01) 0.67 (0.01) 1.00 (0.00)
0.45 (0.00)
0.00 (0.00)
0.17 (0.00)
0.33 (0.00)
ICYM'1
0.93 (0.01) 0.67 (0.01) 1.00 (0.00)
0.45 (0.00)
(0.000 0.17 (0.00)
0.33 (0.00)
Oracle
0.84 (0.01) 0.83 (0.01) 1.00 (0.00) 0.04 (0.00)
0.05 (0.00) 0.27 (0.00)
0.46 (0.00)
Observed
0.78 (0.01) 0.95 (0.01) 1.00 (0.00)
0.00 (0.00)
0.17 (0.00)
0.11 (0.00)
0.23 (0.00)
ICYM2I
0.78 (0.01) 0.95 (0.01) 1.00 (0.00)
0.00 (0.00)
0.17 (0.00)
0.11 (0.00)
0.23 (0.00)
Oracle
0.51 (0.02) 0.49 (0.01) 1.00 (0.00)
0.00 (0.00)
00.000
0.00 (0.00)
0.99 (0.00)
Observed
0.80 (0.02) 0.52 (0.02) 1.00 (0.00) (0.00)
0.07 (0.00)
0.00 (0.00)
0.61 (0.00)
ICYM'I
0.80 (0.02) 0.52 (0.02) 1.00 (0.00)
0.35 (0.00) 0.07 (0.00) 0.00 (0.00)
0.61 (0.00)
24

---

Published as a conference paper at ICLR 2026
E SYNTHETIC DATA RESULTS
Data generation. Our work builds on the example introduced in (Liang et al., 2024a), in which we enforce additional missingness. Three latent variables (21, 22, and ze) are drawn from multi- dimensional clustered data; the observed covariates are a concatenation of 2e and one of the other latent variables, as illustrated in Figure 3. Then, the outcome Y is generated as Y o (p1 + p2)((22 + (1 P1 P2) (2)) with the proportion P: E [0, 1] such that p1 + P2 S 1. We simulate datasets with varying values of Pl and P2- Then, we enforce 50% MAR missingness pattern in X2 by modeling the probability of missingness. We do this by clustering X1 into 100 groups using Kmeans. Then, the probability of missingness is generated using a random forest that regresses X
to predict ej. Y.
21
22
X1
X
Data generating processes for synthetic experiments. 2, denote latent vectors, while all other variables are observed. Filled point nodes are observed variables, while unfilled nodes are unobserved.
Figure 3.
Empirical setting. Data were split into three: 80% for training, 10% for validation, and the rest for testing. We consider neural networks with 2 hidden layers with 32 nodes, trained using an Adam optimizer (Kingma& & Ba, 2014) with learning rate of 0.001 over 100 epochs. Our evaluation relies on discriminative performance measured through AUROC.
Estimating predictive performance under Pobs Table 9 presents the PID obtained under different corrections. These results underline the importance of correcting both training and evaluation, as proposed in ICYM to best align with the PID one would obtain on Ω shown by the smallest Root Mean Squared Error (RMSE) observed when both corrections are applied. Note that this setting, rely on the true IPW correction that one would obtain with properly specified model, as the MAR setting is met.
Table 9. Comparison between estimated PID using training and PID corrections, and oracle PID on Ω. e denotes the RMSE between estimated and oracle PIDs. Underlying (Ω)
Observed (Ω, (9
0.3
c- 0.016 (0.002)
0.3
- 0.016 (0.003)
0.022 (0.004)
0.073 (0.004)
Orocie PID
0.038 (0.002)
P1D
(0.003)
0.2
0.019 (0.007)
0.2
= 0.022 (0.002)
Ω abs
0.1
Unique 2
Dn Corrrectiin
Shared
Complementary
0.00
0.05
0.20
0.25
0.00
0.05
0.10
0.25
Estimated
Estimated
0.3
C- 0.0015 (0.002)
0.3
- 0.001 (0.002)
0.018 (0.003)
0.018 (0.003)
Orazie PID
0.025 (0.002)
Dradle Dha
0.025 (0.003)
0.2
0 0.023 (0.002)
0.2
0.019 (0 003)
0.1
0.00
0.25
0.05
0.10
Estimated
Estimated PID
25

---

Published as a conference paper at ICLR 2026
F SEMI-SYNTHETIC DATA RESULTS
F.1 UR-FUNNY
We illustrate the impact of missingness on estimating the informativeness of different modalities on real-world data with UR-FUNNY (Hasan et al., 2019), multimodal dataset for humor detection from human speech used in affective computing. The dataset comprises text, audio, and visual modalities from 10 20 second videos sourced from TED talks, and the task is to detect whether a punchline would trigger laugh. Labels were generated using the markup "(Laughter)" (Chen & Lee, 2017) from the transcript.
Dataset. The processed dataset from MultiBench (Liang et al., 2021) a modality-complete dataset with 10.166 samples of paired audio, text, and vision embeddings, Audio embeddings were gener- ated with COVAREP (Degottex et al., 2014), text with Glove (Pennington et al., 2014), and visual features through the Facet (Yuan et al., 2008) library and OpenFace (Baltrušaitis al., 2016), and aligned using the Penn Phonetics Lab Forced Aligner (P2FA) (Yuan et al., 2008).
F.1.1 MISSING AT RANDOM (MAR)
Enforcing missingness. To explore the impact of missingness on informativeness, we simulate MAR missingness pattern on the audio and visual features given the textual modality. We vary the missingness from 30% to 70%, using the same mechanism as deseribed for synthetic data. This semi-synthetic setting enables the evaluation of the proposed correction as the missingness mech- anism is known. Note that the original dataset does not contain missing values, the source data (TED Talks) have transcripts, and data labeling was generated based on these transcripts. How- ever, settings with systematic transcripts rare and may reflect shift from the audio and textual modalities observed online for which such match may not exist.
Following the same empirical setting as in the synthetic experiment for each missingness rate, we measure the impact of missingness on PID decomposition. Figure 4 displays the PID values obtained under three strategies:
Results.
Observed: All quantities estimated using 2ass
ICYM²I: All quantities are estimated using 22t but corrected for the distribution shift through IPW.
Oracle: All quantities are estimated Ω.
This figure shows that the proposed strategy is consistently closer to the Oracle's PID values. This demonstrates that under Assumption A, the proposed correction yields better estimates of each modality's informativeness specifically, the audio-visual modality (Unique 1) carries in- formation.
0.100
Observed
PID Values
0.075
ICYM²1
Oracle
0.050
Uniquel
X
0.025
Unique2
Complementary
0.000
Shared
30.0%
40.0%
50.0%
60.0%
70.0%
Missigness Rate
Figure 4: Comparison between estimated PID values under increasing missingness in UR-FUNNY.
F.2 MISSING NoT AT RANDOM (MNAR)
Enforcing missingness. A central assumption of our method is that missingness is MAR. We pro- pose to analyze the impact of violations of this assumption, specifically the presence of MNAR
26

---

Published as a conference paper at ICLR 2026
patterns, on the quality of estimates obtained using our correction. To this end, simulate audio and visual missingness as a function the modality itself. Similarly to the previous analysis, we vary missingness from 30% to 70%. To estimate propensity in this setting, we rely a logistic regression model based on the fully observed modality.
Results. Figure 5 shows the PID values obtained under the three previously described strategies. Critically, the proposed correction leads to performance similar to the model without correction, the missingness probabílities cannot be estimated from the observed modality. This example illustrates the importance of assessing the plausibility of Assumption B in real-world settings, as no theoretical guarantees hold in such settings.
0.100
Observed
PID Values
0.075
ICYM'I
Oracle
0.050
Uniquel
0.025
Unique2 Complementary
0.000
Shared
30.0%
40.0%
50.0%
60.0%
70.0%
Missingness Rate
Figure 5: Comparison between estimated PID values under increasing missingness in UR-FUNNY.
27

---

Published as a conference paper at ICLR 2026
F.2 HATEFUL MEMES
We run experiments using the dataset from the Hateful Memes Challenge (Kiela et al., 2020), which investigates text-image multimodal reasoning in the context of hate speech detection in online memes. The dataset comprises text-image pairs with an associated label indicating hate speech.
Dataset. We utilize the Kaggle version of the Facebook Hateful Memes dataset, as referenced in the Holistic Evaluation of Multimodal Foundation Models (HEMM) (Liang et al., 2024b) repos- itory. Our analysis focuses on the 9,000 samples with associated labels. For each sample, em- beddings were extracted for both modalities using ResNet--50 (He et al., 2016) for images and BERT-base-uncased (Devlin al., 2019) model for text. The proposed ResNet-50 was pre- trained on ImageNet (Deng et al., 2009) with the final layer replaced to extract 2048-dimensional feature vectors, and BERT-base SERT-base-uncasee (Devlin al., 2019) extracts embeddings of dimen- sion 784 from the penultimate layer.
Enforcing missingness. Similarly to the previous experiment, we vary the missingness from 30% to 70% by enforcing the same MAR missingness mechanism on the text modality, given the image modality, as we assume not all memes may contain text. Note that memes in the dataset were created by combining text from collected online memes with images sourced from stock images Getty Images. Consequently, the dataset did not contain missing modality, but may not match the true distribution of memes would observe online.
As above, we measure the impact increasing percentages of missingness on PID esti- mates. While the missingness mechanism results in limited distribution shift, and therefore small differences in estimates between the corrected and observed strategies, the difference at 70% miss- ingness shows the superiority of the proposed methodology in recovering the Unique contributions.
Results.
0.15
Observed
X
ICYM²1
PID Values
0.10
Oracle
Uniquel
0.05
Unique2
Complementary
0.00
Shared
30.0%
40.0%
50.0%
60.0%
70.0%
Missingness Rate
Figure 6. Comparison between estimated PID values under inercasing missingness in Hateful Memes.
28

---

Published as a conference paper at ICLR 2026
G STRUCTURAL HEART DISEASE DATA PROCESSING
G.1 EMBEDDING GENERATION
We generate embeddings using modality-specific foundation models-ECG embeddings are gener- ated using ECG-FM (McKeen et al., 2024) and CXR embeddings with ELIXR-C (Xu et al., 2023).
All electrocardiograms were 10-second, standard 12-lead ECG signals collected and abstracted to 250 Hz, which we resampled to 500 Hz, and standard normalized by channel to match the inputs ECG-FM (McKeen al., 2024). We used the version of ECG-FM with weights pretrained on MIMIC-IV (Johnson et al., 2023; Goldberger et al., 2000) and PhysioNet 2021 (Reyna et al., 2021; 2022). We averaged the output feature embeddings along the temporal dimension and flattened them to produce vectors of length 768.
The chest radiographs used our study were all postero-anterior (PA) view CXRs. We extracted pixel values from the DICOM files grayscale images, center-cropped each image along the shorter dimension, applied contrast- contrast-limited adaptive histogram equalization (CLAHE) (Pizer et al., 1987) with clip limit of 0.2, and resized each image 1284 1284 pixels. All outputted embeddings were flattened to 4098-dimensional vectors.
G.2 IPW CORRECTION
To address missingness in the observed CXRs, we apply the proposed propensity- based correction. The propensity scores are obtained from logistic regression model using the ECG embedding, along with sex and age as predictors, serving as proxies for the socio-medical factors that influence whether CXR is collected. Controlling for these covariates aims to render the MAR assumption more plausible. In practice, all relevant covariates, even outside of modalities being modeled, can be used for the correction.
H COMPUTE INFRASTRUCTURE
All experiments were performed on server with an AMD EPYC 7313 CPU, 256 GB of memory, and two NVIDIA RTX A6000 GPUs, as well as a server with an Intel Xeon E5-2640 CPU, 128 GB of memory, and NVIDIA GTX Titan X GPU. Our software stack includes Python 3.12, PyTorch 2.2.1 (Paszke et al., 2019), and standard Python scientific libraries. Chest radiograph embeddings used Tensorflow 29 (Abadi al., 2015) and Tensorflow-Ters 2.19 based the requirements for the ELIXR models (Xu et al., 2023). Electrocardiogram embeddings were generated using an environment with Python 3.9 and fairseq-signals 1.0 to match the requirements for haiseq-signals and ECG-FM (McKeen et al., 2024). Generating embeddings for our structural heart disease data took approximately hours on our server with Titan X GPU. All synthetic experiments require 12 hours of compute time using one NVIDIA RTX A6000 GPU.
29


Reviews:
[{"review_id": "94jkcEIane", "reviewer": "Reviewer_7Hat", "summary": "The paper studies an issue in multimodal learning: when training/evaluating only on samples where all modalities are present, we bias both performance estimates and conclusions about each modality’s \"informativeness\". \n\nThe authors formalize missingness as a distribution shift and propose $\\text{ICYM}^2\\text{I}$, a double inverse-probability weighting (IPW) correction—apply IPW at training time and again at evaluation time—to recover estimates with respect to the target (fully observed) population. \n\nThey further adapt Partial Information Decomposition (PID) to high-dimensional settings to decompose a modality’s contribution into unique, shared, and complementary/synergistic information, and show that naive PID computed on complete-case data can be substantially biased.", "questions": "Please refer to Weaknesses", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- **Methodologically sound correction**. Leveraging IPW for both training and evaluation addresses two common sources of bias.\n- **Information-theoretic perspective**. Connecting PID to missingness is insightful; prior PID work [1] did not consider selection bias from partial observation.\n- **Thorough experiments**. The synthetic/XOR table shows great insights about how complete-case analysis can invert conclusions (even negative \"shared\" due to bias) and how correction aligns with oracle; this pattern recurs in UR-FUNNY and Hateful Memes.\n\n[1] Williams, Paul L., and Randall D. Beer. \"Nonnegative decomposition of multivariate information.\" arXiv preprint arXiv:1004.2515 (2010).", "weaknesses": "- **Assumption strength and scope**. The core guarantee hinges on MAR + positivity. I wonder whether this holds true in most multimodal systems. For instance, missingness can be MNAR, which violates the MAR assumption. The paper simulates MNAR in the appendix but does not probe MNAR in the clinical study. I suggest authors add sensitivity analyses on real-world data.\n- **Breadth of baselines**. Empirically, the paper compares observed vs. corrected vs. oracle. It would help to include strong missing-modality baselines (e.g., SMIL [2], modality-incomplete prompt/adapters [3]) to situate ICYM2I among robustness methods; also relate to missingness-shift DA.\n\n[2] Ma, Mengmeng, et al. \"Smil: Multimodal learning with severely missing modality.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 3. 2021.\n\n[3] Lee, Yi-Lun, et al. \"Multimodal prompting with missing modalities for visual recognition.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6Kf1Q31K1f", "reviewer": "Reviewer_VFgX", "summary": "This paper introduces a framework to correct for bias in multimodal learning arising from missing modalities. The authors argue that the informativeness and predictive value of a modality are often misestimated when the missingness process differs between the source and target environments, a realistic but underexplored issue. ICYM2I leverages IPW to adjust both model training and evaluation under the MAR assumption. The method also extends to information-theoretic decomposition, allowing unbiased estimation of modality informativeness through a corrected version of PID. Experiments on synthetic and semi-synthetic datasets demonstrate that ignoring missingness can substantially bias conclusions about modality utility, while ICYM2I provides more accurate informativeness estimates.", "questions": "Could you elaborate on how  $p_{\\Omega_\\phi}(m|C)$  is actually modeled and estimated? How sensitive is performance to misspecification of this model? \n\nHave you considered cases with unpaired modalities, where instances across modalities are not aligned? Could you comment how the method could be extended to such cases?   \n\nWould the method still hold for other forms of fusion beyond $q_\\theta = \\exp(f_1 f_2)$ , e.g. attention-based models especially when large in parameter number? \n\nCould one imagine a generative formulation where C and Y are governed by a single latent variable, and would this affect how modality informativeness is estimated?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "The paper is conceptually strong and addresses a highly relevant but largely overlooked problem in multimodal learning: the impact of missing modalities on the estimation of informativeness and performance. It offers a clear and rigorous formalization of missingness as a type of distribution shift, which is an important contribution to the field. The proposed use of inverse probability weighting is theoretically grounded in classical missing-data and statistical literature, and its adaptation to multimodal settings is elegant. The integration with information-theoretic concepts, particularly the extension of PID under missingness, is well-motivated and contributes to a deeper understanding of modality interactions. Overall, the paper is well-written, the motivation is convincing, and the theoretical framing is solid and potentially impactful.", "weaknesses": "Despite its strong conceptual foundation, the paper’s methodological presentation is somewhat difficult to follow. Understanding the full framework requires frequent reference to the appendix, which makes it challenging to reconstruct the complete algorithmic pipeline from the main text. In particular, Step 2 and Step 10 of Algorithm 1, which concern the estimation of the missingness mechanism $p_{\\Omega_\\phi}(m|C)$ and its integration into the weighting scheme, are crucial for comprehension and should be explicitly detailed in the main body. It also remains unclear how exactly this missingness model is parameterized or estimated in practice. Moreover, the objective of minimizing $I(Y; (X_1, X_2))$ appears counterintuitive and should be clarified, as mutual information is typically maximized in multimodal fusion. From an empirical perspective, while the synthetic and semi-synthetic experiments are carefully designed and demonstrate the motivation, the work lacks convincing real-world evidence that the proposed correction improves downstream predictive performance. Correcting modality informativeness estimates is valuable, but readers are left wondering whether a practitioner facing genuinely missing modalities would actually see performance gains by applying ICYM2I. Finally, the framework’s dependence on a particular parameterization $q_\\theta = \\exp(f_1 f_2)$ is restrictive; demonstrating that the approach generalizes across different fusion architectures would strengthen the empirical case and enhance credibility. Lastly, a weakness regarding the submission is that the code has not been shared, either in the supplementary material or via an anonymized GitHub repository, which limits reproducibility and verification of the claims. Lastly, a weakness regarding the submission is that the code has not been shared, either in the supplementary material or via an anonymized GitHub repository, which limits how much reviewers can verify and fully assess the implementation and results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wWSSrTRkQF", "reviewer": "Reviewer_2rF7", "summary": "I'm not an expert in this area.", "questions": "No question.", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I'm not an expert in this area.", "weaknesses": "I'm not an expert in this area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "48zu08rG7f", "reviewer": "Reviewer_pctf", "summary": "This paper addresses missingness-induced distribution shifts in multimodal learning, a largely overlooked issue where source and target environments differ in observed modalities. It notes that naive estimation without accounting for this shift biases modality informativeness and predictive utility estimates. To solve this, it proposes the $ICYM^2I$ framework, which uses inverse probability weighting correction under the MAR assumption to evaluate performance and information gain under missingness. The framework is validated on synthetic, semi-synthetic, and real-world datasets to demonstrate its utility.", "questions": "none", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Framing missingness in multimodal learning as an intrinsic distribution shift between source and target environments is novel, as well as formalizing its mechanisms (MCAR, MAR, and MNAR) and showing unaddressed shift biases modality informativeness and predictive utility estimates.\n2. This paper proposes a double inverse probability weighting (IPW) framework under the realistic MAR assumption to correct both training and evaluation, enabling unbiased assessment of predictive performance and information gain under missingness.\n3. The proposed method integrates IPW into Partial Information Decomposition (PID) to adjust for $\\Omega_{obs} \\mapsto \\Omega$ shift, designing an autodifferentiable algorithm with modified Sinkhorn-Knopp to handle high-dimensional data for unbiased information decomposition \n4. This paper applies the framework to structural heart disease detection, revealing chest radiographs have minimal independent informativeness, bridging methodological advances with real-world healthcare.", "weaknesses": "1. The paper relies heavily on the Missing At Random (MAR) assumption, but real-world multimodal scenarios often involve Missing Not At Random (MNAR). It provides no solutions for MNAR and only briefly mentions its limitations, lacking sensitivity analysis for MAR violations. The authors could add experiments on MNAR with plausible assumptions (e.g., simulating unobserved covariates) or explore semi-parametric methods to relax MAR.\n2. The PID in $ICYM^2I$ focuses on two modalities, but real multimodal tasks involve more, i.e., 3+ modalities. This paper only mentions a \"one-vs-all\" approach without detailed implementation or validation, leaving scalability unclear. \n3. Experiments simulate MAR using existing modalities (e.g., X1 predicts X2 missing) but ignore real drivers of missingness. For example, in healthcare, CXR missingness may relate to patient insurance status, which is unrelated to existing modalities. \n4. The SHD case study uses data from a single academic hospital system, with no validation on external multi-center datasets. This limits the clinical generalization of conclusions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)