Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification
Authors: Siyi Du
Xinzhe Luo
Declan O'regan
Chen Qin
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Learning
Dynamic Neural Network
Missing Modality
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose DyMo, a novel inference-time dynamic modality selection framework that adaptively selects and integrates reliable recovered modalities for each test sample, maximizing task-relevant information in incomplete multimodal data.

Abstract: Multimodal deep learning (MDL) has achieved remarkable success across various domains, yet its practical deployment is often hindered by incomplete multimodal data. Existing incomplete MDL methods either discard missing modalities, risking the loss of valuable task-relevant information, or recover them, potentially introducing irrelevant noise, leading to the discarding-imputation dilemma. To address this dilemma, in this paper, we propose DyMo, a new inference-time dynamic modality selection framework that adaptively identifies and integrates reliable recovered modalities, fully exploring task-relevant information beyond the conventional discard-or-impute paradigm. Central to DyMo is a novel selection algorithm that maximizes multimodal task-relevant information for each test sample. Since direct estimation of such information at test time is intractable due to the unknown data distribution, we theoretically establish a connection between information and the task loss, which we compute at inference time as a tractable proxy. Building on this, a novel principled reward function is proposed to guide modality selection. In addition, we design a flexible multimodal network architecture compatible with arbitrary modality combinations, alongside a tailored training strategy for robust representation learning. Extensive experiments on diverse natural and medical image datasets show that DyMo significantly outperforms state-of-the-art incomplete/dynamic MDL methods across various missing-data scenarios. Our code is available at https://github.com//siyi-wind/DyMo.

Published as a conference paper at ICLR 2026
INFERENCE-TIME DYNAMIC MODALITY SELECTION FOR INCOMPLETE MULTIMODAL CLASSIFICATION
Siyi Du Xinzhe Luo' Declan P. O'Regan" Chen Qin"
'Department of Electrical and Electronic Engineering & I-X, MRC Laboratory of Medical Science Imperial College London, London, UK
ds.du23, x. x.uuo, orglaan c. c.qinn5jeimperian ac.uk
ABSTRACT
Multimodal deep learning (MDL) has achieved remarkable success across various domains, yet its practical deployment is often hindered by incomplete multimodal data. Existing incomplete MDL methods either discard missing modalities, risk- ing the loss of valuable task-relevant information, or recover them, potentially introducing irrelevant noise, leading to the ierrarddinniipptaation dilemma. To address this dilemma, in this paper, propose DyMo,
inference- serennee-imeeeee dy- namic modality selection framework that adaptively identifies and fuses reliable recovered modalities, fully exploring task-relevant information beyond the con- ventional discard-or-impute paradigm. Central to DyMo is a novel selection algo- rithm that maximizes multimodal task-relevant information for each test sample. Since direct estimation of such information at test time is intractable due to the unknown data distribution, we theoretically establish connection between infor- mation and the task loss, which we compute at inference time as tractable proxy. Building on this, a novel principled reward function is proposed guide modality selection. In addition, we design flexible multimodal network architecture com- patible with arbitrary modality combinations, alongside a tailored training strategy for robust representation learning. Extensive experiments diverse natural and medical image datasets show that DyMo significantly outperforms state-of-the-art mncomplte//dynmicc MDL methods across various missing -ata scenarios. Our code is available at https: : / com/ / wttsssiiy///yyaataaaa... wwwd/Dynd.......////
INTRODUCTION
Multimodal multi-view deep learning (MDL), which integrates various modalities/views to achieve multisensory perception akin to humans, has gained increasing attention and made significant ad- vances in various domains such as healthcare (Acosta et al., 2022), marketing (Liu et al., 2024), and embodied intelligence (Duan et al., 2022). However, real-world deployment of these MDL models remains limited due to their simplified data assumptions. Existing MDL approaches typically pre- suppose full modality availability during inference. In practice, however, samples often lack more modalities due to heterogeneous collection protocols across centers, sensor malfunctions, or transmission enrors (Wu et al., 2024a), leading degraded model performance. Therefore, develop- ing MDL models robust to incomplete multimodal data has become a critical research focus.
Current methods addressing missing modality can be broadly categorized into types: (1) recovery-based approaches aim to impute missing modalities at the input level or in latent space via retrieval or generation, enabling the MDL model operate as if all modalities were present (Ma et al., 2021; Xu et al., 2025); (2) recovery-free approaches are designed to ignore missing modalities and make predictions using only available ones (Lee et al., 2023; Wu et al., 2024b).
However, these incomplete MDL methods encounter intrinsic challenges in capturing task-relevant information under modality heterogeneity (Zhang et al., 2024). Specifically, modalities vary their task relevance, due to differences in the strength of task-relevant signals and the degree of interfering,
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
(a-1) Missing 15. (Aec: 0.98]
1a-2) Missing (M, T3 (Ace: 0.82)
Input
Reconstracioo
DyMo
Prediction
g
8
?
Multmodal network trary modaltiess
O
O
O
O
O
Conami selection
8
3
8 ?
? A
!
!
/
Amocallit recovery mothod
?
[a] veuullzation ModDrap''s moodlltt-inoompllteeee festures
modallty peneration MoFoE on PolyMN ST dataset
Id) DyMo dynamic modality selection over missing modaltties inference
MNISTIMM SUHNS))-TECT TEXT()) (MST) training dataset
Figure I: (a-b) Evidence of the discarding- imputation dilemma: (a-1) vs. (a-2) recovery- frree meth- ods (e.g., ModDrop (Neverova et al., 2015)) learn less discriminative features because they ignore highly task-relevant missing modalities (M,T): (b) recovery-basee methods (e.g., MoPoE (Sutter et al., 2021)) generate unreliable reconstructions, e.g., low-fidelity (orange) or misaligned (yellow). (c) Our DyMo, which addresses the dilemma by dynamically fusing task-relevant recovered modal- ities, improving accuracy by 1.61 % on PolyMNIST, 1.68% on MST, and 3.88% on CelebA (Tab. I).
task-irrelevant noise (Huang et al., 2022b). As shown in Fig. (a)), when highly informative modali- ties are missing, earlier recovery -fre methods rely solely on the less distinguishable features of the remaining modalities, without considering the valuable task-relevant information contained in the missing ones, resulting in decreased model performance. Prior recovery-based methods appear to mitigate this issue by reconstructing missing modalities. Yet, imputation qualities often vary across samples due to cross-modal heterogeneity and diverse missing scenarios. As in Fig. 1(b). some recovered modalities may be low-fidelity (i.e., blurry or corrupted, orange boxes) or semantically misaligned (i.e., their labels are inconsistent with the input modalities, yellow boxes). Integrating such unreliable modalities can inject task-irrelevant noise, impairing decision-making. Thus, dis- carding missing modalities risks losing valuable task-relevant information, whereas restoring them may inject harmful information, a trade-off limitation we term the fissaadding-imputaai dilemma.
To address this dilemma in incomplete MDL, we introduce a new perspective that dynamically se- leets and fuses recovered modalities conditioned their task relevance, moving beyond the conven- tional dichotomy of discarding vs. imputing missing data. A key technical challenge in such dynamic systems is how estimate the task-relevant informativeness of each modality and identify unreliable modalities at inference time. Nevertheless, existing dynamic fusion approaches (Cao et al., 2024; Gao et al., 2024) are primarily designed for low-fidelity modalities and rely on modality-specific features for weighting, and thus are limited in identifying semantically misaligned modalities.
In this work, we propose DyMo, novel inference-tine dynamic modality selection framework that adaptively fuses reliable recovered modalities to maximize multimodal task-relevant information in incomplete MDL (see Fig. I(c)). To avoid integrating unreliable (low-fidelity or misaligned) re- covery, we propose new dynamic algorithm that iteratively selects the most informative recovered modality based on the incremental multimodal relevatt information gain it provides. Since the data distribution is unknown at inference, we theoretically derive that reducing task loss can increase the lower bound on task-relevant information. This motivates using the loss decrease as a practical proxy for information gain and introducing a novel principled reward function to guide modality selection. Additionally, to support flexible multimodal input, we design a multimodal network ar- chitecture capable of predicting task targets with arbitrary modalities. We further propose a tailored training strategy to learn robust latent features suitable for DyMo's dynamic selection process.
Our contributions be summarized as follows. (1) To the best of our knowledge, we the first to investigate the irrrdinggimmptattin dilemma in incomplete MDL and introduce dynamic neural networks address it. (2) We propose DyMo, novel dynamic framework that adaptively fuses recovered modalities via a new selection algorithm formulated on multimodal task-relevant infor- mation gain, together with a multimodal network and tailored training algorithm for robust feature extraction from arbitrary combinations of modalities. (3) Experiments on 5 diverse datasets demon- strate DyMo's 's remarkable outperformance over incomplete/dynamic MDL SOTAS. especially under severe missing scenarios. DyMo is also easy-to-use (flexible with various modality recovery meth- ods) and readily deployable without additional architecture overhead for its dynamic algorithm.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Deep Learning (MDL) methods (Zhan et al., 2025; Wu et al., 2024a) broadly fall into two categories: recovery-based and recovery-free. Recovery-based approaches can be further divided into offiine and online, depending on when the imputation occurs. Offline methods typically pre-train separate reconstruction networks, e.8., variational auto-encoders (VAEs) (Sutter al., 2021; Gao & Pu, 2025), and then use reconstructed data for downstream tasks. Online meth- ods (Wang et al., 2023b; Shou et al., 2025) jointy learn modality recovery and downstream tasks during training (Wang et al., 2023a). For instance, SMIL (Ma et al., 2022) employs Bayesian meta- learning to impute latent features, while M3Care (Zhang et al., 2022) retrieves similar samples to compensate for missing modalities. recovery-free approaches, paissing-agnostic techniques learn modallty-invariann features via random modality dropout, e.g.. ModDDrop (Neverova et al., 2015) and MMANet (Wei et al., 2023), or contrastive learning (Wu et al., 2024b), while missing-aware methods introduce missingness-speciinn parameters to handle different missing patterns (Ma et al., 2022; Li et al., 2025). However, previous incomplete MDL methods either fully recover or ignore missing modalities, without considering the discarding -imptitttiiintaiinnnnn dilemma.
Incomplete Multimodal
Dynamic Deep Neural Networks, in contrast to static models, can adapt their architectures and pa- rameters based individual input, yielding substantial improvements in computational efficiency, accuracy, and interpretability (Han et al., 2021; Guo et al., 2024). Dynamic architecture methods adjust network depth, width, or routing paths a per-sample basis (Fedus et al., 2022; Yue al., 2024; Bayasi et al., 2022: Zhao et al.. 2025), while dynamic parameter algorithms modulate network weights or feature scaling without altering the architecture (Zhang et al., 2023; Bolya et al., 2023; Du et al., 2023). Our DyMo, which dynamically routes modality data conditioned on each instance, falls into the dynamic architecture category.
Multimodal Fusion is a critical research problem in MDL (Baltrušaitis et al., 2018). Existing meth- ods can be grouped by their fusion stage into early (data level), intermediate (feature level), and late (decision level) fusion. Recently, dynamic multimodal fusion methods have emerged to address instance-specifee modality reliability variations. These methods assign different weights to each modality (Zhang et al., 2023; Gao et al., 2024) or selectively use a subset of modalities (Xue & Mar- culescu, 2023; Ma et al., 2025). However, such methods typically assume all modalities are available and mainly focus on intra-modality noise (i.e., low-fidelity data) during training, lacking robustness against inter- modality errors (i.e. semantic misalignment). While MICINet (Zhang et al., 2025) explores both types of noise, it requires ground-truth labels and cannot be applied during inference. By contrast, DyMo dynamically fuses observed and recovered modalities to handle incomplete data, while effectively addressing intra- and inter-modality noise without relying labels.
3 METHOD
In this section, we present DyMo, a novel inference-time dynamic modality selection framework for incomplete multimodal classification by fully exploring task-relevant information from recovered modalities, while addressing the discarding imputation comprises 3 key components: (1) ficxible multimodal architecture capable of making predictions from arbitrary modalities (Sec. 3.1, Fig. 2); (2) a novel dynamic selection algorithm that integrates valuable recovered modalities to maximize multimodal task-relevant information for each sample (Sec. 3.2, Algorithm 1); and (3) a tailored training strategy that enhances representation learning to ensure feature robustness under dynamic modality configuration during inference (Sec. 3.3).
discardingiimputation dilemma (Fig. ((c)). To achieve this, DyMo
3.1 MULTIMODAL ARCHITECTURE FOR ARBITRARY MODALITIES
Our multimodal network is designed to produce reliable predictions from any subsets of input modal- ities, enabling adaptive fusion based on the task relevance of each recovered modality. To achieve this, we construct a multimodal network f (Fig. 2), consisting of modality-speciifi encoders for unimodal feature extraction, multimodal transformer for modeling cross-modal interactions and learning a multimodal representation of available modalities, and a classifier for the final prediction.
A multimodal classification dataset can be defined as (X,, Yi N where N is the number of sam- ples, y/ E (1.... K} is the class label, and X, (m) (efmm))me,,, Jmzz represents (potentially incomplete)
3

---

Published as a conference paper at ICLR 2026
multimodal imput comprising modalities such images and text. Here, I₁ denotes subset of the complete modality indices [M]] = {1, ............... As shown in Fig. 2, each available modality m E I4 is encoded by a modality-specific encoder f((m), producing a sequence of feature tokens ( E (2L"")xc xC, where ((m) is the sequence length and C' denotes the feature dimension. We then concatenate these tokens together with a learnable [CLS] token to form multimodal sequence embedding. The [CLS] token's 's output embedding & serves the multimodal representation for classification as in (Devlin et al., 2019). To preserve the sequence structure across samples, we assign dummy tokens to the positions of missing modalities.
Driven by transformer's ability to capture long-range dependencies from variable lengths of the sequence, build multimodal transformer network & composed of T stacked trans- former layers (Vaswani et al., 2017) to learn cross-modal relations in observed modalities. These transformer layers conduct self-attention the mul- timodal sequence embedding and apply attention masks ensure that missing modalities do not distort represen- tation learning. The extracted multimodal representation (X.)) from the transformer is passed through lin- ear softmax classifier Ç to yield the final prediction.
[CLS]
classft
prediction
m
2210
Multirodal Trans
k
00S
3.2 DYNAMIC MODALITY SELECTION AT INFERENCE
dilemma, we propose a novel dynamic modality selection algorithm that adap- tively discovers valuable task-relevant recovered modalities at inference. The core idea of our approach a novel re- ward function that estimates the incremental multimodal task-relevant information contributed by each recovered modality. We establish theoretical connection between information and task (i.e,, classification) loss, leading to an effective reward formulation linked to representation shift in the latent space. To further enhance robustness, we propose an intra-class similarity calibration for reward refinement based on training data. Finally, we introduce an iterative selection mechanism for reliable dynamic multimodal fusion.
To address the
[CLS] Classificttion Token
NA
Dummy
NA Missing modality
Figure Multimodal network architec- ture f for arbitrary modalities.
Multimodal Rask-Relevant Information Reward (MTIR) is designed to estimate the incremental multimodal task-relevant information gained by adding recovered modality to the existing ob- served modalities. MTIR is inspired by the notion of information gain (Ma et al., 2019; Jolliffe, 2011) and aims to indicate the marginal impact of each recovered modality on the multimodal rep- resentation: (1) Positive reward suggests that the recovered modality introduces additional task- relevant information that enhances the representation; (2) Zero reward indicates that the recovery is of low fidelity, mainly introducing noise and providing negligible benefit; and (3) Negative reward implies that the recovery contains task-relevant but semantically inconsistent information, poten- tially degrading the representation. This formulation enables the identification of both low-fidelity and misaligned modalities, which have been largely overlooked in prior dynamic MDL. work.
To quantify task-relevant information contained in the multimodal representation, we consider mu- tual information between the multimodal representations Z and the target labels Y:
ply. 2)
T(Y; Z) Ep(x.)) log p(y)p(=)
Since the true data distributions are unknown at inference, we propose to derive a lower bound of I(Y; Z), estimated using the empirical test-time classification cross-entropy (CE) loss Lugi
l1(1/)) 2/D]
I(Y; > H(Y) - Lae GV
with probability at least 1 -,
(2)
where HYY) is the entropy of the target labels, G is conservative upper bound on the per-sample CE loss, [D] is the size of a test dataset, and E (0, 1) controls the probability of the bound holding (detailed derivation in Appendix A.1). This bound formalizes the intuition that reducing E.e can increase the lower bound on T(Y: ; Z), thereby potentially increasing the task-relevant information
4

---

Published as a conference paper at ICLR 2026
in Z. Motivated by this insight, we propose use the empirical test-time CE loss decrease tractable proxy for information gain, forming the theoretical foundation of our MTIR reward.
mez, (notations follow Sec. 3.1), we recover the missing modalities as X3 = ( (we((M)) using a recovery function Y (e.g., VAE). The test-time empirical CE loss of DyMo 9. which includes multimodal network and dynamic selection algorithm, is then defined as:
Given an incomplete multimodal test sample X, = {a((m) {a((m))
1 Σ D
1
foo 91 (x xy),
To minimize Lee, DyMo should adaptively integrate the recovered modalities in X that reduce per- sample loss fee. Accordingly, for each sample, we define MTIR of u-th recovered modality as:
R(x(") (w) x0) = fee [s(x)),) ).2) fee [ a ((()), ), v) = log (ul=) + (ogl)()lz"),
(4)
where x denotes the observed modalities (initially X), lee logp;(ulz); z (11(x and x" , ( Since the true label is unknown at inference, substitute it with the predicted labels argmax f(X²) and 5" = argmax f(x², _(()). However, such substitution may undermine the reliability of MTIR, especially under mispredictions or overfitting. To mitigate this, inspired by the robustness and generalizability of metric learning (Vinyals et al., 2016; Chen et al., 2020), we further investigate the representation shifts with respect to the training distribution, measured in the latent space. Specifically, treat classification problem in feature space, where each class corresponds to component. Suppose equal class prior probability and exponential family distributions, the posterior probability of y k given x
mixture density estimation
exp( (o ck)) Ek'=1 exp(= (o.
ply k]])
Ca Ee
Z6
(5)
Ty: k]
where de is a Bregman-divergence type distance function (Banerjee et al., 2005) (e.g., squared Euclidean distance), and C₁ is the class prototype estimated from the training dataset. The derivation of this expression is provided in Appendix A.2. Substituting Eq. 5 into Eq. 4 yields:
exp( (xp(-ds(z,c))
+ log exp(-dg Cg-))
R(ā (5) xx) - -log Σ (xp(-ddp(z))
(6)
_ exp cx))
This formula indicates that higher MTIR rewards be obtained when the representation moves closer to the class prototype after incorporating a recovered modality, which is consistent with the in- tuition that fusing effective information increases the model's predictive certainty (Dai al., 2023).
Intra-Class Similarity Calibration: Eq. 6 defines the MTIR reward based on changes in the dis- tance between sample representation and its predicted class prototype. A challenging case arises when ÿ and g" differ, while and z" lie at similar distances from their respective prototypes, yield- ing a near-zero reward. To address this and enhance the reliability of the MTIR reward, we introduce a novel calibration term a, which refines the reward by accounting for how representative a sample within its predicted class cluster. The calibrated MTIR reward R* is:
R (a xx) log exp(-ds(2,c)) cg))
+ a x log exp(-do(2" cg"
Σ
)w))
exp( -ddddd dd(("",cw)) cw))
To compute a, we first define the intra-class similarity (ICS) score of for class k. Unlike many non-pan parametreeic statistical approaches (Hastie et al., 2009) that require computing distances to all training samples in class k, we propose an efficient approximation. Specifically, we approximate the distribution of distances from the samples class the class prototype e2 as a truncated normal distribution de NO, of), dæ > 0, where og is estimated from the training data. The ICS score is
then written as follows:
(8) where & is the cumulative distribution function of the normal distribution. ICS quantifies the rep- resentativenss of z within a class cluster, with a higher value indicating closer alignment with the training samples in that cluster.
ICS( k x) Idss > dz(z, cd)ld 0) 2(1 2 (ddg(z,
5

---

Published as a conference paper at ICLR 2026
The calibration term is defined as the ratio between the ICS scores of x" and x:
ICS(y=0".2* Since ICS()=ū,2)
X are observed, task-relevant modalities, while ((w) is synthetic and can introduce unreliable information, DyMo should be conservative when [CS - ,²" ICS = D, 2). To this end, we introduce an asymmetric a:
if ICS(y = x²) > ICS(y = U, z),
a
(") ICS(y=Baa)
otherwise,
which is used to calculate R in Eq. 7. Thus, 2" is less representative than z within its predicted class cluster (i.e.. < 1), the second term in R is down-weighted, reducing the calibrated MTIR.
Iterative Modality Selection: To improve the reliability of DyMo's dynamic process, we introduce an iterative selection algorithm to maximize multimodal task-relevant information for each sample (Algorithm 1). At each step, given an observed set X and a candidate set x we add the recovered modality with the highest MTIR reward to X while removing all ineffective modalities with non- positive rewards from xC_ This stepwise selection ensures that DyMo incorporates only the most informative modalities, effectively mitigating noise accumulation.
Algorithm I: DyMo Adaptive Inference
Input: Test dataset D. network f (including /, and (). and recovery method T for each test sample X =
Jmsi E D do JuEpaN +- T(X);
X
(
If Recover missing modalities I Initialize observed cundidate modality sets
x0
x < 8;
while xC do
(0)) (x,)x, Yal ex;;
I/ Calibrated MTIR reward (Eq. 7) Best candidate modality index
arg max, r(t;;
if > then
x - x (2())
#f Integrate the most informative modality
end
x x (a(t ( 203:
Remove ineffective modalities
end
j f(XO)
end
return All test predictions y collected over
3.3 TRAINING ALGORITIM
DyMo's dynamic inference relies on representation shifts, making it essential to train our multimodal network / properly to learn robust latent feature space where samples of the same class cluster together despite missing modalities. To achieve this, we design incomplete-modality simulation training and an auxiliary missing-agnostic contrastive loss.
Incomplete Simulation Training, To guarantee that f extracts robust task-relevant features across various missing patterns, we propose a simple yet effective random sampling strategy during train- ing. In specific, each complete multimodal sample (x(1), ...,)((() has 2 - non-empty modality subsets. In each minibatch, we randomly sample A such subsets for classification loss calculation:
Celass
1 1 Á B SS-MA
Es-ie
En logps (1) {m)m))mes)
Jmes).
(10)
1-1
where B is the batch size, and UA Uniformwo rep (P([M) \ A). P([M)) denotes a collection of all modality subsets. This strategy reduces computational cost compared to prior work (Ma et al., 2022) that requires all missing patterns in each training step.
Auxiliary Missing- Aggossic Contrastive Loss is designed to further enhance intra-class clustering and inter-class separation regardless of missing scenarios:
Laux
1 1
Σ S-MA
Σ
Cy)/t) i=1 log E exp (-do(z;, (x;,
(11)
A B
where t is the temperature parameter. In experiments, we test two common distance functions: the squared Euclidean distance de(u, v) ||u - x112 and the cosine distance dz(u,, v) 1 (u /)/([[/]]) (112)) where u v is the dot product of the vectors.
Overall Loss. The final training loss for DyMo expressed as Loverall Letase + Laux-

---

Published as a conference paper at ICLR 2026
4 EXPERIMENT
Datasets and Evaluation Metrics: We conduct extensive experiments on 5 different datasets with varied modalities (i.e., image, text, and structured table), including simulated benchmark datasets (Sutter et al., 2021): PolyMNIST, MST, and biomodal CelebA, and two large real-worth datasets: a natural image dataset, Data Visual Marketing (DVM) (Huang et al., 2022a), and a medi- cal image dataset, UK Biobank (UKBB) (Sudlow et al., 2015). For UKBB, we foeus on two cardiac disease classification tasks: coronary artery disease (CAD) and myocardial infarction, using mag- netic resonance (MR) images and disease-related tabular features. Following (Du et al., 2025; Sutter et al., 2020), we report the area under the curve (AUC) for UKBB, and accuracy for the remaining datasets. Dataset details are presented in Appendix B.I.
For the recovery method in DyMo, we leveraged MoPoE (Sutter et al., 2021), a multimodal VAE network, for PolyMNIST, MST, and CelebA. Then, for DVM and UKBB, we used TIP (Du et al., 2024), an image-tabular reconstruction framework. Note that DyMo can be deployed with any recovery method. To ensure a fair comparison, all compared methods used the same modality-specifie encoders as DyMo. Models were trained on complete datasets and evaluated under various missing scenarios: (i) For PolyMNIST. we set 5 missing rates = {0, 0.2, 0.4, 0.6, 0.8]. where each sample randomly misses 1) x 100% modalities; (ii) For MST and CelebA, we tested different combinations of missing modalities; (iii) For DVM and UKBB, since TIP is table reconstruction network, we evaluated both full- and intra-table (i.e., partial within- modality) missing. Specifically, we set 7 missing tabular rates = {0, 0.3, 0.5, 0.7, 0.9, 1), where each sample randomly misses 7 x 100% tabular features. Note that DyMo can also handle incomplete modality training, as our multimodal network supports arbitrary modalities. Detailed implementation settings for all models are provided in Appendix B.2.
Implementation Details:
4.1 OVERALL RESULTS
To assess DyMo's effective- ness in selecting valuable task-relevant recovered modalities at inference, compared it with SOTA multimodal fusion techniques, including static concatenation-based fusion (CONCAT) (Bal- trušaitis et al., 2018) and dynamic fusion methods, QMF (Zhang et al., 2023), DynMM (Xue & Marculescu, 2023), and PDF (Cao et al., 2024). All methods were provided with the same set of mon-missing and recovered modalities inference. Notably, prior dynamic methods typically require additional modality-specific branches for modality contribution estimation multi-stage training, whereas DyMo operates directly on multimodal representations without extra modality- specific parameters and relies on single-stage training. We report DyMo's results using cosine dis- tance (DyMo.) and squared Euclidean distance (DyMo.).
Comparing Against Multmodal Static Dynamic Fusion SOTAs:
As shown in Fig. 3, DyMo achieves substantial improvements on most datasets, especially under se- vere missing scenarios. For example, DyMo achieves 13. 12% higher accuracy on Poly with 80% missing modalities and 4.11% % higher accuracy DVM when the full table is missing. Both DyMo, and DyMo, consistently outperform prior SOTAs on most datasets, demonstrating DyMo's robustness to the choice of distance metric. We further observe that (1) performance decreases vary across different missing modality combinations (Fig. 3(b,)), indicating the existence of varying modality task relevance; (2) prior dynamic methods outperform static fusion on DVM (Fig. 3()) but achieve limited gains on the 3 simulated benchmarks (Fig. 3(a-c)), This limitation arises be- cause VAB-based reconstruction tends to produce visually plausible but class-misaligned recovered modalities, which prior dynamic methods struggle to handle. By contrast, DyMo addresses this via estimating incremental multimodal task-relevant information gain, resulting superior results. Additionally, DyMo and CONCAT perform similarly on CAD, likely due to the consistent informa- tiveness of the recovered table modality across samples, leaving limited room for dynamic fusion to further improve performance.
To evaluate DyMo's efficacy in handling missing modalities, we compared it with SOTA incomplete MDL approaches, including 5 recovery-based (MultiAE (Ngiam et al., 20,1), MoPoE (Sutter et al., 2021), M3Care (Zhang et al., 2022), On- lineMAE (Woo et al., 2023), and CMVAE (Palumbo et al., 2024)) and 4 accovery-free methods (ModDrop (Neverova et al., 2015), MTL (Ma et al., 2022), MAP (Lee et al., 2023), and MUSE (Wu et al., 2024b)). For models without simulating different missing scenarios during training, ad-
Comparing Against Incomplete MDL SOTAs:
7

---

Published as a conference paper at ICLR 2026
[n] PolyMNIST
[51 MMSSTOO0-SSMM
(c1 GeMbA
100
100
95
ACUrRey th
90
90
95
COMLAI
CORCAA
65
DIMAA
82
PDT
60
8
FDF
72
75
75
0.2
0.4
0.6
2-2
10
(91)
1M 11) 09-33 D--1 (KT))
medulfyyis
70
0
(les)
(d1) DVM
(e) CAD
(11 interction
100
80
AcC.NNy (h)
90
05
COMCAT
CONCOCT
OOACA
02
PDT
FDr
22
70
0.1
0.5
0.5
0.7 0.3
3.0
60
0.3
0.5
0.7
012
1.0
00
0.3
0.3
0.3
0.7
0.0
1.0
Maxsng
Figure 3: Comparison of DyMo with static/dynamic multimodal fusion techniques on 6 multimodal classification tasks under various missing scenarios. DyMo, and DyMo, denote the use of cosine and squared Euclidean distances, respectively.
Table I: Comparison of DyMo with recovery-based and recovery-free incomplete MDL methods on multimodal classification tasks under various missing scenarios. Models marked with were trained using our proposed incomplete modality simulation. Complete results for all missing sce- narios are provided in Appendix C.I.
Model
MNNST Ace (9)
MST Ace (8)
CelebA Ace (9)
DVM Acc (5)
CAD (U)
Infarction AUC (8)
Missing Rate
Missing Modalityfsss
Missing Tabular Rate
(S.T)
[M.T]
(T)
0.7
Recovery-bosee Methods for Missing Modality
MulcAE
99.77
95.36
84.39
99.96
97.00
51.50
39.98
15.51
89.71
MultiAE
83.44
72.04
87.44
McPul
99.79
01.93
79.81
99.62
90.56
79.01
38.97
13.91
34.81
MoPoE!
99.63
87.06
99.39
96.50
56.90
M3Care
99.93
56 66
10.53
99.99
16.03
9.34
92.33
99.92
51.75
98.41
11.92
85.62
61.99
70.61
70.53
M3Care!
09.99
97.27
87.92
99.98
98.27
85.16
98.73
97.14
91.32
98.94
93.43
72.48
72.48
83.27
68.44
OnlineMAB
84.05
CMVAE
94.93
95.13
95.11
CMVAE!
94.59
95.17
95,20
Recovery-fres Methods for Missing Modality
MedDhop
99.97
97.66
68.44
100
98.21
82.47
99.93
99.93
87.32
99.02
93.23
87.97
85.10
76.65
09.18
84.76
74.64
12
MTL.
99.97
91.14
99.96
95.60
99.36
99.44
92.32
84.87
77.72
83.59
75.73
MAP
99.86
43.00
23.19
100
9.83
10.13
99.98
99.92
85.33
98.86
$6.96
63.15
84.39
75.61
70.11
84.62
71.73
69.17
MAP!
99.99
96.74
76.20
99.99
97
11.36
100
99.99
86.06
99.37
94
91.15
85.26
75.84
68.76
85.49
75.88
70.81
99.93
94.73
97.14
99.96
84.40
Dynamic Recovery Metholl for Missing Modality
DyMa,
100
99.71
96.61
100
98.22
85.31
100
100
95.20
59. 30
96 31 93.
85.14
78.49
71.02
85.10
77.85
71.58
DyMo,
24
83.16
ditionally report their results with our incomplete simulation training. Since ModDDrop is training scheme rather than a standalone architecture, we applied it to the same multimodal backbone as DyMo for a fair comparison. Methods requiring task-specific decoders or restricted full-modality missing settings were evaluated only on datasets where such decoders are provided and under full- modality missingness.
In Tab. 1, integrating our incomplete simulation training improves model performance (w/ vs. w/o t), demonstrating its effectiveness in learning features robust missing data. We also observe discarding-impuuaaiin the discarding imputation dilemma: (i) recovery-fire methods suffer large performance drops when
highly task-relevant modalities are missing, e.g., a 61. 188 accuracy reduction for MUSE on MST with missing (M,T} vs. {S,T) (note that CMVAE is relatively stable across missing rates, as it performs classification using a single randomly selected observed modality); (ii) recovery-based methods struggle under severe missing scenarios, e.g., OnlineMAE's accuracy on PolyMNIST de- creases by 9.91% % with 77 = 0.8 vs. 17 = 0, indicating the generation of unreliable recoveries. In contrast, DyMo effectively addresses this dilemma, significantly outperformigg prior SOTAs on full- and intra- modaltyy missing conditions, egg., 5.67% higher accuracy on PolyMNIST with 80% missing modalities and 1.97% higher AUC on Infarction with 70% missing tabular features. Per-
8

---

Published as a conference paper at ICLR 2026
Table Ablation study of DyMo. Baseline integrates all recovered modalities without selection. integrates all modalities with positive reward (r 0) simultnneussy.. iteratively adds the modality with the highest r. C uses the calibrated reward r", obtained via intra-class similarity calibration.
Dynamic
Pol:MNIST
MST
ColebA
DVM
CAD
Infarction
selection
0.6
0.8
[S.T]
[M.T]
(T)
(T)
0.7
0.7
0.7
Baseline
96.80
84.21
96.43
80.73
100
96.67
96.27
88.36
79.15
71.84
78.31
74.89
99.59
94.33
97.21
82.08
100
86.67
96.54
92.83
79.44
72.20
77.83
75.27
99.60
94.50
97.26
82.12
100
86.62
96.54
92.83
79.44
72.20
27.83
75.27
99.71
96.61
98.22
85.31
100
95.20
96.31
93.14
78.49
71.02
77.85
71.58
mon-iiisdng modaltiis (Acz- az)
recovrnd modal tea Aic
integrate selected by ZyMo (Aez: 9.85)
bins
2o-33 Dihto in mispredk 1 +0
Dyboo presaves prodicton er 0.77)
Overpp
(al 1-EE visualication dataset missing
PCA visualitaioon DVM micing
Figure 4: (a) I-SNE visualization of DyMo, MST with different modality inputs: (a-1) using only non-missing modalities; (a-2) integrating all recovered modalities without selection; (a-3) incorpo- rating recovered modalities selected by DyMo (b) PCA visualizations of two successful DyMo,'s test cases on DVM: (b-1) I) a misprediction corrected by incorporating a recovered modality; (b-2) a correct prediction maintained by disregarding unreliable recovered modality.
Sankay diugram 145T samples with DyMo- intg
(R) Case study on FalYMNTT
Classification accuracy (%) of DyMo, (firs three rows) and DyMo, (last three rows) using different modality recov- ery methods PNlyM under various missing modality rates 1).
Table 3:
ecfors uoliztt
After upolate
O
O
Asc
0.85
3
>
5
00.9 DyMo
Conect
?
1
3
0.2
0.4
0.8
5
5
FDF 3
MoPoR
100
99.99
99.71
96.61
MMVAF+
99.99
99.97
99.79
97.40
CMVAE
100
99.97
99.80
97.69
Figure 5: (a) Sankey diagram for DyMo: prediction transitions on MST with missing (M.T). (b) Case study on PolyMNIST, where yellow indicates non-missing modalities, while blue dicates modalities selected by DyMo.-
MoPoE
100
99.99
99.87
96.81
MMVAE+
100
100
99.87
97.39
CMVAE
100
99.99
97.39
formance matches M3Caret
DVM and CAD when v 1, likely due TIP's limited full-table reconstruction; stronger recovery could further enhance results.
4.2 ABLATION STUDY AND VISUALIZATION
Effectiveness of Key Model Components: We ablated the key components of DyMo in Tab. 2, including the MTIR reward, iterative selection, and the calibration term. The results show that in- tegrating all recovered modalities without selection can introduce task-irrelevant information and degrade performance. Each component contributes positively, and DyMo, which combines all of them, achieves the best overall results. For CAD and Infarction, I outperforms /+C. This is likely because the calibration term, bounded between ( and 1, makes the model more conservative in modality selection. In these cases, the recovered table consistently provides task-relevant infor- mation, so omitting C allows more samples to benefit from it. Tuning dataset-specific scaling before applying may mitigate this issue, which we leave for future work. Ad- ditional ablation studies on incomplete simulation training, generalizability analysis of DyMo, and adaptive inference analysis can be found in Appendix C.2-C.3. Test-time task loss analysis is reported in Appendix. C.4.
Robustness of DyMo Across Different Modality Recovery Methods: We conducted incomplete multimodal experiments using 3 different modality recovery approaches, including MoPoE, MM-

---

Published as a conference paper at ICLR 2026
VAE+ (Palumbo al., 2023) and CMVAE (Palumbo et al., 2024), and quantitatively evaluated their reconstruction performance on PolyMNIST. As shown in Tab. 3, DyMo combined with any of these recovery methods consistently outperforms prior SOTA dynamic/incompllte MDL methods (see Fig. 3 and Tab. 1), despite the varying reconstruction quality of CMVAR and MoPoF (Fig. S6 in Appendix C.6). To further assess DyMo's robustness to reconstruction quality, additionally eval- uated an alternative an extreme simulation experiment with a controlled correct recovery rate. The results and detailed analyses are provided in Appendix. C.2.
Seahhinn--lanninnbbaseeee recovery method on DVM and UKBB, and conducted
Visualization of Multimodal Representations and Case Studies: To examine the effect of inte- grating recovered modalities selected by DyMo in shaping the latent space, we used t-SNE (Maaten Hinton, 2008) to visualize the multimodal embeddings of the test set. Fig. 4(a) shows that inte- grating all recovered modalities increases inter-class separation; however, samples with unreliable recoveries can be embedded within incorrect class clusters, leading to misclassifications and de- graded performance. In contrast, DyMo's 's dynamic fusion alleviates this issue, producing a more discriminative latent space and improved classification results. Moreover, we conducted case studies at both the feature level (PCA (Jolliffe, 2011) visualization, Fig. 4(b)) and the input level (Fig. 5(b)), illustrating that DyMo effectively selects reliable, task-relevant recovered modalities that enhance model performance. A challenging example is shown in the fourth row of Fig. 5(b), where all dy- namic methods struggle due to noisy observed modalities and semantically misaligned recovered modalities. Additional visualizations and detailed analyses are provided in Appendix C.5.
Prediction Transition: We analyzed prediction changes before and after dynamic modality selec- tion using& Sankey diagram. As shown in Fig. 5(a), DyMo corrects many initial mispredictions and achieves improved performance, highlighting the benefit of recovered modalities. A small frac- tion of correct predictions become incorrect after updating, likely due to limited recovery quality, suggesting that stronger recovery methods could reduce such errors. Reconstruction analyses of modality recovery methods are Appendix C.6.
CONCLUSION
In conclusion, present the first study on dynamic multimodal fusion after modality recovery for addressing missing modalities. We propose DyMo, a new inference-time dynamic modality selection framework that fully explores task-relevant information while addressing the discarding- imputation dilemma by adaptively identifying and fusing valuable recovered modalities. DyMo introduces novel selection algorithm grounded in maximizing multimodal task-relevant informa- tion and proposes a principled reward function. We further design a flexible multimodal network architecture and tailored training strategy to enable robust multimodal representation learning un- der arbitrary modality combinations. Experiments on various natural and medical datasets showed DyMo's 's SOTA performance and the efficacy of its components. With the growing demand for mul- timodal deep learning, DyMo offers significant potential for real-world deployment on incomplete data. Future work will explore extensions to other tasks (e.g., segmentation) and modalities (e.g., video). An additional discussion for task extension is provided in Appendix. D.
ACKNOWLEDGEMENTS
This research has been conducted using the UK Biobank Resource under Application Number 40616.
Ethics Statement: This research complies with the ICLR Code of Ethics. The study makes of multiple datasets: publicly available benchmarks and the UK Biobank (UKBB) Resource under Application Number 406/6. All datasets are de-identified and contain personally identifiable information. No direct human subject recruitment was involved, and the work does not raise fore- secable risks to participants.
Reproducibility Statement:
We provide detailed descriptions of datasets, preprocessing steps, model architectures, training procedures, and evaluation protocols in Sec. 3 and Sec. 4 of the
10

---

Published as a conference paper at ICLR 2026
manuscript and Appendix B to ensure reproducibility of our model and compared approaches. Source code and model weights are available at https: http:: //tph/uggtth////// com/ wiiddDyoo..
LLM Usage: We note that large language models (LLMs) were used solely to improve language clarity; all scientific content and ideas were developed by the authors.
REFERENCES
Julián N Acosta, Guido J Falcone, Pranav Rajpurkar, and Eric J Topol. Multimodal biomedical AI. Nature Medicine, 2022.
Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE TPAMI, 2018.
Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, and Joydeep Ghosh. Clustering with breg- man divergences. JMLR, 2005.
Nourhan Bayasi, Ghassan Hamamnh, and Rafeef Garbi. BoosterNet:
Improving domain general- ization of deep neural nets using culpability-ranke features. In CVPR, 2022.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. In ICLR, 2023.
Bing Cao, Yinan Xia, Yi Ding, Changqing Zhang, and Qinghua Hu. Predictive dynamic fusion. In ICML. PMLR, 2024.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML.. PMLR, 2020.
Thomas M Cover. Elements of information theory. John Wiley Sons, 1999.
Yinglong Dai, Zheng Yan, Jiangchang Cheng, Xiaojun Duan, and Guojun Wang. Analysis of mul- timodal data fusion from an information theory perspective. Information Sciences, 2023.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In ACL., 2019.
Siyi Du, Nourhan Bayasi, Ghassan Hamameh, and Rafeef Garbi. AVIT: Adapting vision transform- ers for small skin lesion segmentation datasets. In MICCAIW. Springer, 2023.
Siyi Du, Shaoming Zheng, Yinsong Wang, Wenjia Bai, Declan O'Regan, and Chen Qin. TIP: Tabular-image pre-training for multimodal classification with incomplete data. In ECCV. Springer, 2024.
Siyi Du, Xinzhe Luo, Declan P O'Regan, and Chen Qin. STiL: Semi Semi-supervised tabular- labular-image learning for comprehensive task-relevant information exploration in multimodal classification. In CVPR, 2025.
Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied Al: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022.
William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. JMLR, 2022.
Xin Gao and Jian Pu. Deep incomplete multi-view learning via cyclic permutation of VAEs. In ICLR, 2025.
Zixian Gao, Xun Jiang, Xing Xu, Fumin Shen, Yujie Li, and Heng Tao Shen. Embracing unimodal aleatoric uncertainty for robust multimodal fusion. In CVPR, 2024.
Jifeng Guo, CL Philip Chen, Zhulin Liu, and Xixin Yang. Dynamic neural network structure: A review for its theories and applications. IEEE Transactions on Neural Networks and Learning Systems, 2024.
11

---

Published as a conference paper at ICLR 2026
Paul Hager, Martin J Menten, and Daniel Rueckert. Best of both worlds: Multimodal contrastive learning with tabular and imaging data. In CVPR, 2023.
Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang. Dynamic neural networks: survey, IEEE TPAMI, 2021.
Trevor Hastie, Robert Tibshirani, Jerome Friedman, et al. The elements of statistical learning, 2009.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. NIPS, 2017.
Wassily Hoeffding. Probability incqualities for sums of bounded random variables. Journal of the American Statistical Association, 1963.
Jingmin Huang, Bowei Chen, Lan Luo, al. DVM-CAR: large-scale automotive dataset for visual marketing research and applications. In 2022 IEEE International Conference on Big Data (Big Data). IEFE, 2022a.
Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang, and Longbo Huang. Modality competition: What makes joint training of multi-modal network fail in deep (proaiig.(.roabbyy).... In ICML..
PMLR, 2022b.
Ian Jolliffe. Principal component analysis. In International encyclopedia of statistical science. Springer, 2011.
Diederik P Kingma and Jimmy Ba. ADAM: A method for stochastic optimization. arXiv preprint 1r12v 6980, 2014.
Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu, and Chen- Yu Lee. Multimodal prompting with missing modalities for visual recognition. In CVPR, 2023.
Junnan Li, Caiming Xiong, and Steven CH Hoi. CoMatch: Semi-supervised learning with con- trastive graph regularization. In ICCV, 2021.
Sijie Li, Chen Chen, and Jungong Han. SimMLM: A simple framework for multi-modal learning with missing modality. ICCV, 2025.
Jingehen Liu, Andrew Gelman, Jennifer Hill, Yu-Sung Su, and Jonathan Kropko. On the stationary distribution of iterative imputations. Biometrika, 2014.
Qidong Liu, Jiaxi Hu, Yutian Xiao, Xiangyu Zhao, Jingtong Gao, Wanyu Wang, Qing Li, and Jiliang Tang. Multimodal recommender systems: A survey. ACM Computing Surveys, 2024.
Chao Ma. Sebastian Tschiatschek, Konstantina Palla, Jose Miguel Hernandez-Lobato, Sebastian Nowozin, and Cheng Zhang. EDDI: Efficient dynamic discovery of high-value information with partial vae. In JCML. PMLR, 2019.
Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. SMIL: Multimodal learning with severely missing modality. In AAAI, 2021.
Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal transform- robust to missing modality? In CVPR, 2022.
Yingyao Ma, Yifan Xue, Jiasong Wu, Lotfi Senhadji. Huazhong Shu, and Jian Yang. Multimodal en- tity linking with dynamic modality selection and interactive prompt learning. IEEE Transactions on Knowledge and Data Engineering, 2025.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. JMLR, 2008.
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.
Natalia Neverova, Christian Wolf, Graham Taylor, and Florian Nebout. ModDrop: adaptive multi- modal gesture recognition. IEEE TPAMI, 2015.
12

---

Published as a conference paper at ICLR 2026
Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, Andrew Y Ng, et al. Multi- modal deep learning. In ICML, 2011.
Emanuele Palumbo, Imant Daunhawer, and Julia E Vogt. MMVAE+: Enhancing the generative quality of multimodal vaes without compromises. In ICLR. OpenReview, 2023.
Emanuele Palumbo, Laura Manduchi, Sonia Laguna Cillero, Daphné Chopard, and Julia E Vogt. Deep generative clustering with multimodal diffusion variational autoencoders. ICLR. Open- Review, 2024.
Yuntao Shou, Jun Yao, Tao Meng, Wei Ai, Cen Chen, and Keqin Li. GSDNet: Revisiting incomplete multimodaliy- -iffuusin emotion recognition from the perspective of graph spectrum. In IJCAI, 2025.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning, NIPS, 2017.
Cathie Sudlow, John Gallacher, Naomi Allen, Valerie Beral, Paul Burton, et al. UK Biobank: an open access resource for identifying the causes of wide range of complex diseases of middle and old age. PLoS Medicine, 2015.
Thomas Sutter, Imant Daunhawer, and Julia Vogt. Multimodal generative learning utilizing jensen- shannon-divergence. NIPS, 2020.
Thomas M Sutter, Imant Daunhawer, and Julia E Vogt. Generalized multimodal ELBO. In ICLR, 2021.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW). IEEE, 2015.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. NIPS, 2017.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. NIPS, 2016.
Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro. Multi- modal learning with missing modality via shared-specific feature modelling. In CVPR, 2023a.
Yuanzhi Wang, Yong Li, and Zhen Cui. Incomplete multimodality-diffuse emotion recognition. NIPS, 2023b.
Shicai Wei, Chunbo Luo, and Yang Luo. MMANet: Margin- distillation and modality aware regularization for incomplete multimodal learning. In CVPR, 2023.
Sangmin Woo, Sumin Lee, Yeonju Park, Muhammad Adi Nugroho, and Changick Kim. Towards good practices for missing modality robust action recognition. In AAAI, 2023.
Renjie Wu, Hu Wang, Hsiang-Tine Ting Chen, and Gustavo Carneiro. Deep multimodal learning with 2024a.
missing modality: A survey. arXiv preprint
Zhenbang Wu, Anant Dadu, Nicholas Tustison, Brian Avants, Mike Nalls, Jimeng Sun, and Faraz Faghri. Multimodal patient representation learning with missing modalities and labels. In ICLR, 2024b.
Yingxue Xu, Fengtao Zhou, Chenyu Zhao, Yihui Wang, Can Yang. and Hao Chen. Distilled prompt learning for incomplete multimodal survival prediction. In CVPR, 2025.
Chonghua Xue, Sahana S Kowshik, Diala Lteif, Shreyas Puducheri, Varuna H Jasodanand, Olivia Zhou, Anika Walia, Osman B Guney, Diana Zhang, Serena Poésy, al. Al-based differential diagnosis of dementia ctiologies on multimodal data. Nature Medicine, 2024.
Zihui Xue and Radu Mareulescu. Dynamic multimodal fusion. In CVPRW, 2023.
13

---

Published as a conference paper at ICLR 2026
Weiyi Yang, Richong Zhang, Junfan Chen, Lihong Wang, and Jaein Kim. Prototype- Prototype-guided guided pseudo labeling for semi- uumi-supervised text classification. In ACL, 2023.
Yang Yue, Yulin Wang, Bingyi Kang, Yizeng Han, Shenzhi Wang, Shiji Song, Jiashi Feng, and Gao Huang. Deer-VLA: Dynamic inference of multimodal large language models for efficient robot execution. NIPS, 2024.
Yifan Zhan, Rui Yang, Junxian You, Mengjie Huang, Weibo Liu, and Xiaohui Liu. A systematic literature review on incomplete multimodal learning: techniques and challenges. Systems Science & Control Engineering. 2025.
Chaohe Zhang, Xu Chu, Liantao Ma, Yinghao Zhu, Yasha Wang, Jiangtao Wang, and Junfeng Zhao M3Care: Learning with missing modalities in multimodal healthcare data. ACM SIGKDD, 2022.
Qingyang Zhang, Haitao Wu, Changqing Zhang, Qinghua Hu, Huazhu Fu, Joey Tianyi Zhou, and Xi Peng. Provable dynamic fusion for low-quality multimodal data. In ICML. PMLR, 2023,
Qingyang Zhang, Yake Wei, Zongbo Han, Huazhu Fu, Xi Peng, Cheng Deng, Qinghua Hu, Cai Xu, Jie Wen, Hu, et al. Multimodal fusion on low-quality data: A comprehensive survey. arXiv preprint arXiv:2404. 18947, 2024.
Tong Zhang, Shu Shen, and CL Chen. MICINet: Multi-level inter-class confusing information removal for reliable multimodal classification. arXiv preprint 1907744422. 2025.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dynamic diffusion transformer. In ICLR, 2025.
14

---

Published as a conference paper at ICLR 2026
Appendices
The appendices are structured to provide additional details and supporting evidence for the main manuscript. In Appendix A, we provide the detailed formulations of the proposed DyMo. Appendix B describes the datasets used in our experiments, together with the implementation de- tails for both DyMo and the bascline algorithms to ensure reproducibility. Appendix C presents additional experimental results, including complete results under various missing-date scenarios, extended ablation studies, generalization analyses (e.g., additional robustness evaluation for recon- struction quality), adaptive inference analysis, test-time task loss analysis, additional visualizations, and reconstruction performance assessments of modality recovery approaches. Finally, Appendix. D briefly discusses potential extensions of our DyMo beyond classification tasks.
Overview:
A DETAILED FORMULATION
A.1 RELATIONSIIIP BETWEEN TASK LOSS FUNCTION TASK-RELEVANT INFORMATION
Our dynamic selection algorithm leverages novel multimodal task-relevant information reward (MTIR) to rank the incremental multimodal task-relevant information gained from adding each re- covered modality. Since the underlying data distribution is unknown at inference time, we introduce a tractable proxy based on the task (i.e., classification) loss function. In this section, we derive the connection between task loss and task-relevant information, demonstrating that reducing the task loss can increase the lower bound on task-relevant information.
Let the true conditional distribution of labels given representations be p(y|z), and the model- predicted distribution be ()()zz), where 0 denotes model parameters, is the classification label, and z is the multimodal representation. The true classification cross-entropy (CE) loss over a test dataset D is:
Lee CE(p(y) z); g)(()()
- / p(z) I/S p(u|z) log (o(l=))d dz.
(S1)
We can decompose the cross-entropy into conditional entropy and conditional KL divergence:
L. = - p(())
p(x) [s,
[[xte) (n ds+ + p(x) [[ (ogl=))ogpppls)
[((t))
I plu pog de
=- - [(x)
p(x) 1/, lo
log
p(y|2) p(z) 1/, piolz) p (>>))
H((IZ)+ (L(p(s)=)]oadde
(S2)
Since conditional KL divergence non non-negative (Cover, 1999), we have
Loe 2 ZY..
(S3)
On the other hand, the task-relevant information contained in multimodal representations 2 can be quantified by the mutual information between Z and the labels Y (Tishby & Zaslavsky, 2015):
T(Y: Z) = Epp...) log ply, 2) p(u)p(z)
(S4)
-pp)) log p(g) Fpp(...) logr(y)=)
H(Y) H())
where HYY) is the entropy of labels. Combining this with Eq. S3 yields lower bound on mutual information:
IY; 2) 2 H(Y) Lee
(S5)
This shows that the lower bound on mutual information increases as the expected CE loss Lee de- creases, motivating the use of the loss tractable proxy for task-relevant information. In practice
15

---

Published as a conference paper at ICLR 2026
since the true distributions p(z) and p(y|z) are unknown, estimate the bound using the empirical test-time CE loss computed from one-hot labels. The empirical test-time CE loss is written as:
D
P
Lee
log go (15111))
(S6)
PP
Assuming that the model is well-trained, we follow the common bounded-loss assumption in prior work that the per-sample test-time CE loss is bounded in practice (Cao et al., 2024; Zhang et al., 2023; Mohri al., 2018). Therefore, we adopt a conservative bound:
5 eez = log (v 12, - G, Vj E {1. 2,
[DI}-
(S7)
Then, by Hoeffding's ' inequality (Hoeffding, 1963), with probability at least 6,
In(1/5)
Lee S Lee GV
(S8)
2|D|
Substituting this into Eq. S5, we obtain high-probabillty lower bound on the mutual information in terms of the empirical test-time CE loss:
In(1/5) 2/D]
I(Y; 2 H(Y - Eee G
with probability at least 1 - 6.
(S9)
In this equation, the last term GV
Im(1/5) is constant and will be nearly to zero when the test dataset 2 -P2'
size sufficient large. Therefore, reducing the empirical test-time CE loss can increase the lower bound on task-relevant information contained in multimodal representations which provides the theoretical foundation of our MTIR reward in the dynamic modality selection algorithm.
A.2 FORMULATION ON THE FEATURE SPACE
A regular Bregman divergence (Banerjee et al., 2005) d, is defined as:
dz(2,z') = ((z) ((2') (z z') Z')''a
(S10)
where & is a differentiable, strictly convex function of the Legendre type, e,g., squared Euclidean distance or Mahalanobis distance. Inspired by prototypical networks (Snell et al., 2017), the poste- rior probability p(y|z) can be interpreted exponential family density defined in the feature space.
mixture density estimation the training set with an
Any regular exponential family distribution pv (2, 9) with natural parameters O and cumulant func- tion can be written in terms of a uniquely determined regular Bregman divergence:
pv (z (0) exp( (z. 0) (O))))oo exp ( (- do ((0)))
(S11)
where (z) exp( (z) and (0) is the mean parameter. Consider regular exponential family mixture model with parameters {0k, ]
K
p(z[T) Σ T&PV (2(0a) = Σ mx exp (-d)(2,,(00))))ba(f)
(S12)
Given I, the posterior probability of assigning unlabeled data point cluster k: is:
ply = k|2) ply k)p(zly = k)
T
(S13)
p(z
Ex=1 गe exp - do (2, ((Ow)))
Following Snell al. (2017), we assume an equally-weighted mixture model with one cluster per class, where and ) = c denotes the class prototype estimated from the training data. The posterior probability then simplifies to:
expl exp(-ds(2) (d, cx))
ply k|2)
Ck ΣN i= [b
Σ ki
2
(S14)
- exp( - -d ( cHr))
= k)
16

---

Published as a conference paper at ICLR 2026
During training, we utilize the class prototypes to calculate auxiliary trastive loss Laxx. To avoid storing instance embeddings, we maintain the cumulative sum of em- beddings and sample counts for each class during training and compute the prototype at the end of each epoch.
Note that the representations of sample may differ across different modality subsets. To account for this, at inference time, we further define subset-specific class prototypes. In specific, for each class (I,..,, K and each non-empty modality subset C [M]. + 0, we construct:
Ck Em
Σ 1 ((( (1a
(m) mes))
(S15)
Ty
where h denotes sodallty-speccffe encoders and & is the multimodal transformer (notations follow Sec. 3.1). We then aggregate these prototypes into an averaged class prototype by considering all possible non-empty modality subsets:
CK
CkSS-
(S16)
2
SC
In practice, E₁ is substituted for c₁ in Eq. S14. For the ICS score calculation of a sample represen- tation 2, we apply its associated modality subset S to all training samples and compute Eq. 8 using these subset-specific representations.
In experiments, evaluated two common distance metrics: the squared Euclidean distance and the cosine distance. While cosine distance does not strictly belong to the class of Bregman divergences, is widely used in high-dimensional embedding spaces, where directional similarity is often more informative than vector magnitude (Li et al., 2021; Yang et al., 2023; Du et al., 2025). Our experimental results demonstrate that cosine distance achieves performance comparable Bregman divergences, suggesting its suitability as metric for our modality selection algorithm.
B DATASET AND IMPLEMENTATION DETAILS
B.1 DETAILED DATA DESCRIPTION
As summarized in Tab. S1, we conduct extensive experiments on 5 datasets with diverse modalities, e.g. image, text, and structured tables. These comprises simulated benchmark datasets (Sutter et al., 2021; 2020): PolyMNIST, MNIST-SVHN- -ext (MST), and biomodal CelebA (CelebA), as well as two large real-world datasets: a natural image dataset, Data Visual Marketing (DVM) (Huang et al., 2022a) and medical image dataset, UK Biobank (UKBB) (Sudlow et al., 2015).
PolyMNIST consists of 5 images per data point, all sharing the same digit label but with different backgrounds and handwriting styles. MST is trimodal dataset combining MNIST, street view house numbers (SVHN), and synthetic text features. CelebA contains facial images paired with descriptive text annotations of facial attributes. The DVM dataset includes 2D RGB images of cars along with tabular data describing vehicle characteristics. Following (Du et al., 2024; Hager et al., 2023; Du et al., 2025), used 17 tabular features, including 4 categorical features (e.g., color), and 13 continuous features (e.g., width). The UKBB dataset consists of cardiac magnetic resonance images (MRIs) accompanied by tabular data related to cardiovascular diseases. Following prior work (Hager et al., 2023), we used mid-ventricle slices from MRIs at three time points, i.e., end- systolic (ES) frame, end-diastolic (ED) frame, and an intermediate time frame between ED and ES. In addition, we employed 75 disease-related tabular features, including 26 categorical features (e.g.. alcohol drinker status) and 49 continuous features (e.g., average heart rate). Notably, due to low disease prevalence, similar to (Du et al., 2025), constructed 2 balanced training subsets for CAD and Infarction tasks, respectively. Detailed benchmark information for DVM and UKBB, including the complete list of tabular feature names, can be found in the supplementary material of (Du al., 2024).
B.2 IMPLEMENTATION DETAILS
For both DVM and UKBB, adopted the data augmentation strategies described in (Hager et al., 2023; Du et al., 2024). For image data, we applied random scaling, rotation, shifting, flipping. Gaus- sian noise, as well brightness, saturation, and contrastive changes, followed by resizing all images
17

---

Published as a conference paper at ICLR 2026
Table S1: Summary of the multimodal datasets for evaluation.
Daraset
Classiiiation Tisk
aModality
Modality Type
#Train
#Val
#Test
#Class
PolyMNIST
Digit
RGB image
60,000
3,000
7,000
10
MST
Digit
RGB image, Text
1,121, L1211360
60,000
140,000
CelebA
image,
162,770
19,962
19,867
DVM
model
RGB image, Table
70,565
17,642
88,207
283
Corouay anty disesse (CAD)
MR image. Table
3,482
6,510
3,617
Myncantial inforction
MR image, Table
1,552
6,510
3,617
2
Table S2: Aapeer-arrmmeter configurations for DyMo.
preee-prrmmeter
PolyMNIST
MST
CelebA
DVM
CAD
Infarction
layens of the multimodal transformer
2
2
2
2
2
2
attention heads multimodal transformer
4
2
2
8
8
8
Hiddlen dimension of multimodal transformer
128
32
32
256
256
256
Sequence length per modality
(4,4,4,4,4)
(1.1.1}
{1.1}
(1:16,7:17)
(1:16,7:75)
(1:16,7:75)
Lateut space dimension
64
16
16
128
128
128
sampled modality subsets (A)
5
2
2
2
2
2
Learning rate
1c-3
1c-3
1c-3
3e-4
3e-4
3e-4
Batch size
256
256
256
256
128
128
Maximum I epochs
100
20
20
300
300
300
to 128 x 128. For tabular data, categorical values (e.g., yes, no, and blue) were converted into or- dinal integers, while continuous (numerous) values were standardized using z-score normalization. To further enhance data diversity, we randomly replaced 30% of the tabular values for each subject with randomly sampled values from the corresponding columns. Hyper-parameter settings for the proposed DyMo and all compared approaches are detailed below.
Our MoMo: For the recovery method in DyMo, we leveraged MoPoE (Sutter et al., 2021), mul- timodal VAE, for PolyMNIST, MST, and CelebA. For DVM and UKBB, we used TIP (Du et al., 2024), an image-tabular framework capable of reconstructing tabular features from images and in- complete tables. Importantly, DyMo is agnostic to the choice of recovery method and can be de- ployed with any recovery method. For modality-specific encoders, as done in (Sutter et al., 2021), we used convolutional neural networks (CNNs) for images and multi-layer perceptrons (MLPs) for text in PolyMNIST, MST, and CelebA. For DVM and UKBB, we followed (Du et al., 2024), using ResNet-50 the image encoder and transformer-bast coder consists of 4 transformer layers, each with 8 attention heads and a hidden dimension of 512. To ensure fairness, all comparing approaches employed the same encoders as DyMo. To mitigate the curse of dimensionality, multimodal representations z were projected into a low-dimensional latent space using a 2-layer MLP before distance computation. The temperature parameter for distance metrics was set to 0. Hyper-parameter configurations for DyMo are summarized in Tab. S2.
encoder for tabular data. The tabular
CONCAT (Baltrušaitis et al., 2018): This static fusion algorithm has been widely adopted in MDL tasks due to its simplicity and effectiveness. CONCAT concatenates modality-specifff features into a unified multimodal representation, which is then used for the classification task.
QMF (Zhang et al., 2023): This dynamic MDL framework performs modality fusion through uncertainty-awe -ware weighing.
podaaltt-sspeiffi uncertainty estimated via an energy score com- puted from the output logits of each unimodal network. In addition, QMF incorporates regular- ization loss based on the historical training trajectory. Following the original paper, the number of gradient accumulation steps was set to 12 for CAD and Infarction, and 24 for the remaining tasks.
DynMM (Xue Marculeseu, 2023): This dynamic MDL approach employs a learnable gating network. Given M modalities, it constructs M + network branches: M modality-speciie net- works and one concatenation-base multimodal network. The gating network adaptively selects one branch for decision- each network branch independently on the targe task, and (ii) jointly fine-tuning all network branches together with the gating network.
Note that this approach requires two-stage training: (i) pre-training
PDF (Cao et al., 2024): This dynamic MDL framework introduces a modality confidence score to fuse unimodal predictions. The score is derived from the predicted class probability of each
18

---

Published as a conference paper at ICLR 2026
unimodal network. In addition, to address potential uncertainty, relative calibration strategy is further applied to calibrate the confidence scores.
MultiAE (Ngiam et al., 2011): This incomplete MDL model is an offline recovery- method. first pre-trains a multimodal autoencoder (AE) with a reconstruction objective, then freezes the AE encoder and uses the extracted latent features to train classifier for downstream tasks. Since the original design does not simulate missing modalities during training, we incorporated our incom- plete simulation training to obtain MultiAE The hyper-parameter settings of this strategy were aligned with our DyMo (see Tab. S2). The number of pre-training epochs was set to 300 for PolyM- NIST and 200 for MST and CelebA. The same learning rate was used for both pre-training and fine-tuning, matching the learning rate used by DyMo in its one-stage training.
(Sutter et al., 2021): This incomplete MDL approach is also offline recovery-based. It first pre-trains a multimodal VAE with an ELBO formulation tailored for incomplete multimodal data. The encoder then frozen, and its extracted latent features are used train classifier for downstream tasks. Pre-training was performed for 100 epochs, and the learning rates for pre-training and fine-tuning were identical, equal to that used by DyMo
MoPoE
M3Care (Zhang et al., 2022): This incomplete MDL framework is an online recovery-based method. It imputes missing modality information the latent space using auxiliary information from similar patents, identified via task-guided, modality-adaptive similarity metric. Since the original design does not simulate modality missing during training, we incorporated our incomplete simulation training to obtain M3Care The syper-parameter settings of this strategy were aligned with our DyMo
OnLineMAE (Woo et al., 2023): This incomplete MDL model is online recovery recovery-based method. It jointly optimizes a feature-level reconstruction task and the target classification task by randomly dropping modality features and reconstructing them from the remaining modalities. In reproduction, we observed that training from scratch often led to model collapse, likely because the network initially fails to extract reliable features, making feature-level reconstruction unstable. To mitigate this, we initialized the encoders with weights from MultiAE (after downstream task training) for PolyMNIST, MST, and CelebA, and weights from TIP (after downstream task training) for DVM, CAD, and Infarction.
ModDrop (Neverova et al., 2015): This incomplete MDL framework is aggnostic, recovery-free method. It applies random modality dropping during target task training to encourage representation learning. Each modality is dropped according to Bernoulli distri- bution with probability p. For a fair comparison, we used the same multimodal network architecture as our DyMo. Following the original paper, p was set to 0.5.
MTL (Ma et al., 2022): This incomplete MDL model is missing-aware, recovery-free method. It introduces a multimodal transformer architecture with missing-awes aware [CLS] tokens, which encode different missing patterns. Attention masks those [CLS] tokens ensure that they only aggregate information from observed modalities. To optimize missingness-specifii parameters, a multi-task learning strategy is used compute loss across all missing patterns in each training step. The sequence length per modality and transformer configurations matched those of our DyMo
MAP (Lee et al., 2023): This incomplete MDL approach is also missing- aware and recovery-free It designs a multimodal transformer architecture with missing-aware prompts. Prompts are assigned based on the missing pattern of each input and injected into multiple transformer blocks. Since the original paper does not simulate missing modalities during training, we applied our incomplete simulation training to obtain MAP The sequence length per modality and transformer configura- tions followed those of our DyMo. Following the original paper, prompt lengths were set to 12 for PolyMNIST, 2 for MST and CelebA, 8 for DVM, and 16 for CAD and Infarction. Prompts were applied to 2 transformer layers.
MUSE (Wu et al., 2024b): This incomplete MDL framework is a missing-agnostic, recovery- free method. It models patient-modality relationships using flexible bipartite graph that supports patterns. The vertex set includes patient nodes and modality nodes, and edges are defined by the modality missingness matrix. To learn poodality-agnoosse features, a mutual- consistent contrastive loss is applied, where each edge is dropped with probability p. Following the original paper, p was set to 0.5.
arbitrary missing-modaliin
19

---

Published as a conference paper at ICLR 2026
Table S3: Complete results on PolyMNIST and MST under various missing scenarios, compar- ing DyMo with incomplete MDL methods, including both recovery-based and recovery-free ap- proaches. Models marked with were trained using our proposed incomplete-modalle tion training. In addition to the results reported in Tab. 1, we also include PolyMNIST results for 7 = (0.200.4) and MST results for missing modality subsets (M). (S), (T), and (M.S).
simula-
Model
Publicution
MONST Acc (5)
MST Acc (5)
Missing Rate
Missing Modalityles
0.2
0.4
0.6
0.8
(M}
{S}
(T)
(M.S)
(S.T)
(MLT)
(a) Recovery- based Methods for Missing
Modilltt
MuldaA
ICML'11
99.77
99.39
98.56
95.36
84.39
99.96
99.74
99.98
98,30
100
97.00
81.60
MultiAB!
[CML'11 11
99.94
99.83
99.03
97.50
89,86
99.87
98.65
99.93
98.66 60
100
98.33
83.44
MoPoT
ICLR 21
99.79
99.41
98. 13
93.94
79.84
99.62
91.32
99.33
92.28
90.11
90,86
79.01
MoPoE
ICLR'21
99.63
99.57
98.96
96.81
87.06
99.39
99.42
99.18
97.62
100
96.50
82.54
M3Care
KDD 22
99.93
89.26
74.93
56.66
40.53
99.99
40.64
85. 18
23.48
41.69
16.03
9.34
M3Care
KDD'22 22
99.99
99.93
99.40
97.27
87.92
99.98
100
99.98
99.17
100
98,27
85. 16
OnlineMAR
2AAT223
100
99.89
99.61
98.29
90.09
99.90
99.70
99.96
97.94
100
14
84. 84.14
CMVAE
ICLR'24
94.93
95.50
95.21
95.13
95.11
CMVABT
ICLR'24
94.59
95.23
94.93
95.17
95,20
(b) Recovery-free Methods Missing Modility
ModDrop
ICML'15
99.97
99.77
99.43
97.66
88.44
100
100
100
99.14
100
98,21
82.47
MTT.
CVPR'22
99.97
99.91
99.61
98.43
91.14
99.96
99.97
99.96
98.82
98.60
84.37
MAP
CVPR'23
99.
66.40
43.00
23.19
100
65.32
58.14
15.28
80.29
9.83
10.13
MAP
CVPR'23
99.99
99.86
96.74
76.20
99.99
99.99
97.87
97.84
11.36
MUSE
ICLR'24
99.93
98.61
94.73
77.56
99.86
99.99
83
97.51
97.14
35.96
(c) Dynamic Recovery Method Missing Medality
DyMo,
99.99
99.71
96.61
100
100
99.01
98.22
85.31
DyMo,
100
100
99.99
99.87
96.81
100
99.98
100
99.22
98.43
86.84
Table S4: Complete results on DVM, CAD, and Infarction under various missing scenarios, compar- ing DyMo with incomplete MDL methods, including recovery based and recovery- approaches. Models marked with were trained using proposed incomplete -odaltity simulation training. In addition to the results reported in Tab. 1, we also include results for = {0.1, 0.3, 0.5, 0.9}.
Model
DVM Acc (%)
CAD AUC
Infarction (S) Missing Tabular Rate
Missing Tabular Rate
Missing Tabular Rate
03
03
07
0.
03
03
0.7
0.9
03
0.5
07
0.9
(0) Recovery- Methodi for Missing Modality
MrCare
64.99
70.61
70.53
M3Can!
98.94
93.43
72:48
72.48
83.27
68.44
OnlineMAE
(b) Recovery face Mechods for Missing Modality
ModDDoo
99.02
98.73
95.38
89,30
87.97
$5.10
92.96
M0.39
76.65 70.777 69.18
54.76
H1.01
74.64
22.16
MTL.
20.44
99.43
98.70
97.42
95.53
93.38
92.32
84.87
25.22
83.82
50.08
77.72
73,05
70.23
83.59
83.00
81.33
79.36
75.73
69.82
69.30
MAP
98.86
98.53
95.83
86.96
74,55
63.15
$4.29
83.81
81.52
78.58
75.61
71.39
70:11
54.62
83.40
75.30
75.92
71.73
68.47
69.17
MAP!
99.17
99.10
98
20
96.30
94.83
92.43
91.15
85.26
64.66
W2.76
80.14
75.84
70.36
68.76
85.40
$4.45
82.08
78.94
75.88
71.58
70.81
MUSE
96.86
83.47
53.23
84.40
66.78
Dynamic Recovery Method Missing Modillit
DyMa;
99.30
99.10
98.50
47.62
96.31
94.30
98.14|
85.14
84.53
N2.77
$1.10
78.49
75,92
71.02
$5.10
84
82.34
80.76
77.85
75.72
71.58
DyMa,
99.50
98.84
98.08
96.81
94.76
93.36
86.17 N5.73 83384 $1,49 78.24 74.01
22.17
16
82.70
79.05
26.14
72.47
We trained all models using the Adam optimizer (Kingma & Ba, 2014) without weight decay and ran experiments on a single NVIDIA A5000 GPU. To mitigate overfitting. similar to (Du et al., 2024; Hager al., 2023), we employed an early stopping strategy, with a minimal divergence threshold of 0.0001, a maximal number of training epochs (see Tab. S2), and a patience (stopping threshold) of 20 epochs. All models were trained with the same learning rate and batch size as our DyMo (see Tab. S2). We ensured convergence of all methods under this configuration.
C ADDITIONAL EXPERIMENT
C.1 COMPARING AGAINST INCOMPLETE MDL SOTAS (COMPLETE RESULTS)
In Sec. 4.1 of the manuscript, we report results for the most challenging missing scenarios on PolyM- NIST, MST, DVM, CAD, and Infarction in Tab. 1, due to space constraints. Here, we provide the complete results in Tab. S3 and Tab. $4.
20

---

Published as a conference paper at ICLR 2026
[a] multimodal network MST
MST (M.T) missing:
100
MTL DyMo
05
[1,2,47]
Accuracy (%6
60
Accuracy
40
WDO.
Am1
70
Aw2
A-4
(5)
(M,
Observed
All recovery
DyMo's selection
Missing modalitylies
Figure S1: (a) Performance of DyMo,'s multimodal network on MST under various missing sce- narios, evaluated with different numbers sampled modality subsets (A), w/o denotes training without our incomplete-modalie simulation strategy. (b) Comparison between MTL and DyMo, on MST with {M.T) missing, under different modality inputs: (1) using only non non-missing modalities; (2) integrating all recovered modalities without selection; (3) integrating only the recovered modal- ities selected by DyMo..
Table S5: Classification accuracy (%) of DyMo using different modality recovery methods on DVM and Infarction under various missing modality rates.
Model
Recovery Method
DVM
DVM
Infarction
Infarction
7 0.7
= 0.9
1) = 0.7
f = 0.9
CONCAT
IMI
81.85
67.13
73.75
69.39
DyMo.
IMI
95.97
94.28
75.96
71.09
CONCAT
TIP
94.74
86.08
75.91
72.04
DyMo.
TIP
96.81
94.76
76.14
72.96
C.2 ABLATION STUDY & GENERALIZABIILTY ANALYSTS
Effeet of the Incomplete Simulation Training Strategy: We examined the impact of the number of sampled modality subsets (A) during incomplete-modality simulation training MST under various missing scenarios. As shown in Fig. Si(a), removing our incomplete simulation training leads to substantial performance drop, e.g., accuracy decreases by 71.91% % on MST with missing (M,T), compared to models trained with = 2. In contrast, when this training strategy is applied, performance remains stable across different values of A, suggesting that even small A is sufficient to achieve both efficiency and strong performance.
Efficacy and Applicahility of DyMo's Selected Recovered Modalities: We compared DyMo's multimodal network with MTL. a recovery-free transformer-based method, under three input set- tings: (1) using only non-missing modalities; (2) integrating all recovered modalities without lection; and (3) integrating the recovered modalities selected by DyMo. As shown in Fig. S1(b), for both models, naively integrating all recovered modalities results in lower accuracy than using only the non-missing modalities, indicating that some imputed modalities are musk-irrelevan and can negatively affect decision-male making. In contrast, integrating only the recovered modalities se- lected by DyMo improves performance of both models, demonstrating the effectiveness of DyMo's selection algorithm and the applicability of its selected recovered modalities to other models. More- DyMo consistently outperforms MTL across all input settings, which showcases the efficacy of DyMo's 'ss network architecture and training strategy.
Robustness of DyMo to Modality Recovery (Reconstruction) Quality: In Sec. 4.2 of the manuscript, we evaluate robustness of DyMo across different modality recovery methods on PolyM- NIST. Here, further assess an alternative recovery method DVM and UKBB. Specifically, replace TIP with the iterative multivariate imputer (IMI), a widely-used tabular imputation method that recovers missing values using information from other table columns (Liu et al., 2014). Since IMI
21

---

Published as a conference paper at ICLR 2026
(a) MST (missing (M, T})
(b) CelebA (missing {T))
100
100
80
95
Accuracy (%)
Accuracy [%)
85
CONCAT
CONCAT
75
DyMo
DyMo
0.7
0.5
0.3
0.0
0.7
0.5
0.3
0.0
Correct recovery rate
Correct recovery rate
Figure S2: Results MST (missing {M. ТІ) and CelebA (missing (T)) with different correct
recovery rates r.
Distribution of iteration steps on PolyMNIST (Avg: 1.38)
Number of iteration steps
4 -0.2%
3
3.1%
2
31.1%
65.5%
80
100
Percentage of test samples (%)
Figure S3: Distribution of iterative selection steps per test sample for DyMo on PolyMNIST with missing modality rate r 0.8 (i.e., each sample randomly misses 4 out of 5 modalities). The average number of steps per sample is 1.38.
relies only on table information, it yields lower-quality reconstructions than TIP (Liu et al., 2014). As shown in Tab. S5, with this weaker reconstructor, CONCAT suffers a large accuracy drop com- pared its performance with TIP (e.g. 18.95% accuracy decrease on DVM with 90% missing tabular features). In contrast, DyMo remains stable and achieves the best performance under both recovery methods. These results show that DyMo is not constrained by the reconstruction-qualie bottleneck.
To further stress-test DyMo's under noisy recovery quality, we conducted an extreme simulation ex- periment with controlled correct recovery rate For r 100% of samples, their missing modalities are imputed using their ground-truth versions, while for remaining (1 r) x 100%, their missing modalities are replaced with zero-valued noise. We evaluate DyMo MST (missing M,T) and CelebA (missing T). As shown in Fig. S2, prior static and dynamic fusion methods degrade sharply recovery quality deteriorates. In contrast, DyMo remains relatively stable across wide range of correct recovery rates, suggesting its ability to effectively disregard unreliable recovered modalities.
C.3 ADAPTIVE INFERENCE ANALYSIS
To study how DyMo dynamically selects task-relevant recovered modalities at inference time, we visualized the distribution of iterative selection steps per sample PolyMNIST, when 80% of modalities are missing. Fig. S3 shows that samples require different numbers of iterations, reflecting variations in the quality of recovered modalities and the adaptive nature of DyMo. At each itera- tion, only recovered modalities that provide a neglnegligi incremental multimodal task-relevant information added. In addition, most samples require only 1-2 steps, and the average number of
22

---

Published as a conference paper at ICLR 2026
Table S6: Test loss range PolyMNIST with 80% missing modalities (6 0.1).
Dataset
Dataset
Size
Min
Max (G in
G ( in Eq. 2
[D]
Eq. 2)
PolyMNIST 7,000
0.00
4.14
0.0053
iterations 1.38, suggesting that although iterative selection introduces some additional computa- tion, the extra cost compared to DyMo w/o iteration selection is moderate.
C.4 TEST-TIME TASK Loss ANALYSIS
We show the test CE loss range for DyMo on PolyMNIST with 60% missing modalities in Tab. $6. The results show that the test CE loss stays within well-behaved numerical range. The G-related term extremely small (< 0.006), and thus will not affect the tightness the lower bound in practice (Eq. 2).
C.5 VISUALIZATION
Latent Feature Space Visualization on Training Data: The MTIR reward a recovered modality in DyMo is based on the representation shift in the latent space relative to the training distribution after adding that modality. We used t-SNE to visualize the multimodal latent space learnt by DyMo's multimodal network training data under various missing scenarios. As shown in Fig. S4, samples from different classes are generally well-separated, demonstrating that DyMo effectively learns a structured feature space suitable for reward calculation.
Input-I.evel Case Study: To understand which recovered modalities are selected by DyMo, we conduct input-level case studies comparing DyMo with other dynamic/static fusion methods, as shown in Fig. 5(b) in the manuscript and Fig S5. The results indicate that (1) the recovery method may generate modalities of varying qualities across samples; (2) existing dynamic MDL methods information for estimating modality importance often fail to identify semantically misaligned recovery (e.g., right case in Fig. S5(b)), which can degrade model performance; and (3) DyMo, however, alleviates this limitation by selectively incorporating benefi- cial recovery while disregarding unreliable one, thereby improving overall performance. Moreover, Fig. S5(d) illustrates particularly challenging cases for all models, where both non-missing and covered modalities provide limited rele-nelevant information. In these rare and difficult cases, DyMo may not fully correct its initial mispredictions, suggesting that the use of more advanced recovery methods could further enhance performance.
that rely solely on modality-specins
C.6 RECOVERED MODALITY ANALYSIS
VAE-based Reconstruction Analysis: Fig. 5(b) in the manuscript and Appendix C.5 present amples of recovered modalities. We further provide quantitative analysis of these reconstructions. Specifically. We evaluated the reconstruction performance of VAE-based modality recovery meth- ods (i.e., MoPoE, MMVAE+, and CMVAE) on PolyMNIST. Following prior studies (Palumbo et al., 2023; Sutter et al., 2021), ), assessed cross-modal generation using two complementary metries: (i) semantic generative coherence, measured by the accuracy of generated modalities (i.e., conditional coherence accuracy); and (ii) generative quality, measured by the similarity between generated and real samples using the Fréchet Inception Distance (FID) score (Heusel et al., 2017). Additional de- tails of these metrics can found Appendix D.3 of (Palumbo al., 2023). As shown in Fig. $6, CMVAE consistently outperforms the other methods in both cohenrence and quality across different numbers of input modalities. This finding aligns with the observation that DyMo combined with CMVAE achieves the best classification performance (see Tab. 3). Notably, while MoPoE performs substantially worse than MMVAE+ and CMVAE, DyMo exhibits smaller performance gaps across all three recovery methods. This demonstrates that DyMo is robust to different recovery techniques by dynamically integrating tell-aelev recovered modalities while disregarding unreliable ones, highlighting its selective and adaptive behavior.
23

---

Published as a conference paper at ICLR 2026
(a-1) PolyMNIST (17
(a-2) PolyMNIST (1 0.2)
(a-3) PolyMNIST 0.8)
(b 1) MST (missing (})
(b 2) MST (missing (S,T))
(b 3) MST (missing (M, T))
(c-1) CelebA (missing ())
(c-2) CelebA (missing (1)
(c-3) CelebA (missing (T))
(d-1) DVM (missing ())
(d-2) DVM (missing (1))
(d-3) DVM (missing (T))
Figure S4: t-SNE visualization of DyMo,'s 's multimodal network on the training data of (a) PolyM-
NIST, (b) MST, (c) CelebA, and (d) DVM under various missing scenarios. Colors denote different class labels.
24

---

Published as a conference paper at ICLR 2026
(at Tadi-miinaan recoeeeed reddiiiion
(MM 1335 0.0.0.1. 01:11
7
7
7
J Ca00 bu 1.26-0 0.10
MM03
022.022 02000.2..300
ACONCSt:7
COMCATIC
SMF- (0:44 022, 0.1,, 028,022)
X 234 10 D.1 023,0.25 6.04
Dawoss
Butlous
Ground Inll: coNCATt
Ground mrutt KT12
7
Ladie p.t. 022, 922, 236, Aua)
20011 207,
216 Ti:5 0.9, 0.99,070,0000
Bade
Bibto
the kanging deserts modaltie watted reowaned
Becandbuth222
Ground trutto3
CONCAT
Dnvist
Fcc 15.20
022.0.24 025, OCT)
X TDC. 0.10 025, 0.22, 0.24,
Bitto
Figure S5: Extending the cases shown in Fig. 5(b), this figure presents additional representative ex- amples of model predictions on PolyMNIST with missing rate ry = 0.8 for DyMo and static/dynamic multimodal fusion methods. The sub- sub-figures illustrate: (a) all recovered modalities are task-relewe relevant; (b) recovered modalities vary in quality, and DyMo preserves correct predictions by disregarding unreliable recoveries; (c) recovered modalities vary quality, and DyMo corrects initial mispre- dictions by incorporating task-relevant recovered modalities; (d) particularly challenging cases with limited task-relevant information both non-missing and recovered modalities. Yellow boxes in- dicate non-missing modalities, and blue boxes indicate modalities selected by DyMo,- v denotes correct predictions, while x denotes incorrect predictions. For QMF and PDF, which perform dy- namic weighted fusion, report the weights assigned to each modality for every sample.
(a) Generation coherence (higher is better)
(b) Generation quality (lower is better)
Conditional coherence accuracy (%6
250
100
Conditional FID score
80
150
MMVAE+
100
60
MoPoE
MaPoE
50-
MMVAE
CMVAE
MMVAE+
CMVAE
CMVAE
40
0
Number of input modalities
Number of input modalities
Figure S6: Reconstruction performance of different VAE-based recovery methods on Poly MNIST: (a) reconstruction coherence (higher is better), (b) reconstruction quality measured by FID (lower is better).
Table Reconstruction Analysis: Tab. S7 and Fig. S7 report the reconstruction results of tabular features for DVM and UKBB under full-table missingness, respectively. Categorical features evaluated using accuracy, while continuous features are assessed with mean squared error (MSE). The results show that most tabular features are accurately reconstructed, e.g., color in DVM (Ace: 82.46%), suggesting that their integration can benefit the target classification task. Some tabular features, however, exhibit low reconstruction performance, likely due their relatively weak corre- lations with the image modality. Examples include fuel type in DVM (Ace: 64.45%) and smoking status in UKBB (Acc: 4.4%). In real-world applications, though, tabular features are more often partially missing rather than entirely absent (Wu et al., 2024b; Xue et al., 2024). Under tabular missing rate y = 0.3, we observed that the reconstruction performance of these challenging features improves substantially, e.g., fuel type in DVM (64.45% 87.13%) and smoking status in UKBB (4.4% + 86.99%). These findings suggest that the proposed DyMo can generalize well to practical scenarios, where partial missingness is common than than full-table missingness.
25

---

Published as a conference paper at ICLR 2026
Table S7: Reconstruction performance of 17 tabular features (4 categorical and 13 continuous) using TIP on DVM under full-table missingness (i.e., 7 = 1). Cat denotes whether tabular feature categorical, and N Nung represents the number of unique values for each categorical feature.
Feuture
Mony MSE L
Feuture
Mang
Advertisement month (Advv_oonth)
1.2340
Hoight
0.3234
Advertisement year (Adi.yycar)
Length
Bodyrype
13
82,24
Price
0..9211
Color
22
82.46
Registration yoar (Rog.ycar)
0.2622 X
Number of doors (Door.num)
0.3641
Miles runned (Runned
0.6962
Pagine size (Fagine_sive))
Number of (Sem)num)
Entry prize (Eatry.prize)
0.2908
Wheelbase
0.5938
Fuel type (ypel_yye)
12
64.45
Width
0.5873
Gearbox
(s) UKBB's categorical feature reconstruction
100
80
Acourery
20
Canegorical frabure
continuous reconstruction
1.5
1.0
0.5
your perche
Figure S7: Reconstruction performance of 75 tabular features (26 categorical and 49 continuous) using TIP on UKBB under full-table missingness (i.e., ? = 1).
26

---

Published as a conference paper at ICLR 2026
D DISCUSSION
Beyond Classification Tasks: While this paper primarily focuses on classification tasks, the DyMo's framework can be extended broader range of multimodal tasks. The key adaptation is to replace the cross-entropy (CE) term with the appropriate likelihood--aase loss for the target task. Many multimodal tasks, e.g., detection, segmentation, and can be trained using probabilistic losses, and thus the same mutual information decomposition, I(Y: HYY (Y remains applicable. In specific: (1) segmentation: MTIR can operate on the averaged per-pixel CE loss; (2) detection: MTIR can incorporate both classification and local- ization likelihoods as the task loss; and (3) sequence-to-sequeee modelling: MTIR can aggregate token-level CE losses to guide modality selection.
modelling,
27


Reviews:
[{"review_id": "XxAR1Q8Uto", "reviewer": "Reviewer_orCx", "summary": "The paper tackles incomplete multimodal classification. It introduces an inference-time dynamic modality selection framework that (i) reconstructs missing modalities with an external recovery model, then (ii) selects only the “reliable” reconstructions to fuse with available modalities. The key idea is a multimodal task-relevant information reward (MTIR) as a proxy for information gain, plus a latent-space metric calibration to guard against low-fidelity reconstructions. Experiments show gains over both recovery-free and recovery-based baselines, especially in severe missingness.", "questions": "1. In Section 3.1, could the “dummy token” embeddings still introduce bias into positional encodings if not learned or masked carefully?\n\n2. In Section 3.2, the computation of class prototypes as arithmetic means assumes locally Euclidean and unimodal feature geometry. This assumption may not hold in practice. Could you provide statistics or visualizations showing that class cluster means are representative?\n\n3. Why model intra-class distances with a truncated normal distribution? Heavy-tailed or multi-modal classes may violate it. Have you explored nonparametric alternatives (e.g., kernel density) to avoid this assumption? Is there empirical evidence that ICS meaningfully quantifies the representativeness of z within its class cluster?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Clear problem framing (“discarding–imputation dilemma”) and a practical angle: selecting only useful reconstructions instead of always discarding or always imputing.\n2. The method is grounded in an information-theoretic heuristic that links mutual information and cross-entropy\n3. The method has flexible architecture and straightforward training recipe. \n4. Experiments cover five datasets and multiple missing-modality regimes. Reported gains are meaningful in high-missingness settings.", "weaknesses": "1. Loose MI–CE bound.\n(a) Heuristic bound. The proposed lower bound linking task-relevant information I(Y;Z) to the empirical CE loss is very loose and largely heuristic. The bound involves constants G, yet G can be arbitrarily large since the CE loss is unbounded in practice. As a result, the lower bound can collapse to a small or even meaningless value. Moreover, a reduction in CE loss does not necessarily imply increased mutual information.\n(b) Dataset-level rather than per-sample validity.\nThe high-probability guarantee in the bound applies to the randomness of the training dataset \\mathcal{D}, not to individual test samples. It does not provide theoretical support for the per-sample reward computation used during inference without many further assumptions.\n\n2. Dependency on recovery quality.\nDyMo’s selection quality depends entirely on the fidelity of recovered modalities. When all reconstructions are poor, the model effectively reverts to observed modalities while still incurring extra inference cost. \n\n3. Task scope.\nThe method is evaluated for classification. Since many multimodal applications are detection/segmentation/seq-to-seq, a short discussion of what’s needed to extend MTIR beyond CE classification would strengthen the impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nf1dfVcDyp", "reviewer": "Reviewer_V4dA", "summary": "This paper tackles incomplete multimodal classification by explicitly framing the discarding–imputation dilemma and proposing DyMo, an inference‑time dynamic modality selection framework. The method recovers missing modalities via any recovery model, then iteratively selects only those recoveries that yield positive task‑relevant information according to a reward grounded in a connection between mutual information and cross entropy, refined with an intra‑class similarity (ICS) calibration, and implemented through a lightweight greedy procedure. The paper pairs this with a transformer‑based multimodal backbone that accepts arbitrary modality subsets and a simple incomplete‑modality simulation training scheme. Extensive experiments on five datasets (natural and medical) show consistent gains, especially at high missing rates.", "questions": "- Could the authors provide a more in-depth analysis of the choice of distance metric, either theoretical or empirical?\n\n- Please justify the selection of the greedy approach.\n\n- Would the selection mechanism remain effective independent of the chosen reconstructor?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- This paper perceptively identifies discarding–imputation dilemma in incomplete multimodal learning, which is an interesting and practical research problem.\n\n- Selection criterion grounded in standard MI-CE relations and rendered computable via prototype‑based energies (Eq. 5–7), yielding an interpretable \"move‑toward‑the‑prototype\" test. \n\n- Broad empirical coverage with competitive results on five datasets, strong robustness under severe missingness, and ablations demonstrating the contribution of each component. \n\n- Good practicality: recovery‑method agnostic with positive results across MoPoE/MMVAE+/CMVAE and modest extra inference steps on average.", "weaknesses": "- Prototype posterior in Eq. 5–6 presumes Bregman divergences; the cosine distance used in $\\text{DyMo}_c$ lacks that guarantee, and the authors claim that both $\\text{DyMo}_c$ and $\\text{DyMo}_e$ achieved similar results, indicating that DyMo is robust to the choice of distance metric. However, as shown in the results of Table 1, the performance of  $\\text{DyMo}_c$ and $\\text{DyMo}_e$ across different settings (datasets/missing rates/metrics) does not seem to be consistent. \n\n- Sub-optimality of the greedy strategy: The algorithm relies on a greedy approach, selecting at each step the single modality that currently yields the highest reward. This strategy cannot guarantee finding the optimal combination of all missing modalities. There can be scenarios in which adding modality A alone is less beneficial than adding modality B, yet adding the combination of modalities A and C yields far greater gains than adding B alone. The effect of this kind of modality synergy can be significant in different medical imaging modalities. A greedy selection would overlook such combinatorial effects, leading to a sub-optimal subset of modalities.\n\n- The author acknowledges that TIP has an upper limit for full-table reconstruction and uses this to explain why it performs on par with or worse than CONCAT in certain scenarios. However, this means that the \"selection strategy\" capability becomes strongly constrained by the reconstruction quality bottleneck. I highly recommend to include at least one alternative reconstructor in DVM/UKBB.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JOOPLDawna", "reviewer": "Reviewer_eaDt", "summary": "This paper proposes a Dynamic Modality selection method, DyMo, for the missing modality machine learning setting. The key insight is that existing approaches either discard missing modalities (losing valuable information) or recover all missing modalities (potentially introducing noise), creating a fundamental trade-off the authors refer to the discarding-impuattion dilemma. DyMo overcomes this by adaptively selecting only the reliable recovered modalities that provide task-relevant information for each test sample.​\n\n- Core Innovation: A Multimodal Task-Relevant Information Reward (MTIR) function that estimates incremental information gain from each recovered modality using task loss as a tractable proxy​\n\n- Theoretical connection: MTIR is based in a connection between mutual information I(Y;Z) and classification loss, enabling inference-time selection without ground-truth labels​\n\n- Evaluation: Extensive experiments on 5 datasets (PolyMNIST, MST, CelebA, DVM, UKBB) showing significant improvements, especially under severe missing scenarios​", "questions": "- Recovery Method Dependencies: How sensitive is DyMo's performance to the quality of the underlying recovery method? Could you provide analysis on performance degradation when recovery methods produce consistently low-quality reconstructions? For example, if the recovery method always produces noise how would DyMo perform. If it produces accurate recoveries 50% of the time, how well would DyMo perform?\n\n- Computational Scalability Concerns: How does the computational overhead scale with the number of modalities and missing patterns? What is the practical upper limit for real-time applications?\n\n- Hyperparameter Sensitivity: The framework introduces several hyperparameters (temperature t=0.1, calibration threshold alhpa). How sensitive is performance to these choices, and how should they be set for new domains?\n\n- Theoretical Limitations: The mutual information lower bound assumes bounded loss values with conservative upper bound G. How is G estimated in practice, and how does this choice affect the bound's tightness?\n\n- Class Imbalance: How does DyMo perform on highly imbalanced datasets where the equal class prior assumption may be violated? Could you provide analysis or modifications for such scenarios?\n\n- Generalization Beyond Classification: While focused on classification, could this approach be extended to other multimodal tasks like regression, generation, or structured prediction? What modifications would be required?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Strong Problem Formulation: This work substantiates discarding-imputation dilemma in incomplete multimodal learning providing a strong motivation for DyMo\n\n- Comprehensive Technical Design: MTIR reward function handles both low-fidelity and semantically misaligned recovered modalities​ (e.g. when the image is blurry or the recovered image is not representative of the class)\n\n- Strong additional features: Intra-class similarity calibration enhances reward reliability​, iterative modality selection to minimize noise, incomplete simulation training, auxiliary contrastive loss tested with 2 distance functions\n\n- Extensive Experimental Validation: Evaluation across diverse domains (natural images, medical data, synthetic benchmarks)​, Consistent improvements over 12 baseline methods​, particularly strong performance under severe missing scenarios (e.g., 13.12% improvement on PolyMNIST with 80% missing modalities)​\n\n- Thorough Analysis: Comprehensive ablation studies and visualization analyses (Figure 4 with the TSNE and PCA visualizations were particularly convincing that DyMo’s MNITR successfully adds recovered modality features when helpful and does not use them when it would hurt performance.", "weaknesses": "- Limited Recovery Method Diversity: While claiming generalizability, experiments primarily use VAE-based recovery methods (MoPoE, MMVAE+, CMVAE from Table S5) with limited evaluation of fundamentally different recovery approaches​\n\n- Computational Overhead: The authors claim DyMo introduces minimal additional parameters and relies on a relatively simple training scheme. However, it seems give the features of the method including computing the MITR, which includes intra-class similarity calibration, and with iterative selection (average 1.38 iterations per sample from  appendix C.3), DyMo would be more computationally intensive. The inference-time latency, parameter count, or training time computational cost were not thoroughly analyzed​ in this work.\n\n- Calibration Term Limitations: The intra-class similarity calibration shows inconsistent benefits across datasets (improves some tasks but hurts CAD/Infarction performance), suggesting the approach may not be universally optimal​\n\n- Limited Analysis of Edge Cases and limitations: Insufficient discussion of common failure modes or limitations of when DyMo performs poorly or what assumptions DyMo makes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Je4hH8aU9x", "reviewer": "Reviewer_LF6X", "summary": "**Summary**\nTo address the discard-fill dilemma faced by multimodal deep learning (MDL) in real-world scenarios due to modality loss， discarding the missing modality easily loses task-critical information, while filling the modality easily introduces low-fidelity/semantic misalignment noise. This paper proposes DyMo, a dynamic modality selection framework for inference, which aims to balance the utilization of missing modal information and noise avoidance.", "questions": "see the weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Strength**\n1. The motivation is clear and convincing\n2. The dataset and tasks are comprehensive. It covers five significantly different datasets, digital classification, attribute classification, disease diagnosis, and missing scenarios, fully verifying DyMo's adaptability in different scenarios.", "weaknesses": "**Weakness**\n1. The primary concern is whether the noise in cross-modal generated data actually harms, especially with the advancement of diffusion model generation technology in recent years. The author should discuss and compare his work with the related recovery-based methods[1][2][3].\n\n2. The comparison method is a bit outdated, and the author should consider comparing it with more 2024s methods of incomplete multimodal learning. \n\n3. The experiments should contain some classical multimodal datasets, such as CMU-MOSI or CREMAD, for a fair comparison. especially the CMU-MOSI, in which the text is the dominant modality, when it misses the performance degree. This can help to verify the effectiveness of the DyMo.\n\n4.The method requires the label information to select the modality, but when the modality is missing, the prediction will be very incorrect. How can the performance of the model be guaranteed? The representation prototypes also cannot avoid the problem of representations being misclassified.\n\n\n\n[1] Yuanzhi Wang.  Incomplete Multimodality-Diffused Emotion Recognition NIPS\n[2] S Wei. Mmanet: Margin-aware distillation and modality-aware regularization for incomplete multimodal learning CVPR\n[3] Yuntao Shou. GSDNet: Revisiting Incomplete Multimodality-Diffusion Emotion Recognition from the Perspective of Graph Spectrum", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)