Deep Multimodal Learning with Missing Modality: A Survey
Authors: Renjie Wu
Hu Wang
Hsiang-Ting Chen
Gustavo Carneiro
Venue: Accepted by TMLR ()

Abstract: During multimodal model training and testing, certain data modalities may be absent due to sensor limitations, cost constraints, privacy concerns, or data loss, which can degrade performance. Multimodal learning techniques that explicitly account for missing modalities aim to improve robustness by enabling models to perform reliably even when certain inputs are unavailable. This survey presents the first comprehensive review of Multimodal Learning with Missing Modality (MLMM), with a focus on deep learning approaches. We outline the motivations and key distinctions between MLMM and conventional multimodal learning, provide a detailed analysis of existing methods, applications, and datasets, and conclude by highlighting open challenges and future research directions.

Published in Transactions on Machine Learning Research (02/2026)
Deep Multimodal Learning with Missing Modality: A Survey
Renjie Wu
renjie. wwwaanaaaaaaa..o.mmmmmmm www..aaaaa.................
The Australian National University
Hu Wang
fwa www.wan@@bbzaa.ccom
Mohamed bin Zoyed University of Artificial Intelligence
Hsiang-Ting Chen
tim, chen Wadelaide.e....... Sin.chen@adelaiic
edu on
Adelaide University
Gustavo Carneiro
w.caruciro@surrey.cccc uk
The University of Surrey
Reviewed on OpenReview: https: //
// openreview. net/ forum? id= 7RFecmh/hT
Abstract
During multimodal model training and testing, certain data modalities may be absent due to sensor limitations, cost constraints, privacy concerns, or data loss, which can degrade performance. Multimodal learning techniques that explicitly account for missing modalities aim to improve robustness by enabling models to perform reliably even when certain imputs unavailable. This survey presents the first comprehensive review of Multimodal Learning with Missing Modality (MLMM), with a focus on deep learning approaches. We outline the motivations and key distinctions between MLMM and conventional multimodal learning, provide a detailed analysis of existing methods, applications, and datasets, and conclude by highlighting open challenges and future research directions.
1 Introduction
Multimodal learning has become crucial field in Artificial Intelligence (AI). It focuses jointly analyz- ing various data modalities, including visual, textual, auditory, and sensory information. This approach mirrors the human capacity to combine multiple senses for better understanding and interaction with the environment. Modern multimodal models leverage the robust generalization capabilities of deep learning uncover complex patterns and relationships that single-modal systems might not detect. This capability is advancing work across multiple domains (Bayoudh et al. 2022 Hu et al., 2023 Streli al. 2023). Re- cent surveys show that multimodal learning significantly boosts performance and enables more advanced AI
applications (Baltrušaitis al. 2018 Zhao et al. 2024 Wu et al. 2024a
However, real-world multimodal systems frequently encounter scenarios where certain data modalities are missing or incomplete. Such cases arise from sensor malfunctions, hardware limitations, privacy concerns, environmental interference, and data. transmission failures. Interest in addressing this challenge has grown considcrably in recent years, as reflected the publication trends shown figure la For instance, in a three-modality setting, data samples can be classified as either full-modality (containing all three modalities) missing- modality (lacking to deployment, and can substantially degrade model performance.
more). Missing modalities can occur at any stage, from data collection
Early works in affective computing (Cohen et al. 2004 Sebe et al. 2005) reported that when collecting facial and audio data, some image or audio samples were not useful due to excessive microphone noise or camera obstructions. This has forced them to propose andio-visual models that can handle the missing modality recognize human emotional states. In the field of medical AI, privacy concerns and the challenges of obtaining certain data modalities during surgeries invasive treatments often lead to missing modalities in

---

Published in Transactions on Machine Learning Research (02/2026)
60
Mod
Mod
Mod
Full- Modality Samples
50
Mod 2
Mod 2
...
Mod
Numher
40
Mod 3
Mod 3
Mod
30
20
Missing- Modality
Mod
Mod 1
Mod
10
Mod
Mod 2
Mod
Samples
Mod
Mod 3
Mod
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
(a) Trends in Missing Modality Publications
(b) Full-Modality vs. Measing-MMdaaing Samples
Figure 1: (a) The trend of papers published on deep multimodal learning with missing modality in the past 10 years (2014-2025.8). The number of publications has increased over time and has received widespread attention from the community. (b) Description of a three-modality scenario with full- and missing missing-moddality samples. We abbreviate "Modality" as "Mod" in all figures of this paper and use dashed boxes with fading colors to represent missing oodalities/moddlees
multimodal datasets (Zhang et al. 2023b Li et al. 2025b). Similarly, in space exploration, NASA's Ingemu- ity Mars helicopter (Donaldson] 2024) faced a missing modality challenge when its inclinometer failed due to extreme temperature cycles on Mars. To address this issue, NASA applied a software patch that modi- fied the initialization of navigation algorithms (Team) 2022). Beyond such domain-specific cases, structural quantitative heterogeneity across sensor types, versions, brands can also yield inconsistent modality availability. Combined with the unpredictability of real-world scenarios and the diversity of data sources, these factors make robustness to missing modalities an essential requirement for multimodal systems. Con- sequently, developing models that can effectively handle incomplete inputs has become a central research focus.
In this survey, we focus on the challenge of handling incomplete inputs in multimodal learning, which we refer to the missing modality problem. We term solutions to this challenge Multimodal Learning with Missing Modality (MLMM). Unlike the conventional full-modality setting, where all N modalities are assumed to be consistently available, MLMM must operate when one or more modalities are absent during training or testing. The central goal of MLMM is to robustly exploit the available modalities while maintaining performance comparable to systems trained with complete modality information.
This survey reviews recent advances in MLMM and its applications across diverse domains, including in- formation retrieval (Malitesta et al. 2024 Pipoli al.
2025), remote sensing (Wei et al. 2023), Pervasive
Computing [Wweke et al.) 2018 Van Der Donckt et al.)
2024 Chen et al. (023a)), robotic vision Gunasckar
et al. 2020: Park et al.
2025 (bio-)medical domain (Zhou et al. 2023] Shah al. 2023 Azad et al.
2011 Wang et al. 2025b), and multi-vies clus- tering (Chao et al. 2021 We also introduce a fine-grained taxonomy of MLMM methodologies, application scenarios, and corresponding datasets.
2022 Zhu et al. 2025
sentiment analysis (Wagner et al.
(1) A comprchensive survey of MLMM methods across diverse domains, accompanicd by an extensive compilation of relevant datasets, highlighting the versatility of MLMM in addressing real-world challenges. (2) A novel, finegrained taxonomy of MLMM methodologies, with multi-faceted categorization framework based on multi-modal integration stages and missing modality recovery strategies. (3) An in- depth analysis of current MLMM approaches, their challenges, and future research directions, contextualized within the proposed taxonomical framework.
Contributions:
Paper Collection: In our literature scarch methodology, we primarily source papers from Google Scholar and major conferences/jounaat in AI, Machine Learning, Computer Vision, Natural Language Processing, Audio Signal Processing, Data Mining, Multimedia, Medical Imaging, Remote Sensing and Pervasive Com-
2

---

Published in Transactions on Machine Learning Research (02/2026)
Deep Multimodal Learning with Missing Modality
Data Processing Aspect
Strategy Design Aspect
Modality Composition
Coordinated- Representation
Attention
Ensemble
-Based
Modality Generation
Representation Composition
Distillation- Based
Dedicated
Modality Imputation
Representation Generation
Graph Learning -Based
Discrete
Scheduler
Representation-Fooused
Model Combinations
MLLLMs
Architecture-oocused
Figure Our laxonomy of deep multimodal learning with missing modality methods. We categorize ex- isting methods into two aspects: data processing and strategy design. Data Processing: we differentiate between modality imputation (handling the modality data level) and representaalion-oouseed models (deal- ing with at the data representation level). Strategy Design: we distinguish between archtectturefoousee models (model architecture adjustments) and model combinations (combining multiple models externally). "MLLMs": multimodal large language models.
puting. The collected papers are from, but not limited to, top-tier conferences (e.g., AAAI, IJCAI, NeurIPS, ICLR, ICML, CVPR, ICCV, ECCV, ACL, EMNLP, KDD, ACM MM, MICCAL, ICASSP) and journals (e.g.. TPAMI, TIP, TMI, TMM, JMLR, TMLR). Please refer to Section A in the Supplementary Materials for the full names of these conferences and journals. We have compiled a total of 354 significant papers from the period spanning 2012 to August 2025. Our search strategy involved using keywords such as "incomplete," "missing," " "partial," " "absent," " and "imperfect," " combined with terms like "multimodal learning," "deep learning," "representation learning." "multi-view learning," " and "neural networks."
Firstly, we explain the background and motivation of this survey in section In section introduce our taxonomy and categorize existing deep MLMM methods from methodological perspective, detailing them in two aspects and four types (figure 2). In the following section 3 and section we introduce various methods from the aspects of model data processing and strategy design respectively. We then summarize current application scenarios and corresponding datasets used in this field in section In section 7 we discuss unresolved challenges and future directions. Finally, we present our conclusions drawn from the exploration of deep MLMM in section
Survey Organization:
2 Methodology Taxonomy: An Overview
We review current deep MLMM methods from two key aspects: data processing and strategy design, based on the contributions of existing work.
2.1 Data Processing Aspect
Methods that focus on exploring the data processing aspect of a model or framework can be divided into Modalily Impulation and citresenlationFoowseee Models, depending on whether the model's handling of missing modalities occurs at the modality data level or at the data representation level.
(1) Modality Imputation operates the modality data level and fills in the missing information by compositing (Chen al. 2020; Sun et al.] 2024b Parthasarathy & Sundaram 2020 Yang et. al.) 2024
3

---

Published in Transactions on Machine Learning Research (02/2026)
Campos et al.
2015
(modality composition methods) generating (Chen et al. 2024a
Ma et al 2021c
Wang et al., 2024c Tran et al. 2017 Zhang et al. 2023ch absent modalities (modality generation methods) from available modalities. Those approaches are rooted in the idea that if missing modality can be accurately imputed, downstream tasks can continue as "full" modalitics were available.
Arya & Saha 2021
Models are designed address missing modalities at the representation level. In some cases, the coordinated representation methods (Liang al 2019) Wann et al. 2020b] Ma et al.] 2021b) impose some specific constraints to help align the representations of available modalities in the same semantic space, so that the model can be effectively trained even when facing missing modalities. Other representation-focur
(2) Reppessentation-Pocusse
methods either generate the missing-modality representation using available data (Zheng et al.] 2021 Li et al.) 2024b Hoffman et al.) [016 Ma et al.) |022c]] Park et al.) 2024 or
combine the representations of existing modalities {Zhi et al. 2024 Sun et al.) 2024b Zhou et al. 2021
Delgado-Escand et al. 2021 Zheng et al. 2024 Zhang et al. 2024] to fill the gaps.
2.2 Strategy Design Aspect
Methods that explore the strategy design aspect are based on models that can dynamically adapt to different missing-modality cases during training and testing through flexible adjustments of the model architecture (internal model architecture adjustment) and the combination of multiple models (external model combina- tions). We name them Architecture-Focuus Models and Model Combinations.
(1) Architetture-Focused Models address missing modalities by designing flexible model architectures that can adapt to varying numbers of available modalities during training or inference. A key technique here
is based on attention mechanisms (Ge et al. 2023: Yao et al. 2024 Gong et al.
2023 Mordacq et al. 2024
Radosavovic et al.) 2024 Jang et al. 2024 Qiu et al. 2023) which dynamically adjusts the modality fusion
and processing, allowing the model to handle any number of input modalities. Another approach is based
on knowledge distillation (Wang et al. 2020a Saha et al. 2024 Zhang et al., 2023b Chen et al. 2024b;
Wang et al. 2023a Shi et 2024; Wang et al. 2023c), where the model is trained to accommodate missing modalities by transferring knowledge from full-modality models to those operating with incomplete data
distilling between different branches inside the model. Additionally, graph learning- beaseing-based methods (Zhao
ct al. 2022 Zhang et al.) 2022 Lian et al.) 2023 Malitesta et al., 2024) exploit the natural relationships
between modalities, using graphs to dynamically lly fuse and process available modalities while compensating
for missing ones. Finally, MLLMs (Zhan et al. 2024 Wu et al., 2023b Li et al. 2023a) also play a crucial role in this category, as their ability to handle long contexts and act as feature processors enables them to accept and process representations from any number of modalities. These architectural strategies collectively allow models to maintain performance even when dealing with incomplete multimodal inputs.
(2) Model Combinations tackle missing modality problems by employing strategies that leverage multiple models or specialized training techniques. One approach is to use dedicated training strategies (Zeng et al. 2022 Chen et al. 2022 Xue & Marculescu 2023) tailored for different modality cases, ensuring that each case 18 trained for optimal performance. Another approach involves ensemble methods (Hu et al. 2020) Wang et al.] 2021 Lee et al. 2023b), where models trained on either partial/full sets of modalities are combined, allowing the system select the most suitable model based on the available modalities to do joint predictions. Additionally, discrete scheduler methods (Wu et al. 2023a Shen et al. 2024 Surís et all, 2023) can incorporate various downstream modules to flexibly process any number of modalities and handle specific tasks. These schedulers intelligently select and combine the outputs of multiple models or modules to manage missing-modality scenarios, offering a versatile solution for multimodal tasks.
Our taxonomy (figure can reflect different aspects and levels of multimodal ity data to data representations, architectural design, and model combinations-cant providing distinct way to approach the problem of missing modalities based the task requirements and available resources.
from modal-
4

---

Published in Transactions on Machine Learning Research (02/2020)
Mod
Zero/Random Values
Downstream Task
Mod 2
DNN
Representation
Mod 3
Figure 3: Zero/Randier Random values composition methods. If we assume modality is missing, then this modality will be replaced with zero/random values. "DNN" in all figures of this survey means different kinds of deep neural networks.
Sample Library
Mod
KNN...
Retrieval Samples
Process.
Mod
Representation
Downstream Task
Mod 3
Retrieval-bbsee modality composition methods search for one or more samples by randomly selecting using simple retrieval algorithms like KNN, or its variants, from same-category samples that have the required missing modalities, and then compose them with the input missing-modality sample to form a "full" "fulli-modality sample.
Figure 4:
3
Methodologies in Data Processing Aspect
3.1 Modality Imputation
Modality imputation refors to a technique used by MLMM methods that fills in missing-modalit, samples generales missing modalities to complete the dalasel with missing modalities by performing various transformations operations existing modalities. Modality imputation methods addressing the missing modality problem at the data modality level can be categorized into two types. (1) Modality composition
methods use gero/randon values or data copied from similar instances as the input for the missing modality data. The data produced via these methods to represent, the missing data are then composited with the data from the available modalities to form a modality sample.
(2) Modality generation methods generate the missing modality data using generative models such as Auto-Encoders (Hinton & Zemel Generative Adversarial Networks (GANs) [Goodfellow al., 2014), or Diffusion Models (Ho al. 2020
1993)
The generated data is then composited with the data from the available modalities to form "full"-modality sample. We provide more details about these two methods in the next sub-sections.
3.1.1 Modality Composition Methods
Modality composition methods are widely employed for its simplicity and effectiveness maintain the original dataset size. method that replaces missing modality with zeros random values, shown in figure 3 In recent research (Chen et al. 2020 Sun et al. 2024b; Liu et al. 2023a: Malitesta et al. 2024), they are often used as a baseline method for comparison with other more sophisticated methods. For the missing sequential data problem, such as missing frames in videos, the similar Frame- Zero method (Parthasarathy Sundaram 2020) was proposed to replace the missing frames. These methods are prevalent in typical multimodal learning procedures and can be used to balance and integrate information from different modalitics when making predictions. This prevents the model from over-relying on dominant modalities available for each sample, enhancing its robustness by encouraging more balanced integration of information across all available modalities.
Values Composition Methods represent type of modality composition
Retrieval-Based Representation Composition Methods (figure 4 represent another lype of modality compo- sition method, which replaces the missing modality data by copying or averaging data from retrieved samples with the same classification. Some other methods randomly select sample that has the same classification and required missing modality from other samples. The selected modality data then composed with the missing-modality sample to form full-modality sample for training. But those retrieval retrieval-based modality
5

---

Published in Transactions on Machine Learning Research (02/2026)
Mod
Downstream
Mod 2
GFN-2
Mod 2
DNN
Representation
TTak
Mod 3
(a) Individual Modality Generation Methods
Mod 1
Mod!
Downstream
Mod
GEN
Mod 2
DNN
Representation
Task
Mod 3
Mod 3
(b) Unified Modality Generation Methods
Figure 5: Description of two typical modality generation methods. We set modality 2 as the missing modality for examples and use other available modalities to generate modality 2. "GEN" in both figures represents modality generation networks. (a) We set up a modality-2 generator (GEN-2) from other modalities. (b)
All modalities are input and generated together by using a single GEN.
composition methods are not applicable to pixel-level tasks, such segmentation, and are only suitable for simple tasks (e.g., classification) because they may lead to the overfitting of noisy data, if mismatched samples are combined. For example, Yang et al. (2024) proposed Modal-mixup to complete the training datasets by randomly complementing same-category samples with missing modalities. However, such meth- ods cannot solve the missing modality problem during the testing phase because they rely on known labels of training data samples. In some multimodal streaming data classification tasks, such as audio-visual ex- pression recognition, video streams may experience frame drops due to network communication packet loss, etc. Trame-Repeat (Parthasarathy & Sundaram 2020 was proposed to to make up for the missing frames by using past frames.
Other methods Yang et al. 2024 Campos et al. 2015 also used K-Nearest Neighbors (KNN), or its variants, to retrieve the mast-matched samples for composition. For these matched samples, they select the sample with the highest score or obtain the average values of these samples to supplement the missing modality data. Their experiments have shown that KNN-based methods generally perform better than the above methods, and can handle missing modality during testing. Nevertheless, such KNN retrieval-based modality composition methods often sulfer from high compulational complexity, sensitivity to imbalanced data, and significant memory overhead. In pervasive computing, some researcher (Hussein et al. 2024a proposed to cluster sensor data and pre-learns the best imputation pattern for each cluster combination, enabling fast lookup-based completion of missing modality at runtime.
All methods above can complete datasets with missing modalities, but they reduce the diversity of the dataset because they may introduce duplicated training samples. This especially problematic with datasets having high rate of modality missing, where most samples have of overfitting to certain classes with few full-modality samples, if they pad missing-modality samples with duplicated ones.
data, as it increases the risk
3.1.2 Modality Generation Methods
With deep learning, synthesizing missing modalities has become more elfective by leveraging powerful rep- resentation learning and generative models that capture complex cross-modal relationships. Current methods for generating missing-modality data are divided into individual and unified generative methods.
Individual Modality Generation Methods train an individual generalive model for each modality in case any modality is missing, as shown in figure 5a
Early works used Gaussian processes (Mario Christoudias 2014) to generate missing modalities from available data. (AEs) (Hinton & Zemel 1993) can be used for missing modality generation. Li et al. (2014) used 3D-CNN to generate positron emission tomography (PET) data
ct al. 2010) or Boltzmann machines (Sohn et al.
With deep learning, models like auto-Encoders

---

Published in Transactions on Machine Learning Research (02/2026)
Constraint Terms
egg, Deep Canonical- Correlaanoooelatiionn Analysis or Tensor Rank Rogularizatio...
Mod
DNN-1
Representation-
Downstream Task
Mod
DNN-2
Representation-2
Representation
Mod
DNN-3
Representution-3
Figure 6: Illustration of coordinnteed-repreentlal methods, which impose constraints make the learned representations semantically consistent. Assuming that modality 2 is missing, these methods constraints terms between the features of different modalities.
from magnetic resonance imaging (MRI) inputs. Chen et al. (2024a) addressed missing modalities in MRI segmentation by training U-Net models to generate other two modalities from MRI. A recent work (Ma) et al. 2021c) used AEs one of the baselines to complete datasets by training one AE per modality, In domain adaptation, Zhang et al. (2021) proposed a U-Net-basod Multi-Modality Data Generation module with domain adversarial learning to generate each missing modality by learning domain-invarianl features. In immunohistochemicall DeepLIIF (Ghahremani et al. 2022) uses ResNet to generate the various modalities such as Hematoxylin, mpTF DAPI, mpIF Lap2, and mpIF Ki67. HE2RNA (Schmauch et al.) 2020) cross- modal model that predicts gene expression (transcriptomic modality) from H&E whole-slide images (visual modality), generates spatialized gene-expression heatmaps, and transfers this representation to downstream clinical tasks.
GANs significantly improved image generation quality by using a generator to create realistic data and a discriminator to distinguish it from real data. Researchers therefore began replacing above models with GANs for modality generation. For example, GANs generate missing modalities using latent representations of existing ones breast cancer prediction (Arya & Saha 2021). In remote sensing, Bischke et al. used GANs generate depth data, improving segmentation over RGB-only models (Bischke et al. 2018 GANs were also used in robotic object recognition models help generate missing RGB and depth images (uunasckar al. 2020). Recent studies (Ma et al. 2021c) show that GANs outperform AEs in generaling more realistic missing modalities and can lead to better downstream- model performance. SensorGAN (Hussein & Bhat 2024 introduces a GAN model that restores absent sensor channels, directly targeting missing-modality recovery in wearable humaa-acttvitty-rrcoiiito pipclines. Recently, the introduction of Diffusion modcls has further improved image generation quality. Wang et al. proposed the IMDer method (Wang el al. (0224c), which uses available modalities as conditions make diffusion models generate missing modalities. Experiments showed it can reduce semantic ambiguity between recovered and missing modalities and achieves good generalization performance than previous works.
Unified Original Data Generative Methods train unified model that can generate all modalities simul- laneously (figure 5b]. One representative model is Cascade AE (Tran et al. 2017). which stacks AEs to capture the differences between missing and existing modalities for generating all missing modalities. Re- cently, Zhaag et al. (2023c), have attempted use attention and max-pooling to integrate features of existing modalities, enabling modality-specifie decoders accept available modalities and generate other missing modalities. Experiments demonstrated that this method is more effective than using max-pooling alone (Chartsias et al. 2017) to integrate features from any number of modalities for generating all missing modalities together. Above methods for generating missing modalities can mitigate performance degradation to some exlent. However, when facing a scenraio where one of the modalities is severely missing, training a generator that can produce high-quality missing modalities remains challenging. Additionally, the modality generation model significantly increases storage and computational requirements. As the number of modali- ties grows, the complexity of these generative models also increases, further complicating the training process and resource demands.

---

Published in Transactions on Machine Learning Research (02/2020)
Mod
DNN-I
Representation-1
e.g., Add, Pooling...
Arithmetic
Downstream Task
Med 2
DNN-2
Representation-2
Represeutation
DNN
Operations
Mod
DNN-3
Representation-
Figure The general idea of the arithmetic operation-based representation composition methods. The reprosentation of any number of modalitics can be combined to fixed dimension that is the same as the dimension required by the subsequent DNN layer.
3.2 Representation-Foauuk Models
Reprresentation-focusee models address the missing modality problem at the representation level. We in- troduce such models by first presenting two pepriiatted-eppreenttattooooooo learning of more discriminative and robust representations by imposing specific constraints (figure 6 The next type of representation--oousedd methods that we discuss are the representation imputation mets which can be categorized into representation composition and representation generation methods. Rep- resentation composition methods can borrow the solutions described in section 3.1.1 and operate at the representation level of the modalities or employs arithmetic operations (e.g.. pooling) fuse a dynamic number of modalities. Finally, we introduce the representation generation methods, which usually small generative models to produce the representations of missing modalities.
approaches that enhance the
3.2.1
Bhordinated-Represeeie
Methods
Coordinated--epreeetaaee methods focus on introducing certain constraints between representations of dif- ferent modalities to make the learned representations semanticalll consistent. They are usually divided into two categories, one based on regularization and the other driven by correlation. An example of regularization is Tensor Rank Regularization (TRR) (Liang et al. 2019). TRR achieves multimodal fusion by taking the outer products of different-modality tensors and taking the sum of all their outer products. In order to solve the high rank of multimodal representation tensors caused by imperfect modalities (missing or noisy) in time series, Paul al. introduced tensor rank minimization to try to keep the multimodal representation low rank to better express the tensors of true correlation and potential structure in multimodal data, thereby alleviating the imperfection of the input.
Some methods train models by coordinating the correlation between different modal features. For exam- plc, WWan et al.] (2020b) use a Deep Canonical Correlation Analysis (CCA) module to maximize feature associations of available modalities by using Canonical Correlation Coefficient, which enables training on incomplete datasets. Ma et al. (2021b) proposed Maximun likelihood function to characterize conditional distributions of full-modality and missing-modality samples during training. Other research efforts {Liu ct al. 2021b) have added constraints based on the Hilbert-Schmidt Independence Criterion (HSIC), which helps models learn how to complete missing modality features by enforcing independence between irrelevant fcatures while maximizing the dependence between relevant ones. These methods (Matsuura et al. 2018 Ma et al. 2021a Li et al., 2022a) aid models in learning how to complete missing modality features or training on incomplete datasets by learning the similarity or correlation between features. A drawback of the above methods is that they perform well only when two or three modalities and their effectiveness tends to degrade as the number of modalities increases. Even when trained on dataset with missing modalities, some methods still struggle to elfectively address unseen missing modality combinations or cases during testing.
used input (Zhao al.) 2024
3.2.2 Representation Composition Methods
There are two types of representation composition methods, which are explained below. Retrieval-Based Representation Composition Methods attempt to recover the missing modality rep- resentation by retrieving the modality data from existing samples, similarly to the modality composition method in section 3.1.1 They typically use pre-trained feature extractors generate features from available
8

---

Published in Transactions on Machine Learning Research (02/2020)
Auxiliury Reconstruction Task
Mod
Mod
Mod 2
DNN
Representation
DNN
Mod 2
Mod
Mod 3
Downstream Task
(a) Indirect-to- Task Representation Generation Methods
Mod
DNN-1
Representation-1
Downstream Task
Mod
GEN-2
Representation-2
DNN
Representation
Mod
DNN-3
Representation-33
(b) Direct-to-Taast Representation Generation Methods
Figure 8: Description of two typical representation generation methods. We assume modality 2 is missing here. (a) Indirect-lo-task representation generation methods are supervised by the loss of two tasks, namely: auxiliary reconstruction and downstream tasks. Note that these two tasks sometimes can be trained sep- arately, first training the reconstruction task, discarding the generator, and then training the downstream task, such as the training progress of ActionMAB (Woo et al. 2023). While (b) direct-to-task representation generation methods directly use modality generalion nel ("GEN-2" in figure) to generate representa- tions of the missing modality 2 during training and inference without reconstructing the modality 2.
samples, storing them in feature pool Zhi et al. 2024 Sun et al. 2024b). Cosine similarity is then used to retrieve matched features for the input sample, with the average of these missing-modalli features in top-K samples filling the representation of missing modalities. Additionally, some methods, such as Missing Modality Feature Generation (Wang al.j (023b)) replace missing modality features by averaging the representations of available modalities, assuming similar feature distributions across modalities.
Arithmetic Operation-Based Representation Composition Methods flexibly fuse any number of modal- ity representations through arithmetic operations such as pooling without learnable parameters (figure 7 Researchers (Zhou et al. 2021 Wang et al.) 2023b) have fused features through operations like average/max pooling or addition, offering low computational complexity and efficiency. Howevor, drawbacks include po- tential loss of important information. To address this, merge operations (Delgado-Escanoo al. 2021 use a sign-max function, which selects the highest absolute value from feature vectors, yielding better results by preserving both positive and negative activation values. TMFormer [Zhang al. 2024) introduces token merging based on cosine similarity. Similar approaches (Zheng et al. 2024] focus on selecting key vectors merging tokens to handle missing modalities. A recent work (Sarkar et al. 2025) propose to perform weighted fusion of these pooled features to learn a fixed-size multimodal embedding
Therefore, those representation composition approaches not only allow the model to flexibly handle features from any number of modalities but also enable learning without. introducing new learnable parameters. However, they are not good capturing the inter-modality relationships through learning, methods like selecting the largest. vector the feature with the highest score is difficult to fully represent the characteristics of all modalities.
3.2.3 Representation Generation Methods
Compared to modality generation methods in section 3.1.2 representation generation methods can be in- tegrated seamlessly into existing multimodal frameworks. Current methods fall into two categories: (1) Indirect-to-task representation generation methods (figure (aa) treat modality reconstruction as an auxiliary task during training, helping the model intrinsically generate missing modality representations for down- stream tasks. Since the auxiliary task aids in representation generation during training but is dropped

---

Published in Transactions on Machine Learning Research (02/2026)
during inference of downstream task, it is termed "indirect-to-task" (2) Direct-to-tase representation gener- ation methods (figure 8bp train small generative model to directly map available modality representations to the missing modalities. We provide more details about these two categories of methods below.
Indirect-to- tasskkk Representation Generation Methods can indirectly generate missing modality representa- tions and often employ the "encoder-decoder" architecture. During training, modality-specifie encoders ex- tract features from available modalities, reconstruction decoders reconstruct the missing modality, and the downstream module is supervised by downstream task loss for prediction. Those reconstruction decoders are discarded during inference, where predictions rely both the generated missing modality representations and the existing ones. This approach typically employs Multi-Task Learning, training both downstream tasks and auxiliary tasks (modality reconstruction) simultaneously. Some methods, inspired by Masked Autoen- coder [e et al. 2022), split recoustruction and downstream task training. Typically, in the training process of this type of methods, the features of the available modalities are usually input into the downstream task module and the reconstruction decoder of different modalitics after being fused. However, some methods accept the outputs of different modality encoders into their respective reconstruction decoders. Based on this distinction, we divide this type of methods into before-fusion and after-fusion methods.
Before Fusion methods receive the features from modality-specific encoders directly into each specific re- construction decoder for missing modality reconstruction. For example, MGP-VAE (Hamghalam et al. 2021) uses output of the VAE encoder of each modality as the imput of each reconstruction decoder and the fused output for Gaussian Process prediction. Similarly, PFNet (Zheng et al. (021) employs RGB and thermal-infrared modalities for pedestrian re-identification, feeding outputs of each encoder into correspond- ing decoders before fusing them together. Li et al. (2024b) adopted a similar approach for brain tumor segmentation, reconstructing each modality before fusing encoder outputs.
After Fusion methods use the fused features from modelity-speciffe encoders as the imput of reconstruction decoder. Tsai et al. (2018) introduced a Multimodal Factorization Model where modality-speciffc encoder- decoders reconstruct the missing modality, while the downstream task module employs a separate multimodal encoder-decoder. To recover the missing modality features, the outputs of the downstream fask model's encoder are not only used for predictions of the downstream task decoder, but also fused with the outputs of each modality-spociffe encoder for inputting to the cach corresponding roconstruction decoder. Chen et al. (2019) and Li et al. (2025ap also decoupled the encoders for reconstruction and downst lasks, showing that disentangling features helped eliminate irrelevant features. [Jeong et al. (2022) proposed a simpler approach using multi-level skip connections in a U-Net architecture to fuse features for missing modality reconstruction, achieving better generalization in brain tumor segmentation than (Chen et al, 2019). Further, /Zhou| (2023) and [Sun al.] (2024a) demonstrated effective cross- modal attention fusion techniques could help enhance reconstruction and segmentation. Most of these methods train reconstruction and downstream tasks concurrently. However, some approaches, such as ActionMAE (Woo et al.) 2023) and M3AE (Liu et al. 2023a). first train reconstruction tasks and then fine-tune the pre-trained encoders for downstream tasks.
Although these methods achieve strong performance, they still require the missing modality data to be present during training to supervise the reconstruction decoders.
Direct-to-Tase Representation Generation Methods usually use simple generative models to directly map information from avaiable modalities to the missing modality representation. The general concept is illus- trated in figure Sb Early work (Mario Christoudias et al. 2010 using Gaussian Processes to "hallucinate" missing modalities inspired Hoffman et al. (Hoffman et al. 2016) to propose a Hallucination Network (HN) that predicts missing depth modality features from RGB images. The HN aligns intermediate layer features with another network (trained on depth images) using an T.2 loss, allowing RGB inputs to generate depth features to address missing modality challenges. This method has been extended to pose estimation (Choi et al. 2017) and land cover classification (Kampffneyer et al. 2018), where the HN can be used to generate missing heat distribution and depth features.
In tasks like sentiment analysis, translation module can map available-modality representations to the missing ones, with L2 loss applied to align outputs with those from networks trained on missing modali- ties (Huan et al. 2023) A notable recent work, SMIL (Ma et al. 2021c). employs Bayesian learning and
10

---

Published in Transactions on Machine Learning Research (02/2026)
meta-regularization to directly generate missing modality representations. SMIL has shown strong gener- alization on datasets with high rate of missing modality samples. Subsequent work (Li et al. 2022d addressed challenges such as distribution inconsistency, failure to capture specific modality information, and lack of correspondence between generated and existing modalities. Researchers have also explored cross-modal distribution transformation methods to align distributions of generated and existing modality representations, enhancing discriminative ability (Wang et al.) 2023d). In audio-visual question answering (AVQA), Park et al.] (2024) proposed a Relation-aware Missing Modal Generator (RMM) that consist of to generate pseudo features of missing modalities, improving robustness and accuracy. The RMM consists of a visual generator, auditory generator, and text generator, which generate missing auditory representations by analyzing the correlations between visual and textual modalities and utilizing learnable parameter vectors (slots) for reconstruction and synthesis of missing-modality features.
Recently, to address the potential absence of different modalities in sentiment analysis, Guo et al. (2024) proposed a Missing Modality Generation Module for prompt learning. This module maps prompts of available modalities to prompls of missing modalities through set of projection layers. Lin et al. (2024) proposed Uncertainty Estimation Module which can identify useless tokens in different modality tokens to facilitate better use of C-Addapee-asssiseed pre-trained models to better utilize the tokens of available modalities when downstream tasks face missing modalities. Liao et al. (0025) explore a variety of different conditions of sensor impairment are and the capabilities of various existing multimodal segmentation models are evaluated. Park et al. 2025 use the Multi-modal Mixture of Expert, which not only enables the model to dynamically handle multi- modliitodality, but also allows different branches to focus on their own modalities by sharing query.
Compared to indirect methods, direct generation methods avoid the eumbersome reconstruction of modality data, using feature generators that are easier to train and integrate into multimodal models.
In summary, representation generation methods are limited by the imbalance of missing modalities in the training dataset or the limited number of full-modality samples, which may lead to training procedures that overfit to the existing modalities. In addition, indirect-to-task representation generation methods can access complete multimodal data samples during training, which means they can simulate arbitrary missing modality cases during training by dropping modalities. As a result, they generally achieve better performance than direct-to-task methods, which must handle incomplete multimodal data samples during training.
4 Methodologies in Strategy Design Aspect
4.1 Architecture-Focused Models
Different from above methods of handling missing modality problems at the modality or representation level, many researchers adjust the model training or lesting archilecture to adapt to the missing- missing-modalityy cases. We divide them into four categories according to their core contributions in dealing with missing modalities: attention-based mothods, distillation-based methods, graph lcarning-base based methods, and multimodal large language models.
4.1.1 Attention Based Methods
In the self-attention mechanism (Vaswani et al. 2017). each input is linearly transformed to generate Query, Key, and Value vectors. Attention weights are computed by multiplying the query of each element with the keys of others, followed by scaling and softmax to ensure the weights sum to Finally, weighted sum of the values generates the output. We classily atlention-based MLMM methods into two categories. (I) Attention fusion methods put attention on modality fusion, integrating multimodal information, which do not rely on any specific model type and can be suitable for various model types as its input and output dimensions are the same. (2) Others are transformer-based methods, which stack attention layers to handle large-scale data with global information capture and parallelization. We provide more details of these two methods below.
Attention Fusion Methods have the powerful ability to capture key features and can be seen as plug-and- play modules. We categorize them into two types: intra- and inter-modality attention methods. Intra-
11

---

Published in Transactions on Machine Learning Research (02/2020)
Fusion Representation
Intra Alica-1
Mod
DNN-I
Representation-
Mod
DNN-2
Representation-2
Intra Allio=2
Representation
Downstream Task
Intra Atten-3
Mod
DNN-3
Representations-
(a) Intra-Modality Attention Methods
Inter Atlen
Downstream
Mod
DNN-I
Representation-
Representation
Task
Ignore Modality
Mod
DNN-2
Representarion-2
Missing
Modality
Mod
DNN-3
Representation- 3
Mask
(b) Inter-Modality Attention Methods
Figure 9: Two typical attention fusion methods, assuming modality 2 as the missing modality. (a) The antra-modality attention "lterr--Atenn" in figure) of missing modality 2 can be skipped. Since each Intra- Atten of different modalities shares the fusion reprosentation (like Icarnable query in (Ge et al. 2023) or bottleneck tokens in (Nagrani et al.] (2021)), after calculating them one by one, the output representation can be regarded as the fusion of available modalities. (b) "Missing Modality Mask" is the custom mask that is generated according to the missing modality 2 and can help inter-modality attention ("Inter-Atten" in figure) ignore the missing modality 2. By sctting the tokens of modality to zero negative infinity through masking, can force the attention mechanism to ignore the missing modality 2.
Modality Attention Methods compute attention for each modality independently before fusing them, as shown in figure Da This approach focuses on relationships within single modality, and the fusion between modalities is achieved by sharing partial information. For instance, in 3D detection, the BEV-Evolving Decoder (Ge ct al.] 2023) handles sensor failures by sharing same BEV-query with each modality-specific attention modules, allowing fusion of any nunber of modalities. Similarly, in clinical diagnosis, [Le et al.) (2023a) proposed modality-aware attention to perform intra-modaliy attention and predict decisions by using shared bottleneck fusion tokens (Nagrani et al., 2021). Inter-Modality Attention Methods, often based on masked attention, treat missing modality features as masked vectors (using zero or nogative infinity values) to better capture dependencies across available modalities, as illustrated figure 9b Unlike convent ional cross-modal attention, masked attention models share the same parameters across all embeddings, allowing flexible handling of missing modalities. For example, Qian & Wang (2023 designed attention mask matrix to ignore missing modalities, improving model robustness. Similarly, DrFuse (Yao et al. 2024) decouples modalities into specific and shared representations, using the shared ones to replace missing modalities, with a custom mask matrix to help model ignore the specific representations of the missing modality.
Transformer Based Methods can be divided into two types: joint representation learning (JRL) and parame- ter efficient learning (PEL.) according to full parameter training and a small amount of parameter fine-tuning. Since transformers have long context lengths to handle many feature tokens, the multimodal transformer can learn joint representations from any number of modality tokens (figure 10 Some methods first use modality encoders to extract feature tokens from available modalities and then feed them into multimodal transformer. Gong et al. (2023) introduced an egocentric multinodal task, proposing transformer- fusion module with a flexible number of modality tokens and a cross-modal contrast alignment loss to map features into a common space. Similarly, Mordacq et al.) (2024) leveraged Masked Multimodal Transformers, treating missing modalities as masked tokens for robust JRL. Ma et al. (2022) proposod an optimal fusion strategy search strategy within the multimodal transformer to help find the best fusion method for dif- ferent. missing/ meaiinggfullmmonn modalmyyllyyy representations. Radosavovic et al. (2224) introduced specially designed mask
12

---

Published in Transactions on Machine Learning Research (02/2026)
Modality Engoder Tokens
Mod
DNN.I
DNN-2
Multimodal Transformer
Downstream Task
Representation
Set Missing Modality Tokens
Mod 2
DNN-2
Figure 10: Description of general idea of joint representation learning methods, assuming Modality is the missing modality. The missing modality tokens can be replaced by specific masked tokens. The modality encoders (red dashed box) can be replaced by linear projection layers in some methods.
tokens for an autorogressive transformer to manage missing modalities, successfully deploying the model in real-world scenarios.
With the rise of pre-trained transformer models, PEL methods have been developed to fine-tune these models by training few parameters. Two common PEL methods for pre-trained models prompt and adapter tuning. Initially used in natural language processing, prompt tuning optimizes input prompts while keeping model parameters fixed. It has since been extended to multimodal models. [Jang et al. modality-specifie prompts to address limitations in earlier methods, which merge when all modalities are
(2024) introduced
present and allow the model to update all learnable prompts during training. Liu et al. (2024b) further improved this by proposing Fourier Prompt, which uses fast Fourier transform to encode global spectral information of available modalitics into learnable prompt tokens which can be used to supplement missing- modality features, enabling cross-allention with feature tokens to address missing modalities. Adapter tuning, the other hand, involves inserting lightweight adapter layers (e.g.. MLPs) into pre-trained models to adapt to new tasks without modifying the original parameters. Qiu et al. (2023) proposed method that used a classifier to identify different missing-modd modality cases and used intermediate features of that classifier as the missing-modality prompt to cooperate with lightweight. Adapters to address missing modality problems.
Although attention-based fusion mechanisms above methods can effectively help deal with the missing modality problem in any framework, none of them cares about the missing modality that may contain important information required for prediction. In the transformer-based methods, the JRL methods are often limited by large amount of computing resources and require relatively large datasets to achieve good performance. The PEL methods can achieve efficient fine-tuning, but their performance is still not comparable to that of JRL.
4.1.2 Distillation Based Methods
Knowledge distillation (Hinton et al. 2015) transfers knowledge from a teacher model a student model. The teacher model, with access to more information, helps the student reconstruct missing modalities. Below, we categorize two types of distillation methods for addressing this problem.
Representation- Based Distillation Methods transfer rich representations from the teacher model to help the student capture and reconstruct missing modality features. We classify them based on whether they use logits or intermediate fealures. figure [T illustrates this type of method. Response distillation mot focus on transferring teacher models' logits to students, helping it mimic probability distributions. [Wang] et al. (2020a) trained modality-specific teachers for missing modalities, then used their soft labels to guide multimodal student. Hafner & Ban (2023) employed logits from teacher model trained with optical data to supervise reconstruction network for approximating missing optical features from radar data. A
13

---

Published in Transactions on Machine Learning Research (02/2026)
Mod
Mod 2
Teacher
Representation-T
Downstream
Task
Mod
Intermediate Distillation
Response Distillation
Mod
Student DNN
Downstream
Mod
Representation
Task
Mod 3
Figure 11: Description of representation distillation methods. Here we modality 2 is missing. In order to distinguish the representations of teachers and students, we add 4-T" and "-S" after their representations in the figure. We refer to intermediate distillation using features from any combination of intermediate layers within models.
Mod
Downstream Tiisk
Mod
Teacher DNN
Representation-T
Mod
Mod
Student
Downstream Task
Mod 2
Roprescntation-
DNN
Mod
(a) Mean Teacher Distillation Methods
Self-Distillation
Student/Teacher DNNN-
Representation- 1/S.1
Mod
Student/ Teacher DNN-2
Representation- T/S-2
Downstream Task
Mod
Mod
Student/Teadee DNN-n
Representation-TSSS T/S-n
(b) Self Distillation Methods
Figure 12: Description of two process-based distillation methods. (a) "EMA" means exponential moving average. To demonstrate the general architecture Mean Teacher Distillation, we set the loss on the logits representation. (b) Self-distillation methods for missing modality problems involve the use of models that act teachers and students, where these models use their own soft labels/ropresentationn from each branch refine themselves during training.
-owality-Aware Distillation was proposed by (Shaa et al. (2024). leveraging both global and local teacher knowledge a federated learning setting to help student models to learn how handle missing modalities.
Intermediate distillation methods align intermediate features between teacher and student models. [Shnn & Gao 2019) used Domain Adversarial Similarity Loss to align intermediate layers of the teacher and student, improving segmentation in missing modality settings. Zhang et al. (2023b) applicd intermediate distillation in endometriosis diagnosis by distilling features from a TVUS-trained leacher to student using MRI data. Some researchers propose MultiPro (Xu et al.) 2025) to learn the distribution of the missing modality by distilling unimodal knowlodge into Icarnable unimodal prompts for each modality from all remaining samples available of this modality. Some also tackles the missing-modality problem in multimodal federated mobile sensing systems by introigg FedMobile [Liu et al. 2025a), know/elge-cotrrbbtiinnaaaa federated
14

---

Published in Transactions on Machine Learning Research (02/2026)
Gruph Fusion
Mod!
DNN-I
Represeniaation I
Downstream
Mod
DNN-2
Representation
DNN
Task
Mod.3
DNN-3
Represcrtatioon-
Figure 13: Illssraaioon of graph fusion methods. We set modality 2 (white circle in the graph) as the missing modalities. Features from modality 1 and 3 can be aggregated via this graph fusion method and fused features maintains the same dimension.
learning framework that reconstructs missing features through cross-node multimodal knowledge transfer and robust aggregation.
Process- Distillation Methods focus on overall distillation strategies, like Mean Teacher Distillation
(MTD) (Tarvainen & Valpola, 2017) and sclf-distillation (Zhang et al., [019). These methods emphasize
procedural learning over direct representation transfer. MTD enhances stability by using the exponential
moving average of the student model's parameters teacher (figure 12ah. Chen et al. (2024b) applied this
to missing modality sentiment analysis, treating missing samples as augmented data. Li et al. 2022c) used
MTD for Radar segmentation, improving robustness against missing modalities.
Self-distillation helps a model improve by learning from its own soft representations (figure 12b Wang et al. proposed the ShaSpec (Wang et al. 2023a),, which utilizes distillation between modality-shared branches and modality-specific branches of available modalities. Based on ShaSpec, rescarchers proposed Meta-learned Cross-moda Knowledge Distillation (Wang et al. (0244) to further weigh the importance of different available modalities to improve performance. [Ramazanova al. (024) used mutual information and ooff-disilllation for egocentric tasks, making predictions invariant to missing modalities. Shi et al. pro- posed PASSION (Shi et al.) 2024), a self-distillation method designed to use the multi-modal branch to help other uni-modal branches of available modalities to improve multimodal medical segmentation performance with missing modality.
Hybrid Distillation Methods combine various distillation approaches improve student performance, For medical segmentation, Yang et al. (2022) distilled teacher model knowledge (including logits and intermediate features) every decoder layer, outperforming ACNet (Wang et al. 2021). Wang et al. (022cc) introduced ProtoKD, which captures inter-class feature relationships for improved segmentation under missing modality conditions. Recently, CorrKD (Li et al. 2024a) leverages contrastive distillation and prototype learning to enhance performance in uncertain missing modality cases.
Aforementioned methods address the missing modality problem and achieve good generalization during testing. However, except for some intermediate and self-distillation methods, most assume the complete dataset is available for training (means teacher can access full-modality samples during training), with missing modalities encountered only during testing. Therefore, most distillation methods are unsuitable for handling incomplete training datasets.
4.1.3 Graph Learning Based Methods
Graph-learning based methods leverage relationships between nodes and edges in graph-structured data for representation learning and prediction. We categorize approaches for addressing the missing modality problem into two main types: graph fusion and graph neural network (GNN) methods.
Graph Fusion Methods integrate multimodal data using graph structure (figure 13 making them adapt- able to various networks. For example, Angelou et al. (2019) proposed method mapping cach modality to a common space using graph techniques to preserve distances and internal structures. Chen & Zhang (2020) introduced HGMF, which builds complex relational networks using hyper edges that dynamically connect available modalities. Zhao et al. (2022) developed Modality-Adattive Feature Interaction for Brain Tumor Segmentation, adjusting feature interactions across modalities based on binary existence codes. More recently, Yang et. al. (2023a) proposed graph attention based Fusion Block to adaptively fuse multimodal
15

---

Published in Transactions on Machine Learning Research (02/2026)
Process Each Mouillty Separately
Graph Construction
Med
Downstream Task
GNN
Mod 2
DNN-2
Representation-2
Mod
DNN-3
Representarion-3
(a) Individual GNN Methods
Graph
Graph Construction
Completion
Mod
Downstream Task
Mod
GNN
Mod 3
(b) Unified GNN Methods
Figure 14: Illustraiion of graph neural network (GNN) based methods. We set modality 2 as the missing modalities. (a) Each modality first passes through DNN or builds its graph, and then they are uniformly processed using GNNs. (b) This method first complete the graph from missing modality samples via ap- proaches like FeatProp (Malitesta et al. 2024), then process it by GNNs.
features, using attention-based message passing to share information between modalities. These fusion methods can be flexibly inserted into any network for integrating multiple modalities.
Graph Neural Network Methods directly encode multimodal information into a graph structure, with GNNs
used to learn and fuse this information. Early approaches (Wang et al. 2015 employed Laplacian graphs to connect complete and incomplete samples. Individual GNN methods (figure 14a] such as DESAlign (Wang ett al., 2024b), extract features using neural networks or GNNs and fuse them for prediction. Unified GNN methods ( figure 14b) first complete the graph and then use GNNs for prediction, such as in Zhang al.'s M3Care (Zhang al. 2022). which uses adaptive weights to integrate information from similar pa- tients. Lian et al. (2023 proposed the Graph Completion Network, which reconstructs missing modalities by mapping features back to the input space. In recommendation systems, TeatPPoo et al. (2024)
propagates known multimodal features to infer missing ones, while MUSE (Wu et al. 2024b) represents modicntymodality relationships as bipartite graphs and learns unified patient representations. In knowledge graphs, Chen al. (2023b) introduced Entity-Level Modality Alignment to dynamically assign lower weights missing/ missing/uncertaain /nnerrain modalities, reducing the risk of misguidance in learning. Some work (Liu al. 2020) addresses the missing-sensor
moddaitty problem in topology-aware IoT systems by introducing graph-based feature reconstruction module that uses neural message passing to recover features from avail- able sensors. Some people Chen et al. (2023a) proposes a multi-graph convolutional framework for human activity recognition that explicitly handles asynchronous and incomplete IMU, magnetometer, and physio- logical signals, addressing missing- modaaltty challenges in heterogencous multi-sensor environments.
The methods above can leverage the graph structure to better capture relationships both within/betwee modalities and samples. However, these methods tend have lower efficiency, scalability, and higher devel- opment complexity compared to other kind of approaches.
4.1.4 Multimodal Large Language Models (LLM)
The impressive transformative power of LLMs, like ChatGPT (Bubeck et al. 2023), can be explained by their impressive generalization capabilities across many tasks. However, our understanding of the world depends not only language, but also other data modalities, like vision and audio. This has led
16

---

Published in Transactions on Machine Learning Research (02/2026)
rescarchers to explore MLLMs, designed to handle diverse user inputs across modalities, including cases with missing modalities, leveraging the flexibility of Transformers. Their architectures are similar to that shown in figure [0] Due to the idiosyncrasies of MLLMs, we distinguish them from normal transformer-basee joint representation learning methods in scction 4.1.1 and introduce them below. In some current MLLM architectures, LLMs act as feature processors, integrating feature lokens from different modality-specifie encodors and passing the output to rich inter-modal dependencies, but also naturally carry the ability to handle any number of modalities, that is, the ability to solve the missing modality problem.
decoders. This enables LLM to not only capture
Most MLLMs employ transformer-basee modality encoders, such as CLIP, ImageBind (Girdhar et al. 2023
and LanguageBind (Zhu et al.) 2023), which encode multimodal inputs into a unified representation space. Examples inchude BLIP-2 (Li et al. 2023b), which bridges the modality gap using a lightweight Querying Transformer, and LLAVA (Liu et al. 2024a), which enhances visual-language understanding with GPT-4- generated instruction data (Bubock et al. 2023). These models are optimized for tasks like Visual Question Answering, Dialogue, and Captioning. Recent, advancements exlend output generation to multiple modal- ities, such as images. AnyGPT (Zhan et al. 2024) and NEXT-GPT (Wu et al. 2023b) unify modalities like text, speech, and images using discrete representations and multimodal projection adaptors, enabling seamless multimodal interaction. CoDi (Tang et al., 2024) introduces Composable Multimodal Condition- ing, allowing arbitrary modality generation through weighted feature summation. There are also some other methods that discard the modality encoders and use linear transformation directly, such as Fuyu (Bavishi et al. 2023) and OtterHD (Li et al.) 2023a). Although MLLMs can flexibly handle any number of modalities, they have many disadvantages, such as the inconsistent multi-modal positional encoding, training difficulty, and high GPU resource requirements. Additionally, no specific MLLM benchmarks on missing modality problems have been proposed.
4.2 Model Combinations
Model combinations aim to use selected models for downstream tasks. These methods can be categorized into ensemble, dedicated training, and discrete scheduler methods. Ensemble methods combine predictions from multiple selected models through different types of aggregation methods, such as voting, weighted averaging, and similar approaches improve the accuracy and stability. Dedicatod training methods allocate different sub-tasks (e.g., different missing modality cases) to specialized individual models, focusing on specific sub- tasks or sub-datasets. In the discrete scheduler methods, users can use natural language instructions to enable the LLMs to autonomously select the appropriate model based on types of modalities and downstream tasks. We provide more details about these methods below.
4.2.1 Ensemble Methods
As detailed in the next paragraphs, ensemble learning methods allow flexibility in supporting different numbers of expert models to combine their prodictions, as shown in figure [5] Multimodal Model Ensemble Methods: Early work (Wagner et al. 2011) in sentiment analysis employed ensemble learning to handle missing modalities, averaging predictions from uni-modal models. With deep learning advancements, the Ensemble-basee Missing Modality Reconstruction network ŞZeng et al. 2022) was introduced, leveraging weighted judgments from multiple full-modality models when generated missing modality features exhibit semantic inconsistencies. This type of multimodal model ensemble method, depicted in figure 15a integrates various full-modality models to aid decision-making.
The general architecture of this method is shown in figure 15b where each modality is processed by uni-modal model, and only the available modalities contribute to decision- making. In multimodal medical image diagnosis, carly studies found uniformly weighted methods performed better than weighted averaging and voting approaches (Yuan et al. 2012). In multimodal object detection, Chen et al. (2022) proposed a probabilistic ensemble method. This method does not require training and can flexibly handle missing modalities through probabilistic marginalization, demonstrating high efficiency in experiments. Li et al. (2023c) recently proposed Uni-Modal Ensemble with Missing Modality Adapta- tion, training models per modality and performing late-fusion training. Other approaches Miech et. al.
Unimodal Model Ensemble Methods:
17

---

Published in Transactions on Machine Learning Research (02/2020)
Multi-Mod DNN-1
Mod
Representation-
Downstream
Mod
:
Ensemble
Task
Multi-Mod DNN-n
Mod
Representation-a
(a) Multimodal Model Ensemble Methods
Uni-Mod
Mod
Representation-1
DNN-1
Uni-Mod
Downstreum
Mod
Representation-2
Ensemble
DNN-3
Task
Uni-Mod
Mod
Representation-3
DNN-3
(b) Unimodal Model Ensemble Methods
Figure 15:
Two general architectures for ensemble methods. (a) The multimodal model ensemble methods
contain three-modal DNNs and produce final predictions based the outputs of all DNNs. (b) Each modality is processed by unimodal DNN in unimodal model ensemble methods, where final predictions are produced by aggregating the outputs of all accessible unimodal DNNs
DNN Models Library
Dedicated
Mod 1
DNN Model
ooo 00
OO
Sclect
Downstream Task
Mod 2
Representation
O
Mod 3
Figure 16: General idea of dedicated training methods. Assuming that modality 2 is missing, dedicated methods select the sample-suited model from DNN models library for training and testing. In the figure, the three modalities can form a total of seven models with different modality combinations. For easy understanding, we colored circles in purple rectangles to represent the modalities that the model can handle. In order to adapt to the missing modality 2 (light green circles), dodicated methods usually select the DNN model (purple rectangle) that can handle modality 1 (light orange circles) and modality 3 (light blue circles) for training and testing.
2018 calculate feature-based weights to fuse modalities, where weights reflect feature importance for final predictions.
Hybird Methods: Dynamic Multimodal Fusion (DynMM) Xue & Marculescu 2023 uses gating mecha- nisms to select uni/multi-mon modal models dynamically. Recently, Cha et al. (2024] proposed Proximity- based Modality Ensemble (PME) use a cross-attention mechanism with an attention bias to integrate box pre- dictions from different modalities. PME can adaptively combine box features from uni-/multi-model models and reduce noise in multi-modal decoding.
4.2.2 Dedicated Training Methods
Dedicated training methods assign different tasks to specialized models. We show the general idea of those methods in figure 16 KDNet (Hu et al. 2020) was the first dedicated method proposed to handle different combinations of missing modalities. It treats uni-modality specific models as student models, lcarning multimodal knowledge from the features and logits of a multimodal teacher model. Those trained uni- modals can be used for different missing modality cases. Following KDNet, ACNet (Wang et al. (021]] also utilizes this distillation method but introduces adversarial co-training, further improving the performance. prompts to address missing modality problems based the prompt. learning by using input- and attention- level prompts for each kind of missing modality case.
Lee et al.] (2023b) proposed moosinigmodalityy..
18

---

Published in Transactions on Machine Learning Research (02/2026)
Large Language Model
Downstream
User
Text Command
Call & Schedule
Modules Library
Module
Mod
Module
Mod 2
Module
Mod 3
Data Sample
Module
Return Results
General procedure of discrete schoduler methods. The user types text instructions make an LLM call and schedule the selected downstream task modules to process data samples and return results to the user. As long as there are sufficiently diverse downstream task modules, it is possible to handle the same downstream tasks with different numbers of modalities.
Figure 17:
This method only needed 1% of the model parameters to be fine-tuned for downstream tasks. Similarly, some researchers Reza et al. 2023 introduce different adapter layers for each missing modality case.
4.2.3 Discrete Scheduler Methods
In discrete scheduler methods (figure 17). LLMs act controllers, determining the execution order of differ- ent discrete steps broken down from the major task/instruction. While the LLM does not directly process multimodal data, it interprets language instructions and orchestrates task execution across uni- and multi- modal modules. This structured yet flexible approach is particularly effective for outputs requiring sequential tasks, enabling the system to handle any number of modalities and naturally addressing missing modality problems. For example, Visual ChatGPT (Wu et al. 2023a) integrates multiple foundation models with ChatGPT to enable interaction through text or images, allowing users to pose complex visual questions, provide editing instructions, and receive feedback within a multi-step collaboration framework. Hugging- GPT (Shen et al., 2024) is an LLM-driven agent that manages and coordinates various Al models from Hugging Face. It leverages LLMs for task planning, model selection, and summarization to address complex multimodal tasks. ViperGPT (Surís et al. 2023 combines visual and language lasks into modular sub- programs, generating and executing Python code for complex visual queries without additional training to achieve effective outputs. There are other similar discrete scheduler approaches, such as MM-REACT Yang et al. 2023b and LLaVA-Plus [Liu et al. 2023c
Some aforementioned dedicated and ensemble methods can flexibly handle the missing modality problem without additional training, but most of them require more model storage space, which not feasible for many resource-constraiee models. As the number of modalities increases, the required number of models also rises. Also, unimodal model ensemble methods struggle to adequately model the complex inter-modality relationships to make final predictions. Although the some dedicated fine-tuning methods do not require too much time and consumption of resources such as GPU, is still difficult to compare with full parameter training. addition, discrete schoduler methods can solve a variety of tasks when there are sufficient types of downstream modules, but they usually require LLMs to respond quickly and understand human instructions accurately in real world scenarios.
devices. For instance, DynMM requires storing various uni- and multi-modality
5 Methodology Discussion
In the above section 3 and scction 4 we divide the existing MLMM methods into four types from the aspects of data processing and strategy design: modality imputation, representation-foussee mnaitecture-focusse and model combinations. We also further subdivide these four types into twelve categories, exploring
19

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Comparison of deep multimodal learning with missing modality methods from four types and two aspects.
Aspect
Type
Method
Pros and Cons
Modality
Pros: Simple and effective for data augmentation.
Modality Imputation
Composition
Cous: Unsuitable for pixel-level tasks (Composition);
Modality
Modality number 1, generative model number
Data Processing
Generation
training complexity f-
Coordinated-Reprcoaatte
Pros: Flexible for uovel modalities with better generalization.
Representation
Representation Composition
Cons: Hard halance constraints and handle
Focused
Representation Generation
dataset imbalance.
Attention-base
Pros: Attention methods scale well;
Distillation- based
Distillation and graph learning effective.
Architecture
Graph Learning- based
Cons: Attention methods often demand large computation &
Focused
Multimodal Large
datasets; Distillation and graph learning struggle
Strategy
Language Model
with incomplete datasets and training complexity.
Design
Ensemble
Pros: Effective for specific tasks.
Model Combinations
Dedicated
Cons: Grows complex with more modalitics;
Discrete
Suffers from modality imbalance;
Scheduler
Demands significant storage space.
fin-grained methodology taxonomy proposed by us. Table summarizes the overall pros and cons of these methods. Generative and distillation methods are the most common approaches; they are easy to implement and deliver strong performance. With the rise of Transformers, Transformers methods have become more popular due to their larger receptive fields and parallelism. However, indirect-to-task generation methods and most distillation methods (except for some intermediate (Zhang et al.) (023b) and self-distillation meth- ods (Wang et al., 2023a Shi et al. 2024h) are currently unable to handle incomplete training datasets (means cannot access missing modality data during training). Below provide a concise analysis of these twelve methods from two aspects and four lypes.
5.1 Data Processing Aspect
Modality composition methods operate directly on the input modality data level to address missing modality problems by combining existing data, samples or filling missing data. However, they are typically not good at pixel-level downstream tasks and heavily rely on available modalities. On the other hand, modality generation mothods employ generative models to synthcsize missing modalitios, miligating the performance degradation caused by missing data. But these methods are often limited by the availability and number of full-modality samples and the increased training complexity due to the additional modalities. They also require extra storage for generative modcls.
(1) Modality Imputation:
(2) Representation Focused: Coodlnateed- methods allow novel modalities to be intro- duced by simply adding corresponding branches. However, as the number of constraints and modalities increases, balancing them becomes challenging. Correlation-driven methods often fail to handle missing modality samples in the test environment, as they are typically designed for pre-training with incomplete datasets. Similar to modality composition methods, representation composition methods attempt to recover missing modality representations by combining available-modality representations. Since representations typically carry more generalized information than modality data, these methods tend yield better results. Representation generation methods further improve this process by generating representations based re- lationships between modalities, allowing reconstruction of missing modality representations. Many studies have confirmed the evveess of this approach in handling missing modalities. However, if the dalaset contains missing modality samples, indirect- missing modality representations through reconstruction of modality data are not feasible. When the datasct has severely imbalanced modality combinations, generative methods may also fail, becoming overly reliant existing modalities.
tont--oo-aaee task representation generation methods that aim to recover
20

---

Published in Transactions on Machine Learning Research (02/2026)
5.2 Strategy Design Aspect
Some methods focus designing model training or inference architecture to alleviate the performance degradation caused by missing modalities. Attention-based methods are valued by researchers because they can effectively capture the relationships between various modalities, are scalable in terms of dataset size, and are highly parallelizable. Single allention mechanisms used for modality fusion have the advantage of being plug-and-play The drawbacks transformer-based methods in this category are also relatively obvious: training multimodal transformers from scratch requires excessive training time and many GPU resources. Also, those allention-based methods generally require larger datasets due to their high model capacity and reliance on token-level interactions. While this enables stronger representation learning in large-scale settings, it also limits their applicability in data-scarce domains such as medical imaging. This trade-off highlights the need for data-cfficient attention mechanisms or pretraining strategies in missing-modality scenarios. Recently popular PFI. methods can effectively mitigate this draw back, but the generalization performance of this method is still not as well as full-parameter training or tuning, Although MLLMs can handle an arbitrary number of modalities with flexibility, they are constrained by training complexity and require substantial computational resources.
(1) Architecture Focused:
Distillation-based methods are relatively easy to implement, with student models learning how to reconstruct missing modality representations and inter-modal relationships from teacher models. Since the leacher model typically receives full-modality samples as input, it can simulate any missing modality cases during training, ensuring strong performance. However, most distillation-bass methods are limited to the requirement of complete datasets and are inapplicable to datasets with inherent missing modalities. To our knowledge, only one intermediate distillation method (Zhang et al.] 2023b) has attempted to input. mismatched samples of the same class into the teacher and some self-distillation methods (Wang et al., 2023a Shi et al. 2024) use reprosentations of available modality branches for distillation purposes.
Graph learning-based methods capture intra- and inter-modal relationships more effectively, but the number of modalities increases, the development complexity and inefficiency of these methods become more pronouned. They are also currently unsuitable for large-scale datasets.
These methods select models for prediction. Ensemble and dedicated train- ing methods can be affected by imbalances in modality combinations within the dataset. As the number of modalities grows, the complexity and models to be trained also increase, especially for dedicated methods. Discrete scheduler methods requires an LLM to coordinate, but if callable modules are insufficient, down- stream tasks may not be completed. Additionally, inference speed is limited by LLMs and modules. Model combinations methods also require significant model storage.
(2) Model Combinations:
5.3 Recovery and Non- Rec-vecyvyyy Methods & Some Statistics
Table 2: Another common taxonomy of recovery and non-recovery methods in multimodal learning is based on three stages: carly, intermediate, and late stage, as seen in the conventional taxonomy. In this context, recovery and non-recovery methods refer to how missing modalities are handled at each stage.
Mothoes
Endly
Intermodiite
Late
HG boid
Reewwe
Modallly Compatinn kn
Modaliy Grivration Vindg
Grap leerning Ion
Rettlatien-aaad Sley d
Eepecssiatan Compaskng Rung
Kote
Conelimated Begrosctinnn
Repecestaaiinn Groenation Deellarkm-bae Live al.
Mave al D00N
ME
Dastilation loadd
Siede al 2013
Non-Bucowtry
Dirntt Suroller Wo
Dedicated Wange al Jm
Gaaph Leurning loce
Atretton-owe howd Gevtal al
Repemerraton Comp
gelr Wontraa 1070
Conclnntee Begncatnia
Dertlarimphhed
Ling
5000
Appleaale
Molinohh Larg: Lang
We have further classified MLMMs using the taxonomy in Table D to facilitate researchers to better dis- tinguish MLMMs. This classification is based on the three-stage (carly, intermediate, and late) classic multimodal learning laxonomy. Table 2 leverages this classic (axonomy of MLMMs and specifies whether they recover missing modalities or not. We also list reference papers for different methods. In our analysis of 315 papers, we found that 75.5% of the works focus on recovering missing modality information, while only 24.5% explore inference without modality recovery. Among the recovery methods, early-stage and intermediate-stage modality recovery methods account for 20.3% and 45.8%, respectively, with late-stage
21

---

Published in Transactions on Machine Learning Research (02/2026)
60
Modality Imputation
Rxpresentation-Foussee
50
Architecture-ooouee
Model Combinations
40
Number
30
10
0
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
Figure 18: The annual statistics are based on our taxonomy of four methods: Representation- Foocuttseed Architecture Forreetetrr-Fouudd,,,,ee and Model Combinations.
Modality Imputation,
and multi-stage recovery methods accounting for 4.7%, each. For non-recovery methods, early, intermediate, and late-stage modality fusion methods represent 4.2%, 14.1%, and 6.3%, respectively. We were not able find any non-recovery methods that combine multiple stages. We observe that methods recovering missing modality features in the intermediate stage account for the largest proportion. We think this is because recovering features, as opposed to the raw data, can avoid much noise and bias while providing more heliaaity-ssecifc/hhhatt sshared information. Additionally, compared to later-stage features, intermediate-stage features tend to be more enriched.
5.4 Technical Evolution Discussion
The development of MLMM methods has closely followed broader trends in machine learning and deep learn- ing, with distinct methodological phases reflected in the taxonomy and publication statistics (2014-2025). Based on the papers we collected, we count the papers for each year according to the four major categories in our taxonomy mentioned above, and plotted them using stacked bar chart (shown in figure 18
Early research was dominated by data-driven heuristics and ensemble-style solutions, where missing modal- ities were addressed by retrieving similar samples from the dataset or by training separate unimodal models and combining them through ensemble stratogics. This period corresponds to the initial emergence of Modal- ity Imputation and Model Combinations, though the overall scale research activity was still limited. With the rise of deep learning frameworks (e.g., PyTorch, TensorFlow) and the success of architectures such as ResNet, the community shifted its attention toward representation learning, leading to the rapid growth of Representation- Focused Models. From 2017 onward, this category became the most stable and persistent line of research, as learning aligned, shared, or generative representations enabled models to remain effective even when certain modalities were absent. At the same time, modality-level imputation methods evolved from simple sample-based matching to deep generative models, first through autoencoders and GANs and later through diffusion models, greatly improving the quality and generality of imputed modalities.
A notable transition occurred after 2018, possibly driven by the popularity of attention mochanisms, deep graph neural networks, and knowledge distillation. These techniques allowed models to dynamically adapt to arbitrary combinations of available modalities, leading to the rise of Architecture-Fouusee Models category that grows sharply in the statistics. These methods reframed missing modalities not merely as a data problem but increasingly as an architectural and training-strater, challenge, enabling flexible fusion, conditional routing, and intra/inter-modd knowledge transfer.
Most recently, the emergence of transformer-baser foundation models has opened a new direction: applying large pretrained models to diissing-modalln conditions by using them as feature processors, task schedulers, backbone architectures capable of consuming variable sets of modalities. This shift substantially expands
22

---

Published in Transactions on Machine Learning Research (02/2026)
the design space of multimodal systems and suggests future where meissing-modlltt robustness is a built-in capability of general-purpose models rather than specialized add-on.
Finally, although Model Combinations has historically remained a niche line of work, which we suspect is due to the cost of storing and maintaining multiple models, recent interest in Agentic AI systems may renew attention to this class of methods. As agents increasingly integrate multiple specialized tools, dynamic model selection and combination strategies may become more relevant in practical puissig-mmddaliee scenarios. Together, these developments illustrate clear technical evolution: from early heuristic data-level fixes to representation learning, to flexible architecture design, and finally toward foundation-mode modeoo-ooolllbb-bbe general multimodal intelligence. We note that applications and architectures are only weakly linked in this area, largely because modern deep learning architectures (e.g., attention) are highly versatile and transferable across tasks and applications. For instance, aumerraaiee-avvesarilllleeae basen recovery methods are used in both breast-cancer prognosis (Arya Saha
person ro-identification (Li et al., 2022d), and masked modeling strategy in human action recognition (Woo al., 2023) closely parallel those used in brain-lumor segmentation (Liu et al. 2023a). As result, once an architectural idea proves effective for handling missing modalities, it generalizes widely with minimal task-specific adaptation, which naturally weakens the direct coupling between applications and architecture.
2021) and
Limitations: Due to the varicty of settings in most misinn-mmdalitty works, it is difficult to compare performances of different methods across datasets. This represents a research gap that we encourage the community to address.
6 Applications and Datasets
The collection of multimodal datasets is often labor-intensive and costly. In certain specific application directions, issues such as user privacy concerns, sensor malfunctions on data collection devices, and other factors can result in datasets with missing modalities. In severe cases, up to 90% of the samples may have missing modalities, making challenging for conventional ML.FM achieve good performance. This has given rise to the task of MLMM. Since the factors causing incomplete datasets usually stem from differ- ent application directions, we introduce the following datasets based the applications in MLMM tasks: Affective Computing, Medical Applications, Information Retrieval, Remote Sensing, Pervasive Computing, Robotic Vision and Sensing, and others. We categorize those datasots according to the applications and data type in Table 3 It should be noted that for the medical field, we only show part datasets due to their large number.
6.1 Affective Computing
Typically, the goal of affective computing is to classify the current emotional state by combining information from multiple modalities such as textual, auditory, and RGB information. This field has garnered significant attention due to its promising applications in various specific scenarios, including market research, health monitoring, and advertising. In the early stages of audio-visual affective computing research, researchers discovered that facial detection algorithm in data collection devices sometimes failed to capture faces (e.g., due occlusion) or the recorded audio too noisy to use (Cohen et al. 2004: Sebe et al. 2005), resulting in some samples that only contained audio or image. Consequently, researchers began exploring methods to enable affective computing models to continue functioning effectively when facing missing- modality samples. Currently, the datasets for applying affective computing can be roughly divided into two categories. One
missing-modality
type of dataset MMartin et al. 2006
Livingstone & Russo 2018 Cao et al. 2014 Stappen et al.] 2021a b
Ringeval el al. 2013 Zheng & Lu 2015 Zadeh et al.; 2018 Busso el al. 2008 2016 Poria el. al.
2018
et al. 2020 Wöllmer et al. 2013 Morency et al.) 2011 involves using video, audio, text, and biosensors to
determine human emotional states real life or in movies. The other type (Guérin al., 2013 Nguyen
let al.) 2018 Matsui et al. 2017) involves using text and images to assess emotional states contexts such
comic books. Please refer to Table 3 for details.
23

---

Published in Transactions on Machine Learning Research (02/2026)
Table 3: Common datasets used by deep MLMM methods divided by main applications and modality types. "Vision" includes data from visual sensors such as RGB images, Depth, Infrared, LiDAR, Radar, Event, and Optical Flow. "Bio-Sensors" includes data from sensors like: electrophysiological, respiratory sensors and on. "Motion Sensors" includes data. from sensors like accelerometers, gyroscopes, magnctometers, IMUs, barometric sensors, and gail sensors. "Physiological Signals" including ECG, PPG/BVP, EDA/ /GSR, RESP, EEG, EMG, EOG, skin temperature, and SpO2. Other modalities include audio, text, CT scans, MRI, and skeleton.
Application
Modality Types
Dutaeela [Lavhvstone & Rosen p0ts]
Commson and
Typical
Vieion-Anih Aniio
INTRRPACEVO
Martin at al
RAAVDESS
CRPMA-D Ct
2014
DCM
Affective
ME-TST
Stagpor
Computing
MU-MDII
Moden-4-Audlle
MSI-IMPRI
Vi
Wallow
YorTube
Scur
ERA
Momaar
AUNT
Sttoe //bratn decelcpaent. wagg//tggtt dataxet
Applications
Medical
Blo Semsors
Sceesel
BCDaia
Ninag 1 mi 2000
wackk Roolbatatii di
Garrrr Romip al 1
2040
LYON19
Shiliner Ciadla ot 2010
Elettonte Health Record
XCB (La mt Ae 2la2 Po
KIDMI Xie Uniter el
MIM
Missiee Silmwa of
2121
Information Retrieval
VisioneTie Text
Auacia Statis
MeAukd ot Allin de Mokuley 2016 PronpCCral 2017
Valin+Aallig firct
VSR.VTT No
WEL Lo
Sunsing
Oppeet
ant Rororet
2010 Reaaoa Uhires PW DSMMS
Ragginn Alpi Aote PhORD
Rebes
Computing
Pervuive
Plytooogol Signnls
suseas
Smuth
WISAD (Shmikh el dr pirst Normarr (karll (224), Cill Ti
EMG Pinzz aormmn Hunkeeee
Stads 1i pilsi Mar Sayle Veriii Arin KOHAIR
2010
Sel Eopar-mestt 21 dirtomann aral 1990) MAAEAMI Schalk dt A. 24H
Robotic Vision and Sensling
Vision
Delivey Mpinccem 1i2h NiSrener Cossu
leg Vest al 2121 XYSt--MMl
del bil liwahh Ialaaan
2020 HV33 ore
5016
Visson Audho
EgolD-AA [Gounn (00) Epe Spu-rru
Farss Jolu Kavamili 2002
Vition HSackiin lastial Dataa
31M Eoel
Neliission
2019
NTUM
6.2 Medical Applications
Medical domain requires comprehensive analysis from various modalitics such as medical history, physical examination, imaging data, cell and gene data, which is exactly what multimodal learning is good at, many researchers developed multimodal intelligent systems. The current datasets for multimodal medical domain that contain data with missing modality problems are focused on below arcas: Neuroimaging and Brain
Disorders (Menze al.) 2014, Myronenko 2019 Young al.) 2017 Jack Jr et al. 2008 Lesjak et al., 2018
Cardiovascular (Liu et al. 2016; Moody & Mark 2001
Cancer Detection (Arya & Saha 2020 Tomezak
et al.
2015
Cheerla & Gevaert 2019: Farooq et al.
2025
Women Health Analysis (Zhang et al.
2023b Xu
ct al.
2021
, Ophthalmology (Li ct al. 2021 Zhang et al.
2022), Sleep Disorders Lee et al. 2022
Marcus
let al. 2013
Clinical Predictions (Johnson et al.
2016
2023
2019), Biomedical
Analysis
Chaptoukaev
ctt al. 2024
Schmidt ot al. 2018b Kemp et al. 2000), Computational Pathology
Cui ct al.
(2022) Peng
et al. 20024;
Wang el al. (2025a); Xu et al. (2025), Radiology Chen et al. 2025 Cui et al.
(2022); Liang
et al. (0025).
and Gene Schmauch et al. (2020) and Cell Ghahremani al. [002]]
; Liu et al.
(2025b); Palma
et al.
[2024]; Tu et al. (2022) Representation learning. We have categorized some commonly used and
representative datasets from list above by modality type, as shown in Table 3
6.3
Information Retrieval
Information retrieval is a technology that automatically retrieves relevant content or data. based on queries, historical behavior, preferences, and attribute data. It has received widespread attention due to its success various platforms such search engines and social media. It improves user satisfaction and informa- tion access experience through algorithmic analysis and prediction of content that users may find relevant. Multimodal learning makes modern retricval systems possible because it can effectively process and analyze multiple types of information, including lext, images, audio, etc. Due to user privacy concerns and data sparsity issues, some researchers have begun exploring MLMM approaches in retrieval systems. Many of these studics leverage datasets [Lakkaraju et al. 2013 McAulcy et al.) 2015 He & McAuley 2016 Prompt-|
Cloud. 2017 Duin Chua et al.) July 8-10, 2009 Xu et al. 2016)] Lin et al. (023)) from websites such
Amazon and TikToo for their research. We list those datasets based on modality type in Table
24

---

Published in Transactions on Machine Learning Research (02/2026)
6.4 Remote Sensing
By leveraging multimodal learning, we can integrate various types of visual data, such as Syuthetic Aperture Radar data and multi- hhpprssshyppr--pettrallll data. As result, various datasets have emerged (Yokoya et al. 2020;
Shermeyer et al. 2020: Rasti et al., 2017: Piacentini 2023a Hong et al. 2021 Debes et al., 2014). Those datasets enable the assessment and analysis of different environmental conditions, resources, and disasters on earth from satellites or aircraft. Such capabilities can significantly aid in environmental protection, resource management, and disaster response. In practice, remote sensing tasks, such as multimodal land cover classification lasks, optical modalities may be unavailable due to cloud cover or sensor damage. Therefore, it is also necessary address the missing modality problem for remote sensing problems.
6.5 Pervasive Computing
The pervasive computing community has also developed a wide range of multimodal sensing datasets that capture the complexity of real-world mobile, wearable, and IoT environments. They can be meaningfully organized by task category and reflect the dominant research problems across pervasive and mobile comput- ing. (1) Human Activity Recognition. A broad class of datasets focuses on fine-grained activity recognition,
gesture understanding. and daily-living behavior modeling. Representative collections include Opportu-
nity (Rggenn et al. 2010), RealDisp (Banos et al. 2014). DSADS (Barshan & Altun 2010 PAMAP2 (Reiss]
2012 Shoaib (Shoaib el al.] 2014), RealWorkd-HAT (Sxtyler & Stuckenschmicl, 2016). w-HAR Buurzer et al. 2025 ERing (Wilhelm et al. 2015), and EMG Physical Action (Theodoridis, 2.11h. These datasets
capture heterogencous, body-mounted signals-accelcronn accelerometry, gyroscope, magnetometer,
in naturalistic environments that inherently involve sensor dropout, misaligument, and variable sampling. (2) Physiological Monitoring. Many datasets target long-term physiological monitoring. Examples include WESAD (Schmidt et al., Epillpss (Villar et al., 2016), and combining biosignals such as ECG, EDA, BVP, RESP, or skin temperature, with motion sensors contextual smartphone streams. Their free-living collection protocol results in multimodal recordings with non-wear intervals, irregular sampling, and missing physiological channels. (3) Cognitive State Inference. Datasets for cognitive-state modeling, mental workload analysis, and motor imagery- such as Self RegulationSCP1 Birbaumer et al., 1999) and EEG-MMID (Schalk ett al. 2004] contain high-dimensional EEG signals. EEG's susceptibility to noise, channel corruption, and abrupt signal loss naturally produces structural missingness at both temporal and spatial (channel) levels. Taken together, these lask-driven dataset categories illustrate that pervasive computing data are inherently noisy, asynchronous, and incomplete, with many recordings containing long gaps, irregular sampling, and sensor-level failures. As result, they serve not only as benchmarks for multimodal recognition, context understanding, and mobile health monitoring, but also as natural testbeds for evaluating MLMM methods under real-world structural missingness. Their modality heterogeneity, device-specific characteristics, and episodic sensor dropout further make them valuable for assessing robustness, cross-device generalization, and lightweight recovery strategics necessary for deployment on resourco-constrained edge platforms. The classification of above dataset on modalities can be found in Table R
6.6 Robotic Vision and Sensing
Robotic vision is the field that enables robots to perceive and understand their enviromment through visual sensors by acquiring and processing image data. This includes tasks such as object and facial recognition, environment modeling, navigation, and behavior understanding, allowing robots to autonomously perform complex operations and interact with humans. Typically, the modality combination formula in robotic vision tasks can be expressed as RGB+X, where X can include, but are not limited to, LiDAR, radar, in- frared sensors, depth sensors and auditory sensors. Both RGB and other sensor data can be missing for various reasons. Below, we introduce five common tasks in robotic vision concerning the missing modality problems. (1) Multinodal Segmentation: This task aims to inpul multinodal data and use segmentation masks to locate objects of interest. In the context of missing modalities, this task is common in datasets for autonomous vehicles, where sensors might fail due to damage or adverse weather conditions. Typi- cal data combinations include (Depth/D Optical Flow Flow/LiDAA LLer/LLDD///Rdd Raaaaraaarrrreedd /Infrared/ //vent data) (Takumi
et al. 2017 Liu et al. 2023b Caesar et. al. 2020 Zhang et al. 2023a). Other datasets focus on indoor
25

---

Published in Transactions on Machine Learning Research (02/2026)
scene segmentation [iiberman et al. 2012 Ha et al. 2017 (RGB, Depth, Thermal) and material segmen- tation [Liang et. al. 2022) (RGB, Angle of Linear Polarization, Degree of Linear Polarization ion, Near-Infrared images). (2) Multimodal Detection: Similar to multimodal segmentation, multimodal detection aims to
locate objects of interest using bounding boxes. Commonly used multimodal detection combinations in the context, of missing modalities include: RGB with Depth (Silberman et. al.; 2012 Lai et al. 2011) and RGB (3) Multimodal Activity Recognition: This task can input data from the alorementionee visual sensors and also use audio cues to recognize human activities. Common
with Thermal (Waang et al.) 2015 kaggle 2020).
datasets (Grauman et al.) 2022 Huh et al. 2023 Damen et al. 2022; Gong et al. 2023; Wang et al 2014:
Shahroudy et al.) (0066) for this task are combinations of RGB with Audio, Depth, Thermal, Inertial and skeleton data. (4) Multimodal Person Re-Identiffiation: This task aims to use depth and thermal (Ye et al.
2021 Wu et al. 2017) information to robustly identily individuals under varying lighting conditions, such as at night. (5) Multimodal Face Anti-Spoofing: This task utilizes multiple sensors (such as RGB, infrared,
and depth cameras) to dctect and prevent spoofing in facial recognition systems (John & Kawanishi 2022; Zhang et al.) 2020 Liu et al., 2021a).. Its purpose is to enhance the system's ability to identify fake faces (such as photos, videos, masks, etc.) by comprehensively analyzing different modalities, thereby increasing the security and reliability of facial recognition systems.
Other Applications: There are many other areas where MLMM methods are being explored. For
example, conventional audio-visual classification (Vielzeuf et al.] 2018): multimodal large language models in visual-dialogue (Bubeck ct al., 2023); ;; audio-visual question answering (Park ct al.
2024); captioning {Liu
et al.
2023c); hand pose estimation using depth images and heat distributions Tompson et al. 2014
Choi et al. 2017;; knowledge graph completion utilizing multimodal data with missing modalities [Liu
et
al. 2019): ; multimodal time series prediction, such as stock prediction (Luo et al. 2024) and air quality
forecasting Zheng et al. 2015); multimodal gesture generation on how to create natural animations (Liu
et al. 2022
multimodal analysis of single-cell data (Mimitou et al. 2021 in biology.
Discussion: Current MLMM research primarily focuses on the areas of affective computing, medical applications, remote sensing, information retrieval, pervasive computing and robotic vision and sensing, Among these, a significant portion of research is dedicated to addressing the missing modality problem in RGB, text, and audio-based sentiment analysis, MRI segmentation and clinical prediction, and multi- sensor segmentation for autonomous vehicles. In contrast, research efforts in MLMM for streaming data and scientific fields are relatively limited. In addition, the majority of the publicly available datasets mentioned above are complete in terms of modalities, and naturally occurring datasets with missing modalities are rare. As result, these tasks are often evaluated by considering all possible combinations of missing modalities based on the existing types of modalities in the dataset, and then performing training and testing on the performance of the missing modalities, supplemented by different missing modality rates. Perhaps more future work could try to emulate Pervasive Computing by collecting multimodal datasets with real-world modal data from different models, including those with real sensor damage or different sampling rales of time-series information, for model validation. In the works we reviewed, 38% controlled for varying degrees of missing modality rates and tested accordingly, while the remaining 62% used random modality missing rates for training and testing. When categorizing missing modality rates into mild (<30%), moderate (30%- 70%), and severe (>70%) levels, the proportions are 72.6%, 82.2%, and 69.9%, respectively. Tt is worth noting that a single study may perform validation across multiple missing modality rate levels. Additionally, we found that 36.4% of the works trained on incomplete training datasets (cannot access missing modality data during training), while 63.6% actually focused on testing with missing modalities. We can see that there is still small number of rescarch on incomplete training datascts, so it is important to emphasisc the need for more research to this field.
Beyond the above observations, we also note several underexplored yet crucial research gaps. First, current benchmarks rarely model realistic missingness patterns such as systematic or asynchronous sensor failures, despite their prevalence in pervasive and wearable computing. For example, in wearable activity recogui- tion (Neeke et al.) 2018), IMUs and physiological sensors frequently sample asynchronously or drop entire segments, producing structured missingness that cannot be captured by random masking. Sccond, the se- mantic importance of different modalities is typically assumed to be uniform, and there is no principled framework for measuring modality contribution or uncertainty under missing conditions. Third, evalua-
26

---

Published in Transactions on Machine Learning Research (02/2026)
tion protocols remain highly inconsistent across studies, with varying definitions of missing rates, train-test configurations, and dataset preprocessing. For example, SMII. (Ma et. al.) 2021ch considers 90% of the mul- timodal samples in the training set with missing modalities, but DrFiuse (Yao et al. 2024) only considers the case of random missing modalities and does not consider the performance at different missing modal- ity rates. Fourth, most existing models struggle to generalize across domains, devices, or spatiotemporal distributions, highlighting the need for scalable and transfcrable foundations for MLMM. Finally, there is a lack of rigorous real-world case-study evaluations and systematic testing on out-of-distribttion multimodal datasets, both of which are essential for assessing robustness under practical deployment conditions. Ad- dressing these gaps will be critical for developing MLMM systems that can reliably operate in real-world multimodal environments.
7
Open Issues and Future Research Directions
7.1 Accurate Missing Modality//eppreenttaioo Data Generation
Some researchers (Ma et al. 2021ch have shown that recovering, reconstructing, or generating missing modal- ities or their intermediate reprosentations can improve model robustness when encountering incomplete mul- timodal inpuls. However, the generated content often contains artifacts or hallucinations,, stemming either from the generative models themselves or from imperfections and biases in their training data. Beyond im- proving generative fidelity, key future direction is to better understand informational complementarity and redundancy across modalities: not all modalities contribute equally to downstream tasks, and some provide overlapping signals that can be safely inferred, while hers contain unique information that is difficult to reconstruct without introducing bias. Therefore, exploring principled ways to generate accurate, unbiased, and task-relevant missing modalities- while leveraging modality complementarity and avoiding over-rcliance redundant cues will be an important avenue for future research.
7.2 Recovery or Non-Recovery Methods ?
In MLMM, "to recover or not to recover the missing modality", that is the question. According to our obsorvations, most current approaches handling MLMM can be broadly categorizod into two types: one focusing on recovering the missing modal information so that the multinodal model can continue function- ing (Wang al. 2023a Ma et al., 2021c), and the other attempting to make predictions using only the data modalities available (Hu et al. 2020 Zhang et al., 2024). However, some studies (Yao et al. (024) have shown that the recovered modality information might be ignored or dominated by the existing modality information due to the imbalance missing rates of different modalities in the training set,, leading the model to still rely on the available modalitics when making predictions. Morcover, Lin ct al. (2024) have found that in siluations where crucial modality information missing, the model might be unduly influenced by less important existing modality information, resulting in incorrect outcomes. In such cases, attempting to recover missing modality information can potentially alleviate this problem. Consequently, determining under what circumstances recovery non-recovery methods are more beneficial for model predictions, well as how to measure whether the recovered modality information is not dominated by other modality information or is unbiased, remains a significant challenge. Addressing these issues is essential for advancing the offectiveness and reliability of deep multimodal learning with missing modality methods.
7.3 Benchmarking and Evaluations for Missing Modality Problems
Benchmarking and fair evaluation play crucial roles in guiding the field. Many works traine//essted under different missing modality settings, which makes it difficult for researchers to find works with the same settings for comparison. Also, the recent risc of large pro-trained models, such as GPT-4, has given researchers hope for achieving Artificial General Intelligence. Consequently, more large models (Liu et al., 2024a) are integrating visual, auditory, and other modal information to realize the vision of Multimodal Large Language Models. It is important to note that recent multimodal models (especially transformer- based models) are typically trained on significantly larger and more diverse datasets than earlier approaches. Early studies missing modality learning often relied constrained, domain-specific datasets (e.g., medical
27

---

Published in Transactions on Machine Learning Research (02/2026)
imaging dataset) to validate model architectures and theoretical assumptions. In contrast, recent works leverage large-scale multimodal pretraining, sometimes involving billions of image-text or audio-visual pairs collected from the web. This expansion in data scale not only improves model generalization but also changes the nature of the learning problem, thus some recent models can implicitly infer missing information from learned priors rather than explicit modality alignment. Therefore, when comparing results across methods, it crucial to account for these disparities dataset scale, diversity, and accessibility, as they directly affect both performance and reprodueibility. Although some research has been done based on those models, detailed analysis and benchmarks are urgently needed to evaluate the performance of MLLMs on missing modality problems. We call on the community to build a work similar to MultiBench (Liang et al. 20211, covering common datasets mentioned in section 0 and settings of missing modality problems, to help researchers conduct more systematic research.
7.4 Method Efficiency
Current MLMM methods often overlook the need for more lightweight and efficient methods. For instance, some model combinations methods (Hu et al. 2020 Xue & Marculescu 2023) require training an independent model for each modality combination. Similarly; methods (Chen al. 2024a Zhang et al.] 2023c that aim to recover missing modality information typically involve using distinct model for each kind of missing information or large unified model to generate all modalities. Although these approaches generally perform well, they are often too heavy. In the real world, many multimodal models need to be deployed on resource- constrained devices, such as space or disaster-response robots. These devices cannot accommodate high- performance GPUs and are prone to sensor damage, which can be difficult/ difficul/ccsttly costlyy to repair. Therefore, there is an urgent need to explore efficient and lightweight solutions for MLMM that can operate effectively under these constraints.
7.5 Multimodal Streaming/Tempora Data with Missing Modality
Currently, only a small portion of research focuses handling missing modalities in multimodal streaming temporal data. An example is sentiment analysis (Lin & Hu 2023 Long sequences of multimodal data- such as RGB+X video streams and multimodal time-series signals are common in real-world settings and are essential for tasks like anomaly detection and vidco tracking, especially in robotic systems and pervasive computing. In such scenarios, missing modalities often occur asynchronously, creating not only spatial information gaps but also temporal misalignment between surviving modalities. This misalignment makes recovery, fusion, and prediction more challenging, as models must infer missing information while maintaining consistent temporal dependencies. Therefore, MLMM needs to address both missing-modallt problems and temporal alignment issues in streaming data to advance this field further.
7.6 Multimodal Reinforcement Learning with Missing Modality
Multimodal reinforcement learning leverages information from different sensory modalities, enabling agents to learn effective strategies in environments. This approach has wide applications in areas such as robotic grasping, drone control, and driving decisions. However, these agent-based tasks often face practical chal- lenges, as rcal-world scenarios frequently have sensor fallures or restricted access to cortain modality data, compromising the robustness of the algorithms. Currently, only a limited nunber of papers (Vasco et al., 2021 Lee et al. 2021 Awan et al. 2022 aim to address the problem of missing data modalities in rein- forcement learning. Therefore, we hope the community can focus more on the practically significant missing modality problem in reinforcement learning.
7.7 Multimodal Al with Missing Modality for Natural Science
In scientific fields such as drug prediction (Dng et al. 2020) and materials science (Muroga et al., 2023 multimodal learning plays a key role by integrating diverse data types like molccular structures, genomic sequences, and spectral images. This integration can enhance predictive accuracy and uncover new insights. However, implementing multimodal learning in these domains is challenging due to data access restrictions,
28

---

Published in Transactions on Machine Learning Research (02/2026)
high acquisition costs, and the incompatibility of heterogeneous data. These issues often result in missing modalities, hindering the potential of multimodal models. Despite the critical need, there has been limited research (He et al. 2024) on MLMM in scientific applications. Addressing this gap is essential for advancing Al-driven discoveries, requiring new methods to handle incomplete multimodal datascts and create robust models capable of learning despite missing data.
7.8 Handling Practical Missing Modality Problems in Real-World
A promising yet still insufficiently explored direction for MLMM lies in real world scenarios- inciiuinn IoT infrastructures, wearables, smart homes, mobile sensing, and distributed environmental sensor nelworks. Although several studies in pervasive computing have begun address missing- modality challenges in these settings, current efforts remain fragmented and lack holistic evaluation. These environments constitute one of the most consequential real-world deployment contexts for multimodal models, where missingness is not occasional or synthetic but structural and persistent due to heterogeneous devices operating under con- straints such as limited battery life, bandwidth restrictions, hardware degradation, asynchronous sampling, and intermittent connectivity. As a result, multimodal streams often exhibit long gaps, irregular inter- vals, and complete sensor dropout- conditions rarely captured by existing benchmarks. Empirical findings from mobile and wcarable sensing further highlight the gap: human activity recognition systems frequently encounter inconsistent sampling, device failures, and user-dependent variability (Nweke al.) 2018); wrist- worn sensing studies report prolonged non- wear periods and dropout of inertial or physiological channels channells (Van Der Donckt et al. 2024); and systems integrating IMUs, magnetometers, and physiological sensors exhibit structural missingness induced by asynchronous fusion (Chen et al. 2023a). While these studics provide valuable insights, MLMM still lacks general frameworks capable of handling long-term partial observability, heterogeneous devices, asynchronous and irregular multimodal streams, and multi-agent inconsistencies in a unified manner. More systematic evaluations and standardized benchmarks are urgently needed. A comple- mentary future direction involves developing resource-aware MLMM robustness strategies. Although prior work has explored lightweight methods such as signal augmentation and cross-modality distillation (Jeon et al. 2021), and energy-efficient imputation pipelines tailored for mobile health applications (Hussein et al. 2024b), these techniques remain application-specifi and do not yet form a generalizable toolkit. Likewise, generative approaches such as SensorGAN (Hussein & Bhat 2024) show promise for reconstructing missing sensor channols, but broader assessments of their scalability, latency, and reliability in real-world cnviron- ments are still lacking. Overall, handling practical missing modality problems in real world presents a crilical frontier for MLMM research. Despite early progress, the field still needs more comprehensive evaluations, standardized benchmarks, and general-purpose robustness principles. Ultimately, enabling multimodal AI systems that remain reliable under persistent sensor failures and dynamically evolving, heterogeneous en- vironments will require unified methodologies and architectures explicitly designed for real-world sensing constraints.
missing-modality
8 Conclusion
In this survey, we present the first comprehensive survey of Deep Multimodal Learning with Missing Modality. We begin with a brief introduction to the motivation of the missing modality problem and the real-world reasons that underscore its significance. Following this, we summarize the current advances based on our fine-grained taxonomy and review the applications and relevant datasets. Finally, we discuss the existing challenges and potential future directions in this field. Although more and more rescarchers are involved in studying the problem of missing modality, we are also concerned about some urgent issues that need to be addressed, such unified benchmark for testing and the need for wider range of applications With our comprehensive and detailed approach, we hope that this survey will inspire more rescarchers to explore missing modality problems.
Limitations: Because our survey aims to cover deep multimodal learning with missing modalities across as many domains as possible from 2012 to 2025, our taxonomy focuses on the practical machine learning perspective rather than analysing methods in a theoretical way. Moreover, although we separately review how large models address missing modalities, the rapid development of foundation model ecosystems suggests
29

---

Published in Transactions on Machine Learning Research (02/2026)
that this taxonomy may require further refinement new architectures, training paradigms, and modality agnostic interfaces continue to emerge.
In addition, method efficiency remains an underexplored yet increasingly critical challenge. Currently, most existing MLMM approaches rely on computationally heavy architectures, expensive cross modal alignment modules, or complex imputation pipelines. Their scalability to edge devices, resource constrained deploy- ments, and long horizon streaming scenarios still limited. Future research should systematically examine efficiency trade offs, including latency, memory footprint, modality specific bottlenecks, and asymmetry between training and inference, to enable practical and deployable MLMM systems.
Additionally, current benchmarks rarely capture realistic missingness patterns such as systematic, asyn- chronous, intermilient, or catastrophic sensor failures. Existing studies also lend to assume uniform modality importance and lack principled frameworks for quantifying modality contribution, uncertainty, or redundancy under missing conditions. Evaluation protocols remain inconsistent across works, hindering fair compari- son. Beyond this, most methods still struggle to generalize across domains, devices, and spatiotemporal distributions.
Finally, while MLMM research is most active in robotic vision and sensing, medical applications, affec- tive computing, remote sensing, and pervasive computing, its application to real-world sensing systems and natural science domains such as ToT infrastructures, wearables, smart homes, mobile sensing, distributed environmental sensor network, climate modeling, earth observation, molecular biology, and materials dis- covery remains sparse. These fields feature rich, heterogencous, and often severely incomplete modalities, for example missing experimental measurements partially observed molecular conformations. Extending MLMM rescarch to these areas will require domain aware missingness models, physics and biology informed constraints, and better theoretical understanding.
Addressing these limitations, along with developing stronger theoretical frameworks, more realistic bench- marks, efficioncy aware architectures, and broader cross domain applicability, will help guide the design of more robust, transferable, and scientifically grounded future MLMM research.
Acknowledgments
We appreciate all the valuable feedback from the anonymous reviewers and the TMLR editors. In addition, Renjie would like to thank all co-authors for their continued support.
References
Michalis Angelou, Vassilis Solachidis, Nicholas Vretos, and Petros Daras. Graph-based multimodal fusion with metric learning for multimodal classification. Pattern Recognétion, 95:296-230 3076, 2019.
Nikhilanand Arya and Sriparna Saha. Multi-modal classification for human breast cancer prognosis predic- tion: proposal of deep-learning based stacked ensemble model. IEEE/ACM transactions on computational biology and bioinformalics, 19(2):1032-14) 1041, 2020.
Nikhilanand Arya and Sriparna Saha. Generative incomplete multi-view prognosis predictor for breast cancer: Gimpp. IEEE/ACM Transactions on Computational Biology and Bioinformatics, (0(4):2252-2266) 2021.
Saqib Ejaz Awan, Mohammed Bennamoun, Ferdous Sohel, Frank Sanfilippo, and Girish Dwivedi. A rein- forcement learning- based approach for imputing missing data. Neural Computing and Applications, 34 (12):9701-977) 2022.
Reza Azad, Nika Khosravi, Mohammad Debghanmanshadi, Julien Johen-Adade and Dorit Merhof. Medical image segmentation on mri images with missing modalitics: A review. arXiv preprint arXiu:2003.06211 2022.
Tadas Baltrušaitis, Chaitanya Ahuja, and Louis Philipp Morency. Multimodal machine learning: A survey and taxonomy. IEFE transactions on pattern analysis and machine intelligence, 41(2):423 443, 2018.
30

---

Published in Transactions on Machine Learning Research (02/2026)
Oresti Banos, Mate Attila Toth, Miguel Damas, Hector Pomares, and Ignacio Rojas. Dealing with the effects of sensor displacement in wearable activity recognition. Sensors, 14(6):9995-10222 2014.
Billur Barshan and Kerem Altun. Daily and Sports Activities. UCI Machine Learning Repository, 2010.
DOl: https://doi.orr/ 10/ 22442//555555
Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sagnak Taşrlar. Introducing our multimodal models, 2023. URL httth://tp// http://wwwwwdpp.../////// httpbblllffuuu-8
Khaled Bayoudh, Raja Knani, Fayçal Hamdaoui, and Abdellatif Mtibaa. A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. The Visual Computer, 38(8): 2939-2970, 2022.
Niels Birbaumer, Nimr Ghanayim, Thilo Hinterberger, Iver Iversen, Boris Kotchoubey, Andrea Kübler, Juri Perelmouter, Edward Taub, and Herta Flor. A spelling device for the paralysed. Nature, 398(6725): 297-298, 1999.
Benjamin Bischke, Patrick Helber, Florian Koenig, Damian Borth, and Andreas Dengel. Overcoming missing and incomplete modalities with generative adversarial networks for building footprint segmentation. In 2018 International Conference on Content- Based Multimedia Indezing (CBMT), pp. 1-6. IEFE, 2018.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Fldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint KVv:2233.12712, 2023.
Maximilian Burzer, Tobias King, Till Riedel, Michael Beigl, and Tobias Röddiger. Whar datasets: An open source library for wearable human activity recognition. arXi preprint arXiv:2508.1660; 2025.
Carlos Busso, Murtaza Bulut, Chi-Chun Loc, Abe Kazemzadch, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth Narayanan. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evoluation, 42:335 359, 2008.
Carlos Busso, Srinivas Parthasarathy, Alec Burmania, Mohammed AbdelWahab, Najmeh Sadoughi, and Emiil Mower Provost. Msp-improv: An acted corpus dyadic interactions to study emotion perception. IEEE Transoctions on Affective Computing, 8(1):67 80, 2016.
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11621-11631, 2020.
Sergio Campos, Luis Pizarro, Carlos Valle, Katherine R Gray, Daniel Rueckert, and Héctor Allende. Evalu- ating imputation techniques for missing data in adni: a patient classification study. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 20th Iberoamerican Congress, CIARP 2015, Montevideo, Uruguay, November 9-12, 2015, Proceedings 20, pp. 3-10. Springer, 2015.
Houwei Cao, David Cooper, Michael K Keutmann, Ruben C Gur, Ani Nenkova, and Ragini Verma. Crema- d: Crowd-sourced emotional multimodal actors dataset. IEEE transactions on affective compuling, 5(4): 377-390, 2014.
Juhan Cha, Minseok Joo, Jihwan Park, Sanghyeok Lee, Injae Kim, and Hyunwoo J. Kim. Robust multimodal 3d object detection via modality-agnostic decoding and proximity-based modality ensemble, 2024. URL
https://arxiv.. wwtp://.wgg/aa/aaeaeeeeaeee 2407 191556
Guoqing Chao, Shiliang Sun, and Jinbo Bi. A survey on multiview clustering. IEEE Transactions on Artificial Intelligence, 2(2):146 168, 2021. doi: TA1109//AA.2021..000
Hava Chaptoukacv, Valeriya Strizhkova, Michele Panaricllo, Bianca Dalpaos, Aglind Reka, Valeria Manera, Susanne Thümmler, Esma Ismailova, Massimiliano Todisco, Maria A Zuluaga, et al. Stressid: multimodal dataset for stress identification. Advances in Neural Information Processing Systems, 36, 2024.
31

---

Published in Transactions on Machine Learning Research (02/2026)
Agisilaos Chartsias, Thomas Joyce, Mario Valerio Giuffrida, and Sotirios A Tsaftaris. Multimodal mr syn- thesis via modality-invariant latent representation. IEEF transactions on medionl imaging, 37 (3):8:003 814, 2017.
Anika Cheerla and Olivier Gevaert. Deep learning with multimodal representation for pancancer prognosis prodiction. Bioinformatics, 35(14):446 i454, 2019.
Cheng Chen, Qi Dou, Yueming Jin, Hao Chen, Jing Qin, and Pheng-Ann Heng. Robust multimodal brain tumor segmentation via. feature disentanglement and gated fusion. In Medical Image Compuling and Computer Assisted Intervention MICCAI 2019: 22nd International Conference, Shenzhen, China, October 18-17, 2019, Proceedings, Port III 22, pp. 447-456. Springer, 2019.
Jiayi Chen and Aidong Zhang. Hgmf: heterogencous graph-basod fusion for multimodal data with incom- pleteness. In Proceddings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1295-1305, 2020.
Ling Chen, Yingsong Luo, Liangying Peng, Rong Hu, Yi Zhang, and Shenghuan Miao. A multi-gran graph convolutional network based wearable human activity recognition method using multi-sensors. Applied Intelligence, 53(23): 28185, 2023a.
Qianqian Chen, Jiadong Zhang, Runqi Meng, Lei Zhou, Zhenhui Li, Qianjin Feng, and Dinggang Shen. Modality-specific information disentanglement from multi-parametric mri for breast tumor segmentation and computer-aided diagnosis. IEEE Transactions on Medical Imaging, 2024a.
Sishuo Chen, Lei Li, Shuhuai Ren, Rundong Gao, Yuanxin Liu, Xiaohan Bi, Xu Sun, and Lu Hou. Towards multimodal video paragraph captioning models robust to missing modality. arXiv preprint 2000 19221, 2024b.
Weixing Chen, Yang Liu, Ce Wang, Jiarui Zhu, Guanbin Li, Cheng-Lin Liu, and Liang Lin. Cross-modal causal representation learning for radiology report generation. IEEE Transactions on Image Processing, 2025.
Yi-Ting Chen, Jinghao Shi, Zelin Ye, Christoph Mertz, Deva Ramanan, and Shu Kong. Multimodal ob- ject detection via probabilistic ensembling. In European Conference on Computer Vision, pp. 139-158. Springer, 2022.
Zhengyang Chen, Shuai Wang, and Yanmin Qian. Multi-modality matters: A performance leap voxcoleb. In INTERSPEECH, pp. 2252 2256, 2020.
Zhuo Chen, Lingbing Guo, Yin Fang, Yichi Zhang, Jiaoyan Chen, Jeff Z Pan, Yangning Li, Huajun Chen, and Wen Zhang. Rethinking uncertainly missing and ambiguous visual modality in multi multi-modal modal entity alignment. In International Semantic Web Conference, pp. 121-139. Springer, 2023b.
Chiho Choi, Sangpil Kim, and Karthik Ramani. Learning hand articulations by hallucinating heat distribu- tion. In Proceedings of the IEEE International Conference on Compuler Vision, pp. 3104-33 3113,,13, 2017.
Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-Tao Zheng. Nus-wide: A real-world web image database from national university of singapore. In Proc. of ACM Conf. on Image and Video Retrieval (CIVR (09), Santorini, Greece., July 8-10, 2009.
Tra Cohen, Fabio Gagliardi Cozman, Nicu Sebe, Marcelo Cosar Cirelo, and Thomas S Huang. Semisupervised learning of classifiers: Theory, algorithms, and their application to human-compuler interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(12):1553-155) 2004.
Can Cui, Han Liu, Quan Liu, Ruining Deng, Zuhayr Asad, Yaohong Wang, Shilin Zhao, Haichun Yang, Bennett A Landman, and Yuankai Huo. Survival prediction of brain cancer with incomplete radiology. pathology, genomic, and demographic data. In International Conference on Medical Image Computing and Computer-Assie Intervention, pp. 626- (3--655.6 Springer, 2022.
32

---

Published in Transactions on Machine Learning Research (02/2026)
Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et. al. Rescaling egocentric vision: Collec- tion, pipeline and challenges for epic-kitchens-100 International Journal of Computer Vision, pp. 13, 2022.
Christian Debes, Andreas Merentitis, Roel Heremans, Jürgen Hahn, Nikolaos Frangiadakis, Tim van Kasteren, Wenzhi Liao, Rik Bellens, Alcksandra Pižurica, Sidharta Gautama, et al. Hyperspectral and lidar data fusion: Outcome the 2013 grss data fusion contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6):2405-2418, 2014.
Ruben Delgado- Belgado-Escano, Francisco M Castro, Nicolás Guil, Vicky Kalogeiton, and Manuel J Marin-Jimenez. Multimodal gait recognition under missing modalities. In 2021 IEEE International Conference on Image Processing (ICIP), pp. 3003 3007. IEEE, 2021.
Yifan Deng, Xinran Xu, Yang Qiu, Jingbo Xia, Wen Zhang, and Shichao Liu. multimodal deep learning framework for predicting drug-drug interaction events. Bioinformatics,
2020.
Abbey A. Donaldson.
After Three Years on Mars, NASA's Ingenuity Heli- copter Mission Ends NASA
nasa.gov.
https://wwy nasa gov gow.gwgw//wwweeeeee news-releasee
peear-tthre--yrs-oorre nasas- www---ansssanninaaee htg.saaeennnayyaloooooeeeeetoooe 13-05-2024).
Jan 2024. [Accessed
Jonas Van Der Donekt. mbrain21, 2024. URL http://wwwwwaggg.cco
Robert Duin. Multiple Features.
UCI
Machine
Learning
Repository.
DOL:
httpss////oooor//1... /CH47000
Aiman Farooq, Deepak Mishra, and Santamu Chaudhury. Survival prediction in lung cancer through multi- modal representation learning. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 3907-3915. TEEE, 2025.
Chongjaan Ge, Junsong Chen, Enze Xie, Zhongdao Wang, Lanqing Hong, Huchuan Lu, Zhenguo Li, and Ping Luo. Metabev: Solving sensor failures for 3d detection and map segmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 8721-8731, 2023.
Parmida Ghahremani, Yanyun Li, Arie Kaufman, Rami Vanguri, Noah Greenwald, Michael Augelo, Travis J Hollmann, and Saad Nadeem. Deep learning-inferral multiplex immunofhureescenee for immunohisto- chemical image quantification. Nalure machine intelligence, 4(4):401 412, 2022.
Rohit Girdhar, Alaacldin El-Nouby, Zhuang Lin, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVP /CVF Conference Computer Vision and Pattern Recognition, pp. 15180-15190, 2023.
Xinyu Gong, Sreyas Mohan, Naina Dhingra, Jean-Charler Bazin, Yilei Li, Zhangvang Wang, and Rakesh Ranjan. Mmg-ego4d: multimodal generalization in egocentric action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 6481 6491, 2023.
Tan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, el al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 18995-19012, 2022.
Clément Guérin, Christophe Rigaud, Antoine Mercier, Farid Aimar--Bodjeedl Karell Bertet, Alain Bouju, Jean-Christe Chan-Chrittophe Buric, Georges Louis, Jean-Mare Ogier, and Arnaud Revel. ebdtheque: representative database of comics. In 2013 12th International Conference on Document Analysis and Recognition, pp. 1145-1149 IEFF, 2013.
33

---

Published in Transactions on Machine Learning Research (02/2026)
Kausic Gunasekar, Qiang Qiu, and Yezhou Yang. Low to high dimensional modality hallucination using aggregated fields of view. IEEF Robolics and Aulomalion Letlers, 5(2):1983-1 19900, 2020.
Zirun Guo, Tao Jin, and Zhou Zhao. Multimodal prompt learning with missing modalities for sentiment analysis and emotion recognition, 2024. URL http:://aaxxxxoo//////2001111
Qishen Ha, Kohei Watanabe, Takumi Karasawa, Yoshitaka Ushiku, and Tatsuya Harada. Minet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes. In 2017 IEFF/RS) International Conference on Intelligent Robots and Systems (IROS), 2017.
Sebastian Hafner and Yifang Ban. Multi-modal deep learning for multi-tempooaa urban mapping with partly missing optical modality. In IGARSS 2023-2023 IEEE International Geoscience and Remote Sensing Sympostum, pp. 6843 6846. IEEE, 2023.
Mohammad Hamghalam, Alejandro F Frangi, Baiying Lei, and Amber L Simpson. Modality completion via gaussian process prior variational autoencoders for multi-modal glioma segmentation. In Medical CitteenttooMMCCCCA 2021: 24th International Conference,
Image Computing and Computer Assisted Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VII 24, pp. 442-452. Springer, 2021.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference computer vision and pattern recognilion, pp. 16000-16000 2022.
Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web, pp. 507-517, 2016.
Zhen He, Shuofeng Hu, Yaowen Chen, Sijing An, Jiahao Zhou, Runyan Liu, Junfeng Shi, Jing Wang, Guohua Dong, Jinhui Shi, et al. Mosaic integration and knowledge transfer of single-cell multimodal data with midas. Nature Biotechnology, pp. 1-12, 2024.
Gooffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowlodge in neural network. arXiv preprint (tlp:1503.0253.. 2015.
Geoffrey E Hinton and Richard Zemel. Autoencoders, minimum description length and helmholtz free energy. Advances in neural information processing systems, 6, 1993.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Judy Hoffman, Saurabh Gupta, and Trevor Darrell. Learning with side information through modality hallucination. In Proceedings of the IEEE conference on compuler vision and pollern recognition, pp. 826 834, 2016.
Danfeng Hong, Jingliang Hu, Jing Yao, Jocelyn Chanussot, and Xiao Xiang Zhu. Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model. ISPRS Journal of Photogrammetrry and Remote Sensing, 178:68-80, 2021.
Minhao Hu, Matthis Maillard, Ya. Zhang, Tommaso Cicori, Giammarco La Barbera, Isabelle Bloch, and Pietro Gori. Knowledge distillation from multi-modal to mono-modal segmentation networks. In Medical Image Computing and Computer Assisted Intervention -MICAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part T 23, pp. 772-781. Springer, 2020.
Yihan Hu, Jiazhi Yang, Li Chen, Keyu L.i, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, autonomous driving. In Proceedings of the IEEE/CVF Conference
Wenhai Wang, et al. Planning- on Computer Vision and Pallern Recognition, pp. 17853 17862, 2023.
Ruohong Huan, Guowei Zhong, Peng Chen, and Ronghua Liang. Unimf: a unified multimodal framework for multimodal sentiment analysis in missing modalities and unaligned multimodal sequences. JEEE Transactions on Multimedia, 2023.
34

---

Published in Transactions on Machine Learning Research (02/2026)
Zhongyi Huang, Yao Ding, Guoli Song, Lin Wang, Ruizhe Geng, Hongliang He, Shan Du, Xia Liu, Yonghong Tian, Yongsheng Liang, et al. Bedata: A large-scale dataset and benchmark for cell detection and counting. In International Conference on Medical Image Computing and Computer-Assisteee Intervention, pp. 289 298. Springer, 2020.
Jaesung Huh, Jacob Chalk, Evangelos Kazakos, Dima Damen, and Andrew Zisserman. Epic-sounds: A large- scale dataset of actions that sound. In ICASSP 2023-20080 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, IEEE, 2023.
Dina Hussein and Ganapati Bhat. Sensorgan: A novel data recovery approach for wearable human activity recognition. ACM Transactions on Embedded Computing Systems, 23(3):1-28, 2024.
Dina Hussein, Taha Belkhouja, Ganapati Bhat, and Jana Doppa. Sensor-aware data imputation for time- series machine learning on low-power wearable devices. ACM Transactions on Design Automation of Electronic Systems, 30(1): 1-127 2024a.
Dina Hussein, Taha Belkhouja, Ganapati Bhat, and Janardhan Rao Doppa. Energy-effeient missing data imputation in wearable health applications: A classifier- statistical approach. In Proceedings of the Thirty-Third Third International Joint Conference on Artificial Intelligence AI for Good, pp. 7296-7304, 2024b.
Soommin Hwang, Jaesik Park, Namil Kim, Yukyung Choi, and In So Kweon. Multispectral pedestrian detection: Benchmark datasct and basclinc. In Proceedings of the IEEE conference on computer vision and pollern recognilion, pp. 1037- 1045, 2015.
Clifford R Jack Jr, Matt A Bernstein, Nick C Fox, Paul Thompson, Gone Alexander, Danielle Harvey, Bret Borowski, Paula J Britson, Jennifer L. Whitwell, Chadwick Ward, et al. The alzheimer's discase neuroimaging initiative (adni): Mri methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, 27(4):685 691, 2008.
Jachyuk Jang, Yooseung Wang, and Changick Kim. Towards robust multimodal prompting with miss- ing modalities. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8070 8074. IEEE, 2024.
Eun Som Jeon, Anirudh Som, Ankita Shukla, Kristina Hasanaj, Matthew P Buman, and Pavan Turaga. Role of data augmentation strategies in knowledge distillation for wearable sensor data. IEEE internet of things journal, 1244:128448- 128460, 2021.
Seungwan Jeong, Hwanho Cho, Junmo Kwon, and Hyunjin Park. Region-offinterest attentive heteromodal variational encoder-decoder for segmentation with missing modalities. In Proceedings of the Asian Con- ference on Computer Vision, pp. 3707-3723, 2022.
Wei Ji, Jingjing Li, Cheng Bian, Zongwei Zhou, Jiaying Zhao, Alan I, Yuille, and Li Cheng. Multispectral video semantic segmentation: A benchmark dataset and baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1094-1104, 2023.
Vijay John and Yasutomo Kawanishi. A multimodal sensor fusion framework robust to missing modalities for person recognition. In Proceedings of the 4th ACM International Conference on Multimedia in Asia, pp. 1-5, 2022.
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-ili, a freely accessible critical care databasc. Scientific data, 3(1):1-9, 2016.
Alistair EW Johnson, Tom J Pollard, Seth Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger Mark, and Steven Horng. Mimic-exr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific dala, 6(1):317, 2019.
Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, al. Mimic-iv, freely accessible electronic health record dataset. Scientific data, 10(1):1, 2023.
35

---

Published in Transactions on Machine Learning Research (02/2026)
kaggle.
kaggle fir thermal, 2020.
URL
flir- -aaemmal-maagee-daata
Michael Kampflmeyer, Arnt-Borre Salberg, and Robert Jenssen. Urban land cover classification with missing data modalities using deep convolutional neural networks. IEEE Journal of Selected Topics in Applied 2018.
Earth Observations and Remote Sensing, 11(6):1758-11
Bob Kemp, Aeilko H Zwinderman, Bert Tuk, Hilbert AC Kamphuisen, and Josefien JL Oberye. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontimuity of the eeg. IEEE Transactions on Biomedical Engineering, 47(9):1185-119M) 2000.
Navid Alemi Koohbanani, Mostafa Jahanifar, Neda Zamani Tajadin, and Nasir Rajpoot. Nuclick: deep learning framework for interactive segmentation of microscopic images. Medical Image Analysis, 65:101771, 2020.
Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. A large-scale hierarchical multi-view rgb-d object dataset. In 2011 IEEE international conference on robotics and automation, pp. 1817-1824. IEEE, 2011.
Himabindu Lakkaraju, Julian McAuley, and Jure Leskovec. What's in a name? understanding the interplay between titles, content, and communities in social media. In Proceedings of the international AAAT conference on web and social media, volume 7, pp. 311 320, 2013.
Harlin Lee, Boyue Li, Shelly DeForte, Mark I. Splaingard, Yungui Huang, Yuejie Chi, and Simon I. Linwood. A large collection of real-world pediatric sleep studies. Scientific Data, 9(1):421, 2022.
Kwanhyung Lee, Soojeong Lee, Sangchul Hahn, Heejung Hyun, Edward Choi, Byungeun Ahn, and Joobyung Lee. Learning missing modal electronic health records with unified multi-modal data embedding and modality-aware attention. In Machine Learning for Healthcare Conference, pp. 423-442. PMTRR 2023a.
Michelle A Lee, Matthew Tan, Yuke Zhu, and Jeannette Bohg. Detect, reject, correct: Crossmodal compen- sation of corrupted sensors. In 2021 IEEE international conference on robotics and automation (ICRA), pp. 909 916. IEEE, 2021.
Yi-Lum Lee, Yi-Hsuan Tsai, Wei-Chen Chiu, and Chen Yu Lee. Multimodal prompting with missing modal- ities for visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14943 14952, 2023b.
Ziga Lesjak, Alfia Galimzianova, Ales Koren, Matej Lukin, Franjo Pernuš, Boštjan Likar, and Ziga Spielin. A novel public mr image datasct of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics, 16:51 63, 2018.
Bo Li, Peiyuan Zhang, Jingkang Yang, Yuanhan Zhang, Fanyi Pu, and Ziwei Liu. Otterhd: A high-resolution multi-modality model. arxiv preprint or Xrv::2331 1.04219, 2023a.
Haitao Li, Ziyu Li, Yiheng Mao, Zhengyao Ding, and Zhengxing Huang. De-seg: Disentangled rastrastive learning for brain tumor segmentation with missing modalities. arXiv preprint arXiv:2505.1192, 2025a.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023b.
Mingeheng Li, Dingkang Yang, Xiao Zhao, Shuaibing Wang, Yan Wang, Kun Yang, Mingyang Sun, Dongliang Kou, Ziyun Qian, and Lihua Zhang. Correlation-deccuplee knowledge distillation for multimodal sentiment. analysis with incomplete modalities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12458-12468, 2024a.
Mingyang Li, Shao-Lun Huang, and Lin Zhang. A general framework for incomplete cross-modal retrieval with missing labels and missing modalities. In ICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 4763 4767. IEFF, 2022a.
36

---

Published in Transactions on Machine Learning Research (02/2026)
Ning Li, Tao Li, Chunyu Hu, Kai Wang, and Hong Kang. A benchmark of ocular discase intelligent recogni- tion: One shot for multi-disease detection. In Benchmarking, Measuring, and Optimizing: Third Bench- Council International Symposium, Bench 2020, Virtual Event, November 15 16, 2020, Revised Selected Papers 8, pp. 177-193. Springer, 2021.
Rongjian Li, Wenlu Zhang, Houng-Il Suk, Li Wang, Jiang Li, Dinggang Shen, and Shuiwang Ji. Deep learning based imaging data completion for improved brain disease diagnosis. In Medical Image Compuling and Computer-Assisted Intervention-MICC
2014: 1 17th International Conference, Boston, MA, USA, September 14-18, 2014, Proceedings, Part III 17, pp. 305-312. Springer, 2014.
Sijie Li, Chen Chen, and Jungong Han. Simmlm: A simple framework for multi-modal learning with missing modality, arXiv preprint arXiv:2507. 19264, 2025b.
Siting Li, Chenzhuang Du, Yue Zhao, Yu Huang, and Hang Zhao. What makes for robust multi-modal models in the face of missing modalities? arXiv preprint (rXii:2810006335 2023c.
Xue Li, Guo Zhang, Hao Cui, Shasha Hou, Shunyao Wang, Xin Li, Yujia Chen, Zhijiang Li, and Li Zhang, Mcanet: A joint semantic segmentation framework of optical and sar images for land use classification.
International Journal of Applied Earth Observation and Geoinformation, 106:102638, 2022b.
Yu-Jhe Li, Jinhyung Park, Matthew O'Toole, and Kris Kitani. Modality-agnostic learning for radar-lidar fusion in vehicle detection. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 918-927, 2022c.
Yulin Li, Tianzhu Zhang, Xiang Liu, Qi Tian, Yongdong Zhang, and Feng Wu. Visible-infrared person re-identification with modality-specile memory network. IEEE Trunsactions on Image Processing, 31: 2022d.
Zhiyuan Li, Yafei Zhang, Huafeng Li, Yi Chai, and Yushi Yang. Deformation-awar and reconstruction-driven multimodal representation learning for brain tumor segmentation with missing modalities. Biomedical Signal Processing and Control, 91:106012, 2024b.
Zheng Lian, Lan Chen, Licai Sun, Bin Liu, and Jianhua Tao. Genet: Graph complction network for incom- plete multimodal learning in conversation. IEEE Transactions on pattern analysis and machine intelli- gence, 2023.
Paul Pu Liang, Zhun Liu, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhurdinov, and Louis-Philipe Morency. Learning representations from imperfect time series data via tensor rank regularization. arXiv preprint xrXio:190.00001, 2019.
Paul Pu Liang, Yiwei Lyu, Xiang Fan, Zetian Wu, Yun Cheng, Jason Wu, Leslie Chen, Peter Wu, Michelle A Lee, Yuke Zhu, et al. Multibench: Multiscale benchmarks for multimodal representation learning. Advances in neural information processing systems, 2021 (DB1): ):1, 2021.
Yupeng Liang, Ryosuke Wakaki, Shohei Nobuhara, and Ko Nishino. Multimodal material segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 198000-18800 2022.
Zihan Liang, Ziwen Pan, and Ruoxuan Xiong. Causal representation lcarning from multimodal clinical records under non-random modality missingness. In Proceediigg of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 28779-287 77799,,,,,, 2025.
Chenfei Liao, Kaiyu Lei, Xu Zheng, Junha Moon, Zhixiong Wang, Yixuan Wang, Danda Pani Paudel, Lue Van Gool, and Xuming Hu. Benchmarking multi-modal semantic segmentation under sensor failures: Missing and noisy modality robustness. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1576-15866 2025.
Ronghao Lin and Haifeng Hu. Missmodal: Increasing robustness to missing modality in multimodal sentiment analysis. Transactions of the Association for Compulational Länguistics, 11:1686-11 1702, 2023.
37

---

Published in Transactions on Machine Learning Research (02/2026)
Xun Lin, Shuai Wang, Rizhao Cai, Yizhong Liu, Ying Fu, Zitong Yu, Wenzhong Tang, and Alex Kot. Sup- press and rebalance: Towards generalized multi-modal face anti-spoofing. arviv preprint or arXiu:2402. 19298, 2024.
Zhenghong Lin, Yanchao Tan, Yunfei Zhan, Weiming Liu, Fan Wang, Chaochao Chen, Shiping Wang, and Carl Yang. Contrastive intra-and inter-modality generation for enhancing incomplete multimedia recommendation. In Proceedings of the 31st ACM International Conference on Mullimedia, pp. 6234 6242, 2023.
Ajian Liu, Zichang Tan, Jun Wan, Sergio Escalera, Guodong Guo, and Stan Z Li. Casia-surf cela: A benchmark for multi-modal cross-ethnicity face anti-spoofing. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 1179 1187, 2021a.
Chengyu Liu, David Springer, Qiao Li, Benjamin Moody, Ricardo Abad Juan, Francisco Chorro, Francisco Castells, José Millet Roig, Ikaro Silva, Alistair EW Johnson, et al. An open access database for the evaluation of heart sound algorithms. Physiological measurement, 2016.
Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. European conference on computer vision, pp. 612-630. Springer, 2022.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in newral information processing systems, 36, 2024a.
Hong Liu, Dong Wei, Donghuan Lu, Jinghan Sun, Liansheng Wang, and Yefeng Zheng. M3ac: multimodal representation ion learning for brain tumor segmeniation with missing modalities. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 37, pp. 1657-1665, 2023a.
Jinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma, Risheng Liu, Wei Zhong, Zhongxuan Luo, and Xin Fan. Multi- interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation. In Proceedings of the IEEE/CVE international conference computer vision, pp. 8115 8124, 2023b.
Ruiping Liu, Jiaming Zhang, Kunyu Peng, Yufan Chen, Ke Cao, Junwei Zheng, M Saquib Sarfraz, Kailun Yang, and Rainer Stiefelhagen. Fourier prompt tuning for poddllty-incompletee scene segmentation. arXiv preprint ar Xiv:2400 16928, 2024b.
Shengzhong Liu, Shuochao Yao, Yifei Huang, Dongxin Liu, Huajie Shao, Yiran Zhao, Jinyang Li, Tianshi Wang, Ruijie Wang, Chaoqi Yang, and Tarck Abdelzaher. Handling missing sensors topology-aware iot applications with gated graph neural network. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 4(3), September 2020. doi: 10.1145/3411811 URL https:t///// pectpa://doi.. org/ 10. 11145/34118888
Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, et al. Llava-plus: Learning to use tools for creating multimodal agents. arXiv preprint arXiv:2311.05497 2023c.
Yanbei Liu, Lianxi Fan, Changqing Zhang, Tao Zhou, Zhitao Xiao, Lei Geng, and Dinggang Shen. Incomplete multi-modal representation lcarning for alzheimer's discase diagnosis. Medical Image Analysis, 69:101953, 2021b.
Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David S Rosenblum. Mmkg: multi-modal knowledge graphs In The Semantic Web: 16th International Conference, ESWC 2019, Portorož, Slovenia, June 2-6, 2019, Proceedings 16, pp. 459-474. Springer, 2019.
Yi Liu, Cong Wang, and Xingliang Yuan. Fedmobile: Enabling knowledge contribation-awwree multi-modal federated learning with incomplete modalities. In Proceedings of the ACM on Web Conference 2025, pp. 2775-2786, 2025a.
Yuhang Liu, Quan Zou, Ran Su, and Leyi Wei. semomer: modality- pretraining framework for single-cell multi-omics modeling under missing modality conditions. bio bioRriv, pp. 2025 08, 2025b.
38

---

Published in Transactions on Machine Learning Research (02/2026)
Steven R Livingstone and Frank A Russo. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PloS one, 13(5)=e0196391, 2018.
Zihong Luo, Haochen Xue, Mingyu Jin, Chengzhi Liu, Zile Huang, Chong Zhang, and Shuliang Zhao. Me- dbn: A deep belief network-based model for modality completion. arXiv preprint arXiv:200.. 09782, 2024.
Fei Ma, Shao-Lun Huang, and Lin Zhang. An efficient approach for andio-visual emotion recognition with missing labels and missing modalities. In 2021 IEEE international conference on multimedia and Expo (ICME), pp. 1 6. IEEE, 2021a.
Fei Ma, Xiangxiang Xu, Shao-Lun Huang, and Lin Zhang. Maximum likelihood estimation for multimodal learning with missing modality. arXiv preprint arXiv:2108. 2021b.
Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. Smil: Multimodal learn- ing with severely missing modalily. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 35, pp. 2302 2310, 2021c.
Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal transformers robust to missing modality? In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18177 18186, 2022.
Daniele Malitesta, Emanuele Rossi, Claudio Pomo, Fragkiskos D) Malliaros, and Tommaso Di Noia. Dealing with missing modalities in multimodal recommendation: feature propagation-based approach. arXiv preprint arx::2403. 19841, 2024.
Carole L Marcus, Reneé H Moore, Carol Rosen, Bruno Giordani, Susan L Garetz, H Gerry Taylor, Ron B Mitchell, Raouf Amin, Fliot S Katz, Raanan Arens, et al. A randomized trial of adenotonsillectony for childhood sleep apnea. New England Journal of Medicine, 2013.
C. Mario Christoudias, Raquel Urtasun, Mathieu Salzmann, and Trevor Darrell. Learning to recognize objects from unseen modalities. In Computer Vision-ECCV 2010, pp. 677-691. Springer, 2010.
Olivier Martin, Trene Kotsia, Benoit Macq, and Toannis Pitas. The enterface audio-visual emotion database. In 22nd international conference on data engineering workshops (ICDEW'06), pp. 8. LEEE 2006.
Yusuke Matsui, Kota Ito, Yuji Aramaki, Azuma Fujimoto, Toru Ogawa, Toshihiko Yamasaki, and Kiyoharu Aizawa. Sketch- based manga retrieval using manga109 dataset. Multimedia tools and applications, 76: 21811-21838, 2017.
Toshihíko Matsuura, Kuniaki Saito, Yoshitaka Ushiku, and Taisuya Harada. Generalized bayesian canonical correlation analysis with missing modalities. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0-0, 2018.
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based recommen- dations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and develoyment in information retrieval, pp. 43 52, 2015.
Bjoern Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, al. The multimodal brain tumor image segmentation benchmark (brats). IEFF transactions on medical imaging, 34(10):1993-22) -0224, 2014.
Antoine Miech, Tvan Taptev, and Josef Sivic. Learning a text-video embedding from incomplete and hetero- geneous data. arXiv preprint arXiv:1804.082 02516, 2018
Eleni P Mimitou, Caleb A Lareau, Kelvin Chen, Andre Marzetto-Feraander Yuhan Hao, Yusuke Takeshima, Wendy Luo, Tsc-Shun Huang, Bertrand Yeung, Efthymia Papalexi, et al. Scalable, mul- timodal profiling of chromatin accessibility, gene expression and protein levels in single cells. Nature biotechnology, 32(10):1246-122) 1258, 2021.
39

---

Published in Transactions on Machine Learning Research (02/2026)
George B Moody and Roger G Mark. The impact of the mit-bih arrhythmia database. IEEE engineering in mediccie and biology magazine, 20(3):45-50, 2001.
Julie Mordacq, Leo Milecki, Maria Vakalopoulou, Steve Oudot, and Vicky Kalogeiton. Adapt: Multimodal
learning for detecting physiological changes under missing modalities. In Medical Imaging with Deep Learning, 2024.
Kouis-Phillipss Morency, Rada Mihalcea, and Payal Doshi Towards multimodal sentiment analysis: Harvest- ing opinions from the web. In Proceedings of the 13th international conference on multimodal interfaces, pp. 169-176, 2011.
Shun Muroga, Yasuaki Miki, and Kenji Hata. A comprehensive and versatile multimodal deep-learning approach for predicting diverse properties of advanced materials. Advanced Science, 10(24):2002508, 2023.
Andriy Myronenko. 3d mri brain tumor segmentation using autoencoder regularization. In Broinlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spuin, September 16, 2018, Revised Selected Papers, Part II 4, pp. 311-320, Springer, 2019.
Arsha Nagrani, Shan Yang, Amurag Arnab, Aren Jansen, Cordelia Schmid, and Chen Sun. Attention bottlenecks for multimodal fusion. Advances in neural information processing systems, 34:14200-1444 2021.
Nhu-Van Nguyen, Christophe Rigaud, and Jean- Soan-Chisttophe Burie. Digital comics image indexing based on deep learning. Journal of Imaging, 4(7):89, 2018.
Henry Friday Nweke, Ying Wah Teh, Mohammed Ali Al-Garadi, and Uzoma Rita Alo. Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Systems with Applications, 105:233 261, 2018
Se Won Oh, Hyuntae Jeong, Seungeun Chung, Jeong Mook Lim, Kyoung Ju Noh, Sunkyung Lee, and Gyuwon Jung. Understanding human daily experience through continuous sensing: Etri lifelog dataset 2024. arXiv preprint ar Xiv:2500 03698, 2025.
Alessandro Palma, Till Richter, Hanyi Zhang, Manuel Lubetzki, Alexander Tong, Andrea Dittadi, and Fabian Thois Multi-modal and multi-attribute generation of single cells with efgen. arXiv preprint arXiv:2407.11794, 2024.
Konyul Park, Yecheol Kim, Daehun Kim, and Jun Won Choi. Resilient sensor fusion under adverse sensor failures via multi-modal expert fusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 6720 6729, 2025.
Kyu Ri Park, Hong Joo Lee, and Jung Uk Kim. Learning trimodal relation for audio-visual question answering with missing modality, 2024. URL https://arxiv/ wtw.x/w.orr//bb//40.... 16171
Srinivas Parthasarathy and Shiva. Sundaram. Training strategies to handle missing modalities for andio-visual expression recognition. In Companion Publication of the 2020 International Conference on Mullimodal Interaction, pp. 400-404, 2020.
Yuanzhe Peng, Jieming Bian, and Jie Xu. Fedmm: Federated multi-modal learning with modality hetero- geneity in computational hologl In ICASSP 2024-2024 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1696 1700. JEEE, 2024.
Lisa RS Phillips, Gaynor Parfitt, and Alex V Rowlands. Calibration of the genea accelerometer for assessment of physical activity intensity in children. Journal of science and medicine in sport, 16(2):124-122, 2013.
Claudio Persello; Saurabh Prasad; Gemine Vivone; Vincent Lonjou Frédéric Bretar ; Raquel Rodriguez- Suquet Pauline Guntzburger; Vincent Poulain Jacqueline Le Moigne; Benjamin Smith Sujay Kumar Thomas Huang Sophie Ricci ; Thanh Huy Nguyen ; Andrea Piacentini. 2024 icce grss data. fusion contest flood rapid mapping. IEEE Dataport, 2023a. doi: 10.21227 n..212277732j-21 7733)3-3333 4303. URL https::/ :/dx doi, www.ddoooo..o..... org/10.
21227 -332
40

---

Published in Transactions on Machine Learning Research (02/2026)
Vittorio Pipoli, Alessia Saporita, Federico Bolelli, Marcella Cornia, Lorenzo Baraldi, Costantino Grana, Rita Cucchiara, Flisa Ficarra, et. al. Missrag: Addressing the missing modality challenge in multimodal large language models. In Proceedings of the 2025 IEEE/CVE International Conference on Computer Vision 2025.
Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihal- cea. Meld: A multimodal multi-party dataset for emotion recognition in conversations. ar preprinl arXiv:1810.0m 02508, 2018.
PromptCloud. Toy products on amazon, Sep 2017. PrompttCooudiiqtoy-prooodoo
URL
on- amazon.
Shuwei Qian and Chongjun Wang. Com: Contrastive maskod-attention model for incomplete multimodal learning. Neural Networks, 162:443 455, 2023.
Yansheng Qiu, Ziyuan Zhao, Hongdou Yao, Delin Chen, and Zheng Wang. Modal-aware visual prompting for incomplete multi-modal brain tumor segmentation. In Proceedings of the 31st ACM International Conference on Mullimedia, pp. 3228 3239, 2023.
Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasogaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, and Jitendra Malik. Humanoid locomotion as next token prediction. arXiv preprint arXiv:2402.1 19469, 2024.
Hossein Rahmani, Arif Mahmood, Du Huynh, and Ajmal Mian. Histogram of oriented principal components for cross-view action recognition. IEEE transactions on pattern analysis and machine intelligence, 38(12) 2430-2444 2016.
Merey Ramazanova, Alejandro Pardo, Bernard Ghanem, and Motasem Alfarra. Combating missing modal- ities in egocentric videos at test time. arXie preprint 2rXiv:2404.151e 2024.
Behnood Rasti, Pedram Ghamisi, and Richard Gloaguen. Hyperspectral and lidar fusion using extinction profiles and total variation component analysis. IEEE Transactions on Geoscience and Remote Sensing, 55(7):3997-4007, 2017.
Attila Reiss. PAMAP2 Physical Activity Monitoring. UCI Machine Learning Repository, 2012. DOL: http://d...orr10.02/////////
Md Kaykobad Reza, Ashley Prater-Bennette, and M Salman Asif. Robust multimodal learning with missing modalities via parameter-efficient adaptation. arXio preprint arXio:2310.0333 2023.
Fabien Ringeval, Andreas Sonderegger, Juergen Sauer, and Denis Lalanne. Introducing the recola multimodal corpus of remote collaborative and affective interactions. In 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG), pp. 1-8. IEEE, 2013.
Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gorald Pirkl, Alois Ferscha, Jakob Doppler, Clemens Holzmann, Marc Kurz, Gerald Holl, Ricardo Chavarriaga, Hesam Sagha, Hamidreza Bayati, Marco Creatura, and José del R. Millán. Collecting complex activity datasets in highly rich networked sensor environments. 2010 Seventh International Conference on Networked Sensing Systems (INSS), pp. 233-240, 2010. URL
https: //api. semanticacholar. wttpaaaaccc..aaaooooorrppaeeaeee
org/CorpusII:993333
Pramit Saha, Divyanshu Mishra, Felix Wagner, Konstantinos Kamnilsas, and J Alison Noble. Examining modality incongruity in multimodal federated learning for medical vision and language based disease detection. Xix preprint arXiv:2402.05220 2024.
Sayan Deb Sarkar, Ondrej Miksik, Mare Pollefeys, Daniel Barath, and Iro Armeni. Crossover: 3d scene cross-modal alignment. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3985-8994 2025.
41

---

Published in Transactions on Machine Learning Research (02/2026)
Gerwin Schalk, Dennis J McFarland, Thilo Hinterberger, Niels Birbaumer, and Jonathan R. Wolpaw. Bci2000: a general-purpose brain-computer interface (bei) system. TEEF Trunsactions on biomedical engineering, 51(6):103441D) 1043,, 2004.
Benoît Schmauch, Alberto Romagnoni, Elodie Pronier, Charlie Saillard, Pascale Maillé, Julien Calderaro, Aurélic Kamoun, Meriem Sefta, Sylvain Toldo, Mikhail Zaslavskiy, et al. A deep learning model to predict raa-ee expression of lumours from whole slide images. Nalure communications, 11 (1(13:8877 2020.
Philip Schmidt, Attila Reiss, Robert Duerichen, Claus Marberger, and Kristof Van Laerhoven. Introducing wesad, multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI '18, pp. 400-408, New York, NY, USA, 2018a. Association for Computing Machinery. ISBN 9781450356923. doi: 10. (11144//222299..24222 7/doi.org 10. 20..114/3322999.333 32449885
URL https:
Philip Schmidt, Attila Reiss, Robert Duerichen, Claus Marberger, and Kristof Van Laerhoven. Introducing wesad, multimodal dataset for wearable stress and affect detection. In Proceedings of the 20th ACM international conference on multimodal interaction, pp. 400-408, 2018b.
Nicu Sebe, Ira Cohen, Theo Gevers, and Thomas Huang. Multimodal approaches for emotion recognition: a survey. In Internet Imaging VI, volume 5670, pp. 56 67. SPIE, 2005.
Deep Shah, Amit Barve, Brijesh Vala, and Jay Gandhi. A survey brain tumor segmentation with missing mri modalities. In International Conference on Information Technology, pp. 299 308. Springer, 2023.
Amir Shahroudy, Jun Liu, Tian-Tbong Ng, and Gang Wang. Ntu rgb+ d: A large scale dataset for 3d human activity analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
1010-1019, 2016.
Yan Shen and Mingehen Gao. Brain tumor segmentation mri with missing modalities. In Information Processing in Medical Imaging: 26th International Conference, IPMI 2019, Hong Kong, China, June. 2 7, 2019, Proceedings 26, pp. 417 428. Springer, 2019.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving tasks with chatgpt and its friends in hugging face. Advances in Newral Information Processing Systems, 36, 2024.
Jacob Shermeyer, Daniel Hogan, Jason Brown, Adam Van Etten, Nicholas Weir, Fabio Pacifici, Ronny Hansch, Alexei Bastidas, Scott Soenen, Todd Bacastow, et al. Spacenet 6: Multi-sensor all weather mapping dataset. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition workshops, pp. 196-197, 2020.
Junjic Shi, Caozhi Shang, Zhaobin Sun, Li Yu, Xin Yang, and Zengqiang Yan. Passion: Towards effective incomplete multi-modal medical image segmentation with imbalanced missing rates, 2024. URL https: //arxiv.org/abs/2200 14796
Muhammad Shoaib, Stephan Bosch, Ozlem Durmaz Incel, Hans Scholten, and Paul JM Havinga. Fusion of smartphone motion sensors for physical activity recognition. Sensors, 14(6):10146-10176, 2014.
Aniruddh Sikdar, Jayant Tcotia, and Suresh Sundaram. Contrastive lcarning- based spectral knowledge distillation for multi-modality and missing modality scenarios in semantic segmentation. arXie preprinl arXiv:2312.022/// 2023.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12, pp. 746-760. Springer, 2012.
Kihyuk Sohn, Wenling Shang, and Honglak Lec. Improved multinodal deep learning with variation of information. Advances in neural information processing systems, 27, 2014.
42

---

Published in Transactions on Machine Learning Research (02/2026)
Lukas Stappen, Alice Baird, Lukas Christ, Lea Schumann, Benjamin Sertolli, Eva-Maria Messner, Erik Cambria, Guoying Zhao, and Björn W Schuller. The muse 2021 multimodal sentiment analysis challenge: sentiment, emotion, physiological-ementl -eotton,, and stress. In Proceedings of the 2nd on Multimodal Sentiment Analysis Challenge, pp. 5-14. 2021a.
Lukas Stappen, Eva-Maria Mcßner, Erik Cambria, Guoying Zhao, and Björn W Schuller. Muse 2021 chal- lenge: Multimodal emotion, sentiment, physiological-emooion and st.ress detection. In Proceedings of the 29th ACM International conference multimedia, pp. 5706 5707, 2021b.
Paul Streli, Rayan Armani, Yi Fei Cheng, and Christian Holz. Hoov: Hand out-of-view tracking for propri- oceptive interaction using incrtial sensing. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1-16, 2023.
Jun Sun, Xinxin Zhang, Shoukang Han, Yu-Ping Ruan, and Taíhao Li. Redcore: Relative advantage aware cross-modal representation learning for missing modalities with imbalanced missing rates. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15173 15182, 2024a.
Yuhang Sun, Zhizhong Liu, Quan Z Sheng, Dianhui Chu, Jian Yu, and Hongxiang Sun. Similar modality completion- based multimodal sentiment analysis under uncertain missing modalities. Information Fusion, pp. 102454, 2024b.
Dídac Surís, Sachit. Menon, and Carl Vondrick. Vipergpt: Visual inference via python exccution for reasoning. In Proceedings of the IEEE/CE /CVF International Conference Computer Vision, pp. 11888 11898, 2023.
Zaneta Swiderska-Chadaj Hans Pinckaers, Mart Van Rijthoven, Maschenka Balkenhol, Margarita Mel- nikova, Oscar Geessink, Quirine Manson, Mark Sherman, Antonio Polonia, Jeremy Parry, et al. Learning detect lymphocytes in immunohistochemistr with deep learning. Medical image analysis, 58:101547, 2019.
Timo Sztyler and Heiner Stuckenschmidt. On-body localization of wearable devices: An investigation of position-aware activity recognition. In 2016 IEEE international conference percosive compuling and communications (PerCom), pp. 1-9. JEEE, 2016.
Karasawa Takumi, Kohei Watanabe, Qishen Ha, Antonio Tejero-De-Pabllee Yoshitaka Ushiku,, and Tatsuya Harada. Multispectral object dctection for autonomous vehicles. In Proceedings of the on Thematic Workshops of ACM Mullimedia 2017, pp. 35 43, 2017.
Zineng Tang, Ziyi Yang, Chenguang Zhu, Michael Zeng, and Mohit Bansal. Any-to-any generation via composable diffusion. Advances in Neural Information Processing Systems, 36, 2024.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
consistency
NASA Science Editorial Team. Keeping Our Sense of Direction: Dealing With Dead Sensor NASA Science - science.nasa.gor hngenutty-elllccopter/eeeee htt.://ppeeoerooooooooeeeeeeaaaa 2022.
nasa.gov. https::/sciii nasa. htwpaaasioooomamaeee--------eeee
JUN
Theo Theodoridis. EMG Physical Action Data Set.
UCI Machine Learning Repository, 2011. DOL:
https///00..or//102443/////
Katarzyna Tomczak, Patrycja Czerwiíska, and Maciej Wiznerowicz. Review the cancer genome atlas (tega): an immeasurable source of knowledge. Contemporary Oncology/Wspptczeem Onkologia, 2015(1):68 77,
2015.
Jonathan Tompson, Murphy Stein, Yann Lecun, and Ken Perlin. Real-time continuous pose recovery of human hands using convolutional networks. ACM Transactions Graphics (ToG), 33 2014.
43

---

Published in Transactions on Machine Learning Research (02/2026)
Luan Tran, Xiaoming Liu, Jiayu Zhou, and Rong Jin. Missing modalities imputation via cascaded residual autoencoder. In Proceedings of the TEEE conference on computer vision and paltern. recognation, pp. 1405 1414, 2017.
Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis- Philippe Morency, and Ruslan Salakhutdinov. Learning factorized multimodal representations. arXiv preprint 1xXi::8066000 16, 2018.
Xinming Tu, Zhi-Jie Cao, Sara Mostafavi, Ge Gao, et al. Cross-linked unified embedding for cross-modality representation learning. Advances in Neural Information Processing Systems, 35:15942 15955, 2022.
Jonas Van Der Donckt, Nicolas Vandenbussche, Jeroen Van Der Donckt, Stephanie Chen, Marija Stojchevska, Mathias De Brouwer, Bram Steenwinckel, Koen Pacmeleire, Femke Ongenae, and Sofie Van Hoecke. Mitigating data quality challenges in ambulatory wrist- wearable monitoring through analytical and practical approaches. Scientific. Reports, 14(1):17545, 2024.
Miguel Vasco, Hang Yin, Francisco Melo, and Ana Paiva. How to sense the world: Leveraging hierarchy in multimodal perception for robust reinforcement learning agents. arXiv preprint 1Xi::2210003608, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Valentin Vielzeuf, Alexis Lechervy, Stéphane Pateux, and Frédéric Jurie. Centralnet: a multilayer approach for multimodal fusion. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0-0, 2018.
Jose R Villar, Paula Vergara, Manuel Menéndez, Enrique de la Cal, Victor M González, and Javier Sedano. Generalized models for the classification of abnormal movements in daily life and its applicability to epilepsy convulsion recognition. International journal of neural systems, 26(06):1650037, 2016.
Johannes Wagner, Elisabeth Andre, Florian Lingenfelser, and Jonghwa Kim. Exploring fusion methods for multimodal cmotion recognition with missing data. IEEE Transactions on Affective Computing, 2(4): 206-218, 2011.
Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro. Multi-modal learning with missing modality via sharex--pecifie feature modelling. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. -188871,87, 2023a.
Hu Wang, Congbo Ma, Jianpeng Zhang, Yuan Zhang, Jodie Avery, Louise Hull, and Gustavo Carneiro. Tearnable cross-modal knowledge distillation for multi-modal learning with missing modality. In In- ternational Conference Medical Image Computing and Computer-Assisted Intervention, pp. 216 226. Springer, 2023b.
Hu Wang, Congbo Ma, Yuyuan Liu, Yuanhong Chen, Yu Tian, Jodie Avery Louise Hull, and Gustavo Carneiro. Enhancing multi-moda learning: Meta-learned cross-modal knowledge distillation for handling missing modalities. Xiv preprint or Xiv:2405.00115, 2024a.
Jiang Wang, Xiaohan Nie, Yin Xia, Ying Wu, and Song-Chun Zhu. Cross-view action modeling, learning and recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2649-2656, 2014.
Qi Wang, Liang Zhan, Paul Thompson, and Jiayn Zhou. Multimodal learning with incomplete modalities by knowledge distillation. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Dala Mining, pp. 1828-18881 1838,1838, 1838, 2020a.
Qianqian Wang, Huanhuan Lian, Gan Sun, Quanxue Gao, and Licheng Jiao. lemse: Incomplete cross-modal subspace clustering. IEEE Transactions Image Processing, 30:305-317 2020b.
Qifan Wang, Luo Si, and Bin Shen. Learning to hash on partial molti-modal data. In 24th International Joint Conference on Artificial Intelligence, 2015.
44

---

Published in Transactions on Machine Learning Research (02/2026)
Shuai Wang, Zipei Yan, Daoan Zhang, Haining Wei, Zhongsen Li, and Rui Li. Prototype knowledge dis- tillation for medical segmentation with missing modality. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1 5. IEEE, 2023c.
Tianyi Wang, Jianan Fan, Dingxin Zhang, Dongnan Liu, Yong Xia, Heng Huang, and Weidong Cai. Mirror: Multi-modal pathological self-supervised representation learning via modality aligument and retention. arXiv preprint arXic:2503.003 00374, 2025a.
Yixin Wang, Yang Zhang, Yang Liu, Zihao Lin, Jiang Tian, Cheng Zhong, Zhongchao Shi, Jianping Fan, and Zhiqiang He. Acn: adversarial co-training network for brain tumor segmentation with missing modali- ties. In Medical Image Compuling and Compuler Assisted Internenlion-MiCES 2021: 24th International Conference, Strasbourg, France, September 27 17-ttober I, 2021, Proceediings, Part VII 24, pp. Springer, 2021.
Yiyu Wang, Hailang Jian, Jian Zhuang, Huimin Guo, and Yan Leng. Sslmm: Semi-supervised learning with missing modalities for multimodal sentiment analysis. Information Fusion, 120:103058, 2025b.
Yuanyi Wang, Haifeng Sun, Jiabo Wang, Jingyu Wang, Wei Tang, Qi Qi, Shaoling Sun, and Jianxin Liao. Towards semantic consistency: Dirichlet energy driven robust multi-modal entity alignment. arXiv preprint arXiv:2401.17853 2024b.
Yuanzhi Wang, Zhen Cui, and Yong Li. Distributionn-consiseett modal recovering for incomplete multimodal lcarning. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 22025- 22034, 2023d.
Yuanzhi Wang, Yong Li, and Zhen Cui. Incomplete mullimooallty-ddiussd emotion recognition. Advances in Neural Information Processing Systems, 36, 2024c.
Shicai Wei, Yang Luo, Xiaoguang Ma, Peng Ren, and Chunbo Luo. Msh-net: Modality-shared hallucination with joint adaplation distillation for remote sensing image classification using missing modalities. IEEE Transactions on Geoscience and Remote Sensing, 61:1-15, 2023.
Mathias Wilhelm, Daniel Krakowezyk, Frank Trollmann, and Sahin Albayrak. ering: multiple finger gesture recognition with one ring using an electric field. In Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction, pp. 1-6, 2015.
Martin Wöllmer, Felix Weninger, Tobias Knaup, Björn Schuller, Congkai Sun, Kenji Sagae, and Louis- Philippe Morency. Youtube movie reviews: Sentiment analysis an andio-visual context. IEEE Intelligent Systems, 28(3):46 53, 2013.
Sangmin Woo, Sumin Lee, Yeonju Park, Muhammad Adi Nugroho, and Changick Kim. Towards good practices for missing modality robust action recognition. Tn Proceedings of the AAAT Conference Artificial Intelligence, volume 37, pp. 2776 2784, 2023.
Ancong Wu, Wei-Shi Zheng, Hlong-Xing Yu, Shaogang Gong, and Jianhuang Lai. Rgb-infrared cross- pross-modality person re-identification. In Proceedings of the IEEE international conference computer vision, pp. 5380- 5389, 2017.
Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXie preprint
2023a.
Renjie Wu, Hu Wang, Feras Dayoub, and Hsiang- Ting Chen. Segment beyond view: handling partially missing modality for audio-visual semantic sogmentation. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 38, pp. 6100-6108, 2024a.
Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal 1lm. arXi::2309005519 2023b.
45

---

Published in Transactions on Machine Learning Research (02/2026)
Zhenbang Wu, Anant Dadu, Nicholas Tustison, Brian Avants, Mike Nalls, Jimeng Sun, and Faraz Faghri. Multimodal patient representation learning with missing modalities and labels. In The T Twelfth Interna- tional Conference on Learning Representations, 2024b.
Feng Xu, Chuang Zhu, Wenqi Tang, Ying Wang, Yu Zhang, Jie Li, Hongchuan Jiang, Zhongyue Shi, Jun Liu, and Mulan Jin. Predicting axillary lymph node metastasis in carly breast cancer using doep learning on primary tumor biopsy slides. Pronliers in oncology, 11:759007, 2021.
Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Mst-vtt: A large vidco description datasct for bridging video and language. In Proceedings of the IEEE conference on computer vision and pallern recognilion, pp- 0288-5296 2016.
Yingxue Xu, Fengtao Zhou, Chenyu Zhao, Yihui Wang, Can Yang, and Hao Chen. Distilled prompt learn- ing for incomplete multimodal survival prediction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1102-5111, 2025.
Zihui Xue and Radu Marculescu. Dynamic multimodal fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2023.
Heran Yang, Jian Sun, and Zongben Xu. Learning unified hyper-networ's for aulti-modal mr image synthesis and tumor segmentation with missing modalities. IEFE Transactions on Medical Imaging, 2023a.
Qiushi Yang, Xiaoqing Guo, Zhen Chen, Peter YM Woo, and Yixuan Yuan. D 2-net: Dual disentanglement network for brain tumor segmentation with missing modalities. IEEF Transactions Medical Imaging, (1(10))2953-2966 2022.
Yanwu Yang, Hairui Chen, Zhikai Chang, Yang Xiang, Chenfei Ye, and Ting Ma. Incomplete learning of multi-modal connectome for brain disorder diagnosis via modal-mixup and deep supervision. In Medical Imaging With Deep Learning, pp. 1006 1018. PMLR, 2024.
Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXi::2303.1 11381, 2023b.
Wenfang Yao, Kejing Yin, William K Cheung, Jia Liu, and Jing Qin. Drfuse: Learning discntangled repre- sentation for clinical multi-modal fusion with missing modality and modal inconsistency. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 16416-16424, 2024.
Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. Deep learning for person re-identification: A survey and outlook. IEFE transactions on pattern onalysis and machine intelligence, 44(6):2872 2893, 2021.
Naoto Yokoya, Pedram Ghamisi, Ronny Haensch, and Michael Schmitt. 2020 ieee grss data fusion contest: Global land cover mapping with weak supervision [technical committecs). IEEE Geoscience and Remote Sensing Magazine, 8(1):154-18) 157, 2020.
Jeffrey T Young, Yundi Shi, Marc Niethammer, Michael Grauer, Christopher L Coe, Gabricle Lubach, Bradley Davis, Francois Budin, Rebecca C Knickmeyer, Andrew L Alexander, et al. The unc-wisconsin rhosus macaque neurodevelopment database: structural mri and dti database of carly postnatal devel- opment. Fronliers in neuroscience, 11:29, 2017.
Wenmeng Yu, Hua Xu, Fanyang Meng, Yilin Zhu, Yixiao Ma, Jicle Wu, Jiyun Zou, and Kaicheng Yang. Ch-sims: A chinese multimodal sentiment analysis dataset with fine- ane-grrined annotation of modality. In Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 3718-3727, 2020.
Lei Yuan, Yalin Wang, Paul M Thompson, Vaibhav A Narayan, Jieping Ye, Alzheimer's Discase Neuroimag- ing Initiative, al. Multi-source feature learning for joint analysis of incomplete multiple heterogeneous neuroinaging data. Neurolmage, 61 (3):622 632, 2012.
46

---

Published in Transactions on Machine Learning Research (02/2026)
Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis- Morency. Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXie preprinl arXiv:1606.0622 2016.
Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, and Louis-Philippe Morency. Multi- attention recurrent network for human communication comprehension. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Jiandian Zeng, Jiantao Zhou, and Tianyi Liu. Mitigating inconsistencies in multimodal sentiment analysis under uncertain missing modalities. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 2924-2934, 2022.
Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, et al. Anygpt: Unified multimodal Ilm with discrete sequence modeling, arXiv preprint ar Xiv:2400. 12226, 2024.
Chaohe Zhang, Xu Chu, Liantao Ma, Yinghao Zhu, Yasha Wang, Jiangtao Wang, and Junfeng Zhao M3care: Learning with missing modalities in multimodal healthcare data. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 2428-2428, 2022.
Jiaming Zhang, Ruiping Lüu, Hao Shi, Kailun Yang, Simon ReiB, Kunyu Peng, Haodong Fu, Kaiwei Wang, and Rainer Stiefelhagen. Delivering arbitrary-modal semantic segmentation. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Paltern Recognition, pp. 1136 1147, 2023a.
Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance convolutional neural networks via self distillation. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 37122,72,,22 2019.
Shifeng Zhang, Ajian Liu, Jun Wan, Yanyan Liang, Guodong Guo, Sergio Escalera, Hugo Jair Escalante, and Stan Z Li. Casia-surf: A large-scale multi-modal benchmark for face anti-spoofing. IEEE Transactions on Biometrics, Behavior, and Identily Science, 2(2):182 193, 2020.
Weichen Zhang, Dong Xu, Jing Zhang, and Wanli Ouyang. Progressive modality cooperation for multi- modality domain adaptation. IEEE Transactions Image Processing, 30:3293 3306, 2021.
Yuan Zhang, Hu Wang, David Butler, Minh-Son To, Jodie Avery, M Louise Hull, and Gustavo Carneiro. Distilling missing modality knowledge from ultrasound for endometriosis diagnosis with magnetic resonance images. In 2023 IEEE 20th International Symposium on Biomedical Imaging, 2023b.
Yue Zhang, Chengtao Peng, Qiuli Wang, Dan Song, Kaiyan Li, and S Kevin Zhou. Unified multi-modal image synthesis for missing modality imputation. Xrxiv preprint arXiv:2304 05340, 2023c.
Zheyu Zhang, Gang Yang, Yueyi Zhang, Huanjing Yue, Aiping Liu, Yunwei Ou, Jian Gong, and Xiaoyan Sun. Tmformer: Token merging transformer for brain tumor segmentation with missing modalities. Tn Proceedings of the AAAI Conference Artificial Intelligence, 2024.
Fei Zhao, Chengeui Zhang, and Baocheng Geng. Deep multimodal data fusion. ACM Computing Surveys, 56(9):1-36, 2024.
Zechen Zhao, Heran Yang, and Jian Sun. Modality-adaptive feature interaction for brain tumor segmentation with missing modalities. In International Conference on Medical Image Computing and Computer-Assisteee Intervention, pp. 183-192. Springer, 2022.
Aihua Zheng, Zi Wang, Zihan Chen, Chenglong Li, and Jin Tang. Robust. multi-modality person re- identification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 3529-3537, 2021.
Wei-Long Zheng and Bao-Liang Lu. Investigating critical frequency bands and channels for bog-basod emotion recognition with deep neural networks. IEEE Transactions on autonomous mental development, 7(3):162- 175, 2015.
47

---

Published in Transactions on Machine Learning Research (02/2026)
Xu Zheng, Yuanhuiyi Lyu, and Lin Wang. Learning modality-agnostic representation for semantic segmen- tation from any modalities, 2024. URL https://arxiv.orr
/210224401.11 11351
Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine- grained air quality based big data. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 2267-2276, 2015.
Zhuo Zhi, Ziquan Liu, Moe Elbadawi, Adam Daneshmend, Mine Orlu, Abdul Basit, Andreas Demosthenous, and Miguel Rodrigues. Borrowing treasures from neighbors: In-context learning for multimodal learning with missing modalities and data scarcity. arXin:2403. 09428, 2024.
Tongxue Zhou. Feature fusion and lalent feature learning guided brain tumor segmentation and missing modality recovery network. Pattern Recognition, 141:109665, 2023.
Tongxue Zhou, Stéphane Canu, Pierre Vera, and Su Ruan. Latent correlation representation learning for brain tumor segmentation with missing mri modalities. IEEE Transactions on Image Processing, 30: 4263 4274, 2021.
Tongxue Zhou, Su Ruan, and Haigen Hu. A literature survey of mr-based brain tumor segmentation with missing modalities. Compulerized Medical Imaging and Graphics, 104:102167, 2023.
Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, Hongla Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, et al. Languagebind: Extending video- language pretraining to n-modality by language- based semantic aligmment. arXiv preprint ar arXiv:2310.014 2023.
Xiaoguang Zhu, Lianlong Sun, Yang Liu, Pengyi Jiang, Uma Srivatsa, Nipavan Chiamvimonvat, and Vladimir Filkov. Causal debiasing medical multimodal representation learning with missing modalities. arXiv preprint arXim:2509.055 05615, 2025.
48


Reviews:
[{"review_id": "eKFHYB9gMn", "reviewer": "Reviewer_fwck", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper reviews Deep Learning approaches to Multimodal Systems, more specifically in the context of missing modalities (which is represented by the MLMM acronym). \nIt reviews 354 papers published between 2012 and 2025, in AI, ML and more content or application focussed titles such as ICASSP, ACM Multimedia or MICCAI.  The review aims to cover various architectural approaches and application areas, which include information retrieval, remote sensing, robotic vision, medical diagnosis and sentiment analysis. \nIt purports to make three contributions: i) a comprehensive survey of MLMM methods across diverse domains; ii) A novel, fine-grained taxonomy of MLMM methodologies and iii) an in-depth analysis of current MLMM approaches, their challenges, and future research directions.\n\nThese contributions are achieved to a variable extent. Contribution ii) is clearly visible and consists in an original and comprehensive taxonomy, even though some details may warrant further discussion such as the inclusion of ‘attention’ as a generic component rather than associated with a specific DL architecture other than a Transformer. Contribution i) is partly achieved, the main question being the scope of the review that might result in an imbalance between most recent approaches and earlier one due to the major evolutions of the discipline (and the growth in submissions across the year which creates another, quantitative, imbalance).  Contribution iii) could be found wanting, primarily for lack of in-depth analysis. In many places the discussion is mostly about categorizing approaches and clustering publications with no real additional conceptual frameworks beyond the above taxonomy, which is not sufficient to that purpose. \n\nThe main strengths of the paper consist in the identification of the MLMM topic; the collection of references; the attempted breadth of coverage in terms of application; the generic presentation; the volume of publications reviewed.  \n\nIts weaknesses include: a reviewing philosophy which primarily appears as clustering and categorization, with direct comments on individual works and their clustering, and limited analysis frameworks apart from the taxonomy; a lack of clarity on the technical evolution of the discipline; some difficulties in articulating architectures and applications. \nAn overall weakness of the paper is not to have developed a conceptual framework for multimodality into which to analyze the issue of missing modalities. Here I am making reference to the kind of frameworks that are pervasive in Multimodal research, which discuss relationship between modalities in terms or temporal relations, informational complementarity vs. redundancy, and fusion techniques.", "claims_and_evidence_yn": "No", "claims_explanation": "In the context of a review paper, we should consider the claims as corresponding to elements of stated contributions, such as the in-depth nature of the technical analysis or the comprehensive treatment of challenges and future directions.   \nBecause the review spans a long period of technical evolution, from early multimodal fusion networks to contemporary foundation models, MLMM and diffusion-based, the treatment occasionally remains at the level of categorization rather than achieving the in-depth analysis advertised in the contribution section. This is understandable given the breadth of material, yet the result is that the discussion tends to enumerate architectural families without fully articulating their respective motivations, assumptions, or comparative trade-offs. \nRelated to this, the paper sometimes encounters difficulties in consistently relating architectures to their application domains. Some applications (e.g., captioning, VQA, retrieval, diagnosis, sentiment analysis) introduce intermediate layers or task-specific modules that function as additional interpretive steps between architecture and modality, which makes the classification less clear-cut (this is visible is some medical example and some affective computing examples). The survey describes these applications in detail, but the relationship between architectural choices and the functional requirements of such tasks could be made more explicit.", "audience_yn": "Yes", "audience_explanation": "Although the publications in this MLMM topic is more impressive in terms of growth than numbers, which remain limited in the broader ML context, the topic can be of interest to a broader audience, who might encounter MLMM issues as part of their Multimodal ML work or Multimodal application development. The actual extent of this interest depends on improvements and clarifications that would make the findings more substantial and more easily accessible. In the paper’s current form there should be an interest in raw references and their comments, MLMM taxonomy, early categorizations of approaches.", "requested_changes": "Overall, the review would benefit from a conceptual framework for analyzing multimodal architectures and their behaviour under missing-modality conditions. Established perspectives in multimodal research mentioned earlier, such as informational complementarity vs. redundancy, temporal alignment between modalities, or distinctions between early, mid-level, and late fusion could provide the theoretical scaffolding needed to interpret the surveyed literature beyond taxonomic grouping. Such a framework would allow the authors to position individual works not only within categories but also within a set of principled multimodal constructs, thereby offering a more coherent analytical synthesis.\n\nThe relationship between applications and architectures needs reworking, whether via the above reframing or through a refined bottom-up approach. Some examples in particular medical ones in 6.2, are barely convincing when they span across so many modalities from HER to omics via many different imaging modalities. Similar comments could be made around what is termed ‘sentiment analysis’ and often seems to refer to various tasks in affective computing: this is such an active field (for instance at ACM Multimedia, which is part of the reviewed venues) that it deserves a more articulate treatment. \nIn terms of individual section contents and in addition to the above broader comments, sections 6.x necessitate significant rewriting making their MLM tasks more differentiated (think robotic vision and sensing vs. affective computing or medical applications). The discussion section is way too generic and needs to be expanded.\n\nSection 7.3 raises the issue of the balance between recent foundation models and previous approaches: as it is written it would almost make some previous sections appear irrelevant. The authors have quite some room for manoeuvre here as they cannot be blamed for refocusing around specific architectures (which do not have to ba all foundational models). \n\nSection 8, the conclusion, needs substantial expansion. It would also be appropriate to cover some limitations of the paper in view of the challenging nature of the task, setting the limits even to the eventually revised framework for analysis.", "broader_impact_concerns": "This reviewer finds no obvious concerns regarding ethical implications.", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "76efFROZUS", "reviewer": "Reviewer_p5sV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors provided the systematic and comprehensive overview of Deep Multimodal Learning with Missing Modality (MLMM), a subfield focused on building multimodal systems that remain robust when one or more input modalities are absent during training, testing, or deployment. \nMotivated by real-world failures--such as sensor malfunctions, privacy-driven data restrictions, environmental interference, hardware inconsistencies, and domain-specific constraints--the paper positions missing modality as a central challenge for dependable multimodal AI across affective computing, medical imaging, robotics, remote sensing, and space exploration.\n\n**S1.** A central strength of the survey lies in its well-structured and fine-grained taxonomy of MLMM approaches. \nThe authors organize existing methods along two intuitive dimensions--data-level processing and strategy-level design--which naturally lead to four coherent categories: modality imputation, representation-focused techniques, architecture-oriented methods, and model-combination strategies. \nTheir explanations within each group are generally convincing and thoughtfully reasoned: they clarify how generative models may introduce hallucinated content, why representation-level recovery can mitigate raw-data noise, how attention-based architectures scale effectively but rely on large datasets, why distillation commonly assumes access to complete modality teachers, how graph-based approaches capture rich structural dependencies but struggle with scalability, and how multimodal LLMs can flexibly handle partial inputs despite heavy pretraining requirements.\n\n**S2.** The survey also proposes a second organizational scheme based on the traditional stages of multimodal fusion--early, intermediate, and late--distinguishing approaches that attempt to reconstruct missing modalities from those that operate without recovery. \nTheir meta-analysis shows that the majority of existing studies emphasize some form of recovery, with intermediate-stage methods being particularly common because they strike a practical balance between representational richness and stability. \nIn contrast, comparatively fewer works tackle the problem directly through non-recovery strategies.\n\n**S3.** The authors further survey MLMM applications across information retrieval, medical diagnosis, sentiment analysis, multimodal time-series, remote sensing, robotic vision, and autonomous driving, providing an extensive dataset compilation for each domain. \nThey also analyze the rise of multimodal transformers and multimodal LLMs (e.g., CLIP, BLIP-2, LLaVA, AnyGPT, ImageBind), highlighting how large-scale pretraining implicitly helps models cope with partial modality inputs.\n\n**W1.** However, the survey overlooks an important and rapidly growing domain where missing-modality issues are intrinsic rather than incidental: IoT, pervasive sensing, and distributed real-world sensor networks. \nIn these systems--ranging from smart homes and industrial monitoring to wearable platforms and environmental sensor grids--modalities fail unpredictably due to battery loss, bandwidth limits, asynchronous sampling, heterogeneous device capabilities, and physical degradation. \nThese settings represent some of the most persistent and structurally inevitable instances of missing modalities, yet the survey touches only lightly on them and provides limited references or methodological discussion tailored to this context. \nAs these real-world sensing ecosystems increasingly rely on multimodal fusion (vision + audio + inertial + environmental + network data), MLMM techniques must address challenges such as long-term partial observability, streaming data with asynchronous modalities, multi-agent inconsistencies, and on-device resource constraints--areas that remain underrepresented in the current review.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Despite offering a broad survey of multimodal learning under missing-modality conditions, the discussion gives relatively little attention to IoT, pervasive sensing, and real-world distributed sensor ecosystems--domains in which missing modalities are not an occasional anomaly but a fundamental and recurring characteristic of the data environment. \nIn practical deployments such as smart homes, industrial monitoring platforms, wearable systems, environmental sensor grids, and mobile robotic networks, data streams are inherently heterogeneous, asynchronous, and failure-prone. M\nodalities often disappear due to battery depletion, bandwidth limitations, variable sampling rates, intermittent connectivity, hardware degradation, or environmental disruption.\n\nBecause these systems routinely operate with partial, degraded, or intermittently available modalities, they present some of the most representative and demanding scenarios for missing-modality learning. \nYet the survey provides only brief coverage of these settings and does not fully articulate the unique challenges they raise--such as cross-device heterogeneity, long-term partial observability, edge-compute constraints, multimodal drift, or the need for continual adaptation in dynamic environments.\n\nAs a result, an important class of real-world applications--where MLMM would be not just beneficial but essential--receives limited conceptual framing and methodological treatment. \nMore explicit discussion of IoT and pervasive multimodal sensing would strengthen the completeness of the field overview, especially given the increasing reliance on multimodal fusion in everyday deployed systems.", "audience_yn": "Yes", "audience_explanation": "The survey on missing-modality multimodal learning will likely draw significant attention because it offers one of the most comprehensive and systematic overviews of a rapidly expanding research area. \nAs multimodal models increasingly power applications in vision–language understanding, medical imaging, robotics, human–computer interaction, and pervasive sensing, practitioners frequently encounter scenarios where one or more modalities are absent, degraded, or unavailable. \nThis survey distinguishes itself by bringing clarity to this methodological landscape through a carefully structured taxonomy spanning modality imputation, representation-centric strategies, architecture-level solutions, and model-combination approaches. \nThe breadth of coverage--integrating over 300 papers across diverse domains--gives readers a holistic understanding of how missing-modality challenges have been addressed historically and where the field is heading.\n\nMoreover, the paper’s dual categorization framework (methodological perspective and fusion-stage perspective) helps unify previously scattered contributions under a coherent conceptual structure, making it easier for researchers and practitioners to navigate the space, compare methods, and identify gaps. \nIts extensive discussion of strengths, limitations, and trade-offs of each method family further enhances its value as a practical reference. Given the accelerating adoption of multimodal large language models and the increasing need for robust, failure-tolerant AI systems, readers across machine learning, computer vision, NLP, medical AI, and IoT sensing communities will find this survey especially timely and informative.", "requested_changes": "A more thorough discussion of missing-modality learning would be strengthened by incorporating insights from wearable sensing, IoT infrastructures, and pervasive multimodal systems. \nPrior work on mobile and wearable sensor networks [1] has long emphasized that sensor noise, fluctuating sampling rates, device failures, and user-dependent variability naturally produce persistent gaps in multimodal streams. Recent analyses of ambulatory wrist-worn devices further illustrate how real-world data are characterized by non-wear intervals, dropout segments, and irregular sensor availability [2].\n\nStudies in multi-sensor human activity recognition similarly highlight the structural difficulties posed by heterogeneous sensing environments. For example, multi-graph modeling approaches [3] show that IMUs, magnetometers, and physiological sensors often sample asynchronously or provide incomplete data, underscoring the importance of techniques that remain effective under partial modality access. Work on robustness and data augmentation for wearable signals [4] likewise demonstrates that signal degradation, corruption, and cross-modality inconsistency can substantially degrade downstream model performance--conditions directly relevant to missing-modality learning.\n\nComplementary research on recovering absent sensor channels provides additional rationale for stronger integration of these domains. Generative reconstruction approaches such as SensorGAN [5] aim to restore missing sensor streams, while recent efforts on lightweight imputation strategies for mobile health systems [6] develop efficient reconstruction pipelines suitable for resource-constrained devices. Collectively, these lines of work make clear that missing modalities are not rare artifacts but a defining characteristic of distributed sensing, ubiquitous computing, and mobile health ecosystems.\n\nIntegrating these references would help situate missing-modality learning within the practical settings where it is most essential--demonstrating that MLMM methods must be designed not only for curated multimodal datasets but for real-world sensor networks where modality loss, degradation, and heterogeneity are intrinsic and unavoidable.\n\n**Reference**\n- [1] Nweke, Henry Friday, et al. \"Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges.\" Expert Systems with Applications 105 (2018): 233-261.\n- [2] Van Der Donckt, Jonas, et al. \"Mitigating data quality challenges in ambulatory wrist-worn wearable monitoring through analytical and practical approaches.\" Scientific Reports 14.1 (2024): 17545.\n- [3] Chen, Ling, et al. \"A multi-graph convolutional network based wearable human activity recognition method using multi-sensors.\" Applied Intelligence 53.23 (2023): 28169-28185.\n- [4] Jeon, Eun Som, et al. \"Role of data augmentation strategies in knowledge distillation for wearable sensor data.\" IEEE internet of things journal 9.14 (2021): 12848-12860.\n- [5] Hussein, Dina, and Ganapati Bhat. \"SensorGAN: A novel data recovery approach for wearable human activity recognition.\" ACM Transactions on Embedded Computing Systems 23.3 (2024): 1-28.\n- [6] Hussein, Dina, et al. \"Energy-efficient missing data imputation in wearable health applications: A classifier-aware statistical approach.\" Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence AI for Good. 2024.", "broader_impact_concerns": "NA", "additional_comments": "NA", "extra_scores": {}, "extra_text": {}}, {"review_id": "49vtJa8DET", "reviewer": "Reviewer_PJyo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The manuscript is a survey of Deep Multimodal Learning with Missing Modality (MLMM). It proposes a two-axis taxonomy: Data Processing and Strategy Design. And it expands these into 12 concrete categories (e.g., modality/representation composition vs. generation; attention-, distillation-, and graph-based methods; ensembles, dedicated training, discrete schedulers; plus a separate bucket for MLLMs). It compiles 354 papers (2012–Aug 2025), outlines trends, catalogs applications and datasets, and concludes with open problems. It also offers clear visualizations of the different categories, which I found useful for understanding the main ideas behind the methods in each category. The text flows naturally and is well written, with only a couple of places that are harder to read. Overall, I found this manuscript useful for readers (potentially new to this field) to get a big picture of different methodologies.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, the manuscript supports the main claims and lists the papers. However, due to the scale of the papers published in the field, it is possible that they missed some papers. This is because I came across some papers in computational pathology that were not included and I list a few of them in the next sections.", "audience_yn": "Yes", "audience_explanation": "The manuscript covers an important problem, and it is useful for people interested in ML in general and those who specialize in ML applications.", "requested_changes": "I have come across multiple papers in the space of computational pathology that the survey doesn't cover, and I believe there are even more. So, I'd strongly recommend reevaluating the paper inclusion scheme for the medical domain to cover fields such as medical imaging, gene representation learning, cell representation learning, and radiology and pathology, as work in these often associate with missing modality:\n1. Distilled Prompt Learning for Incomplete Multimodal Survival Prediction, CVPR 2025 [distilled models category] \n2. A deep learning model to predict RNA-Seq expression of tumours from whole slide images, Nature Comm 2020 [modality generation]\n3. Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification, Nature Mac Intl 2022 [modality generation]\n\nThe other point that I would like the author to clarify, especially for the attention network, is that these methods are generally more data hungry than other older methods. However, it has not been covered in the draft. Also, chronologically speaking, newer methods tend to train and scale with more data in their method (potentially a main reason for better generalization), which makes the head-to-head comparison of different methods even harder. So, I'd like the author to comment on this as well. \n\nThere are some editorial comments: \n1. I found this paragraph hard to read and would recommend rewriting it for better clarity: \"In addition, indirect-to-task representation generation methods can access **the missing modality data of the missing modality samples during training, which means that the training dataset is actually complete**, so this type of method usually performs better than direct-to-task representation generation methods.\" To be specific, I highlighted the part that is confusing to me.\n\n2. I found this \"self-distillationZhang et al. (2019).\" which just needs an extra edit midway in the sentence ;)", "broader_impact_concerns": "N/A", "additional_comments": "It was an interesting read! Good job!", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)