RAG4DMC: Retrieval-Augmented Generation for Data-Level Modality Completion
Authors: Ningxin He
Yongheng Deng
Sheng Yue
Yongjian Fu
Zhangzehui
Tiegang Gao
Venue: ICLR 2026 Poster (2026)
Keywords: Retrieval-Augmented Generation; Missing Modality Completion; Multimodal Learning
Primary area: generative models

Abstract: Multi-modal datasets are critical for a wide range of applications, but in practice, they often suffer from missing modalities. This motivates the task of Missing Modality Completion (MMC), which aims to reconstruct missing modalities from the available ones to fully exploit multi-modal data. While pre-trained generative models offer a natural solution, directly applying them to domain-specific MMC is often ineffective, and fine-tuning suffers from limitations like limited complete samples, restricted API access, and high cost. To address these issues, we propose RAG4DMC, a retrieval-augmented generation framework for data-level MMC. RAG4DMC builds a dual knowledge base from complete in-dataset samples and external public datasets, enhanced with feature alignment and clustering-based filtering to mitigate modality and domain shifts. A multi-modal fusion retrieval mechanism combining intra-modal retrieval with cross-modal fusion then provides relevant context to guide generation, followed by a candidate selection mechanism for coherent completion. Extensive experiments on general and domain-specific datasets demonstrate that our method produces more accurate and semantically coherent missing-modality completions, resulting in substantial improvements in downstream image–text retrieval and image captioning tasks.

Published as a conference paper at ICLR 2026
RAG4DMC: RETRIEVAL-AUGMENR GENERATION FOR DAVA-LEVEL MODALITY COMPLETION
Ningxin He Nankai University
Yongheng Deng"
Tsinghua University
ningxinhe@mal 1 nanka cn
wit.:eettttingguuaaaa edu.
Yongjian Fu
Sheng Yue
Tsinghua University
Sun Yat-sen University
nongjianworkegman gma com
yuesh5@mail. sysu. edu.en
Zehui Zhang
Tiegang Gao"
Hangzhou Dianzi University zhangt aatgttianxii918880 com
Nankai University gaot gaotiieganggnankii..ccmm cn
ABSTRACT
Multi-modal datasets are critical for a wide range of applications, but in practice, they often suffer from missing modalities. This motivates the task of Missing Modality Completion (MMC), which aims to reconstruct missing modalities from the available ones to fully exploit multi-modal data. While pre-trained genera- tive models offer natural solution, directly applying them to domain-specific MMC is often ineffective, and fine-tuning suffers from limitations like limited complete samples, restricted API access, and high cost. To address these is- sues, we propose RAG4DMC, retrineal-augmented generation framework for data-level MMC. RAG4DMC builds a dual knowledge base from complete in- dataset samples and external public datasets, enhanced with feature alignment and clustering-bant based filtering to mitigate modality and domain shifts. A multi-modal fusion retrieval mechanism combining intra-modal retrieval with cross-modal fu- sion then provides relevant context to guide generation, followed by candidate selection mechanism for coherent completion. Extensive experiments on general and domain-specift datasets demonstrate that method produces more accurate and semantically coherent cuisiing-modality completions, resulting in substantial improvements in downstream image- text retrieval and image captioning tasks.
INTRODUCTION
Multi-modal datasets have become indispensable advancing a wide range of applications, includ- ing dision-langual understanding, healthcare analysis, and autonomous systems (Yuan al. 2025 Xu et al. 2023). However, in practice, such datasets often suffer from missing modalities in some samples, due reasons such as sensor failure, annotation costs, data corruption (Tiao al. 2024). This issue motivates the task of Missing Modality Completion (MMC)(Ke et al. (2025). i.c., recon- structing or inferring missing modalities from available ones, thereby enabling more complete and effective utilization of muuli--odaa data.
With the remarkable success of pre-trained generative models, natural approach this task is to leverage them to generate missing modalities conditioned on the observed ones. Nevertheless, directly applying pre-trained generative models to reconstruct domain-specific missing data often yields unsatisfactory results, due to their limited adaptation to specific domains (Ke et al.) (025). straightforward remedy is to fine-tune these models using the subset of complete samples in the dataset. However, this strategy faces several challenges: (i) the amount of complete data is usually limited, making fine-tuned models prone to overfitting and losing generality; (ii) many pre-trained
"Correponniigg author.

---

Published as a conference paper at ICLR 2026
generative models are accessible only through restricted inference APIs, preventing effective fine- tuning: and (iii) fine-tuning large models is often costly and sssuurce-ittessve..
To overcome these limitations, propose retrieval- -uggeeened generation (RAG) based frame- work for MMC. RAG combines retrieval from a knowledge base with generative modeling, allow- ing the model to incorporate external, task-relevant information during generation. While RAG has achieved great success in language modeling and downstream NLP tasks, its potential for MMC remains largely underexplored. In adapting RAG to MMC, two key challenges arise: (i) how to construct an effective knowledge base from the available multi-modal data, and (ii) how to design retrieval strategies that can best augment the generation of missing modalities.
To this end, we propose RAG4DMC, a retrieval -agmeette generation framework tailored for data- level MMC. RAG4DMC leverages a dual- soaal--kwledgg--ba base design that combines an internal knowl- edge base-built from complete samples in the target dataset-win an external knowledge base constructed from publicly available datasets. While the internal knowledge base provides domain- specific information, its limited scale motivates the use of external data as complementary source, despite domain shifts. To make this integration effective, RAG4DMC encodes all samples into embeddings and introduces feature alignment mechunisms to address the embedding misalignment between modalities and knowledge bases. Furthermore, a clustering- -based filtering strategy is ap- plied to prune irrelevant external samples. On top of the dual knowledge base, RAG4DMC develops a multi-modal fusion retrieval strategy, which combines intra-modal retrieval with cross-modal re- ranking to achieve semantically consistent retrieval results. The retrieved information is used to augment the generative model, which produces multiple candidate completions. Finally, a candidate selection mechanism identifies the most semantically coherent output fill in the missing modality.
We extensively evaluate RAG4DMC on both general-domain (MSCOCO, Flickr30K) and domain- specific (RSICD) datasets under various modality missing rates. Downstream models trained on datasets completed using RAG4DMC exhibit improvements of up to +5.0 Avg R@) 1 in image-text retrieval tasks and +5.0 CIDEr in image captioning tasks over those trained on datasets completed via direct generative completion methods.
The main contributions of this paper are as follows: (i) To our knowledge, RAG4DMC is the first RAG-based approach to achieve multimodal completion at the raw-data level, generating the actual missing modality instead of relying on latent-feature imputation. (ii) We design an integrated knowledge base combining internal and external samples, equipped with filtering and alignment, and propose a two-stage multi-modal fusion retrieval with candidate selection to mitigate modality gaps and produce semantically faithful generations. (ii) Extensive experiments on MSCOCO, Flickr30K, and RSICD demonstrate that RAG4DMC consistently outperforms baselines, validating both the effectiveness and robustness of the proposed approach.
2 RELATED WORK
Strategies for handling missing modalities. Existing approaches for handling missing modali- ties can be broadly categorized into two groups: non-completion methods and completion- based methods. Non-completion methods avoid explicit imputation and instead rely fusion strategies or missing-indee missing-indicctor mechanisms to make predictions directly from the observed modalities (Lang
eet al.) 2025 Lee et al. 2023 Guo al.) 2024 Zhao al. 2024 2025). While effective in certain scenarios, these methods cannot generate addlity-guaranted reusable multimodal samples, which limits their applicability across diverse tasks. In contrast, completion-based methods address the issue by reconstructing the missing modalities before downstream processing. These approaches can be further divided into two subcategories: 1) Feature-level completion. which learns to infer missing features in a shared latent space. For example, MMIN (Zhao et al.] (202]) imagines latent features and Smil Ma et al. (202T) meta-learns extreme missing patterns. 2) Data-level completion, which reconstructs missing modalities directly in the data space. For example, Knowledge Bridger Ke et al.] (2025) leverages structured priors from domain knowledge, GTI-MM (Feng et al. 2024) steers diffusion with prompts, and DiCMoR (Wang et al., 2023c) aligns cross-modal distributions via normalising flows. These methods produce more faithful modal datasets that can be fully reused across diverse downstream tasks, but existing methods still suffer from hallucination, poor generalization to rare samples, and heavy resource demands (Wang et al. 2023a). The limitations of existing methods motivate our investigation to RAG-based methods for data-level MMC.
2

---

Published as a conference paper at ICLR 2026
Multimodal Retrieval-Augmente Generation. Retrieval-Augmenter Generation (RAG) has proven highly effective in NLP for knowledge-grue gronenerrddddddddd text generation and QA (Riedler Langer
2024 Lewis et al.] 2020 Iacard & Grave 2020). Recent multimodal extensions adapt RAG to tasks such as captioning, VQA, and reasoning g by retrieving relevant images or texts as additional context for generation models (Abootorabi al. 2025 Lin & Byrne, 2022 Zhang al. 2024 De-
spite these advances, the application of RAG to MMC remains underexplored. The most related work is MissRAGPPipoli al. 2025), wfhich retrieves prototype representations from the training set to approximate missing inputs. However, MissRAG focuses on feature- level completion at infer- ence, while our approach operates at the data level, explicitly reconstructing missing modalities and generating enriched multi-modal training data.
3 METHOD
3.1 PROBLEM STATEMENT
We consider the task of data-level MMC multi-modal dataset. Let [(=/ ; ) denote dataset containing image (7) and text (T) modalities. In practice, many samples are incomplete, i.e., only x' or x" is observed. Our goal is to promote a pre-trained generation model to complete the missing modality f from the available one based on two complementary knowledge bases:
K = {1} 2f) C, Kent {1 ) /m=1
where Kint contains complete samples from the dataset D, and Kent contains paired image-text samples from publicly available datasets (e.g., CC3M, LAION) provide additional knowledge. For an incomplete sample the missing modality is generated as ă = G(I, R(x; Kint U Keat)), where R(-) denotes retrieval from the dual knowledge base Kint U Keett
3.2 OVERVIEW
The design of RAG4DMC is illustrated in Fig. [ (algorithm description is illustrated in Ap- pendix A.I). RAG4DMC follows a attieval-augmenneed generation pipeline with three key com- ponents. It first constructs dual knowledge base, where the internal base is built from complete samples in the target dataset and the external base is derived from publicly available datasets to compensate for data scarcity. Since embeddings from different modalities and from internal and ternal sources are often misaligned, RAG4DMC incorporates feature alignment modules that learn mappings align representations from different modalities and knowledge bases, and applies clustering-based filtering strategy to remove irrelevant external samples. Given an incomplete sam- ple, the system then performs two-stage multi-modal fusion retrieval process that first conducts intra-modal retrieval and subsequently refines candidates via cross-modal re-ranking to ensure se- mantically consistent matches. The retrieved knowledge is used to guide the generative model, which produces multiple candidate completions. Finally, candidate selection mechanism identifies the most semantically coherent and high-quality output reconstruct the missing modality.
3.3 DUAL KNOWLEDGE BASE CONSTRUCTION
To enable effective
generation, RAG4DMC constructs unified dual knowledge base that integrates internal complete samples and external public datasets. Relying solely on the internal dataset is often problematic because the number of complete multi-modal samples is lim- ited and cannot sufficiently cover the diversity of missing cases. For example, in a vision-language dataset collected for autonomous driving. only fraction of scenes may contain both images and detailed textual amnotations, making it difficult to reconstruct missing captions for rare traffic sce- narios. By incorporating external public datasets such as CC3M or LAION, the knowledge base can provide broader visual-textual coverage, offering complementary patterns that mitigate the sparsity of internal data. However, directly combining internal and external data introduces three challenges: (i) misalignment between modalities within the same dataset, (ii) noise and irrelevance in exter- nal datasets, and (iii) domain shifts across internal and external sources. To address these issues, RAG4DMC employs three key techniques: (i) cross-modal bidirectional mapping mechanism to reconstruct missing modalities and mitigate modality gaps in the embedding space, (ii) a clustering- based filtering mechanism prune irrelevant external samples, and (iii) cross-domain alignment
3

---

Published as a conference paper at ICLR 2026
Step 1 Dual Knowledge Base Construction
Step 2 Muiti-Madal Fusion Retrieval
Duat Knowledge Bese
Bi-MLP
dockar bes deiidia pait thtee
Miss
Rerenk
Internal Knowledge Base
A abas skip pamnt beac vhere people walk
Kin Knowledge Filtering
Top-1
nétee biue herry uaader
Prompe
Step Generation and Candidate Selection
ginn shly poocn by beak viere paga
A gost 18 proces
da dedio has.
seels
wiito and Man lvery cresse
wlere peepih rulk diaiss
To wiibi uandy where people
External Knowledge Base
Figure 1: The workflow of RAG4DMC.
strategy to align internal and external embeddings into unified semantic space. Together, these components yield semantically consistent multi-modal knowledge base for subsequent retrieval and generation.
Cross-modal bidirectional mapping. We use fixed pretrained encoders E, x + 12d and Er X + Rdd to extract embeddings only for paired multimodal samples. Let Pint x C D and Pezt [a Fm _T,ext) [ Jm=1 Dezt denote the sets of complete image-text pairs, For these
pairs we compute:
2 = Ex({) E2 (x
2f = =r(x)) Pine) TText Fr (xx)) (m Pezt). Fm
(1)
Lext 2m
(2)
To bridge the gap between modalities, train lightweight MLP for bidirectional cross-modal mapping. We denote the two directions as JI-TT RRd + Rd and fr : IR + Rd, which share parameters:
Smap
Σ (1()-)())) - o II - 2 11)
Panl
(x!
This bidirectional cross-modal mapping reconstructs missing modalities in the internal embedding space using available ones, and producing pseudo emheddings for retrieval. As the external knowl- edge base is subsequently aligned into the internal space, training the MLP on internal pairs suffices for both internal and external samples.
Clustering- based knowledge filtering. External datasets mix relevant and irrelevant information. To reduce noise, we design a clustering based knowledge filter. For the image modality, we denote the internal and external image embeddings as z !int and ,ext, First, we apply K K-means internal and external embeddings to obtain cluster centroids. For K clusters, we compute cluster centroids as the average embedding:
lint
2,
1,int
Leext
Lext Zm
(4)
Lint
Tiit E
Ec
where of and of exx denote the of embeddings assigned to the p-th internal cluster and q-th external cluster, respectively. Next, we match each external centroid 1u Lletxt to its closest internal centroid i based on cosine similarity:
P arg max cos (ML (pf,ext, 11 jff.iut))..
where p denotes the index of the internal centroid /f that is most similar to the external centroid Lext We then define the corresponding centroid-level and instance-level similarities as: M4
1. cent cos (N)
f.int) -1pb
S cos 1eext Lint
(6)
4

---

Published as a conference paper at ICLR 2026
where z is an external embedding in the q-th cluster. Clusters and samples are then filtered by thresholding the similarities:
Kkeep (c/ 8'eent 2 Teent )
Zkooep [xxxxx _ext E c/jext, Sinst 2 Tinst, Cq E Cheepp
Jlext
(8)
We first prune clusters with low centroid similarity and then filter instances within the retained ones to improve efficiency and robustness. The same procedure is applied to text [Appendix(A.2) Further analysis of imbalanced clusters and the sensitivity of Timst and Teent is provided Appendix A.3
alignment. Even after filtering, the internal and retained external embeddings still reside in different semantic spaces due to domain shifts (see t-SNE visualizations in Appendix[A.. (..4). Inspired by MUSE (Lample et al. 2018), we adopt an orthogonal Procrustes alignment strategy, performed separately for the image and text modalities using their internal and filtered external embeddings.
For images, let {(Z) iint Zm } denote the mutual nearest neighbor (MNN) pairs identified via CSLS (analysis in Appendix A.6 Stacking these pairs yields matrices Z and Z For texts, the procedure is analogous, producing 2 and Z The orthogonal Procrustes problems are then solved:
W7 arg min
IZ
Zex WIl
Wi arg min WEO(d)
1
ZT W
(eo(d)
where ((d) { dddd WTW W La} denotes the of d-dimensional orthogonal matrices. Each admits closed-form solution via SVD, with derivation in Appendix A.7. embeddings are then
The aligned external
Ž Z W7, Z
Z W+-
(10)
We iteratively refine Wi and Wi by alternating between updating MNN pairs using CSLS on the aligned embeddings and recomputing Eq. 9 until convergence, i.e., until |[ - w(r || <
After alignment, the internal embeddings remain fixed as the target space, while W1 and Wi- are applied to all external embeddings of the corresponding modality. Each entry in the knowledge base thus contains raw data and its aligned multi-modal embeddings:
Kint fa
Kext = (x/ eext z/,oxt x Text Text
(11)
K K U.xt-
(12)
Thus, modality-leved mapping within D and domain-level alignment across datasets together yield a semantically aligned multimodal knowledge base for retrieval- completion.
3.4 MULTI-MODAL FUSION RETRIEVAL
While RAG has shown strong performance in knowledge-intensive tasks, directly extending it to multimodal missing modality completion poses unique challenges. Conventional approaches of- ten perform cross-modal retrieval (c.g., retrieving images from text). but the inherent modality gap severely limits discriminability, even with powerful joint encoders like CLIP. As shown in Ap- pendix A.4, embeddings of paired image-text samples still occupy distinct subspaces, leading to mismatched candidates and degraded generation quality. This motivates us to design a retrieval mechanism that can both preserve the precision of intra-modal similarity and incorporate cross- modal cues for alignment. To this end, we propose two-stage multi-modal fusion retrieval strategy: (i) mntra-modal top-k retrieval for precise candidate selection, and (ii) multimodal re-ranking with of the missing modality. This fusion mitigates the modality gap and yields more semantically aligned candidates for generation. We illustrate the procedure using the case where an image lacks corresponding text. Let s/f be an unpaired image and 2f Fr(r/)) We obtain pseudo-text embedding via the learned map:
= (1-r(s))
(13)
We first conduct intra-modal retrieval. For each candidate in the knowledge base K. with precom- puted embeddings (z/,, 2f¹) define
Sing (r) cos(2/. =4)-
(14)
5

---

Published as a conference paper at ICLR 2026
Let stable permutation be The top-k candidate index set is
0 = argsort
(- Sing Sing(r); 7).
(15)
= (o(1),..., (kk)), |A|] k.
(16)
For each r E A, compute the fused score
sim = a cos( 2 + (1 - ax) cos ,z,) 2..)
(17)
Re-rank stably by the fused score:
argsort (- sim (r), r), (T));
(18)
rEA
and select the top-1 caption of... Construct the prompt as "Please wrile captions of the image. Caption 1: "x."" Caption 2:" and generate sample with the original image z/:
i (Grz((1/ P).
(19)
The text-only case is symmetric,with detailed formulas provided Appendix A.5
3.5 GENERATION AND CANDIDATE SELECTION
Given the retrieved exemplars, RAG4DMC leverages modality-specinfe generators (e.g., Gizr for image-to-texx and G7æι for text-to-image) to produce multiple candidate completions. This multi- sample generation increases diversity and reduces the risk of degeneration, but also raises the chal- lenge of identifying the most faithful and coherent output. To address this, we introduce a candidate selection mechanism that jointly evaluates semantic consistency and perceptual quality. For image- only inputs, the caption generator G2r produces a set of candidate captions { ...., = We score each candidate with a weighted function that balances semantic alignment and linguistic qual- ity:
8T X - cos (Er(""),,7) + A2 BL",
(20)
where S is the pseudo-text embedding derived from the input image and BLEU measures n-gram overlap with the retrieved exemplar caption off.. For text-only inputs, the image generator Gr2 pro- duces set of candidate images {51 [ 2 Each candidate is evaluated by combining semantic similarity and perceptual quality:
87 5y = cos (Er ~2 NIQB(
(21)
where 2 is the pseudo-image embedding from the input text, and NIQE is no-reference image quality metric (lower is better). The candidate with the highest score is selected as the final comple- tion for the missing modality.
EVALUATIONS
4.1 EXPERIMENTAL SETUP
Datasets. We evaluate RAG4DMC on both general-domain and domain-specific datasets. For the general domain, we adopt MSCOCO (Lin al. 2014) and Flickr30K (Young al. 2014). For the domain-specific setting, we use the RSICD dataset (Lu et al., 2017). From these datasets, we construct modality-mise missing dataset. In our experiments, incomplete samples are introduced in the training phase by applying predefined missing rates to each modality. The evaluation phase is con- ducted exclusively on complete samples. To support retrieval, we adopt external public datasets as knowledge bases. Specifically, CC3M (Sharma et al., (018)) serves the external knowledge base for MSCOCO and Flickr30K, while NWPU-Caption (Cheng et al. (002) used for RSICD, We use BLIP2 (Li et al. 2023) as the text generator and Stable Diffusion XL 1.0 (Rombach et al. 2022) as the image generator across all datasets. We provide more detailed dataset splits and implementation details in the AppendiXA.8
Metrics. Following prior work (Ke et al,, 2025 Jin et al. 2024), we evaluate the quality of the imputed datasets through their utility in downstream tasks. The underlying rationale is that if the

---

Published as a conference paper at ICLR 2026
Table 1: image-Text Retrieval and Image Captioning results on MSCOCO.
Image- Retricval
Image Captioning
Method
12T R@1
T21 R@1
I2T R@5
T21
R@5
CIDEr
BERTScore
Complete
48.9
49.6
82.3
81.0
127.9
92.6
Drop-Incomplete
35.4
35.1
69.5
69.3
109.2
92.1
Direct Generation
41.4
43.9
75.4
76.3
112.2
92.2
GTI-MM
41.0
41.7
74,9
74.9
111.0
92.2
Knowledge Bridger
44.5
76.1
77.5
112.2
Vanilla-RAG
44.9
44.6
77.9
78.0
113.4
92.3
Combined-RRe -RG
44.8
44.6
77.6
77.7
113.0
92.2
46.4
78.1
78.6
RAGADMC
46.6
47.5
79.0
79.7
117.2
92.5
Table 2: Image-Texx Text Retrieval and Image Captioning results on Flickr30K.
Image-Text Text Retrieval
Image Captioning
Method
12T R@I
T21 R@1
I2T R
T21 R
CIDER
BERTScore
Complete
53.2
52.6
81.3
81.1
71.2
91.7
Drop-Incomplete
45.2
47.4
75.9
75.9
64.8
91.3
Direct Generation
47.1
49.6
76.6
77.8
65.9
91.4
GTI-MM
46.3
48.9
75.6
76.0
65.8
91.4
Knowledge Bridger
47.2
49.8
77.3
78.0
66.0
91.4
Vanilla-RAG
48.3
50.2
77.6
78.3
67.3
91.5
Combined- RAG
48.3
50.2
77.3
78.0
66.9
91.5
KFA-RAG
52.7
79.2
79.1
68.2
91.6
RAG4DMC
52.9
53.8
80.6
81.5
70.4
91.7
imputed data preserves semantic fidelity and consistency with the original multimodal samples, models trained on these datasets should achieve better performance on standard benchmarks. We consider two representative tasks: (i) Image-text retrieval, where train CLIP (Radford al.)
2021 model on the augmented datasets and evaluate retrieval accuracy using Recall I and Re- c consistent with prior studies (Hao et al.) 2023] ; Ke et al. 2025 Jin et al. 2024). (ii) Image
captioning, where we fine-tune LLaVA (Liu et al. (023) on the augmented datasets to generate tex-
tual descriptions, and measure caption quality using CIDEr VVdaantam et al. 2013) and BERTScore
(Zhang et al. 2019). following prior work (Li et al. 2022, Wang et al. 2023b Zhang et al. 2019). For training, CLIP is optimized with AdamW (learning
Te-4, 4, batch size 16) for 20 epochs.
LLaVA is fine-tuned with LoRA adapters using AdamW (learning rate 2e-5, 5, effective batch size 2) for 2 epochs. All evaluations are conducted the standard test splits: 1,000 samples for MSCOCO
and Flickr30K, and 1,093 samples for RSICD.
Baselines. Since data-level modality completion based on generative models is largely under- explored, we design the following three categories of baselines: (i) Naive baselines: Drop- Incomplete, discards all samples with missing modalities. (ii) Direct generative-basee methods: Direct Generation, which fills missing modalities using generative models without retrieval ground- ing, GTI-MM et al., al, [2224), which uses generative models to impute missing visual modalities and designs prompts control the generation quality, and Knowledge Bridger (Ke et al.) 2025), which extracts knowledge from the available modalities and then guides a generative model to com- plete the missing ones. (iii) RAG-based methods: Vanilla-RAG. which builds the knowledge base from internal data only and performs naive cross- mroal retrieval: Combined-RACE which builds the knowledge base by directly merging internal and external datasets without filtering alignment: and KFA-RAG, which constructs an integrated knowledge base using the same method in RAG4DMC but performs naive cross-modal retrieval. This design also allows
dissect the contribution of each component: Combined-Re -AAG vs. Vanilla-RAG highlights the benefit of incorporating external knowledge; KFA-RAG vs. Combined-RAg demonstrates the importance of filtering and alignment in knowledge base construction; and finally, RAG4DMC vs. KFA-RAG validates the effectiveness of our proposed retrieval mechanism.
4.2 RESULTS AND ANALYSIS
4.2.1 MISSING MODALITY COMPLETION PERFORMANCE
We first evaluate the missing modality completion performance on MSCOCO, Flickr30K, and RSICD datasets. Tables D summarize performance downstream image- text retrieval and cap- tioning tasks using datasets completed by different methods. It can be observed that removing incomplete samples (Drop-lnccmpplet) leads substantial performance degradation. For example,

---

Published as a conference paper at ICLR 2026
Table 3: Image-Text Retrieval and Image Captioning results on RSICD.
Image- Text Retricval
Image Captioning
Method
12T R@1
T21 R@1
12T R@5
T21 R
CIDEr
BERTScore
Complete
10.3
10.3
30.4
30.1
34,0
89.1
Drup-Incomplete
8.8
25.3
27.9
30,9
88.8
Direct Generation
8.5
9.5
25.9
28.5
32.5
89.0
GTI-MM
8.3
9.1
24.3
27.7
32.1
89.0
Knowledge Bridger
9.5
26.3
28.9
32.8
Vanilla-RAG
4.8
5.1
18.5
18.7
19.4
87.8
Combineed-RAn RAG
4.8
5.2
18.2
18.8
18.1
87.6
5.6
20.2
20.3
2016
88.1
RAG4DMC
8.7
9.7
27.6
29.6
33,9
89.1
Avg R@ on MSCOCO drops from 49.2 to 35.2, which highlights the necessity of modality comple- tion. Introducing retrieval grounding via Vanilla-RAG improves performance (+2.2 Avg R@ +2.1 Avg R@5 on MSCOCO), demonstrating the benefit of exemplar guidance. However, naively adding external data (Combined-RACe vs. Vanilla-RAG) does not yield gains because of noise and domain mismatch. Filtering and aligning external knowledge (KFA-RAG vs. Combined-RAGG consistently boosts performance (+1.5 Avg R@] +0.7 Avg R@5), confirming the effectiveness of our knowl- edge processing. Finally, integrating multi-modal fusion retrieval in RAG4DMC further enhances results (Avg R@ = 47.1, Avg R@5 79.4), approaching the oracle upper bound and validating our retrieval design.
On the domain-specific RSICD dataset, standard RAG variants degrade due representation bias in the knowledge base. Since CLIP encoders are not trained on remoin-sensing data, the extracted image and text features exhibit a modality gap, leading inaccurate retrieval and reduced comple- tion quality. Comparing RAG4DMC with KFA-RAG shows that integrating our multi-modal fusion retrieval effectively mitigates this issue and substantially restores performance.
For image captioning, we observe that Combined-RAGG performs worse than Venilla-RAG, with CIDEr decrease of 1.3% on RSICD dataset. This is because naively merging external samples introduces noise that harms linguistic consistency, captioning requires accurate word-level supervi- sion and is therefore more sensitive to dataset quality. In contrast, KFA-RAG alleviates this problem through filtering and alignment, improving CIDEr from 1 113.0 to 1 115.8 on MSCOCO. Furthermore, training LLaVA VA on datasets completed by our method achieves the highest captioning performance, further boosting CIDEr and BERTScore. These results indicate that our completions provide faith- ful, semantically coherent, and dataset-consistent supervision that benefits not only retrieval but also downstream generative tasks. These findings highlight both the necessity of modality completion and the effecctivnees of our proposed design across general and domain-specific datasets.
4.3 PERFORMANCE UNDER DIFFERENT MISSING RATES
To evaluate robustness against different degrees of missing information, we conduct experiments on MSCOCO with modality missing rates ranging from 10% to 70%. As shown in Table all methods experience performance degradation as the missing rate increases, but the decline rates differ signif- icantly. Generation-base methods (Direct Generation, GTI-MM) deteriorate the fastest, reflecting their reliance on generative filling without sufficient grounding, which amplifies errors when larger portions of data are absent. In contrast, Vanilla- RAG demonstrates greater stability by leveraging emplar grounding from internal data, while KFA-RAG further enhances robustness by filtering noisy external samples and aligning them with internal exemplars. RAG4DMC consistently surpasses all baselines across missing rates, with particularly pronounced gains under high-mi figh-missing settings. For example, at 70% missing rate, RAG4DMC improves Avg R@ by +2.3 over Ainilla-RAG. highlight- ing its ability to deliver faithful and reliable completions even in highly incomplete scenarios. These results demonstrate the effectiveness and robustness of RAG4DMC, showing that its integration of modality completion, retrieval grounding, and knowledge filtering enables consistent performance even under severe data scarcity.
4.4 IMPACT OF GENERATION CANDIDATE NUMBER
In RAG4DMC, the generation and candidate selection module introduces hyperparameter n, which determines the number of generated candidates. To investigate its effect, conduct experiments on MSCOCO with different n values (n E (1, 3, 5, 10)). As shown in Table the performance of RAG4DMC consistently improves increases. For example, Avg @ of RAG4DMC rises from
8

---

Published as conference paper at ICLR 2026
Table 4: Performance of RAG4DMC on MSCOCO under different modality missing rates.
Image- Text Retrieval
Image Captioning
MissRate
Method
12T R@
R@1 1
T21
R@
12T R@5
T21 R@5
CIDEr
BERTScore
Drop-Incomplcte
49.3
49.1
50.7
19.7
128.2
92.4
Direct Generation
50.5
51.3
81.6
81.4
129.7
92.7
GTI-MM
49.9
49.7
80.9
80.2
128.9
92.6
Knowledge Bridger
50.9
81.9
81.9
129.5
92.6
Vanilla-RAG
51.7
518
82.4
82.6
130.4
92.7
Combined-RAG
51.6
51.8
82.3
82,4
129.7
92.6
KBA-RAG
52.7
52.9
83.9
83.6
131.7
92.7
RAG4DMC
53.9
85.6
85.5
133.8
Drop-Incomplete
46.7
46.4
78.7
77.8
120.5
92,4
Direct Generation
47.9
48.1
80.3
80.6
121.7
92.6
GTI-MM
47.5
92.5
0.3
Knowledge Bridger
48.2
49.1
80.5
81.1
121.8
92.6
Vanilla-RAG
48.3
49.3
81.2
81.5
123.2
92.6
Combined-Ran
48.1
49.2
80.9
81.5
92.6
KEA-RAG
50.2
51.7
82.3
83.0
124.7
92.7
RAG4DMC
52.3
52.9
84.1
84.4
125.8
92,7
Drop-Incomplete
42.7
45.6
76.9
78.5
116.9
92.4
Direct Generation
45.5
45.8
77.9
118.5
92.4
GTI-MM
44.3
44,9
76.3
76.6
118.2
92.4
Knowledge Bridger
45.3
45.8
77.3
78.5
118.6
92.5
Vanilla-RAG
45.8
46.1
78.3
78.9
119.8
92.5
Combined-Ra. RAG
45.6
46.1
78.0
78.7
119.6
92.5
KEA-RAG
46.7
47.1
79.4
79.2
121.6
92.5
RAGADMC
92.6
Drop-Incomplete
33.4
35.1
69.5
69.3
109.1
92.1
Direct Generation
41.4
43.9
75.4
76,3
112.2
92.2
GTII-MM
74,9
42.2
0.7
Knowledge Bridger
42.5
44.5
76.1
77.5
112.2
92.2
Vanilla-RAG
44.9
44.6
77.9
78.0
113.4
92.3
Combined-Re
77.7
92.2
KEA-RAG
45.9
46.4
78.1
78.6
115.8
92.3
RAGADMC
46.6
47.5
79.0
79.7
117.2
92,4
Table Performance of RAGADMC on MSCOCO with different generation candidate number n.
Image-Text
Retrieval
Image Captioning
12T R@1
T21 R@1
12T R@S
T2I R@S
CIDEr
BERTScore
46.6
47.1
48.0
79.4
79.8
117.9
92.4
47.4
48.2
119.3
10
47.7
48.5
80.1
80.3
121.5
92.6
47.1 at n to 48.1 at n = 10, while CIDEr rises from 117.2 to 121.5. This demonstrates that
RAG4DMC effectively identifies the best candidate among multiple generations, filtering noisy outputs and ensuring that diversity translates into real performance gains.
4.5 IMPACT OF RETRIEVAL SIZE
We further investigate the effect of the retrieval hyperparameter k. which controls the number of exemplars retrieved for RAG. As shown in Table increasing from 10 steadily improves performance, since larger candidate pool increases the likelihood of including highly relevant exemplars, enhancing the quality of generation. However, when & becomes too large (e.g.. 15), performance slightly degrades, likely due to the introduction of less relevant or noisy exemplars, which can misguide the generator. This suggests that a moderate number of high-quality retrievals strikes good balance between diversity and precision. We set k 10 in all main experiments based on this trade-off.
4.6 SENSITIVITY TO KNOWLEDGE FILTERING THRESHOLDS
We conducted additional sensitivity experiments by varying Tcent and Tinst within reasonable range (0.6-0.8). The results (Table 7p show that retrieval and captioning performance remains highly stable: the changes in R@ 1, R R CIDEr, and BERTScore are minimal, indicating that our filtering strategy robust these thresholds.

---

Published as a conference paper at ICLR 2026
Table 6: Performance RAG4DMC with dif-
Table 7: Performance of RAG4DMC with dif-
ferent retrieval size k.
ferent filtering thresholds.
12T
T21
12T
T21
CIDEr
BERT Score
Filtering
12T
T31
12T
T21
BERT
R@5
R@5
Threshold
R@1
R@1
R@5
R@5
CIDEr
Score
3
46.5
46.1
77.7
76.6
109.4
92.1
0.6
46.3
79.0
117.0
92.4
5
46.6
47.5
79.0
79.7
110.7
92.2
0.7
46.6
47.5
79.0
79.7
117.2
92.5
0.8
46.9
47.6
79.6
79.9
117.2
92.5
15
41.9
42.9
75.0
76.7
104.1
92.5
4.7
IMPACT OF EXTERNAL KNOWLEDGE BASE SIZE
To investigate how performance varies with the size of the external knowledge base, conducted experiments on MSCOCO using external samples ranging from 5k to 15k. As shown in Table g the results demonstrate a monotonic improvement the size of the external KB increases, though with diminishing returns at larger sizes. This suggests that while a larger external KB offers richer exemplars and broader coverage, our method does not depend on extremely large corpora to perform performance is already achieved with just 5k external samples.
Table 8: Performance of RAG4DMC with different number of external knowledge base samples.
Number External KB
12T R@1
T21 R@1
12T R@
Res
CIDEr
BERTScore
5000
46.0
46.6
78.6
116.4
92.4
10000
46.6
47.5
79.0
79.7
117.2
92.5
15000
47.1
48.1
79.3
80.1
117.5
92.7
4.8 COMPLEXITY ANALYSIS
In Appendix A.9 we provide detailed theoretical complexity analysis for each component in pipeline. As shown there, the dominant factors are the number of samples N, the number of clusters k, and the feature dimension d. We supplemented experiments measuring the runtime for each module under varying N, k, and d, NVIDIA GeForce RTX 3090 GPU.
As summarized in Table g under the largest tested configuration (N = 1k+10k, k = 128, d = 1024), the end-to-end inicluine-iccludinn knowledge filtering, cross-domain alignment, and bidirectional MLP mains extremely lightweight (0.0031 s). Thus, although the pipeline involves clustering and nearest- neighbor search, the overall computational overhead is clearly manageable in practice. Table p further breaks down the major sources of computation:(i) K-means is most sensitive to and k consistent with its standard computational behavior; (ii) Cross-domain alignment scales primarily with d, since nearest- neighbo search performed the feature space; (iii) Bidirectional MLP training dominates the total runtime but is largely insensitive to N and k, it operates only in- ternal feature representations and mainly driven by d: (iv) Filtering and retrieval incur negligible cost across all configurations. Overall, these results demonstrate that the scaling behaviors of all modules are predictable and well controlled. Importantly, none of the components introduce unex- pected or prohibitive overhead. Moreover, users may flexibly adjust N, k. and d to trade off between efficiency and accuracy, enabling deployment under different computational budgets.
in approximately 629 s (10.48 minutes), while retrieval re-
Table 9: Time cost (in seconds) of each module under different settings.
K-means Clustering
Donss- Cruss-Modal Alignment
Multi-Modal Retrieval
Configuration
Filtering
Mapping
N
1k 5k,
128, d 512
21.43
0.30
0.40
405.17
0,0012
N 100,
128, d 512
39.14
0.64
0.55
407.14
0.0014
N -200,
128, d 512
112.87
1.35
0.99
406.25
0.0017
N 1k 10K, 64, 512
32.98
0.58
0.52
408.95
0,0015
N 1k 10k, 256, d 512
46.97
0.78
0.54
408.98
0,0015
N 1A 10k, k 128, d 768
53.49
0.82
1.43
499.29
0.0018
N
IK
+ 10k, k 128, d 1024
92.20
0.88
2.01
533.85
0.0031
5 CONCLUSION
This paper proposes RAG4DMC, a earieval-augmented framework for data-level missing modal- ity completion. Our approach leverages dual knowledge base and multi-modal fusion retrieval with semantic-qualin quaitii--uualtttbbeeeeeee -bsed candidate selection to generate faithful and semantically coherent com- pletions. Extensive experiments both general-domain and domain that RAG4DMC consistently outperforms existing baselines, significantly improving the training performance of downstream models on image-text retrieval and captioning tasks.
datasets demonstrate
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research was supported in part by the National Key R&D Program of China under Grant 2024YFB4207203: in part by the National Natural Science Foundation of China under Grant No. 62402267, 62572496, and 52401376; part by the Shenzhen Science and Technology Program under Grant JCYJ20250004115500000 Program of Zhejiang under Grant 2024C03254.
and in part by the "Pioneer" and "Leading Goose" R&D
REFERENCES
Mohammad Mahdi Abootorabi, Amirhosein Zobeiri, Mahdi Dehghani, Mohammadali Mohammad- khani, Bardia Mohammadi, Omid Ghahroodi, Mahdieh Soleymani Baghshah, and Ehsaneddin Asgari. Ask in any modality: A comprehensive survey multimodal retrieval-augmented gen- eration. arxiv preprint arXiv:2502.08820 08826, 2025.
Qimin Cheng, Haiyan Huang, Yuan Xu, Yuzhuo Zhou, Huanying Li, and Zhongyuan Wang. Nwpu- captions dataset and mica-net for remote sensing image captioning. IEEE Transactions on Geo- science and Remote Sensing, 60:1-19, 2022.
Tiantian Feng, Daniel Yang, Digbalay Bose, and Shrikanth Narayanan. Can text-to-image model assist multi-modal learning for visual recognition with visual modality missing? In Proceedings of the 26th International Conference on Multimodal Interaction, pp. 124-133, 2024.
Zirun Guo, Tao Jin, and Zhou Zhao. Multimodal prompt learning with missing modalities for sentiment analysis and emotion recognition. arXiv preprint arXiv:2407.05374, 2024.
Xiaoshuai Hao, Yi Zhu, Srikar Appalaraju, Aston Zhang, Wanqian Zhang. Bo Li, and Mu Li. Mix- gen: A new multi-moda data augmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 379-389, 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Gautier Izacard and Edouard Grave, Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282. 2020.
Tianzhe Jiao, Chaopeng Guo, Xiaoyue Feng, Yuming Chen, and Jie Song. A comprehensive sur- vey deep learning multi-modal fusion: Methods, technologies and upplications. Computers, Materials & Continua, 80(1), 2024.
Xiaomeng Jin, Jeonghwan Kim, Yu Zhou, Kuan-Hao Huang, Te-Lin Wu, Nanyun Peng, and Heng Ji. Armada: Attribute- based multimodal data augmentation. arXiv preprint arXiv:2408. 10086, 2024.
Guanzhou Ke Shengfeng He, Xiao Li Wang, Bo Wang, Guoqing Chao, Yuanyang Zhang, Yi Xie, and HeXing Su. Knowledge bridger: Towards training-free missing multi-modality completion. arXiv preprint arXiv:2502.198.. 2025.
Guillaume Lample, Alexis Conneau, Mare' Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. In International conference on learning representations, 2018.
Jian Lang, Rongpei Hong, Zhangtao Cheng, Ting Zhong, Yong Wang, and Fan Zhou. Redeeming modality information loss: Retrieval-guided conditional generation for severely modality missing learning. In Proceedings of the 31st ACM SIGKDD Conference Knowledge Discovery and Data Mining V. 2, pp. 1241-1252, 2025.
Yi-Lum Lee, Yi-Hsuan Tsai, Wei-Cher Chiu, and Chen Yu Lee. Multimodal prompting with missing modalities for visual recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14943-14952 2023.

---

Published as a conference paper at ICLR 2026
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- -arreval-auggented gener- ation for chooledge-intenssee nlp tasks. Advances in neural information processing systems, 33: 9459-94744 2020.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. In International conference on machine learning, pp. 12888-12900. PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Weizhe Lin and Bill Byrne. Retrieval augmented visual question answering with outside knowledge. arXiv preprint arXiv:2210.03800 2022.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
Xiaoqiang Lu, Binqiang Wang, Xiangtao Zheng, and Xuelong Li. Exploring models and data for remote sensing image caption generation. IEEE Transactions on Geoscience and Remote Sensing, 26(4):2183-2195, 2017.
Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. Smil: Multimodal learning with severely missing modality. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 2302-2310, 2021.
Vittorio Pipoli, Alessia Saporita, Federico Bolelli, Marcella Cornia, Lorenzo Baraldi, Costantino Grana, Rita Cucchiara, Elisa Ficarra, al. Missrag: Addressing the missing modality challenge in multimodal large language models. In Proceedings of the 2025 IEEE/CVF International Con- ference on Computer Vision, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 7768-8763. PmLR, 2021.
Monica Riedler and Stefan Langer. Beyond text: Optimizing rag with multimodal inputs for indus- trial applications. arXiv preprini 14X22:2410.21944, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypemymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume Long Pupers), pp. 2556-2565, 2018
Ramakrishna Vedantam, C Lawrence Zitncck and Devi Parikh. Cider: Consensus-basse image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4566 4575, 2015.
Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Carneiro. Multi- modal learning with missing modality via -hared-specifie feature modelling. In Proceedings of the IEEE/CV conference computer vision and pattern recognition, pp. 15878-15887, 2023a.
Ning Wang, Jiangrong Xie, Hang Luo, Qinglin Cheng, Jihao Wu, Mingbo Jia, and Linlin Li. Ef- ficient image captioning for edge devices. In Proceedings of the AÅAI conference on artificial intelligence, volume 37, pp. 2608-2616 2023b.
12

---

Published as a conference paper at ICLR 2026
Yuanzhi Wang, Zhen Cui, and Yong Li. Distribution-consitterr modal recovering for incomplete multimodal learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20025,2 2023c.
Peng Xu, Xiatian Zhu, and David A Clifton. Multimodal learning with transformers: A survey.
IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(10):12113-22132 2023.
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockemmaier. From image descriptions to visual denotations: New similarity metries for semantic inference over event descriptions. Trunsactions of the association for computational linguistics, 2:67-78, 2014.
Yuan Yuan, Zhaojian Li, and Bin Zhao. A survey of multimodal learning: Methods, applications, and future. ACM Computing Surveys, 57(7):1-34, 2025.
Tao Zhang. Ziqi Zhang, Zongyang Ma, Yuxin Chen, Zhongang Qi, Chunfeng Yuan, Bing Li, Junfu Pu, Yuxuan Zhao, Zehua Xie, al. mr 2 ag: Multimodal aarreval-refieciioo-agmeeet genera- tion for knowledge-bett baaee vqa. arXiv preprint arXiv:2411.1504 2024.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluat- ing text generation with bert. arXiv preprint PXXiv:190400960 2019.
Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recog- nition with uncertain missing modalities. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pp. 2608-2613 2021.
Shu Zhao, Xiaohan Zou, Tan Yu, and Huijuan Xu. Reconstruct before query: Continual miss- ing modality learning with decomposed prompt collaboration. arXiv preprint arXiv:2403. 11373, 2024.
Wenqian Zhao, Kai Yang, Peijin Ding, Ce Na, and Wen Li. Graph attention contrastive learning with missing modality for multimodal recommendation. Knowledge- Based Systems, 31 311:11100 113:51 2025.
A APPENDIX
A.1 THE FRAMEWORK OF RAG4DMC
The complete algorithmic procedures are provided in this section. Specifically, Algorithm | de- scribes the construction of the dual knowledge base, while Algorithm 2 outlines the overall inference workflow of mudti-modal fusion retrieval and candidate selection.
A.2 CLUSTERING- BASSDD KNOWLEDGE FILTERING FOR TEXT EMBEDDINGS
For the text modality, the internal and external text embeddings denoted as gT.iit and exxt and processed analogously to the image modality. We apply K K-means to obtain cluster centroids, computed as the average embedding:
int Hp
Tint z
Text Pg
Txett. Zm
(22)
Tiint
Teext Cig
Cp
Eop
Teett 2m
Txxt Eo
Each external centroid M4
Txxxt is matched to its closest internal centroid p
Tintt
p arg max cos (N² Text, M Tiin)).
(23)
and we define centroid-level and instance-level similarities as ST cos (TT.xxt
Scent cos (n, Text Ppp
T.int),
Texxt T Tiint Ppp
(24)
where 2 ext E cf Txxt Clusters and instances are then filtered hierarchically; we first prune clusters with low centroid similarity and then filter embeddings within the retained clusters:
keep
= [c Txxt
b 2 Tceat)
(25)
2keep
{2TText
Text T 2.m
Textt
T Tinsts oy Teext E )
(26)
13

---

Published as a conference paper at ICLR 2026
Algorithm The Framework of Dual Knowledge Base Construction
Require: Internal data D. external data Dext: frozen encoders E,, ET; mapping MLPs
II-T:: .T--; number of clusters K: thresholds Trent: Tinsti tolerance
Ensure: Multi-modal knowledge base K. trained mapping MLPs [I-T; ST-1
1: Construct paired sets: Pint {(I; .int T.int) }}. Pext {arhe x xxText)) )).
2: for (I, ( Piint, Pint do
224
3: Lint + Ei(r; ,int Tiitt f ET (x, )
>> Eq.
4: end for
5: for (x) I' Txxt)) E Pext do
6: 2m
L.ex < E1 (extt)) Txxtxt Er (x (xx)))
Eg
7: end for
8: // Cross-modal bidirectional mapping
9: Train JT on internal pairs (2g int _.intt)
>> Eq.
10: // Clustering-baser knowledge filtering (image modality)
11: Run K-means on {a (in)) and [2/1ext] to obtain centroids {uz 1p=1
int K (xtt 1 /q=1
Eq
12: for each external cluster q =l...., do
13: P/ aag maxp cos( cos(11g
(og Lext 11..in)
> Eq.
Scent (cg
Pp
7,int)
>> Eq.
if sent Teent then
16: drop cluster y
17:
else
18: keep cluster g, add to Cloep
19: end if
20:
end for
21: for each cluster cy dfeext E keeep do
for each sample IT,ext E cg do L'ustt cos( 2m Lext
23:
Eq.
24:
if / Tinst then %eett
25:
drop pm
26:
else
keep pm 28: end if
27:
Text
29:
end for
30: end for
Repeat the same procedure for embeddings (zT.int, x2Txxtt to obtain CT and Zeeop (see Appendix A.2)
32: // Cross-domain alignment (Procrustes, modality-speccffic)
33: Build image MNN pairs {(2
_L,ext] }} and text MNN pairs {(z] Tiint _T,xxt)) using CSLS. Z
34: Solve W = arg min EO(4) || Zux WI Wi = arg minweo(s) 1 Zoo WII Eq.9
35: Iteratively refine Wf, Wi until convergence
|W(r) w(r <c
36: // Knowledge base construction
37: Project external embeddings: Z _ Wi, 7T = ZT W
Eq. 10
38: Kint = {x-, It ",x"",, f Kext. {a {x-,ext ,an,zzz,,, z1,ext x eext = eext
Eq
39: K. Kint UK Kext
Eg
12
40: return K. fI-TT,f--
A.3 ANALYSIS OF CLUSTERING-BASED KNOWLEDGE FILTERING
Imbalanced clusters. small or imbalanced clusters, outliers may severely bias the centroid pes rendering centroid-based decisions unreliable. For cluster with size |c| and sample embeddings (z(z) x c). its centroid is He. We compute the average cluster radius as
Σ [1(()) - Pell2,
Uc
(27)
lcl
zEc
14

---

Published as a conference paper at ICLR 2026
Algorithm 2 The Workflow of Multi-Modal Fusion Retrieval and Candidate Selection
Require: Knowledge base K; encoders E,, Er; maps fI-T, fr-15 generators G (inage->text,, Gr25 (text=immag); retrieval size k; fusion weight at number of candidates weights A1
Ensure: Completed internal dataset D
1: Image Text
2: 2 =Er(;)) = ())T(())
Eq.
13
3: for candidate r in knowledge base K. do
4: Sing - (05(2),22))
Eq.
14
5: end for
6: a f- argsort, (-mimg
Sing (r),
(a(1),.... (
Eq.
15
16
7: for r A do
8: sim (r) a cos( c 2/) + (1 ax) cos( (07 -2,,)
Eq.
17
9: end for
10: π + ae (r),
Eq.
18
11: Reorder and select top-1 caption x77.
12: Build prompt P
13: Generate n caption candidates [ + G)2r(z),,)) P)
>> Eq.
19
14: // Text - Image
15: z/h = =E((=))
fT-
Eq.
29
for candidate knowledge base K. do
17: Stea(r) t ,
b Eq.
30
end for
19: σ + argsort, S
(r), (-swat A + ((())......kk))
Eq.
30
for A
21: sim (r) < cos( ( of)
+ a) cos((2)) 2+)
Eq.
31
22: end for
23: Reorder A and select top-1 image x/-
24: Generate n image candidates
3 + Gr21 ( xf.))
Eq. 32
25: // Candidate Selection
26: // Text candidates
27: (T
λ cos( Er (T )²)) ABBLEU()))
Eqq
20
28: // Image candidates
sy sz = \₁
21 (IQE
Eq.
21
30: Select the candidate with the highest score as the final completion
31: return D
and let ū denote the average radius across all clusters. A cluster is regarded imbalanced if uu B u, with 3 = 2 our experiments. For such clusters, we adopt more robust representatives instead of plain centroids. Specifically, we use the Ve, defined as
Vc = arg min
E1la(1) - 21v)l2
(28)
DEC
VEc
which is the most central sample c and less sensitive to outliers.
Sensitivity to thresholds. We further analyze the impact of Term and Tims- If the thresholds set too high, only very close samples are retained, reducing coverage and harming downstream per- formance. If the thresholds set too low, many noisy or irrelevant external samples are included, introducing excessive noise and again degrading performance. Our experiments show that perfor- mance improves as the thresholds increase up to a moderate range, but deteriorates once they are either too strict or too relaxed. Based on this observation, we set both thresholds to 0.7, which provides a stable trade-off between precision and recall.
A.4 VISUALIZATION OF KNOWLEDGE ALIGNMENT
To illustrate the necessity and effectiveness of our knowledge filtering and alignment, visualize the embeddings the internal dataset (MSCOCO) and an external dataset (CC3M) using t-SNE before and after alignment.
15

---

Published as a conference paper at ICLR 2026
Before alignment. As shown in Fig. 2 the embeddings from the two datasets form distinct clus- ters, reflecting the domain gap. Although semantically related (both are natural image- corpora), the representations not directly compatible.
After alignment. As shown in Fig. 2 after applying knowledge filtering and alignment, the external embeddings are rotated into the internal space, and semantically similar samples from both datasets become much closer, indicating successful mitigation of domain shift.
MECSETIA Snagerk
1911 Pecaataatto MMECEO
MNCOC
Figure 2: t-SNE visualization of MSCOCO and CC3M embeddings before (left) and after (right) alignment.
A.5 MULTI-MODAL RETRIEVAL FOR TEXT-ONLY CASE
Given an unpaired text TT,,, we extract its embedding zx (x,) and obtain pseudo-image embedding via the learned mapping:
fr
(29)
We perform intra-modal retrieval over by text similarity and take the top-k candidates:
2/ T (r) = cos(
argsort (ren(r), r), A = (o(1), (k)), |A| = k. (30)
For E A, we compute fused score and pick the best exemplar image:
simaue (r) cos (2 2 +
a) cos( 2/, of), r" arg max sim (r). rEA
(31)
Finally, we feed the text nr and the retrieved reference image x/. into the text-to-image generator Gr2 to produce the completed sample:
r' = (Tr
(32)
A.6 DETAILS ON PROCRUSTES ALIGNMENT
In practice, all filtered external embeddings are considered during mutual nearest neighbor (MNN) matching. If an external embedding is linked to multiple internal embeddings, we retain only the pair with the highest CSLS score to ensure a one-to-one mapping. When the internal and external datasets differ greatly in size, we randomly subsample the larger side to balance the number of pairs used for alignment. Since Procrustes alignment is based on one-to-one MNN pairs, the number of effective seed pairs bounded by the size of the internal dataset. Although this means that at most [D] external embeddings are directly used for learning the mapping. having a large external pool remains beneficial as it increases the chance of finding reliable MNN pairs. After learning the alignment from these one-to-one pairs, the resulting Procrustes transformation applied to the entire external knowledge base, projecting all external embeddings into the internal space for subsequent retrieval.
16

---

Published as a conference paper at ICLR 2026
A.7 THEORETICAL DERIVATION OF ORTHOGONAL ALIGNMENT
The goal of cross-domain alignment is to map external embeddings Zex E Raxd into the internal embedding space Zint E Faxd while preserving geometric structure. Formally, this corresponds to solving the orthogonal Procrustes problem defined in Eq. [9] For clarity, the following derivation presented without modality superscripts, but applies equally to both image (Z") and text (Z²) embeddings.
Closed-form solution.
Expanding the Frobenius norm, the optimization is equivalent to
max Tr(WT M), where M Z Zint.
WEO(d)
Let the singular value decomposition (SVD) of M be
M UEVT.
with V E Raxd orthogonal and Bad diagonal. The optimal alignment matrix is
-- UV
which yields the aligned external embeddings given in Eq. TO
Iterative refinement. Since exact correspondence between Zexa and Zime is unknown, we refine W iteratively following (Lample et al. 2018): (i) update mutual nearest neighbor (MNN) pairs using CSLS on the current aligned embeddings; (ii) recompute W via the SVD step above; (iii) repeat until convergence, i.e.,
| w(+ <E,
for a small threshold e, where s indexes the iteration.
Domain-consistent embedding space. After convergence, internal and external embeddings are merged into unified space. The internal embeddings remain fixed the anchor space (standard- ized by their own mean and scale), while the learned mapping W applied solely to external embeddings, ensuring domain-consistent representations.
A.8 DATASET SPLITS AND IMPLEMENTATION DETAILS.
Dateset Splits. From MSCOCO, we sample 10k training instances and construct modality dataset where each modality has 35% missing rate, resulting in a total missing rate of 70%. For Flickr30K, we use 5k training instances with 20% missing rate per modality, yielding an overall missing rate of 40%. For RSICD, all 8,743 training samples are utilized with a 20% missing rate per modality, corresponding to a total missing rate of 40%. Since both CC3M and NWPU-Caption are large-scale, randomly sample 10k entries from each to construct the external knowledge base.
Implementation Details. For knowledge aligmment, we cluster embeddings with KMeans (A: 128), apply thresholds Teent, Tinst 0.7, and refine the alignment using Procrustes analysis (5 iterations). During knowledge-constralne 0.5 for conditioning. For candidate selection, we \ 0.7 and λa 0.3 to balance semantic similarity and quality scores.
completion, we retrieve top-k = 10 candidates and a =
A.9 COMPLEXITY ANALYSIS
We analyze the asymptotic cost of each component. Let N be the number of entries (per modality) in the knowledge base, d the embedding dimension. the number of clusters, s the number of alignment refinements, & the retrieval shortlist size, the number of generated candidates, and P = Punt the number of internal image-text pairs used to train the mapping.
Cross-modal mapping (fr-sT: fr-))). Training the lightweight MLPs on P paired samples scales linearly in both P and d. With epochs (and fixed-width MLP), the cost is O d2).
17

---

Published as a conference paper at ICLR 2026
Knowledge filtering (clustering + thresholding). K-means over N embeddings per modality costs ( Kd). Matching cluster centroids and computing centroid/instance similarities add at most O(K2d) O(Nd), so the filtering step remains dominated by ONN
Cross-domain alignment (MNN + Procrustes). Let Ny be the number of filtered embeddings that enter alignment (per modality). Per refinement step:
Mutual nearest neighhor (MNN) construction with CSLS requires nearest-neighbor search across domains: O(N)d).
Procrustes: form M Z Zim in ONN)d") then SVD on d d in O(d)).
With 8 refinements, the alignment cost is Oss Njd + Nyd + d)
Retrieval-based completion (two-stage retrieval + candidate scoring). Given query embed- ding (image or (ex)):O(Nd) (kd) + ((dd).
Overall. Summing the components yields O (()) + O(N + O(Nd + kd + nd) In typical settings with fixed d < and small k, n, the dominant terms are Od) (filtering), the MNN search inside alignment, and the intra-modal retrieval term.
If +
A.10 EXPERIMENTS WITH STRONGER GENERATORS
To examine the effect of stronger generation models, we conducted additional experiments using enhanced image captioning and text-to-image models. As shown in Table 10 upgrading the gener- ators (e.g., from BI.IP2 Stable Diffusion XI. 1.0 to LLa (Liu improves the performance of Direct Generation across both retrieval and captioning metries. No- tably, RAG4DMC achieves 49.1 / 49,4 R and 117.9 CIDEr, outperforming the enhanced Direct Generation baseline by clear margin. These results demonstrate that while stronger generators can enhance Direct Generation, RAG4DMC consistently provides additional gains on top of any generation backbone. This reinforces that our method improves modality completion beyond what generator quality alone can achieve.
2023 + Juggernaut-XI,
Table 10: Performance of Direct Generation and RAG4DMC with different generation models.
Method
Generation Models
12T R
T21 R@1
12T R@S
T21 R@S
CIDEr
BERTScore
Generation
Alip2 and Stable Diffusion-XI
43.9
76.3
Direct Generation
LLaVA 1.5 and Juggernaut XL.
45.8
47.2
78.3
78.0
113.0
92.2
RAG4DMC
Blip2 and Stable Diffusion-X
RAGADMCC
LLaVA 1.5 and Juggermaut XL.
49.1
49.4
81.7
81.4
117.9
92.6
A. ABLATION STUDY ON MULTI-MODAL FUSION SCORE
To evaluate whether the proposed multi-modal fusion score (simuse) provides additional benefits beyond standard top-k retrieval, we conducted an ablation study comparing three settings: (i) Top-k: One exemplar is randomly selected from the retrieved top-k candidates and fed into the generator; (ii) Top-k-all: All & retrieved exemplars are fed into the generator (i.e., no fusion and no selection); (iii) RAG4DMC: Our proposed fusion score is used to select the best exemplar among the & candidates and feed it into the generator.
As shown Table [ Top-k-all performs worst, likely because feeding all retrieved samples into the generator introduces excessive noisy context. Top-k performs better, but RAG4DMC achieves the best results across all metrics, improving both retrieval (e.g., +0.6 R@ over Top-k) and captioning (e.g., +1.0 CIDEr). These results demonstrate that the fused score effectively selects the most semantically aligned exemplar, yielding consistent performance gains over naive top-k retrieval strategies.
A.12 ABLATION STUDY ON CROSS-DOMAIN ALIGNMENT ITERATIONS
To evaluate the effect of varying the number of cross- domain alignment iterations, we conducted experiments with 3, and 5 iterations. As shown in Table 12 varying the number of iterations
18

---

Published as a conference paper at ICLR 2026
Table 11: Performance of RAG4DMC with different retrieval methods.
Method
12T R@I
T21 R@1
12T
R@B
T21 R
CIDEr
BERTScore
Top-k
46.0
46.7
78.4
78.8
116.0
92.3
Top-k-all
45.4
76.2
115.5
92.2
RAG4DMC
46.5
47.3
78.9
79.5
117.0
92.5
from 3 to 5 results in only marginal performance improvements (e.g., I2T R@I: 46.2 - 46.6; CIDEr: 116.3 + 117.2), demonstrating that the method is not highly sensitive to this parameter.
Table 12: Performance RAG4DMC with different number of cross-domain alignment iterations.
Number Ieeattions
T21
RéS
R
CIDEr
BERTScore
46.2
-6.7
78.5
78.8
116.3
92.4
46.4
46.9
78.8
79.2
116.8
92.4
A.13 ABLATION STUDY CANDIDATE SELECTION WEIGHTS (A1 AND A2)
To examine the effect of candidate selection weights, conducted experiments by varying A and A2. As shown in Table [3] varying these weights leads to moderate but controlled changes in performance. Importantly, RAG4DMC continues outperform baseline methods across all tested settings.
Table 13: Performance of RAG4DMC with different λ and X2 values.
A
A
12T R@I 1
T21 R@1
12T R@S
T21 R@5
CIDEr
BERTScore
0.7
0.3
47.7
48.5
80.1
80.3
121.5
92.6
47.2
48.4
80.1
80.4
121.3
92.6
0.3
0.7
46.4
47.5
79.0
79.7
120.9
92.5
A.14 SEMANTIC AND FIDELITY EVALUATION OF GENERATED DATA
To provide more comprehensive assessment of the generated data quality, conducted direct semantic and fidelity evaluations. Specifically, we computed CLIP- -ii--iiillartty between the gener- ated text and the original modality images, and evaluated image fidelity using FID, comparing generated images with 10k real MSCOCO images. As shown in Table 14 RAG4DMC achieves higher CI.PP-Similarity and substantially lower FID (Heusel al. 2017 compared to Direct Gen- eration, indicating that our method produces text that is more semantically aligned with the original images and generates images with significantly better distributional fidelity. These results confirm that RAG4DMC improves not only downstream performance but also the intrinsic quality of the generated modalities.
Table Semantic and fidelity evaluation of generated data.
Real CLIP-Similarit
Method
FID
0.3040
Direct Generation
0.2915
30.70
RAG4DMC
0.2995
27.92
A.15 EFFECT OF DOMAIN DISTANCE ON PERFORMANCE
To study the effect of domain distance, we compare two external knowledge base choices in Table CC3M (a large, generic caption dataset) and Flickr30K (a smaller dataset more similar in style to our internal datasets). Although Flickr30K is closer in domain to the internal datasets, CC3M yields better performance, suggesting that semantic diversity and scale of the external KB are more important than strict domain matching. Furthermore, the gains achieved with CC3M show that RAG4DMC is robust to moderate domain mismatch and does not require perfectly curated or tightly aligned external corpus.
15
19

---

Published as a conference paper at ICLR 2026
Table 15: Performance of RAG4DMC with different external knowledge bases.
External KB
12T R@1
T21 R&1
12T R@B
T21 R
CIDEr
BERTScore
CC3M
46.6
47.3
79.0
79.7
117.2
92.3
Flickr30K
44.7
45.2
77.4
77.5
112.5
92.3
A.16 VISUAL COMPARISON OF RAG4DMC
In this appendix, we present visual comparison of RAG4DMC's performance in filling missing modalities. Fig. shoows the comparison between real captions and those generated by RAG4DMC when the image modality is missing. On the left, we display real images from the MSCOCO dataset with their corresponding captions. The middle column shows captions generated through Direct Generation, which tend to be brief and less detailed. On the right, RAG4DMC -eggerrtdd captions more descriptive and contextually relevant, demonstrating its ability to fill in missing details.
Image
Real Caption
Direct Generation
RAG4DMC
giraffe standing next rocks on grass.
The giraffe fffee brown and white.
A girafft standing grass by some hig rocks buckgroand.
A man riding skis covered slops.
A skiier skis down slopo.
A clear blue sky,
playing frishos teams at public park.
group backgronnd.
Two men playing frishes.
man riding water skds of water.
The water is bluc.
person who water sliing Inle.
Figure 3: Comparison of generated captions when the image modality missing.
large white bath reh snomboarding knife with black vvery nierly dressed sitting in bothroam mext sink
Caption
flying through the air handle some tomatoes man standing by snoebaard.
and apple
door.
Image
Direct Generation
Figure 4: Comparison of generated images when the text modality is missing.
Fig. Aillustrates RAG4DMC's performance in generating images when the text modality is missing.
The left column shows real images with captions, while the middle column presents images gener- ated through Direct Generation methods, often lacking detail or relevance. On the right, RAG4DMC generates more detailed and contextually accurate images, better reflecting the described scene.
These figures highlight the effectiveness of RAG4DMC in handling missing modalities and gener- ating content that aligns with real scenes in both text and image modalities.
20

---

Published as a conference paper at ICLR 2026
A.17 USE OF LARGE LANGUAGE MODELS
During the preparation of this manuscript, we used OpenAl's GPT-5 as a language-assist tool to help improve the clarity and readability of certain paragraphs. All scientific ideas, experimental design, and data analysis were solely conceived and conducted by the authors. The outputs from the LLM were carefully reviewed and edited to ensure technical accuracy and consistency with our research findings.
21


Reviews:
[{"review_id": "XyZvxAWGue", "reviewer": "Reviewer_nBpJ", "summary": "This paper introduces RAG4DMC, a retrieval-augmented generation framework for data-level Missing Modality Completion (MMC). RAG4DMC overcomes challenges of fine-tuning large models and limited domain-specific data by structuring knowledge for more effective completion, leading to significant gains in downstream tasks. RAG4DMC builds a dual knowledge base —one from comprehensive in-dataset samples and another from external public datasets. It enhances performance by using feature alignment and clustering-based filtering to better manage differences in modality and domain.", "questions": "* Regarding Equations (20) and (21), were the fixed weights for candidate selection empirically optimized for both BLEU (image-to-text) and NIQE (text-to-image)? Additionally, is it appropriate to apply the same weighting scheme given the differences between these metrics?\n* Under multi-modal fusion retrieval, it is not clear in terms of ablation if sim_{fuse} provides additional benefits over top-k retrieval results.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* First framework for RAG tailored specifically for data-level missing modality completion problem. \n* Combination of internal complete samples with external public datasets to generate the dual knowledge base. \n* Two-stage multi-modal fusion retrieval strategy that leverages both intra-modal precision and cross-modal cues via pseudo-embeddings, ensuring the retrieval provides semantically consistent and highly relevant context for generation.\n* Outperforms all baselines across general domain (MSCOCO, Flickr30K) and domain-specific (RSICD) datasets highlighting effectiveness in diverse tasks under high missing rates.", "weaknesses": "* High computational complexity of the proposed knowledge base construction especially the K-means clustering algorithm and iterative nearest neighbor search during cross-domain alignment. \n* In the setting it should be mentioned that the incomplete samples are encountered during the training phase (through missing rates) and the evaluation is performed on the complete samples.\n* High sensitivity of the performance to specific thresholds associated with filtering and retrieval size.\n* How does the quality of results associated with Direct generation baseline change with improvements in image caption generation models (for missing text) and text to image models (for missing image).\n* The negative effects of missing modalities are increasingly significant in multimodal classification tasks such as audio-visual action recognition (for example, Mit-51, UCF-101, and Activity Net as examined in GTI-MM). This study does not address experiments related to audio-visual action recognition with varying rates of missing data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vv6yoU7c3l", "reviewer": "Reviewer_i2QC", "summary": "This paper introduces RAG4DMC, a novel framework that applies retrieval-augmented generation (RAG) to the task of data-level Missing Modality Completion (MMC). The core contribution is a sophisticated system that constructs a dual knowledge base from both in-domain and external datasets, employing techniques like feature alignment and clustering-based filtering to mitigate domain and modality shifts. The proposed multi-modal fusion retrieval, which combines intra-modal retrieval with cross-modal re-ranking, is a clever approach to guide the generation process more effectively. The experimental results on both general and domain-specific datasets demonstrate that this method not only produces more accurate and semantically coherent completions but also leads to significant improvements in downstream tasks like image-text retrieval and captioning.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is among the first to systematically adapt the RAG paradigm for data-level Missing Modality Completion, moving beyond feature-level imputation to generate complete, usable data samples.\n\n2. The dual-knowledge-base design, which leverages both internal and external data, is a key strength. The proposed methods for feature alignment and clustering-based filtering to handle domain and modality gaps are well-motivated and appear effective.\n\n3. The two-stage multi-modal fusion retrieval process is a significant contribution. By first using precise intra-modal retrieval and then refining with cross-modal signals, the method effectively mitigates the inherent modality gap, leading to more semantically relevant context for the generator.\n\n4. The authors have conducted extensive experiments on a variety of datasets (MSCOCO, Flickr30K, and the domain-specific RSICD), evaluating the impact on multiple downstream tasks. The inclusion of several well-designed baselines and ablation studies (e.g., KFA-RAG, Combined-RAG) effectively dissects the contribution of each component of their framework.", "weaknesses": "1. The overall framework has many components and hyperparameters (e.g., clustering parameters, thresholds, fusion weights). This complexity might make the system difficult to tune and reproduce. While the appendix provides some details, a more in-depth analysis of hyperparameter sensitivity would be beneficial.\n\n2. The construction of the knowledge base, particularly the clustering, filtering, and nearest-neighbor search for alignment, could be computationally expensive for very large-scale internal and external datasets. The complexity analysis in the appendix is helpful, but the practical implications on massive datasets remain a potential concern.\n\n3. While the quantitative results are strong, the paper could benefit from more qualitative examples. Showing more side-by-side comparisons of the generated modalities from RAG4DMC versus the baseline methods would provide more intuitive evidence of its superior performance in generating semantically coherent and high-fidelity data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PGOFTlkEZZ", "reviewer": "Reviewer_MkeP", "summary": "This paper proposes RAG4DMC to address the problem of missing modalities in multimodal datasets. Existing generation-based methods suffer from low reliability, while simple RAG-based approaches are ineffective due to noise and domain gaps in external data. To overcome these issues, the authors construct a dual knowledge base by combining internal complete data with external public data. They align and integrate the semantic spaces through cross-modal mapping, clustering-based filtering, and Procrustes alignment. For missing samples, a two-stage fusion retrieval is performed to find highly relevant examples, which are then used with BLIP2 and Stable Diffusion to generate restoration candidates. Finally, CLIP cosine similarity, BLEU, and NIQE metrics are used to select the most semantically consistent and high-quality result.\n\nExperiments conducted on MSCOCO, Flickr30K, and RSICD demonstrate that RAG4DMC outperforms all baseline methods. Notably, as the missing rate increases, the performance gap between RAG4DMC and other methods widens, proving its robustness under data-deficient conditions. The authors further extend the concept of RAG beyond text generation to data-level multimodal restoration, showing that the generated complete datasets significantly enhance the performance of downstream models such as CLIP and LLaVA", "questions": "- The generator used in the proposed framework appears to be implemented based on Stable Diffusion, but it is unclear which specific version was used.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- One of the key strengths of this paper is that it effectively overcomes the generalization limitations of fine-tuning approaches that rely on pre-trained generative models or a small number of fully observed samples. Notably, this work is the first to apply RAG to the Missing Modality Completion problem, leveraging retrieval-based semantic grounding to achieve more reliable and consistent restoration.\n\n- The proposed RAG4DMC framework simultaneously utilizes both an internal knowledge base and an external knowledge base, addressing domain shift between the two through feature alignment and clustering-based filtering. These alignment and filtering procedures reduce noise and unify the semantic space, thereby improving retrieval quality and overall restoration accuracy.\n\n- Through a multi-modal fusion retrieval strategy, the model fuses information across images and text, enabling more fine-grained search results. Additionally, the candidate selection stage ensures semantically coherent outputs, making the restoration process far more stable than simple generation-based reconstruction and enabling faithful modality recovery grounded in real data distribution.\n\n- Even under various missing-ratio settings, downstream models trained on restored data from RAG4DMC achieved consistent and significant performance gains on tasks such as image–text retrieval and image captioning. These results demonstrate that the proposed approach remains robust and effective even in realistic scenarios with severe data incompleteness.", "weaknesses": "- RAG4DMC consists of multiple stages cross-modal mapping, knowledge filtering, cross-domain alignment, and retrieval-based completion which collectively result in high computational complexity. When applied to large-scale datasets, both training and inference may become slow, indicating potential inefficiency in large-scale or real-time environments.\n\n- The framework focuses only on image and text modalities, leaving its applicability to other modalities such as speech or depth insufficiently explored. Further experiments are needed to verify whether the proposed approach can be effectively extended to additional modalities.\n\n- Existing MMC methods generally suffer from limited transferability in out-of-domain scenarios, often requiring retraining to maintain performance on new domains, which incurs high computational cost. Although RAG4DMC mitigates this issue through a dual knowledge base and cross-domain alignment, the bidirectional mapping is learned primarily from internal data, making it difficult to guarantee complete domain transfer. Moreover, due to its reliance on BLIP2 and Stable Diffusion, the generation quality may degrade in domains that differ significantly from the pre-training data, potentially resulting in unnatural or inaccurate outputs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AKaduvpPMH", "reviewer": "Reviewer_ZWWy", "summary": "The paper tackles the Missing-Modality Completion (MMC) problem, where samples in a multi-modal dataset lack one or more modalities (e.g., an image without its paired caption). Instead of fine-tuning large generative models—often impractical due to limited complete samples, API-only access, or high cost—the authors propose RAG4DMC, a retrieval-augmented generation framework.", "questions": "Q1. How does performance vary with the size and domain distance of the external KB? Please provide a scaling or ablation study.\nQ2. What alignment method is used (e.g., Procrustes, MMD, contrastive loss)? Is it learned jointly with retrieval or pre-computed?\nQ3. How many candidate completions are generated per query, and how sensitive are results to this number?\nQ4. Could retrieval introduce harmful or biased content? Any safeguards or toxicity filtering applied?\nQ5. What is the wall-clock cost (GPU hours) and latency per completion compared with direct generation?\nQ6. Have you tested extremely high missing rates (e.g., 90 %) where internal KB is tiny? Does external data fully compensate?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. Important practical problem: MMC occurs frequently in real-world pipelines, yet receives less attention than model-level fusion.\nS2. Novel RAG formulation: combining internal and external KBs and explicitly tackling modality/domain shift via alignment + clustering is fresh in the MMC context.\nS3. No fine-tuning of the large backbone: framework works even with “API-only” generative models, broadening applicability.\nS4. Comprehensive evaluation: three datasets (general + domain-specific), multiple missing-rate settings, and downstream task benefits.\nS5. Ablation on retrieval components and filtering validates each design choice.", "weaknesses": "W1. Scope limited to image-text dyads; unclear whether the approach scales to higher-order or fundamentally different modalities (e.g., audio, LiDAR).\nW2. Generation quality is assessed only indirectly (via downstream tasks). Lack of direct fidelity/semantic metrics (e.g., CLIP-Similarity, FID) or human evaluation.\nW3. External KB dependence: quality and domain bias of external data may dominate results; sensitivity analysis is brief.\nW4. Computational footprint: retrieving from (possibly large) dual KB + generating multiple candidates can be expensive; runtime/memory costs are not reported.\nW5. “First” claim could be overstated—earlier works have combined retrieval and generation for cross-modal reconstruction, though perhaps not under the MMC label; a clearer positioning is needed.\nW6. Feature-alignment technique is only sketched; hyper-parameters and convergence behaviour are not well analysed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)