MoRA: Missing Modality Low-Rank Adaptation for Visual Recognition
Authors: Shu Zhao
Nilesh Ahuja
Tan Yu
Tianyi Shen
Vijaykrishnan Narayanan
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Classification
Missing Modality
Parameter-Efficient Fine-Tuning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: MoRA is a PEFT method for multimodal models that handles missing modalities during inference, resulting in better performance, faster inference, and minimal trainable parameters compared to existing approaches

Abstract: Pre-trained vision language models have shown remarkable performance on visual recognition tasks, but they typically assume the availability of complete multimodal inputs during both training and inference. In real-world scenarios, however, modalities may be missing due to privacy constraints, collection difficulties, or resource limitations. While previous approaches have addressed this challenge using prompt learning techniques, they fail to capture the cross-modal relationships necessary for effective multimodal visual recognition and suffer from inevitable computational overhead. In this paper, we introduce MoRA, a parameter-efficient fine-tuning method that explicitly models cross-modal interactions while maintaining modality-specific adaptations. MoRA introduces modality-common parameters between text and vision encoders, enabling bidirectional knowledge transfer. Additionally, combined with the modality-specific parameters, MoRA allows the backbone model to maintain inter-modality interaction and enable intra-modality flexibility. Extensive experiments on standard benchmarks demonstrate that MoRA achieves an average performance improvement in missing-modality scenarios by 5.24% and uses only 25.90% of the inference time compared to the SOTA method while requiring only 0.11% of trainable parameters compared to full fine-tuning. The code is available at https://github.com/Tree-Shu-Zhao/MoRA.

Published as a conference paper at ICLR 2026
MORA: MISSING MODALITY LOW-RANK ADAPTA- TION FOR VISUAL RECOGNITION
Shu Zhao', Nilesh Ahuja', Tan Yu', Tianyi Shen', Vijaykrishnan Narayanan' 'The Pennsylvania State University PIntel 'NVIDIA smz5505 psu.edu
ABSTRACT
Pre-trained vision language models have shown remarkable performance on vi- sual recognition tasks, but they typically assume the availability of complete multimodal inputs during both training and inference. In real-world scenarios, however, modalities may be missing due to privacy constraints, collection dif- ficulties, or resource limitations. While previous approaches have addressed this challenge using prompt learning techniques, they fail to capture the cross- modal relationships necessary for effective multimodal visual recognition and suf- fer from inevitable computational overhead. In this paper, we introduce MoRA. a parameter-efficient fine-tuning method that explicitly models cross-modes in- teractions while maintaining modality-specific adaptations. MoRA introduces modality-commont parameters between text and vision encoders, enabling bidi- rectional knowledge transfer. Additionally, combined with the modaaltyyspecciffe parameters, MoRA allows the backbone model to maintain inter-modality in- teraction and enable intra-modality flexibility. Extensive experiments on stan- dard benchmarks demonstrate that MoRA achieves average performance im- provement in missing moodalit scenarios by 5.24% and uses only 25.90% the inference time compared the SOTA method while requiring only 0. 0.11% of trainable parameters compared to full fine-tuning. The code is available at hatpps:/gite com/ Tree- Shu-
INTRODUCTION
Pre-trained vision language models (VLMs) integrate multiple modalities (e.g., vision and language) to comprehensively understand their environment, demonstrating remarkable performance on vari- ous downstream tasks, including visual recognition (Hu et al. (2024) and cross-modal retrieval (Li et al. 2025). VLMs like CLIP (Radford et al. 202I) I and ViLT (Kim et al. 2021) leverage large-scale paired data to learn joint representations of images and text. Multimodal large language models, in- cluding GPT-4 (Achiam et al., 2023). Gemini (Team et al., 2024). LLaMA- Vision (Grattafiori et al.) 2024), and LL (Liu al. 2023a), build connections between vision and language and the knowledge within LLMs to establish powerful conversation and reasoning abilities.
Despite their impressive capabilities, deploying them in real-world scenarios presents two significant challenges. First, most multimodal models work under the assumption of modality completeness, requiring all modalities be available during both training and inference. However, this assumption rarely holds in practice due to privacy constraints, collection difficulties, or resource limitations (Ma) 2022). When input modalities are missing. performance degrades substantially (Hu et al., 2024. limiting their applicability in real-world settings where data completeness cannot be guar- anteed. Second, as model sizes grow, fine-tuning becomes increasingly computationally expensive with limited resources and leads overfitting small-scale target datasets (Khattak et al. (023). Although several works (Lee et al. 20233 Hu et al. 2024) have devised prompt- methods to alleviate them, the prompts lead inevitable inference overhead.
To address these challenges, we explore the underlying mechanisms affecting the performance when modalities missing. A critical insight comes from Mind the Gap (Liang al.) 2022). identifying the "modality gap" which is the geometric separation between different modality embeddings the shared representation space. Building on this observation, argue that both the alignment and gap

---

Published as a conference paper at ICLR 2026
between modalities provide valuable complementary information for improving performance dur- ing inference with missing modalities. Specifically, during fine-tuning, the embedding spaces of the visual and text encoders should be related, moving in the same direction to maintain multimodal per- formance. Simultaneously, these encoders need to maintain their own independent update directions to better adapt to downstream tasks without compromising modality-specific characteristics.
Inspired by this, we introduce MoRA, a parameter-efficien fin-tuning method that explicitly mod- els cross-modal interactions while maintaining modality-specific adaptations. MoRA incorporates two key design elements: a shared cross-modal parameter module that enables knowledge trans- fer between modalities through the Gram matrix (Strang 2022) of shared low-rank parameters and modality-speciffe adaptation components that preserve the unique characteristics of each modal- ity. This -ual-structure design allows MoRA to maintain intra-modality flexibility, resulting in robust performance across various mansaittm scenarios.
modality interactions while enabling
To summarize our contributions, we propose MoRA, parameter-effiient fine-tuning method for multimodal models that explicitly addresses the challenge of missing modalities through shared cross-modal parameters and mopelity-specific adaptations, enabling bidirectional knowledge trans- fer between modalities while preserving the directional properties of the original weights. We design an efficient training strategy that requires updating only a small fraction (~ 0.11%) of the model parameters, making it feasible to adapt large pre- pre-trained models even with limited computational resources. Through extensive experiments on standard benchmarks, we demonstrate that MoRA significantly outperforms existing prompt- based and parameter-efficient approaches across various midsing-modality scenarios while maintaining inference efficiency.
2
RELATED WORK
2.1
MISSING MODALITY FOR MULTIMODAL LEARNING
The missing modality issue presents a significant challenge in deploying robust systems, leading to significant performance drop. Previous approaches for addressing missing modality challenges can be broadly categorized into Alignment-brut based and Reconstruction
methods. Alignment-base methods (Wang et al. 2023 Zhang et al.] 2023b; Shhvetooa et al. 2022) embed different modali- ties into shared representation space, enabling the model to operate effectively even when certain modalities are missing by aligning the feature spaces of different modalities during pre- tre-training or fine-tuning. Reconstruution-baase methods (Ma et al. 2022 Zhao et al.) [021 Ma et al.) 2021 use available modalities to reconstruct features of missing modalities explicitly. These approaches typi- cally employ generative models or gross-modas translation networks to synthesize the absent infor- mation, allowing the model to operate on "completed" inputs. However, these methods often suffer from imperfect reconstruction quality, especially when the missing modality contains information that cannot be fully inferred from available ones. More recently, prompt learning techniques (Lee) et al. 2023 Hu et al.) 2024) have emerged as a subset of reconstruction-bee based approaches, handling missing-modalln
scenarios by inserting learnable tokens into transformer layers. Modality-specific information is offloaded to learnable prompts and reused when modalities are missing. MMP (Lee) et al. 2023) treats different missing- modality cases different types of input, adapting the model through learnable prompts while keeping the backbone frozen. However, MMP inserts indepen- dent prompts into each layer, overlooking the complex relationships between modalities. DCP (Hul let al. 2024) and SyP (Zhang et al. (025) devise more prompts to leverage the correlations between prompts and input features across different layers. However, it discards the features of the missing modalities and cannot fully exploit multimodal features for downstream tasks. MoRA preserves the modality information during training and introduces no overhead during inference.
2.2 PARAMETER-EITCIIAE
TUNE-TUNIGG
Chaameter-fficienn Fine-Tuning (PEFT) methods reduce the computational burden of adapting large models by updating only small subset of parameters. These approaches can be classified into three categories. Adapter-ber based methods (Houlsby et al., 2019) insert trainable modules into backbones, either sequentially in parallel with existing layers. Prompt-based methods (Liu et al. 2023b) add trainable tokens to the input while keeping model parameters fixed. Both categories typically intro- duce additional inference latency. Low-Rant Adaptation (LoRA) methods (Hu et al. 2022) approx-
2

---

Published as a conference paper at ICLR 2026
Both
Image
Text
100
65.6
80
80
Performane
60
80
47.7
40
20
15.2
20
1,2
MM-IMDb
Food101
MM-IMDB
Food101
(a) Unaligned: VIT-B/16 BERT-base
(b) Aligned: CLIP VIT-B/16
Gap
Pre- Training
Fine-Tuning
TTxt
(c)
Image O
Figure I: Motivation for MoRA. (a) Performance comparison MM-IMDb and Food101 datasets using unaligned vision and text encoders. (b) Performance comparison using aligned CLIP ViT-B/16 encoder. (c) During pre-training, modalities are aligned in embedding space with gap; during fine- tuning, modalities should maintain their relationship while allowing modality-speciffe adaptations.
imate weight updates using low-rank matrices that be merged with pre-trained weights before inference, thus maintaining inference efficiency. Various extensions have been proposed, including SVD-based approaches (Zhang et al. (023a), orthogonal factorization (Qiu al., 2023 Liu al.) 2024b), and direction decomposition (Liu et al. (024a).. While Multimodal LoRA methods (Shen al. 2024, Ge al., 2025) have focused instruction tuning, they cannot handle missing modal- ities and lack architectural innovations for cross-modal interaction in dual-branch architectures. (Shi al. (2024) address the task medical diagnosis through unidirectional adaptation. MoRA targets general visual recognition tasks with bidirectional knowledge transfer, achieving superior efficiency with smaller trainable parameters and zero inference latency.
METHOD
3.1 PROBLEM FORMULATION
We focus the multimodal classification task with missing modalities during both training and testing. For simplicity, but without loss of generality, we consider a multimodal dataset with text (t) and vision (v) modalities, i.c., D = (D','' DV D.). Specifically, D = {tt, y.),,N, contains text- only data samples; D = (()v,. ) includes image-only data samples; De = ((t,, vi, y the subset containing moddllttycoopllet samples with both text and image, where t, is text, v; denotes image, and y; HR is the label vector where C' is the number of classes. When the image is missing, we set the image input to an all-l matrix; when the text missing, we set the text input to empty string.
3.2 MOTIVATION
Vision Language Models (VLMs) have been pre-trained on massive image-text pairs. Although the pre-training stage aligns the vision and language embedding space, Mind the Gap (Liang et al. 2022) points out that there is still gap between modalities. We argue that both the alignment property and the gap are important for the missing modality task. To demonstrate these, we fine-tune aligned and unaligned models using modality-comples samples and test them using both complete and incomplete samples, as illustrated in Figure (a) (b). The aligned/unaligned models denote whether vision and text encoders trained image-text pairs. Implementation details be found in

---

Published as a conference paper at ICLR 2026
MoRA
S
Vision Layer
Vision Layer -
Vision Layer
B"
W
+
G
A"
Downstream
g
MoRA
MoRA
St
B'
Text Layer
Text Layer
W
+
G
Layer
A'
S
Figure Overview the proposed MoRA architecture.
Compared to the performance drop of -11.1 using the aligned model, the unaligned model shows drop of -4.5, demonstrating that other available aligned modalities can maintain certain level of performance when modalities are missing. Additionally, using both image and text features with the aligned model achieves better performance than using only image features (-14.1) or only text features (-11. This finding suggests that the gap represents different information across modalities, which serves as important complementary information for multimodal tasks.
Section A.3
Therefore, we identify two properties that need to be considered during fine-tuning VLMs, as il- lustrated in Figure [ (c). First, the direction fine-tuning image and text modalities should be the same to maintain their relationship in embedding space for general ability. Second, the image and text modalities should have their own fine- tening direction to enable flexibility for downstream tasks. Inspired by these, we propose MoRA, garameter-effiiicee fine-tuning method that explicitly models cross- modal interactions while maintaining modality modalitt-speccffc adaptations.
3.3 MISSING MODALITY LOw-RANK ADAPTATION
The weight matrix in LoRA can be decomposed into the magnitude and direction, as shown below:
Wo + AW
W = Wo + AW = IW + AW|PP
|Wo AW|FF = W. + AW W AWW (1)
where W F is the Frobenius norm of the matrix, denoting the magnitude; W is the normalized matrix, denoting the direction.
2025) have shown the importance of the direction in fine-tuning models, they focus on large language models, while the cross-modality information interaction, which important for multimodal tasks, is not discussed. Based the analysis in Section
Although recent works (Liu 2024a Wu
3.2 we introduce MoRA, a parameter-efficient fine-tuning method that enables cross-modal interactions and captures
coallity-commoosspeeffiitt information during training. as illustrated in Figure 2 MoRA introduces two types of learnable parameters, including modality-specific pa- rameters A v/t E grxd,/s, B [Rdv xr for independent adaptation, and shared parameters S" grxd, St [dd,xr for cross-modal knowledge transfer, where d, and d, are the dimensions of vision and text encoders respectively, and < d is the rank. The updated weight matrix for image (v) text (t) encoders is:
wv/ w + AWV/t A
(w +AW%) + W +AW* W
w frozen and ignored
AW / AWW!
W
A
/
||
A
| + AW
W
AW (2)
-|
AW
+ AW
AW W
+AW*
= =a + BY/A W + SW/SI/
Sodality-Seeifiie
Modality-Sharee Sharced
4

---

Published as a conference paper at ICLR 2026
where a²/o and a² denote the learnable modality-specific and modality-shared magnitudes; W is the frozen pre-trained weights in vision/text encoders.
However, Equation 2 only works when the dimensions of the image and text encoders the same. For example, the dimension of vision (dy) and text encoders (dz) in the CLIP ViT-B/16 model is 768 and 512, i.e. W E II 768 and Wi |8512x512, Direct multiplication of S and S would yield Rexd x R'xd, which is incompatible with both encoder dimensions. This dimension mismatch challenge significantly limits the applicability of MoRA. Although we can add projection layers to map the image and text embeddings to a common space, the projection layers will significantly increase the number of learnable parameters during training and cannot be absorbed into the pre- trained weights Wo. increasing the inference latency. We resolve this dimension mismatch by operating in the rank space through Gram matrices (Strang 2022). For shared parameters S and St, we compute:
G S' ER"
G = S E E
The key insight is that these Gram matrices capture the structural information of each modality in cinension-aanostee rank space. We then use cross-modal Gram matrices to update each encoder:
W a W B + W S G'SS
(4)
W W + B a W SI G.St.
Since Sy G xd, and St G g4xd, they can be absorbed into the pre-trained weights during inference.
Discussion First, the rank space captures second-order statistics of the low-rank representations, which extracting invariant representations across domains (Arjovsky et al.) (001)). Second, the low- rank structure serves eters to incorporate shared knowledge (Srebro & Shraibman, (005). With Gram matrices, MoRA maintains balance between preserving modality-specific characteristics and enabling cross-modal information exchange. More importantly, all introduced learnable parameters can be absorbed into the original pre-trained weights, which makes MoRA introduce no overheads during inference.
moosl adaptation module that transforms modality-specific param-
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We evaluate MoRA on three benchmarks, including MM-IMDb (Ovalle et al. 2017), UPMC- Food101 (Wang al. 2015). and Hateful Memes (Kiela et al.) 2020). We adopt FI-Macro, top-1 classification accuracy, and Area Under the Receiver Operating Characteristic Curve (AUROC) to evaluate the three benchmarks, respectively, More details can be found Section A
Missing Modality Setting We adopt a rigorous approach wherein modality absence occurs throughout both the training and inference phases. Following previous works (Lee et al., 2023] Hu et al. 2024). designate 17% the missing ratio that quantifies the proportion of incomplete- modality data. In single-modality missing scenarios, the distribution follows a ratio of 1/% chooplete-modallee
samples to 17% complete-modality samples. When addressing dual- modality absences, the dataset consists of 3% image-only instances and 3% text-only instances, complemented by 1 of samples containing both modalities. This configuration effectively simulates real-world modality scarcity conditions and provides robust framework for evaluating performance in missing modality environments. Implementation details can be found in Section
4.2
MAIN RESULTS
As shown in Table MoRA consistently outperforms baseline methods across all missing ratio settings. Several key findings emerge from our experiments. First, the text modality consistently demonstrates greater importance than the image modality across all three datasets. This asymmetry may partially stem from text containing direct label information certain datasets like UPMC- Food101. Second, MoRA achieves particularly strong improvements when the image modality is
5

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison on MM-IMDb, Food101, and Hateful Memes datasets with vary- ing missing ratios. MoRA consistently outperforms all baselines with average improvements of 5.30%, 1.91%, and 8.51% over the next best method DCP.
Datasets
Image
Text
CoOp
MMP
MaPLe
DePT
DCP
MoRA
100%
50%
48.06
48.88
49.58
50.64
52.13
54.62
(+2.49)
50%
100%
49.89
51.46
52.32
52.78
54.32
57.61 (+3.29)
50%
75%
75%
48.37
49.32
49.56
50.87
52.32
55.88 (+3.56)
Average
48.77
49.89
50.49
51.43
52.92
56,04 (+3.12)
MM-IMDb
100%
30%
44.13
45.64
45.52
46.38
48.52
52.56 (+4,04)
30%
100%
48.82
50,52
50.64
52.13
53.14
$6.39 (+3.25)
70%
65%
65%
46.84
48.12
49.16
50.32
51.42
52.97 (+1555)
Average
46.60
48.09
48.44
49.61
51.03
53,97 (+2.94)
100%
10%
44.76
45.32
46.84
47.56
49.26
50.67 (+1.41)
10%
100%
48.32
49.12
50.13
50.88
52.22
53.57 (+1.35)
90%
55%
55%
44.12
44.87
45.12
46.54
48.04
51.64 (+3.60)
Average
45.73
46.44
47.36
48.33
49.84
51.96 (+2.12)
100%:
50%
77.45
77.89
79.64
80.16
82.11
84.41 (+2.30)
50%
100%
87.02
87.16
87.35
82.14
89.12
(+0.51)
50%
75%
75%
81.24
81.72
82.34
83.12
85.24
86.68 (+1.44)
Average
81.90
82.26
83.11
81.81
85.49
86,91 (+ (+1.4
Food101
100%
30%
76.34
76.52
77.02
77.34
78.87
80.85 (+ 1.9))
30%
100%
84.78
85.64
85.89
86.12
87.32
88.01 (+ (+0.69)
70%
65%
65%
78.87
79.12
79.84
81.46
81.87
83.77 (+ (+1))))
Average
80.00
80.43
80.92
81.64
82, 82,69
84.21 (+1.))
100%
10%
71.87
73.14
73.46
74.12
75.26
78.41 (+3.15)
10%
100%
81.67
82,
83.12
83,56
85.78
86,77 (+0.99)
90%
55%
55%
76.46
76.58
77.85
78. 12
79.87
81.09 (+ 1.22)
Average
76.67
77.29
78.14
78.60
80.30
82.09 (+1.79)
100%
50%
60.56
60.31
60.87
61.87
62.32
70.66 (+8.34)
50%
100%
62.41
62.35
63.13
63.88
64.46
71.58 (+7.12)
50%
75%
75%
64.87
65.84
65.46
65.86
66.02
69.58 (+3.56)
Average
62.61
62.83
63.15
63.87
64.27
70.61 (+6.34)
Hateful Memes
100%
30%
60.74
61.12
61.26
61.56
62.82
69.43 (+6.61)
30%
100%
62.74
63.24
63.14
63.48
64.12
70.68 (+6.56)
70%
65%
65%
64.82
65.04
65.23
65.48
66.08
70.15 (+4.07)
Average
62.77
63.13
63.21
63.51
64.34
70.09 (+5.75)
100%
10%
60.03
$7.21
60.74
61.14
62.08
68.52 (+6.44)
10%
100%
61.46
61.52
61.87
62.42
63.87
68.78 (+4.991)
90%
55%
55%
64.32
63.34
64.85
65.37
66.78
68.37 (+ 1.59)
Average
61.94
60.69
62.98
64,24
68.56 (+4.32)
missing. highlighting its effectiveness in addressing the inadequate visual understanding of current methods through cross-modal knowledge interaction. Third, MoRA maintains remarkable robust- ness even under extreme conditions with a 90% missing ratio on Hateful Memes, it achieves perfor- mance comparable to DCP at only 50% missing ratio, demonstrating its superior ability handle severe modality scarcity. These results validate that our dual mechanism of podality-speciffi adap- tation and cross-modal parameter sharing creates a more resilient multimodal learning framework.
4.3
GENERALIZATION
To evaluate the generalization capability of MoRA across different missing- missing-modality scenarios, we conduct cross-scenario experiments where models are trained with one missing-modual modality configu- ration at a 70% missing ratio and tested on different configurations. This evaluation is crucial for real-world deployment where the missing-modality patterns during inference may differ from those seen during training. We consider two training strategies: (1) training on both-missing scenarios where samples randomly have either text or image modality missing, and (2) training on single- modality scenarios where only specific modality is consistently missing. We then evaluate these models on three test configurations: both-missing.
missing, and text-missing scenarios.
As shown in Figure MoRA demonstrates superior cross-scenario generalization compared to DCP across all configurations on the Hateful Memes dataset. When trained on both-missin mitsimissig scenar- ios, MoRA maintains strong performance when tested on specific missing-modality cases, signifi-

---

Published as a conference paper at ICLR 2026
Missing
Missing
72
72
69
70
66
AUROC
AUROC
AUROC
60
66
62
64
[botheeee??
MoRA (bsth- mage
Omagenimage)
DCP
DC> cooth emagel
(sct)
51
10%
30% 50% 70% 90% 100% Missing Rate (%)
10% 30% 50%
70% 90% 100%
10% 30% 50% 70% 90% 100% Missing Rate (%)
Missing Rate (%)
Figure 3: Generalizability Analysis on Hateful Memes dataset. (a) Models are trained on missing- both missing-text cases, and evaluated missing-tex cases. (b) Models are trained on missing- both or missing-image cases, and evaluated on missing-image cases. (c) All models are trained on missing- both cases, and evaluated missing-both cases.
Table 2: Inter-modal Distance Analysis. Aver- age L2 distance and angle between vision and text embeddings on Food101 test set.
Table 3: Modality- SSeecifi Drift Analysis. Av- erage embedding shift from original CLIP rep- resentations.
Method
L2 Dist.
Angle (?)
Vision
Text
Method
CLIP (orig.)
1.18
72.44
L2
Angle
L2
Angle
FFT
22.61
91.64
FFT
8.36
92,16
20.60
87.67
DCP
15.78
86.92
DCP
8.22
65.
13.57
66.26
MoRA
9.99
77.07
MoRA
8.12
43.24
6.04
44.84
cantly outperforming DCP. This advantage persists even in challenging out-of-distribttion scenar- ios-when models trained on text-missing data are tested on image-missing cases, The consistent performance gaps across all train-test combinations demonstrate that MoRA's dual mechanism of maintaining modality-specific parameters while enabling cross cross-modal knowledge transfer through Gram matrices creates more robust representation space, particularly valuable for real-world de- ployments where missing-modalee patterns may vary unpredictably from training conditions.
4.4 DIRECTION PROPERTY IN MORA
To quantitatively validate our motivation illustrated in Figure I c), we conduct comprehensive anal- ysis comparing the embedding space of different approaches. We train models with 70% missing ratio where both modalities are absent, then evaluate on complete test samples to measure how well each method preserves inter-modal relationships while enabling adaptation.
Inter-modal Relationship Preservation. We measure the average T.2 distance and angle between vision and text embeddings for each category in the Food101 test set. As shown in Table 2 MoRA maintains the inter-modal distance and alignment with the original CLIP. In contrast, FFT, which fine-tunes all parameters, severely distorts these relationships, while DCP shows substantial degra- dation. This demonstrates that MoRA successfully preserves the aligned embedding structure cial for handling missing modalities.
Modality-Speciffe Adaptation. We analyze the embedding drift from original CLIP representations to measure -speciif flexibility. Table 3 shows that MoRA achieves balanced adaptation with minimal drift, significantly outperforming FFT which exhibits catastrophic drift. DCP shows moderate drift but fails to maintain the inter-modal alignment as shown above.
4.5 EIGENVALUE ANALYSIS OF MoRA
We conduct an eigenspectrum analysis of the Gram matrices used in MoRA and compare them to the pre-trained weights. We extract eigenvalues from the Gram matrices and singular values from

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison of different parameter-effiieen fine-tuning methods.
Table Comparison with multimodal alignment and fusion methods.
MM-IMDb
Food101
Hateful Memess
MM-IMDb
Food101
Hateful Merees
Low Rank- Braed
MoRA
52.97
83.77
70.15
MoRA
52.97
83.77
70.15
LoRA
51.35
82.14
67.97
Align
51.39
81.14
68.53
DoRA 51.89
82.34
68.28
Fusion
50.72
81.01
68.17
Prompt-Based
Specific
81.32
DePT
50.32
81,46
65.48
w/o Gram
50.41
80.31
68.19
DCP
51.42
81.87
66.08
w/ Learablle Gram
52.25
83.37
69.12
Weight Fine Tianing
w/ Wo
52.88
83.59
70.03
BitFit
48.57
79.38
64.10
FTT
3.01
14.05
46.91
Layer 10
Layer 11
0.35
Normalized Vallee (log scale)
gram
0.3
Soxt pro-trained
-ision gram
0.25
0.25
vision pre-trained
0.2
0.2
Normalized
8.15
0.1
0.1
0.05
0.06
2
Index
Index
Figure Comparison of eigenvalue distributions between Gram matrices and pre-trained weights.
the pre-trained weights in layers 10 and 11 of the vision and text encoders. Figure 4 presents the normalized eigenvalue distributions. Our analysis reveals several critical findings.
information concentration mechanisms. This substantial differ- ence demonstrates that Gram matrices effectively concentrate information in much more compact form. To verify this empirically, we remove the Gram matrix, denoted w w/o Gram in Table 5 The average performance drops by 2.66%, confirming that Gram matrices are essential for effective knowledge transfer between modalities due their information concentration properties.
First, Gram matrices
Second, text and vision modalities exhibit similar structural patterns in their Gram matri- ces. Despite dimensional differences, we observe relatively stable eigenvalue distributions across indices for both modalities, indicating cooss- structural similarities despite their dimensional differences. This structural similarity enables effective cross-modal knowledge transfer through the shared parameter space. To validate this, we use independent parameters instead of shared ones, denoted as w/o Shared in Table The average performance drops by 2.78%, demonstrating that the emergent similar patterns functionally critical for effective knowledge sharing.
Third, we observe converging representational structures in deeper layers. The eigenvalue pattern of Layer 11 shows more convergence compared to Layer 10, suggesting that deeper layers develop more aligned representational structures, which MoRA effectively leverages and maintains information preservation while enabling cross-modal transfer.
4.6 ABLATION STUDIES
Compared to Parameter-Efficiaa Fine-Tuning Methods We compare other pfrfmeter-cefficient fine-tuning techniques, including LoRA (Hu et al.) 2022), DoRA (Liu et al. (2024)), and BitFit (Za-) ken et al. 2022), shown in Table 4 Low-rank-basse based methods achieve the best performance due to their flexibility. MoRA outperforms other methods, demonstrating its effectiveness in enabling modality interaction.
Alternatives for Addressing Dimension Mismatch As shown in Table Align uses two extra linear layers to project modalities into the same embedding, while Fusion concatenates embed-
8

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison on image-to-image retrieval using models trained on multimodal CIRR dataset. Results are evaluated on the MS-COCO validation set.
Method
Recall@1
Recall@ R
CLIP4CIR (Baldrati al. 2024
43.34
76.99
86.49
CLIP4CIR MoRA
60.50
85.00
88.60
400
Inference Time (ms per Sumple)
350
CLIP VIT-B
MMP
[149.79M]
300
(3.48M, 13mm)
DCP
CLP WTLL (427.88M) 8BM]
250
80
SUP VIT-S [104.37M]
200
75
100
MoRA
50mm0
70
50
0
100
200
300
400
500
8
#Parameters
#Truinable Parameters
Figure 5: Performance scaling of MoRA with
Figure 6: Inference time (ms) per sample versus the number of trainable parameters.
different backbone models.
dings from different modalities and uses one linear layer to project. MoRA consistently outperforms them across all datasets. We also conduct experiments removing modality-specific parameters in MoRA, denoted w/o Speci f fic. For Gram matrix construction, removing it and replacing it with a lcarnable one. These results validate the effectiveness of MoRA. We also add the ignored Wo in Equation (2). showing that the learnable magnitude parameters effectively compensate for the omitted frozen weights during training.
report the performance of
Parameter sensitivity analysis can be found in Section D
4.7 EXTENSION TO EMBEDDING TASKS
To demonstrate MoRA's generalizability beyond classification tasks, we evaluate it on Composed Image Retrieval (CIR) based on Baldrati et al.] (2024) using CLIP models, where models use a ref- erence image and text modification identify target images. CIR models are trained on multimodal inputs, making the original image-to-image retrieval as an important and natural missing- missing-modality modality scenario where texts are absent. We train models on the CIRR dataset (Liu et 202T with com- plete image-text pairs and evaluate on the MS-COCO validation set (Lin et al. 2014 for image- to-image retrieval, as shown in Table MoRA achieves substantial improvements across all recall metrics, indicating that MoRA's applicability to tasks beyond classification, where missing modali- ties fundamentally alter the task dynamics. Implementation details can be found Section A.3
4.8 SCALABILITY AND INFERENCE TIME
Figure 5 demonstrates the effectiveness of MoRA integration across various backbone architectures, including SLIP ViT-S (Mu et al.] 2022), CLIP ViT-B, and CLIP ViT-L. (Radford et al. (2021). The results indicate that performance exhibits favorable scaling properties with respect to model capacity, with accuracy improvements correlating positively with the number of parameters. We conduct comprehensive analysis of inference times to evaluate the computational efficiency of MoRA and prompt-based methods, including MMP and DCP. As shown in Figure 6 prompt-based methods significantly increase the inference time. MoRA theoretically introduces no inference overhead, and experimental results demonstrate its efficiency.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
We introduced MoRA, a parameter-effiient fine-tuning method that effectively addresses the missing modality challenge in multimodal learning through shared cross-modal parameters and modality-specific adaptations, By leveraging Gram matrices for Simension-agnossee knowledge transfer, MoRA enables bidirectional information exchange while preserving modality-speciff char- acteristics without introducing inference overhead. Extensive experiments demonstrate that MoRA significantly outperforms existing approaches across multiple benchmarks both on performance and inference time, demonstrating the effectiveness and efficiency of MoRA.
ACKNOWLEDGMENTS
This work was supported in part by Semiconductor Research Corporation JUMP 2.0 PRISM Center.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877/ 2023.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: ar0i:: /tt.//9979.9993 2019.
Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. Composed image re- trieval using contrastive learning and task-oriented clip-based features. ACM Transactions on Multimedia Computing, Communications and Applications (TOMM), 2024.
Chendi Ge, Xin Wang, Zeyang Zhang, Hong Chen, Jiapei Fan, Longtao Huang, Hui Xue, and Wenwu Zhu. Dynamic mixture curriculum lora experts for continual multimodal instruction tuning. arXiv preprint arXiv:2506. 16672, 2025.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21788 2024.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning (ICML), 2019.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Con- ference on Learning Representations (ICLR), 2022.
Lianyu Hu, Tongkai Shi, Wei Feng, Fanhua Shang, and Liang Wan. Deep correlated prompting for visual recognition with missing modalities. Conference on Neural Information Processing Systems (NeurIPS), 2024.
Muhammad Uzair Khattak, Hanoona Abdul Rasheed, Muhammad Maaz, Salman H. Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023.
Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ring- shia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech in multimodal memes. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-anddlannguag transformer without convo- lution or region supervision. In International Conference on Machine Learning (ICML), 2021.
Yi-Lun Lee. Yi-Hsuan Tsai, Wei-Chen Chiu, and Chen-Yie Yu Lee. Multimodal prompting with miss- ing modalities for visual recognition. In IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023.
10

---

Published as a conference paper at ICLR 2026
Haobin Li, Peng Hu, Qianjun Zhang, Xi Peng, Xiting Liu, and Mouxing Yang. Test-time adaptation for cross cross-mode modal retrieval with query shift. International Conference on Learning Representa- tions (ICLR), 2025.
Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y. Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
Tsung- Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects context. In European Conference Computer Vision (ECCV), 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Confer- on Neural Information Processing Systems (NeurIPS), 2023a.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. ACM Comput. Surv. 2023b.
Shih-
Shih-Yang Yang Liu, Chien-Y Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: Weight- weight-decomposed low-rank adaptation. Interna- fional Conference on Machine Learning (ICML), 2024a.
Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Hco, Songyou Peng. Yandong Wen, Michael J. Black, Adrian Weller, and Bernhard Schölkopf. Parameter-efficient orthogonal finetuning via butterfly factorization, In International Conference Learning Representations (ICLR), 2024b.
Zheyuan Liu, Cristian Rodriguez Opazo, Damien Teney, and Stephen Gould. Image retrieval on real-life images with pre-trained vision-and-language models. In International Conference Computer Vision (ICCV), 2021.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations (ICLR), 2019.
Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng. SMILL multimodal learning with severely missing modality. In AAAI Conference on Artificial Intelligence (AAAI), 2021.
Mengmeng Ma, Jian Ren, Long Zhao, Davide Testuggine, and Xi Peng. Are multimodal trans- formers robust to missing modality? In IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2022.
Norman Mu, Alexander Kirillov, David A. Wagner, and Saining Xie. SLIP: self-supervision meets language-image pre-training. In European Conference on Computer Vision (ECCV), 2022.
John Edison Arevalo Ovalle, Thamar Solorio, Manuel Montes-y-Gómez, and Fabio A. González. Gated multimodal units for information fusion. In International Conference on Learning Repre- sentations Workshops (ICLRW), 2017.
Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng. Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf. Controlling rext-to-image diffusion by orthogonal finetuning. In Con- ference on Neural Information Processing Systems (NeurIPS), 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Gob, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Interna- tional Conference on Machine Learning (ICML), 2021.
Ying Shen, Zhiyang Xu, Qifan Wang. Yu Cheng, Wenpeng Yin, and Lifu Huang. Multimodal instruction tuning with conditional mixture of lora. In Annual Meeting of the Association for Computational Linguistics (ACI.), 2024.
11

---

Published as a conference paper at ICLR 2026
Zhiyi Shi, Junsik Kim, Wanhua Li, Yicong Li, and Hanspeter Pfister. Mora: Lora guided multi- modal disease diagnosis with missing modality. Medical Image Computing and Computer Assisted Intervention Society (MICCAI), 2024.
Nina Shvetsova, Brian Chen, Andrew Rouditchenko, Samuel Thomas, Brian Kingsbury, Rogério Feris, David Harwath, James R. Glass, and Hilde Kuehne. Everything at once multi-modal fusion transformer for video retrieval. In IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2022.
Nathan Srebro and Adi Shraibman. Rank, trace-norm and max-norm. In International conference on computational learning theory, 2005.
Gilbert Strang. Introduction to linear algebra. SIAM, 2022.
Gemini Team, Petko Georgiev. Ving lan Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context. arXiv preprint WxXiv:2403.05522 2024.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research (JMLR), 2008.
Hu Wang, Yuanhong Chen, Congbo Ma, Jodie Avery, Louise Hull, and Gustavo Camnerr. Multi- modal learning with missing modality via shared-specifet feature modelling. In IEEE/CVF Com- puter Vision and Pattern Recognition Conference (CVPR), 2023.
Xin Wang, Devinder Kumar, Nicolas Thome, Matthieu Cord, and Frédéric Precioso. Recipe recog- nition with large multimodal food dataset. IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2015.
Yichen Wu, Hongming Piao. Long-Kai Huang, Renzhen Wang, Wanhua Li, Hanspeter Pfister, Deyu Meng, Kede Ma, and Ying Wei. Sd-lora: Scalable decoupled low-rank adaptation for class incre- mental learning. In International Conference on Learning Representations (ICLR), 2025.
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine- tuning for transformer-based masked language-models. In Annual Meeting of the Association for Computational Linguistics (ACL), 2022.
Zhang, Shihan Wu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. Dept: Decoupled prompt tuning. In IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2024.
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efffcient fine-tuning, In International Con- ference Learning Representations (ICLR), 2023a.
Yunhua Zhang, Hazel Doughty, and Cees Snoek. Learning unseen modality interaction. In Confer- ence on Neural Information Processing Systems (NeurIPS), 2023b.
Zhihui Zhang, Luanyuan Dai, Qika Lin, Yunfeng Diao, Guangyin Jin, Yufei Guo, Jing Zhang, and Xiaoshuai Hao. Synergistic prompting for robust visual recognition with missing modalities. arXiv preprini arXiv: 2507. 2025.
Jinming Zhao, Ruichen Li, and Qin Jin. Missing modality imagination network for emotion recogni- tion with uncertain missing modalities. In Annual Meeting of the Association for Computational Linguistics (ACL), 2021.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision (IJCV), 2022.
12


Reviews:
[{"review_id": "gsruxQvXS8", "reviewer": "Reviewer_qZrT", "summary": "This paper presents MoRA (Missing Modality Low-Rank Adaptation), an innovative approach for parameter-efficient fine-tuning (PEFT) that tackles the issue of absent modalities in pre-trained Vision-Language Models (VLMs) during both training and inference. The proposed approach introduces modality specific and shared parameters with minimal computational overhead. The proposed methodology demonstrates superior performance compared to baseline models across multiple benchmarks involving missing modalities.", "questions": "* What is the sensitivity of proposed method MoRA to the initialization of the matrices Sv and St ?\n* Further, have the authors explored any constraints on the shared parameters e.g. effect of orthonormal columns etc ?\n* Have the authors considered a scaling factor between the modality specific and shared modality based updates (Eq 2 and 4) ?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Inclusion of low rank updates for modality specific and shared parameters. \n* Inference time efficiency in terms of minimal computational overhead.\n* Superior performance across multiple benchmarks involving diverse missing modality scenarios.\n* Strong cross-scenario generalization in cases when training time missing-modality patterns differ significantly from the testing patterns.", "weaknesses": "* The proposed low rank update approach is restricted to the dual encoder models i.e. CLIP or SLIP. The applicability of the proposed approach should be explored for single stream models that relies on alignment of visual representations with LLM inputs.\n* The current set of analysis is restricted to image-text datasets with significant text modality dominance. Tasks involving other modality combinations ( audio-visual inputs ) can be considered e.g. audio-visual action recognition.\n* Situations involving partial modality corruptions are not considered. Examples include image corruptions by black patches and text corruption by random word dropouts or redaction by [MASK] tokens.\n* Detailed analysis regarding the layer placement of MoRA updates w.r.t. individual datasets (MM-IMDb and Hateful-Memes) are not mentioned.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0ng4l7iABk", "reviewer": "Reviewer_fEg2", "summary": "This paper addresses performance degradation in Vision-Language Models (VLMs) when a modality is missing, a scenario where existing prompt-based solutions suffer from high inference latency. The authors propose MoRA (Missing Modality Low-Rank Adaptation), a new Parameter-Efficient Fine-Tuning (PEFT) method that combines modality-specific adapters with novel shared low-rank parameters to model cross-modal interactions. Experiments on three benchmarks show MoRA significantly outperforms prior SOTA methods like DCP in missing-modality scenarios while being substantially more efficient at inference time.", "questions": "Please see weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The core mechanism of MoRA is clever. Using Gram matrices to facilitate shared, low-rank adaptation between two frozen encoders of different dimensions is a technical solution to the dimension-mismatch problem.\n- The primary advantage of MoRA over its main competitors (MMP, DCP) is its efficiency. Because MoRA is a LoRA-based method, all its adapter weights can be merged into the backbone model post-training.", "weaknesses": "- The central premise of the paper—that missing modalities are a common, critical \"real-world scenario\"—is weakly motivated. The paper justifies this by vaguely citing \"privacy constraints, collection difficulties, or resource limitations\" but provides no concrete, compelling examples.\n- In most practical VLM applications (VQA, captioning, retrieval), the user provides the modalities. The experimental scenarios, such as classifying a \"hateful meme\" with the text missing, or identifying food from a recipe with the image missing, feel highly contrived and niche.\n- The proposed solution is architecturally specific to dual-encoder models like CLIP. It is not clear how this Gram-matrix-based cross-encoder sharing would be applied to modern, single-stack, fused MLLMs (e.g., LLaVA-style models), which are becoming the standard.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wvV4KCdoia", "reviewer": "Reviewer_3Yzm", "summary": "This paper addresses the missing modality problem in visual recognition tasks and proposes missing modality low-rank adaptation (MoRA), a parameter-efficient fine-tuning approach that explicitly models cross-modal interactions while maintaining modality-specific adaptations. Specifically, the MoRA introduces shared cross-modal parameters based on the Gram Matrix to enable bidirectional knowledge transfer, along with modality-specific adapting parameters to preserve the unique characteristics of each modality. Extensive experiments and ablation studies are conducted to demonstrate the efficacy of the proposed method.", "questions": "Please address my concerns in the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "+ The paper is technically sound, well-motivated, and tackles the missing modality issue, which is an important and practical challenge in multimodal learning.\n+ The proposed MoRA method is conceptually interesting and carefully designed, as it explicitly models cross-modal interactions while retaining modality-specific adaptations.\n+ The proposed MoRA brings impressive improvements and consistently outperforms other SOTA methods across various missing-modality scenarios. \n+ Extensive ablation studies are conducted to validate the contribution of each component and showcase the efficiency of MoRA in both training and inference time and computational cost.", "weaknesses": "- Scalability to more modalities (e.g., beyond two) is a potential concern. While MoRA appears feasible for dual-modality tasks such as image–text or audio–text learning, its current design may not scale efficiently to scenarios involving multiple modalities (e.g., 4 or more), where cross-modal adaptations could grow exponentially in cost and complexity.\n- The generalizability across multimodal architectures is not clearly demonstrated. The proposed MoRA is primarily evaluated on a CLIP-like architecture, which has a separate encoder for each modality. However, for architectures such as ViLT or InstructBLIP, where early fusion occurs and modality interactions are embedded within transformer attention layers, it remains unclear whether the proposed adaptation strategy would still be directly applicable or equally effective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vl2CejQizs", "reviewer": "Reviewer_AxMw", "summary": "This paper introduces MoRA (Missing Modality Low-Rank Adaptation), a parameter-efficient fine-tuning method for vision-language models (VLMs) under missing modality scenarios.\n\nMoRA addresses this by introducing two types of parameters:\n* Modality-specific parameters for independent adaptation of each modality.\n* Shared cross-modal parameters that facilitate bidirectional knowledge transfer via Gram matrices in a low-rank space.\n\nThis allows MoRA to maintain inter-modality alignment while preserving intra-modality flexibility, without introducing inference overhead.", "questions": "Could the Gram matrix mechanism be extended to more than two modalities (e.g., vision, text, and audio)?\n\nHow well does MoRa perform when it uses the same hyperparameters for all datasets (for example, the one for Food101)\n\nHow sensitive is MoRA on the rank?\n\nDoes MoRA’s shared Gram-space adaptation still yield improvements when applied to unaligned encoders?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Novel contribution: The use of Gram matrices for cross-modal low-rank adaptation is original and mathematically elegant, avoiding dimension mismatch between modalities.\n\nComprehensive evaluation: The paper evaluates across multiple datasets, missing ratios (50–90%), and cross-scenario generalization settings.\n\nStrong empirical results: MoRA outperforms baselines in accuracy.\n\nEfficiency: Achieves high performance with minimal trainable parameters and no inference overhead, a key improvement over prompt-based tuning.\n\nInsightful analyses: Includes embedding-space, eigenvalue, and ablation studies supporting claims of inter-modal alignment and modality-specific adaptability.\n\nExtensible framework: Demonstrates that MoRA generalizes beyond classification to retrieval tasks (CIRR→COCO).", "weaknesses": "Evaluation diversity: All experiments use CLIP ViT-B/16 as the backbone; results on other architectures are only briefly summarized (Fig. 5). \n\nLimited modality scope: Experiments are restricted to image–text tasks. The paper claims generality but does not test on other modality pairs.\n\nHyperparameters were likely tuned on the test set and differ from dataset to dataset. Hyperparameters should be either selected on a validation set or on one dataset and shared across all datasets.\n\nExperiments are restricted to image-text tasks, despite claiming generality, the method is not tested on audio–visual or video–text datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)