Adaptive Re-calibration Learning for Balanced Multimodal Intention Recognition
Authors: Qu Yang
Xiyang Li
Fu Lin
Mang Ye
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal data
Modality imbalance
modality valuation
Sample Calibration
Encoder Calibration
Primary area: deep_learning

Abstract: Multimodal Intention Recognition (MIR) plays a critical role in applications such as intelligent assistants, service robots, and autonomous systems. However, in real-world settings, different modalities often vary significantly in informativeness, reliability, and noise levels. This leads to modality imbalance, where models tend to over-rely on dominant modalities, thereby limiting generalization and robustness. While existing methods attempt to alleviate this issue at either the sample or model level, most overlook its multi-level nature. To address this, we propose Adaptive Re-calibration Learning (ARL), a novel dual-path framework that models modality importance from both sample-wise and structural perspectives. ARL incorporates two key mechanisms: Contribution-Inverse Sample Calibration (CISC), which dynamically masks overly dominant modalities at the sample level to encourage attention to underutilized ones; and Weighted Encoder Calibration (WEC), which adjusts encoder weights based on global modality contributions to prevent overfitting. Experimental results on multiple MIR benchmarks demonstrate that ARL significantly outperforms existing methods in both accuracy and robustness, particularly under noisy or modality-degraded conditions.

Adaptive Re-calibration Learning for Balanced Multimodal Intention Recognition
Qu Yang Xiyang Li Fu Lin't Mang Ye
1 School of Computer Science Wuhan University, Wuhan, China
2 Taikang Center for Life and Medical Sciences, Wuhan University, Wuhan, China {yangqu, lixiyang, linfu, yemang)@whu. edu. sttps://githin httpss//gg.bbb...ooooaaasg//g// cow.lyasS@p//leer1PP22-----
Abstract
Multimodal Intention Recognition (MIR) plays critical role applications such as intelligent assistants, service robots, and autonomous systems. However, in real- world scenarios, different modalities often vary significantly informativeness, reliability, and noise levels. This leads to modality imbalance, where models tend to over-rely on dominant modalities, thereby limiting generalization and robustness. Although existing methods address this issue at either the sample or model level, they generally fail to account for multi-level nature. To address this, we propose Adaptive Re-calibration Learning (ARL). novel dual-path framework that models modality importance from both sample-wise and structural perspectives. ARL in- corporates two key mechanisms: Contribution-Invee which dynamically masks overly dominant modalitics at the sample level to en- courage attention to underutilized ones; and Weighted Encoder Calibration (WEC), which adjusts encoder weights based on global modality contributions to prevent overfitting. Experimental results multiple MIR benchmarks demonstrate that ARL significantly outperforms existing methods both accuracy and robustness, particularly under noisy or modality-degradee conditions.
Sample Calibration (CISC),
Introduction
Understanding human intentions is paramount for creating truly intelligent and user-centric systems [ 6) From predicting needs in smart homes enabling natural interactions with robots, the ability to accurately decipher what person wants to achieve is crucial. Multimodal Intention Recognition (MIR) g directly addresses this challenge by integrating information from diverse modalities, including visual cues, linguistic expressions, and auditory signals, to infer users' underlying goals. This capability is not just theoretical; it's the bedrock for advancing wide range of real-world applications, including more intuitive intelligent assistants, sophisticated robotics, and safer, more responsive autonomous systems TO).
However, achieving reliable intention recognition multimodal settings is far from trivial [T 12 Real-words world multimodal data is inherently heterogeneous: different modalities contribute unequally, suffer from varying levels of noise, and may be missing or degraded under certain conditions [5] These challenges make effective integration of modalities both crucial and non-trivial, and they highlight the importance of addressing modality imbalance, which is a core factor that significantly affects MIR model robustness and generalization.
'Equal contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Despite recent advances in MIR, modality imbalance remains key challenge that limits model performance and generalization [T6] In realistic settings, different modalities vary significantly in terms of informativeness, reliability, and noise levels. This heterogeneity often causes models to over-rely on dominant modalities while underutilizing weaker but potentially critical signals. To address this, researchers have proposed various strategies. Early approaches such as gradient modulation [17] aim to balance modality contributions by globally adjusting learning rates during optimization. However, these methods typically lack fine-grained sensitivity to the varying importance of modalities at the sample level. More recently, Shapley value-based methods [14] have been introduced to more precisely estimate the marginal contributions of different modality combinations. While theoretically appealing, these methods suffer from exponential computational complexity with respect the number of modalities, severely limiting their scalability in real-world MIR tasks. This limitation triggers a critical question: How can we enable mne-grained, dynamic modeling of modality importance without incurring prohibitive computational cost?
Moreover, most existing methods focus solely on either sample-level weighting or model-level parameter adjustment [8 18)20), overlooking the multi-level nature of modality imbalance. In reality, imbalance occurs not only in how individual samples leverage modalities, but also in how the model itself encodes and integrates them structurally. Thus, methods that optimize only one level often fall short in handling the complexity of world multimodal scenarios. This observation leads to another fundamental question: How can jointly address modality imbalance both the input and architectural levels to build more robust and generalizable MIR models?
To this end, we propose Adaptive Re-calibration Learning (ARL), novel framework that tackles modality imbalance through dual-path calibration mechanism. ARL introduces two complementary strategies that work in tandem to address imbalance from both the input and model perspectives. The first strategy dynamically masks dominant modalities during training based on per-sample contribution estimates, encouraging the model to pay more attention to underrepresented modalities, component referred to as Contributtion-Inveree Sample Calibration (CISC). Concurrently, the second strategy adjusts the weights of modality-specific encoders according to their overall importance, which helps prevent overfitting to strong modalities and promotes balanced representation learning at the architectural level, known as Weighted Encoder Calibration (WEC). By integrating these two mechanisms, ARL achieves a more holistic and adaptive approach to modality balancing in MIR. Extensive experiments multiple MIR benchmarks demonstrate that ARL outperforms state-of-the- art methods in both accuracy and robustness. Particularly in noisy or modality-deprive settings, ARL shows superior generalization and resilience, highlighting its practical value in real-world deployment. Our contributions are threefold:
We propose Adaptive Re-calibration Learning (ARL), a unified framework that addresses modality imbalance in Multimodal Intention Recognition by jointly calibrating modality importance at both the input and architectural levels.
We design two complementary components within ARL: Contributtion-Invess Sample Calibration (CISC). which dynamically dein-weights dominant modalities at the sample level, and Weighted Encoder Calibration (WEC), which adjusts encoder weights based on modality importance, enabling balanced and adaptive multimodal representation learning.
We conduct extensive experiments on multiple MIR henchmarks, demonstrating that ARL. consistently outperforms state-of- tftt--of-the-art methods in both accuracy and robustness, especially under noisy or modality-degradee conditions.
2 Related Work
2.1 Multimodal Intention Understanding
Multimodal intention recognition (MIR) seeks to infer user intent by integrating diverse signals from
multiple modalities, such as text, audio, and visual information [1] Early unimodal methods, including text- based intent classification [25] and acoustic emotion recognition (26). often simplify the
inherent complexity of human intention by relying on single source of data. In contrast, multimodal learning approaches provide a more comprehensive understanding by leveraging the complementary nature of different modalities. Representative methods include MulT (27), which models inter-modal
interactions through pairwise attention mechanisms, and MISA 28 which captures both shared and

---

G. Adoptive Re calibration Learning
oi c. CISC
Inform
Unimodal Features
Callbreted Features
Lagits
Multi Multi-mocil mocal Input
Encoders
100 Modolity Contribution
Modal Contribution
d. WEC
Fusion Net
Prevent
b. Leove one out (100) Model Contribution
Contribtiion-oweve Somple Colibration (CISC)
o4D-g ou Fusion Net
mask
Similariy blll
Unimodal
Callbrated
Welghted Encoder Calbration (WEC)
D Calibrated Encoders
Masked
Fusion
Fused Feature
Vall
Statistical Variance
The architecture of Adaptive Re-calibration Learning (ARL). ARL integrates three compo- nents: (b) Leave-One Caae--One-Out (LOO) Modality Contribution, (c) Contribution- Inverse Sample Calibration (CISC), and (d) Weighted Encoder Calibration (WEC). LOO assesses modality importance by mask- ing inputs and measuring similarity. CISC refines features using a contribution threshold. WEC adjusts encoders with consistency gap from training and validation sets. The process concludes with Fusion Net for intent prediction.
Figure 1:
modallty-speeciff representations to enhance generalization. MAG-BERT 29 further extends this line of research by integrating nonverbal modalities into the BERT architecture using a multimodal adaptation gate, thereby enabling the model to incorporate contextual signals beyond text.
More recently, researchers have explored advanced strategies to further improve multimodal learn- ing in MIR. Contrastive learning methods (30] have been proposed better align and distinguish representations across modalities by promoting consistency while preserving unique modality charac- teristics. In addition, modality-aware prompting techniques [[T] have been introduced to dynamically guide model behavior based on the specific combination of available modalities. Despite these advancements, one of the major challenges that remains is the issue of sample-specife modality imbalance. In many real-world scenarios, certain modalities may be noisy, incomplete, or less informative for particular samples, which can negatively impact model performance. Addressing this challenge is essential for developing robust and adaptable multimodal intention recognition systems.
2.2 Imbalanced multimodal learning
Imbalanced multimodal learning arises from inherent variations in informativeness and reliability across modalities, often leading models to overemphasize dominant signals while neglecting weaker, yet potentially crucial, complementary cues 31 Traditional techniques, such as loss re- re-weighting and modallty-sppeiifi augmentation, attempt to balance modality influence but often lack adaptability to sample-specific variations (34. Gradient modulation [17] represents more dynamic approach, adjusting training updates to mitigate the dominance of stronger modalities. However, these methods typically operate a dataset level and lack the instance-level adaptivity essential, where modality importance is context-dependent.
are prominent strategies for managing modality imbalance. Modality dropout enhances robustness by randomly masking features from dominant modalities during training, encouraging reliance weaker signals. However, its heuristic nature can inadvertently suppress critical information. Contribution-awal learning offers more principled approach, employing techniques like Shapley values [1] to estimate and balance modality contributions. While theoretically sound, the computational complexity of Shapley value calculation hinders its practicality in real-time MIR applications. Thus, achieving an optimal balance between adaptivity, computational efficiency, and predictive accuracy remains a significant challenge in imbalanced multimodal learning.
Modality dropout 38 40 and contribution aware learning 41
Methodology
The Adaptive Re-calibration Learning (ARL) framework, shown in Fig. I addresses the modality imbalance problem in Multimodal Intention Recognition (MIR) by integrating three key components:

---

Leave-One-Out (LOO) modality contribution estimation, Contribution-Invers Sample Calibra- tion (CISC), and Weighted Encoder Calibration (WEC). As noted in Sec. I modality imbalance, where certain modalities dominate due to differences in informativeness and reliability, poses significant challenge for MIR. ARI. mitigates this through an efficient and adaptive strategy. It starts with unimodal encoders extracting features from language, visual, and acoustic inputs via pre-trained models. The LOO module then evaluates each modality's contribution to the fused representation. Based on these estimates, CISC calibrates features at the sample level by masking dominant modali- ties to encourage the use of weaker ones. In parallel, WEC refines the modality-specific encoders by considering both their contribution and stability during training, promoting more balanced learning. complete list of variables and their definitions can be found in Appendix A.1
3.1 Problem Formulation
To provide clear foundation for ARL, we first present general framework for multimodal intention recognition. Given a source imput m {I. V, A), where L, V, and represent the language, visual, and acoustic modalities, respectively, the goal is to fuse these modalities into unified representation intention recognition. Each modality is encoded into its respective feature representations:
m TextEmbed(L) L) E R4xd,
(1)
ma = AudioEmbed (.A) E R/
(2)
my VideoEmbed(V)) E 1g/xxd.
(3)
where my, my and m, are the extracted feature representations of each modality. Here, lefe/v denotes sequence lengths (text tokens/audio frames/video frames), and dz/a/o represents feature dimensions for respective modalities. After obtaining the shallow features for each modality, a critical step is to temporally align these heterogeneous sequences to enable cross cross-modal interaction. This alignment process maps the features of each modality into a shared space while synchronizing their temporal dynamics. Formally, given the modality-specin features m,, May my the alignment operation can be expressed as:
by he. h SeqAligned(m,, Mai m.) Fxxd
where hes ha and hy are the aligned feature representations of each modality. Here, / indicates the unified sequence length after alignment, and d the projected feature dimension. Once the high-level, aligned features are obtained, they are fused through multimodal fusion to obtain the final representation for intent prediction:
1m F(h,, ba: hw) E Rd,
where F()) denotes the multimodal fusion function. The final intent prediction is made by applying a softmax function the fused representation:
logits Softmax(W" hm + b).
(6)
with W Rd and b RC being trainable parameters, where C' denotes the number of categories. The whole framework preserves modality-specifff patterns while enabling cross-modal interaction through shared alignment. To formally quantify the issue of modality imbalance, introduce the Modality Imbalance Index (MII), a metric derived from our LOO contributions. A detailed definition and formulation are provided in Appendix A.3.
3.2 Leave-One-Out Modality Valuation
Quantifying the contribution each modality (language, visual, and acoustic) is fundamental to tackling modality imbalance in multi-modal intent recognition (MIR) systems. The Leave-One One-Out (LOO) method offers a practical and efficient solution for this, contrasting with the computationally demanding Shapley value approach, which scales exponentially with the number of modalities.
For a given sample i, we start with features from the language (m²), acoustic (m/),, and visual (mx)) modalities. These are combined to produce fused representation that encapsulates the joint information:
F;
m m.)).

---

where Fuse(-) denotes the fusion function provided by the base multimodal learning architecture. Importantly. ARL operates as lightweight plug-in module and does not modify or participate in the modality fusion process. To evaluate the importance of specific modality E {l, a, v), we mask that modality by replacing its features with a zero vector. This simulates the absence of and allows us to compute a modified fused representation:
F m) (useeSeqAlignnedmm m,)).
(8)
where rij, mig and vic denote the inputs with modality m masked (e.g., if m = a, then m - 0, while m m and m = m F(-m) represents the fused representation obtained when the modality m masked, allowing us analyze the impact of on the overall representation.
The contribution of modality m, denoted olm), determined by comparing the original fused
representation F; with the modified one F F(-m) using cosine similarity. This metric measures the angular difference hetween the two vectors, indicating the impact of modality m on the fused output. A high cosine similarity (close to 1) means the representations
nearly aligned, suggesting that masking m has minimal impact and thus m contributes little. A low similarity (closer to 0 or negative) indicates significant shift, implying that m is highly influential. The contribution is formalized as:
(m) exp(-n Sim FF; F. {{-)))
(9)
Σ ml exp(-1) Sim Sim(F,,5 F. (0-m"")
where Sim(.) is the cosine similarity, and 17 is a sensitivity parameter that adjusts the sharpness of the contribution scores. A larger 1j amplifies differences in similarity scores, making the distinction between high- and low-contributing modalities more pronounced.
Unlike Shapley methods requiring 2M combinations (where M is the number of modalities), LOO needs only M forward passes per sample, reducing complexity from exponential to finear-crucht for real-time MIR. This process, depicted in Fig. (b), enables sample-specific modality valuation, allowing the system to adapt dynamically to varying modality importance across different inputs.
3.3 Contribution-Inverse Sample Calibration
Building on the modality contributions derived from LOO, Contribution Innerrs Sample Calibration (CISC) addresses modality dominance by adjusting features at the sample level. The principle is to reduce reliance on overly influential modalities, encouraging the model to utilize weaker but potentially valuable ones, thus promoting balanced multi-modal representation.
For each sample and modality m, we use the LOO contribution mm)) and compare it to a threshold If 5(m) 2 indicating that modality dominates the fused representation, we mask its features by setting them to zero. Otherwise, the features remain intact:
2 (m))
if (m)
(m)
T
(10)
0.
if
5 m)
ZT.
where (m) are the original features, and (m)
are the calibrated features. By removing the influence of a dominant modality (e.g., audio in a noisy environment), the model must rely on the remaining modalities, enhancing robustness (see Fig. (c)). The threshold governs the masking aggressiveness; a lower masks more modalities, potentially improving balance but risking information loss, while a higher is more conservative. The calibrated features then fused:
Fuse (ensAASellig (,))) _!(() ",("))))))
(11)
driving intent prediction. This per-sample adjustment adapts to context-specific modality importance, making the MIR system versatile across diverse scenarios.
Our use of hard masking (zeroing out features) is deliberate design choice. The motivation is to introduce a strong regularization effect, decisively breaking the model's over-reliance on dominant modalities. An alternative could be soft masking, where features are only attenuated. We empirically validate our choice in Sec. 4.2 where we show that hard masking leads to a more robust and balanced representation.
5

---

3.4 Weighted Encoder Calibration
While CISC handles sample-level imbalances, Weighted Encoder Calibration (WEC) ensures long- term stability and balance by adjusting modality-specific encoders based on their global performance. We introduce the purity score as a measure of class consistency within clusters. It quantifies the proportion of the dominant true class in each cluster and is formally defined as:
1 max # J. k),,
Purity(y, c)
(12)
where E N denotes the vector of ground-truth class labels, E No represents the cluster assignments, K is the number of clusters, 3/ the set of all class labels, and # yj = j, G - k} counts the number of samples from class in cluster k. This metric reflects how well the clustering preserves the underlying class structure. Using this metric, we assess the stability of each modality across training and validation sets based on the consistency gap:
P train
val Pm
(13)
where pir and pir are purity scores for modality on training and validation sets. A high Pm indicates inconsistency, such as overfitting.
Combining this with the average LOO contribution 8(m), WEC computes dynamic weight:
Wm tanh(A Pm + a 8(m)),
(14)
where À and a tune the influence of stability and contribution. The encoder parameters are then adjusted:
new
wm
(.()) ini
+ (1 W.) (m)
(15)
shifting unstable or underused encoders toward their initial state while preserving learned parameters
for stable ones. Applied every T epochs (see Fig. 2). WEC prevents long-term bias, enhancing generalization across training.
3.5
The Synergy CISC and WEC
CISC and WEC synergistically mitigate modality im- balance through complementary short-term and long- term calibration. CISC adjusts features per batch, mask- ing dominant modalities to emphasize weaker ones, adapting to sample-specific variations [Fig[[)) WEC, invoked periodically, refines encoders using aggregated contribution and stability metrics, ensuring balanced learning over time. During training, CISC operates at each batch, followed by WEC every T epochs, creating an iterative optimization loop. This dual-phase strat- egy, visualized across Fig. enhances ARL's ability to handle dynamic modality importance, delivering supe- rior accuracy and efficiency, vital for real-world MIR applications like robotics and smart assistants. To pro- vide clear, step-by-step overview of the entire training process, we include a detailed pseudocode of the ARL training procedure in Appendix A.4 A.4
Representation
Origin Weight
ball
M
C
Wesight
Callbrated Encoders
Modal
Aligned Reprosenatiio
Statistical Varance
Figure 2: Overview of WEC. The mod- ule processes training and validation sets compute consistency gap and modal con- tributions, adjusting encoder to ensure bal- ance across modalities.
4 Experiments
Datasets. We evaluate our ARL framework on two multimodal benchmarks: MintRec [21 for intent recognition, and MOSI 43 for sentiment analysis. addition to multimodal intention recognition, we posit that multimodal sentiment analysis also suffers from modal imbalance issues. Therefore, we incorporate MOSI dataset as additional evaluation benchmark. The MintRec dataset comprises 2,224 samples, split into 1,334 for training, 445 for validation, and 445 for testing. It supports two

---

Table 1: Performance comparison of various multimodal intention understanding methods the MIntRec dataset. Our proposed Adaptive Re-calibration Learning (ARL) framework consistently improves the performance of baseline methods (MAG BERT, MulT, and TCL (CLMAPP across all metrics for both twenty-class fine-grained classification and binary coarse-grained classification tasks.
Twenty-class
Binary
Method
ACC
FI(WFI)
P(WP)
R
ACC
FI(WFI)
P(WP)
R
MAG BERT 29
72.65
68.64
69.08
69.28
89.24
89.1
89.1
89.13
w/ ARL
74.38
71.66
71.55
72.89
90.11
90.01
89.92
90.14
A Improvement
+1.73
+3.02
+2.47
+3.61
+0.87
+0.91
+0.82
+1.01
MulT (27
72.52
69.25
70.25
69.24
89.19
89.07
89.02
89.18
w/ ARL
73.71
70.82
72.06
70.45
89.89
89.78
89.70
89.88
Improvement
+1.19
+1.57
+1.81
+ +1.21
+0.70
+0.71
+0.68
+0.70
TCL_MAP 11
73.62
73.31
73.72
70.50
89.66
89.69
89.84
89.84
w/ ARL
75.28
75.16
75.71
73.35
90.56
90.57
90.59
90.54
A Improvement +1.66
+1.85
+2.00
+2.85
+0.90
+0.88
+0.75
+0.70
Table 2: Comparative experimental results of the sentiment analysis models on the public MOSI dataset. The best results are highlighted in bold.
Table 3: Impact of hyperparameter on the MOSI. Best results per metric are in bold.
Method
ACC
FI(WF1)
P(WP)
MAG BERT
42.35
36.54
37.51
37.36
w/ ARL
43.22
37.44
38.23
38.53
a
ACC
F1
AVG
0.01
42.42
36.32
36.77
38.23
38.43
A Improv.
+0.87
+0.90
+0.72
+1.17
0.1
42.56
35.75
37.58
37.35
38.31
MulT
42.42
37.24
37.00
39.05
0.5
42.78
37.40
38.21
38.07
39.11
w/ ARL
43.44
38.50
38.97
40.42
43.80
36.69
37.80
37.91
39.05
A Improv:
+1.02
+1.26
+1.97
+1.37
43.22
37.44
38.23
38.53
39.35
MAP
42.13
40.54
42.68
37.17
41.62
37.30
38.32
37.98
38.80
w/ ARL
43.73
42.60
43.73
38.38
Improv.
+1.60
+2.06
+1.05
+1.21
levels of intent classification: coarse-grained, with binary labels distinguishing between expressing emotions and achieving goals, and fine-grained with twenty labels (11 for expressing emotions and for achieving goals). The MOSI dataset consists of 2,199 samples, divided into 1,284 for training, 229 for validation, and 686 for testing, with sentiment scores ranging from -3 (highly negative) to (highly positive). For MOSI, we discretize the continuous sentiment scores into labels from to 6 for classification tasks.
Evaluation Metrics. For MintRec, we report Accuracy (ACC), F1-score (F1), Precision (P), and Recall (R) to evaluate fine-grained intention recognition performance. For MOSI, we apply the same metrics to assess the 7-class sentiment classification task. Additionally, when integrating with TCL-MAP [II, which uses weighted Fl (WFI) and weighted Precision (WP), we include these metrics to ensure compatibility with its evaluation standards.
Implementation Details. Our -od--and-play module is incorporated into three established methods: MAG-BERT (29] MulT (27), and TCL-MAP [I]. We adopt hyper- paramerrss such as learning rate, batch size, and optimizer settings from the publicly released configurations of these baseline methods. We tune specific hyper-parameters for ARL, including masking threshold CISC, weight adjustment factor in WEC, optimizing them via grid search. For MIntRec, we use pre-extracted features with dimensions 768 for text, 256 for visual, and 768 for acoustie. For MOSI, feature dimensions are 768 for text, 20 for video, and 5 for audio. All experiments conducted on 4 NVIDIA 4090 GPUs.

---

Table 4: Ablation study of the proposed model on MIntRec. The best results are highlighted in bold.
Twenty-class
Binary
Method
CISC
WEC
ACC FI/WFI
P/WP
R
AVG
ACC
FI/WFI
P/WP
R
AVG
72.65
68.64
69.08
69.28
69.91 89.24
89.10
89.10
89.13
89.14
72.81
68.81
69.79
69.32
70.18
89.55
89.43
89.41
89.55
89.48
72.92
69.58
69.94
70.34
70.69
89.66
89.57
89.46
89.79 89.62
74.38
71.66
71.55
72.89
72.62
90.11
90.01
89.92
90.14 90.04
72.52
69.25
70.25
69.24
70.31
89.19
89.07
89.02
89.18 89.11
73.03
69.80
71.04
69.16
70.75 89.44
89.33
89.23
89.48 89.37
MulT
72.80
69.74
70.62
70.00
70.79
89.21
89.05
89.14
88.97 89.09
73.71
70.82
72.06
70.45
71.76 89.89
89.78
89.70 89.88 89.81
73.62
73.31
73.72
70.50
72.78
89.66
89.69
89. 89.84 89.84 89.75
73.93
73.76
74.37
71.50
73.39
90.34
90.30
90.38 89.92 90.23
TCL_MAP
74.16
73.37
73.30
71.01
72.96
90.00
90.02
90.11 90.08 90.05
75.28
75.16
75.71
73.35
74.87
90.56
90.57
90.59
90.54 90.56
4.1 Comparison with the State-of-the-art
As shown in Tabs. I and 2 our experimental analysis on the MIntRec and CMU-MOSI datasets demonstrates the consistent effectiveness of the ARL framework in enhancing multimodal intention recognition by addressing modality imbalance. On MIntRec, we evaluated state-of-the. methods such as MAG-BERT, MulT, and TCL-MAP, and found that integrating ARL improves performance in both twenty-class and binary classification tasks. For example, TCL-MAP combined with ARL achieves a twenty-class accuracy of 75.28 and binary accuracy of 90.56, with consistent gains across accuracy, Fl score, precision, and recall. Similarly, on CMU-MOSL, benchmark for multimodal sentiment analysis, ARL improves all baseline methods. TCL-MAP, in particular, shows the most notable recalibration of modality importance, reaching accuracy of 43.73. These results highlight that ARL not only improves performance but also adaptively recalibrates modality contributions to address -ataset-specific challenges, making it versatile framework for advancing multimodal learning. To further contextualize the performance of ARL, we conducted additional experiments comparing it directly against representative imbalanced learning methods and evaluating its plug and- play capability on other recent state-of-the-an models for multimodal understanding. In all cases, ARI demonstrated significant improvements, validating its effectiveness and versatility. The detailed results of these comprehensive comparisons are provided in Appendix A.5
4.2 Ablation Studies
Effectiveness Proposed Components. To assess the contributions of the core components in the ARL framework, namely CISC and WEC, we conduct ablation studies on the MIntRec dataset for both twenty-class and binary classification tasks. As shown Tab. 4 CISC and WEC each improve baseline performance when applied independently. CISC reduces modality bias by masking dominant modalities, while WEC refines modality-specific encoders based on purity variation and LOO contribution. Their combination yiclds a synergistic effect: in the twenty-class task, MAG BERT's AVG increases from 69.91 to 72.62, and TCL MAP reaches 74.87, For binary classification, despite already strong baselines, ARL consistently boosts performance, with TCL MAP improving to 90.56. Although WEC alone slightly reduces MulT's binary AVG, its combination with CISC recovers and enhances performance. These results highlight ARL's adaptability and its effectiveness in addressing modality imbalance through complementary mechanisms.
Sensitivity Analysis of Calibration Weight a and Masking Threshold r. To evaluate the role of modality contributions estimated by the I.O method adjusting calibration weights within WEC, conducted series of experiments the MOSI dataset. The results, summarized in Tab. 3 focus
8

---

Table 5: Sensitivity analysis of the CISC masking threshold on the MIntRec (twenty-class) dataset using the MulT model. The baseline performance corresponds to r = 1.0. Best results are in bold.
Table Comparison of Hard Masking (our ARL) and Soft Masking on MIntRec with MAG-BERT: The baseline performance corresponds to not us- ing mask. Best results are in bold.
T
ACC
F1
P
R
Method
ACC
F1
P
R
1.0
72.52
69.25
70.25
69.24
Baseline
72.65
68.64
69.08
69.28
0.9
72.58
69.15
70.04
69.15
HardMask
74.38
71.66
71.55
72.89
0.7
71.24
67.98
68.70
68.11
A Improv.
+1.73
43.02
+2,47
43.61
0.5
73.71
70.82
72.06
70.45
0.3
73.26
69.94
71.23
69.64
w/ SoftMask
73.24
67.31
70.14
69.57
0.1
70.79
67.16
68.27
66.52
A Improv.
+0.59
-1.33
+1.06
+0.71
on the effect of different a values (0.01, 0.1, 0.5, 5, 10) on the performance of the MAG BERT model. Here, a controls the influence of LOO-derived contributions on WEC. The results show that when a 1, the model achieves the highest accuracy (43.80) and a relatively high recall (37.91), although the F1 score (36.69) is slightly lower than that at a 5. At = 5, the model obtains the highest F1 score (37.44) and peak recall (38.53), resulting in the best overall performance (AVG 39.35). In contrast, setting a too low (e.g.. 0.01) or too high (e.g., 10) leads to performance degradation across all metrics, indicating the importance of selecting an appropriate a value to balance performance, 1 and o 5 yield the best results, confirming that incorporating LOO- based modality contributions into WEC can effectively enhance model performance when properly calibrated.
To further demonstrate the robustness our ARL framework, we provide a sensitivity analysis for the Contribution- Contribution-Inveree Sample Calibration (CISC) masking threshold, т. This hyperparameter governs the aggressiveness of the masking mechanism; a lower 7 means modalities are masked more frequently. The analysis was conducted using the MulT model on the MIntRec (twenty-class) dataset.
As shown in Tab. 5) the model achieves optimal performance across all metries when 0.5. A value of 1.0 is equivalent to the baseline model without CISC. While performance degrades with very aggressive masking (e.g.. 7 0.1) or overly conservative masking, there stable range where the model benefits from the calibration. We observed similar trends across our other experimental setups. This analysis demonstrates that while the hyperparameter is important, its optimal value can be reliably identified via standard tuning procedures.
Effectiveness of Hard Masking To validate choice of hard masking in the CISC module, conducted an experiment comparing its performance against a soft masking alternative. the soft masking setup, instead of being zeroed out, the features of dominant modality attenuated (e.g..
multiplied by a factor 1 (())
The results, presented in Tab. 6 clearly show that our proposed hard masking approach yields significant and consistent improvements across all evaluation metrics. While soft masking provided a modest improvement in Accuracy and Precision, it was detrimental to the F1-score and Recall. We hypothesize that this is because an attenuated signal insufficient to compel the model to fundamentally alter its strategy. Hard masking, in contrast, forces the model to actively learn from underutilized modalities, leading to superior Fl and Recall scores which indicate a more robust and balanced model.
Generalizability under Missing Modalities. To assess the robustness of the ARL framework, we conduct ablation studies under missing modality conditions on the CMU-MOSI dataset. We test three modality pairs: {a,e) (audio-visual), {v, t} (visual-text), and {a, f} (audio-text), evaluating performance improvements for both MAG-BERT and MulT models. As shown in Tab. 7 ARL consistently enhances accuracy (ACCt, ACC₂) and F1 scores (F17. F12) across all settings. For example, in the {a, y scenario, ARL increases MulT's ACC₁ from 22. 13 to 23.09 and ACC₂ from 59.79 60.67. The largest gains oceur in the {a, I) setting, where MAG-BERT'S ACC; rises by 1.57 (to 42.82) and ACC₂ by 1.08 (to 81.98), underscoring ARL's ability to leverage textual information, which is vital for sentiment analysis. These results highlight ARL's adaptability and effectiveness in mitigating modality imbalance across diverse configurations and architectures.

---

Table 7: Performance under missing modality scenarios CMU-MOSI. Best results are in bold.
{
[vrt
(a.t)
Method
ACC²
Fl7
ACC₂
Fl2
ACC₁
Fl7
ACC₂
Fl2
ACC₂
Fl7
ACC₂
F12
MAG_BB BEERT
17.78
7.50
48.13
45.29
40.00
35.06
81.07
80.79
41.25
35.69
80.90
80.56
w/ ARL
19.30
8.24
48.80
46.77
41.02
35.89
81.63
81.43
42.82
35.70
81.98
81.73
A Improvement
+1.52
+0.74
+0.67
+1.48
+1.02
+0.83
+0.56
+0.64
+1.57
+0.01
+1.08
+1.17
MulT
22.13
15.93
59.79
58.74
40,90
36.57
81.24
81.03
41.19
36.55
81.22
80.88
w/ ARL
23.09
16.22
60.67
60.21
41.89
37.03
81.54
81.19
42.27
36.76
81.51
81.30
Improvement
+0.96
+0.29
+0.88
+1.47
+0.99
+0.46
+0.30
+0.16
+1.08
+0.21
+0.29
+0.42
Computational Cost and Scalability of LOO. To evaluate the practicality of our Leave-One-Out (LOO) modality valuation method compared to Shapley- approaches, we analyze computa- tional cost with respect to two key factors: fea- ture dimension and number of modalities. This addresses scalability concerns raised Sec.
Computation Time Comparison by Feature Dimension
Stap
612 dimensine direreoos
20
LGO
Computation Time (4 log scal))
10
As shown in Fig. 3 both methods scale linearly with feature dimension, as expected. However, they differ significantly in scalability with re- spect to the number of modalities. Shapley- based methods suffer from exponential com- plexity (O(n2")), due to evaluating all possible modality subsets. In contrast, LOO demonstrates linear growth (O(n)), requiring only single for- ward pass per modality. This makes LOO far more scalable and suitable for real-world MIR applications with multiple modalities.
0.
02
01
Figure 3: Computational cost for LOO and Shap- ley methods.
Conclusion and Discussion
In conclusion, we have presented Adaptive Re-calibration Learning (ARL), a novel framework designed mitigate modality imbalance Multimodal Intention Recognition and enhance the robustness of multimodal learning. ARL leverages our proposed Leave-One-Out (LOO) modality valuation method, a computationally efficient alternative to Shapley values, to power a two-phase calibration strategy. Through Contribution-Invenr Inverse Sample Calibration (CISC) and Weighted Encoder Calibration (WEC), ARL dynamically refines model learning by addressing both sample-specific and modality-level imbalances. Our extensive evaluations across MIntRec and MOSI datasets demonstrate ARL's effectiveness in boosting performance over strong baselines. Crucially. ARL overcomes the scalability bottleneck inherent in Shapley-base based approaches, offering a practical and efficient solution for real-world MIR applications.
Limitations of ARL. Although ARI. has achieved remarkable results in alleviating the modality imbalance problem in multimodal intent recognition, still has some limitations. First, the perfor- mance of ARL depends the expressiveness of the underlying model and the design of the fusion mechanism. If the base model itself has biases understanding certain modalities or lacks sufficient feature extraction capabilities, ARL may not be able to fully compensate for these shortcomings. In addition, the computational overhead increases. Although the LOO Srave-One-Oue method has linear complexity compared Shapley values, in large-scat scale datasets multimodal tasks, each forward pass used to assess modality contributions still introduces extra computational costs. The dual calibration mechanisms of WEC and CISC improve performance, but they also lead to increased training time and resource consumption.
10

---

Acknowledgements. This work is partially supported by the National Key Research and Development Program of China (2024YF33305400,, National Natural Science Foundation of China under Grant (62176188, 62361 166629), Major Project of Science and Technology Innovation of Hubei Province (2024BCA003, 2025BEA002).
References
[1] Rongjic Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jia-Bin Huang, Jinglin Liu, Yixiang Ren, Zhou Zhao, and Shinji Watanabe. Audiogpt: Understanding and generating speech, music, sound, and talking head, ArXiv, 2023.
[2] Cecilia Panigutti, Andrea Beretta, Fosca Giannotti, and Dino Pedreschi. Understanding the impact of explanations on advice-taking: user study for ai- bi-dased clinical decision support systems. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022.
[3] Raffaele Filieri, Zhibin Lin, Yulei Li. Xiaoqian Lu, and Xingwei Yang. Customer emotions in service robot encounters: A hybrid machine- humann intelligence approach. Journal of Service Research, 2022.
Mang Ye, Qinghongya Shi, Kehua Su, and Bo Du. Cross-modality pyramid alignment for visual intention understanding. IEEE Transactions on Image Processing, 2023.
[5] Mang Yc, Shuoyi Chen, Chenyue Li, Wei-Shi Zheng, David Crandall, and Bo Du, Transformer for object re-identification: A survey. International Journal of Computer Vision, 2025.
[6] Amir Zadeh, Paul Pu Liang. Soujanya Poria, Prateek Vij, Erik Cambria, and Louis-Philippe Morency. Multi-attention recurrent network for human communication comprehension. In Phirty--econd AAAI Conference on Artificial Intelligence, 2018
[7] Ruining Chong, Cunliang Kong, Liu Wu, Zhenghao Liu. Ziye Jin, Liner Yang. Yange Fan, Hanghang Fan, and Erhong Yang. Leveraging prefix transfer for multi-intent text revision. In Annual Meeting of the Association for Computational Linguistics, 2023.
[8] Yawen Ouyang, Jiasheng Ye, Yu Chen, Xinyu Dai, Shujian Huang, and Jiajun Chen. Energy-based unknown intent detection with data manipulation. ArXiv, 2021.
[9] Jyoti Aneja, Harsh Agrawal, Dhruv Batra, and Alexander G. Schwing. Sequential latent spaces for modeling the intention during diverse image captioning. 2019 IEEE/CVF International Conference on Computer Vision, 2019.
[10] Kuan-Chuan Peng, Amir Sadovnik, Andrew C. Gallagher, and Tsuhan Chen. Where do emotions come from? predicting the emotion stimuli map. 2016 IEEE International Conference Image Processing, 2016.
[11] Qianrui Zhou, Hua Xu, Hao Li, Hanlei Zhang, Xiaohan Zhang, Yifan Wang, and Kai Gao. Token-level contrastive learning with modality-aware prompting for multimodal intent recognition. ArXiv, 2023.
[12] Kaixuan Chen, Lina Yao, Dalin Zhang, Xianzhi Wang. Xiaojun Chang, and Feiping Nic. A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Transactions on Neural Networks and Learning Systems, 2020.
[13] Yake Wei, Siwei Li, Ruoxuan Feng, and Di Hu. Diagnosing and re-learning for balanced multimodal learning. European Conference on Computer Vision, 2024.
[14] Y. Wei, R. Feng, Z. Wang, and Hu. Enhancing multimodal cooperation sample-level modality valuation. In Proceedings of the 2024 IEEESCVF Conference on Computer Vision and Pattern Recognition, 2024.
[15] Tao Meng, Yuntao Shou, Wei Ai, Nan Yin, and Keqin Li. Deep imbalanced learning for multimodal emotion recognition in conversations. IEFE Transactions on Artificial Intelligence, 2023.
[16] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[17] Hao Li, Xin Li, Peng Hu, Yi Lei, Cuicui Li, and Yue Zhou. Boosting modai- model performance with adaptive gradient modulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
11

---

[18] Kailai Zhou, Linsen Chen, and Xun Cao. Improving multispectral pedestrian detection by addressing modality imbalance problems. In European Conference on Computer Vision, 2020.
[19] Arindam Das. Sudip Das, Ganesh Sistu, Jonathan Horgan, Ujjwal Bhattacharya, Edward Jones, Martin Glavin, and Ciarán Fising. Revisiting modality imbalance in multimodal pedestrian detection. 2023 IEEE International Conference on Image Processing 2023.
[20] Fushuo Huo, Wenchao Xu, Jingeai Guo. Haozhao Wang, and Song Guo. C2kd: Bridging the modality gap for cross-modar knowledge distillation. 2024 IEEE/CVF Conference on Computer Vision and Pottern Recognition, 2024.
[21] Hanlei Zhang, Hua Xu, Xin Wang, Qianrui Zhou, Shaojie Zhao, and Jiayan Teng. Mintrec: A new dataset for multimodal intent recognition. In Proceedings of the 30th ACM International Conference on Multimedia, 2022.
[22] Qu Yang. Mang Yc, and Bo Du. Emollm: Multimodal emotional understanding meets large language models. arXiv preprint arXiv:2406. 16442, 2024.
[23] Qu Yang, Mang Ye, and Dacheng Tao. Synergy of sight and semanties: visual intention understanding with clip. In European Conference on Computer Vision, 2024.
[24] Qu Yang, Qinghongya Shi, Tongxin Wang, and Mang Ye. Uncertain multimodal intention and emotion understanding in the wild. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
[25] Gopendra Vikram Singh, Mauajama Firdaus, Asif Ekbal, and Pushpak Bhattacharyya. Emoint-trans: A multimodal transformer for identifying emotions and intents in social conversations. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
[26] Yishuang Ning, Jia Jia, Zhiyong Wu, Runnan Li, Yongsheng An, Yanfeng Wang, and Helen M. Meng. Multi-task deep learning for user intention understanding in speech interaction systems. In AAAI Conference on Artificial Intelligence, 2017.
[27] Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the conference. Association for computational linguistics. Meeting, 2019.
[28] Devamanyu Hazarika, Roger Zimmermann, and Soujanya Poria. Misa: Modality-invariant and-specific representations for multimodal sentiment analysis. In Proceedings of the 28th ACM international conference on multimedia, 2020.
[29] Wasifur Rahman, Md Kamrul Hasan, Sangwu Lee, Amir Zadeh, Chengfeng Mao, Louis Souis-Philippe Morency, and Ehsan Hoque Integrating multimodal information in large pretrained transformers. In Proceddiggs of the conference. Association for Computational Linguistics. Meeting, 2020.
[30] Kaili Sun, Zhiwen Xie. Mang Ye, and Huyin Zhang. Contextual augmented global contrast for multi- modal intent recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[31] Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang, and I Longbo Huang. Modality competition: What makes joint training of multi-modal network fail in deep learning? ArXiv, 2022.
[32] Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. Balanced multimodal learning via on-the-fly gradient modulation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2022.
[33] Yichi Zhang, Zhuo Chen, Lei Liang, Hua zeng Chen, and Wen Zhang. Unlcashing the power of imbalanced modality information for multi- multi-modal modal knowledge graph completion. ArXiv, 2024.
[34] Yunfeng Fan, Wenchao Xu, Haozhao Wang, Junxiao Wang, and Song Guo. Pmr: Prototypical modal rebal- ance for multimodal learning. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[35] Yake Wei and Di Hu. Mmpareto: Boosting multimodal learning with innocent unimodal assistance. ArXiv, 2024.
[36] Nan Wu, Stanislaw Jastrzebski, Kyunghyun Cho, and Krzysztof J. Geras. Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks. ArXiv, 2022.
12

---

[37] Zequn Yang, Yake Wei. Ce Liang, and Di Hu. Quantifying and enhancing multi-modal robustness with modality preference. ArXiv, 2024.
[38] Anbin Qi Zhongliang Liu, Xinyong Zhou, Jinba Xiao, Fengrun Zhang, Qi Gan, Ming Tao, Gaozheng Zhang, and Lu Zhang. Multimodal emotion recognition with vision- language prompting and modality dropout. ArXiv, 2024.
[39] Yusheng Dai, Hang Chen, Jun Du, Ruoyu Wang. Shihao Chen, Jie Ma, Haotian Wang, and Chin-Hui Lee. A study of dropout-inducee modality bias robustness to missing video frames for audio-visual speech recognition. 2024 IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
[40] Saghir Alfasly, Jian Lu, Xu, Yuru Zou. Learnable irrelevant modality dropout multimodal action recognition on modality-specific annotated videos. 2022 IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2022.
[41] Zelei Liu, Yuanyuan Chen, Yansong Zhao, Han Yu, Yang Liu,, Renyi Bao, Jinpeng Jiang, Zaiqing Nie, Qian Xu, and Qiang Yang. Contribution-awwe -ware federated learning for smart healthcare. In AAAI Conference on Artificial Intelligence, 2022.
[42] Shicai Wei, Yang Luo, and Chunbo Luo. Mmanet: Margin aware distillation and modality aware regular- ization for incomplete multimodal learning. 2023 IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
[43] Amir Zadch, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency. Multimodal sentiment intensity analysis in vidcos: Facial gestures and verbal messages. IEEE Intelligent Systems, 2016.
[44] Pietro Morerio, Jacopo Cavazza, Riccardo Volpi. René Vidal, and Vittorio Murino. Curriculum dropout. In Proceedings of IEEE international conference on computer vision, 2017.
[45] Yake Wei, Di Hu, Henghui Du, and Ji- Ri-Rong Wen. On-the-fit fly modulation for balanced multimodal lcarning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024.
[46] Chengxiang Huang, Yake Wei, Zequn Yang, and Di Hu. Adaptive unimodal regulation for balanced multimodal information acquisition. In Proceedings of Computer Vision and Puttern Recognition Conference, 2025.
[47] Yong Li, Yuanzhi Wang, and Zhen Cui. Decoupled multimodal distilling for emotion recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We introduced our contribution in both abstract and Sec.
Guidelines:
The answer NA means that abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match thcorctical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitation of our dataset and experiments in Sec.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed the paper.
The authors are encouraged create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results these assumptions (e.g.. independence assumptions, noiseless settings, model well well-specificen specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
violations of
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should he articulated.
The authors should reflect the factors that inffuence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper has no theoretical analysis.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
14

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix or supplemental material.
Theorems and Lemmas the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided all the information needed to reproduce the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g., in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct
dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome to describe the particular way they provide for reproducibility. In the of cosed-source models, it may be that access to the model is limited some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open faithfully reproduce the main experimental results, described supplemental material?
the data and code, with sufficient instructions
Answer: [Yes]
Justification:
The code used in this work will be announced soon.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... http::://nips.cc/putteee ccpbblicc/ quides/CodeSubeen seibbbbiiisi for more details.
While we encourage the release of code and data, we understand that this might not be possihle, so "Na" an acceptable answer. Papers cannot rejected simply for not including code, unless this is central to the contribution (e.g.. for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: :/ :/nniss//.///////////e:::eee cc/public/ qu..dess/ooeeSuubllesssoool..... for more details.
15

---

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) recommended, but including URLs to data and code is permitted.
6. Experimental sefting/details
Question: Does the paper specify all the training and test details (c.g,, data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We introduced various experimental settings in detail in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please refer to Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should given (e-g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error than state that they not verified.
if the hypothesis of Normality of errors
asymmetric distributions, authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). crror bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, docs the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Please refer to Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
16

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform. every respect, with NeurIPS Code Ethics htttts://neurr// httpp/////eerrp....cppblieeeeeee cctpppublic/tbhiceeeiiielelllll
Answer: [No]
Justification:
Our paper does not include ethical problems.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e-g., if there is a special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to Sec. and appendix A.2
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations. (e.g.. deploy- of technologies that could make decisions that unfairly impact specific groups), privacy considcrations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated relcase of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible relcase of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
There is no risk in using the dataset.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
17

---

12. Licenses for existing assets
Question: Are creators or original owners assets (e.g., code, data, models), used the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected"'
Answer: [Yes]
Justification: We properly credited original assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, ppaperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged reach out asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification:
No new assets were introduced in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training. license, limitations.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, applicable, as as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection. curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
18

---

The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
recognize that procedures this may vary significantly between institutions and Iocations, and we expect authors adhere to the NeurIPS Code of Ethics the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review,
Declaration LLM usage
Does the paper describe the usage of LLMs if it important, original, or non-standard component of the core methods in this rescarch? Note that if the LLM is used only for writing, editing, or formatting purposes and does impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [No]
Justification: This article does not use LLM.
Guidelines:
answer NA means that core method development this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https hattps:/nnerrppe:: http:/upw......ooo.eeerre/////// for what should or should not be described.
19

---

A Technical Appendices and Supplementary Material
A.1 Variable summary
See Tab.
Table 8: Variables and Definitions in the ARL
Variable
Definition
{L, V,
Input source modalities: Language (L), Visual (V), and Acoustic (A).
Text modality feature: m, (ex R4xa,
my
Audio modality feature: m. = AudioEmbeddAA] E Rloxd,
ma
Video modality feature: my VidcoEmbed(()) (V) Rl.xd.
my
Sequence lengths of respective modalities (c.g., tokens, frames). Feature dimensions of each modality.
dufafe
hy Bas he
Aligned multimodal features: Aeggnigg,,,,,, May m.)) E Rixd Unified sequence length after alignment. Projected feature dimension.
d
hm
Final fused representation: hm Fhh,, hay ER. Multimodal fusion function. Intent prediction output: Softmax(W (W hm + b). Weight matrix: W E Raxc
F()
logits
W
b
Bias vector: E R.C Number of classes. Original fused representation for sample i: FuseeSSeAAligned(mm ml, mt]))
C
F,
F mm
Modified fused representation with modality masked: Fuse (SgglSeg/llggned( mi, m.))
Modality m contribution: d!m)) En exp(1y (F
explyy F. p
Sim(-)
Cosine similarity function.
n
Sensitivity parameter to control sharpness of contribution scores.
Calibrated modality feature: zero if 6!(²) r, else unchanged. Threshold to determine whether to mask modality.
F
Fused representation using calibrated features: Fuse Alignddd)) .3(") ..(")) Purity score formula: Purity(y, c) = ΣK= maxjey # [1: M4 -. k} Ground truth class label vector.
Purity (s..c)
y
Cluster assignment vector.
K
Number of clusters.
y
Set of all class labels.
# (i:
J.
k}
Count of samples in cluster that belong to class j.
P m
Modality inconsistency measure: Pm ppain pial
trin
training Purity score of modality m on validation set.
Purity score of modality
Pm
Dynamic weight: Wi. tanh(A - P., a 5((m)) Hyperparameter controlling stability impact.
X
Hyperparameter controlling average 1.0 contribution impact.
glm)
Updated encoder parameters: alm) = w., gp(m) (1 am.)) 0
g(m)
Initial encoder parameters for modality m.
current
Current encoder parameters for modality
A.2 Broader Impacts of ARL
ARL proposes innovative multimodal intent recognition framework that has demonstrated significant effec- tiveness in addressing the issue of modality inbalance, showing great potential for practical applications. Firstly, enhances the robustness and generalization capability of multimodal systems, which crucial for the develop- ment of more reliable intelligent assistants, service robots, and autonomous driving systems. Morcover, ARL introduces a dual- dual--path calibration mechanism (CISC + WEC), offering new perspectives and methodological insights for future research, and contributing to the advancement of multimodal collaborative learning. In partic- ular, when dealing with modality imbalance, this approach exhibits greater flexibility and efficiency compared traditional methods. Furthermore, ARL performs robustly under conditions of modality degradation or absence, which is especially important in real- world scenarios involving incomplete data or device malfunctions.
However, ARL also presents potential drawbacks. First, the increased model complexity poses challenges to interpretability. The inclusion of two additional modules (CISC and WEC) may complicate the model's decision- making process, reducing transparency, which could raise trust issues in applications requiring strict regulatory
20

---

compliance. In addition, privacy concerns must not be overlooked. Multimodal systems lypically involve various perceptual signals (e.g.. speech, images, text), and ARL's goal of better leveraging the synergy among these modalities inherently increases the risk privacy leakage. particularly in tasks involving personal identities or sensitive information. Finally, there a risk misuse. If ARL were to be applicd for malicious purposes such as generating highly convincing but deceptive intent recognition outputs or manipulative sentiment analysis could be exploited to create mislcading content or influence user behavior uncthically.
A.3 Formal Definition and Measurement of Modality Imbalance
During the review process, was suggested that a formal definition and a quantitative metric for modality imbalance would enhance the paper's rigor. We address this by introducing the Modality Imbalance Index (MII).
Formal Definition. We formally define modality imbalance as: "The degree of non-uniformity in the contribu- tions of different modalities to the final prediction for given multimodal sample."
Quantitative Metric (Modality Imbalance Index) We leverage our Leave- One Out (LOO) contribution scores create quantitative metrie. a sample with modalities, we have vector contribution scores A; [ 6/ 8 (/ ), where Σm 6/m) 1. An ideal balance implies that all modalities contribute equally (i.e., 0m L/M m).
quantify the deviation from this ideal state, we introduce the Modality Imbalance Index (MII), defined as the variance these contrihution scores:
M
MII(i) = Var (.))
M mmm1
M
A higher MII value indicates greater imbalance for sample i, signifying that one or few modalities are dominating the prediction.
Connection ARL Our proposed ARI framework is designed implicitly minimize this imbalance.
Contribution-Invvrse Sample Calibration (CISC) acts as a reactive, sample-level mechanism. directly targets samples with high MIl by masking dominant modalities whose contribution 5(m) exceeds threshold т. thereby encouraging the use of underutilized modalities.
Weighted Encoder Calibration (WEC) serves proactive, architectural-levd mechanism. It adjusts encoder weights hased on global contribution statistics (8(m)) and stability, aiming to reduce the average modality imbalance across the entire dataset over the long term.
A.4 ARL Training Algorithm
To enhance clarity and reproducibility. we provide the detailed pseudocode for the complete ARL training procedure in Algorithm| This algorithm outlines the interplay between I.OO valuation, CISC, WEC, and the base model optimization.
A.5 Extended Comparisons with State-of-the-art Methods
To comprehensively validate the effectiveness and generalizability of our proposed ARI. framework, we conducted two additional sets of experiments. First, compare ARL directly with several strong baselines specifically designed for inbalanced learning. Second, we apply ARL plug-and-play module other recent, high- performrfn multimodal models demonstrate versatility.
ARL significantly outperforms other methods designed tackle imbalance, such as gradient modulation (OGM, OPM) and regularization techniques. Furthermore, Tab. shows that ARL consistently enhances the performance of strong, contemporary models like DMD and EMOE. These results collectively underscore the superiority and broad applicability of our approach.
As shown in Tab, 9a
21

---

end for Algorithm Adaptive Re-calibration Learning (ARL) Training Procedure
Require: Training set Dtrain. Validation set Deat, Base model fe
Require: ARL hyperparameters: CISC threshold 5, WEC update frequency T, WEC weights X,
1: Initialize base model parameters 0 (encoders ((m), fusion net, etc.)
2: for epoch t- to max_ max_epochs do
3: for all batch {(m,, ma m v: in Draiin do
4: Compute full-modality fused representation F; for each sample i.
> Eq. (7)
5: for all m {t, a, u} do
Compute masked fused representation F! (m(-)
6:
>> Eq. (8)
7:
end for
Calculate contribution scores T! for each modality.
8:
b Eq. (9)
Calibrate unimodal features _i!")) using 6!m) and threshold т.
9:
Eq. (10)
10:
Compute final fused representation F; using calibrated features. Compute prediction loss C = LossSSoftmax( [F), y/)
Eq. (11)
11:
12:
Update base model parameters & via backpropagation end for
13:
14:
if (edoe 7) 0 then
15:
Calculate purity scores frain piaal and stability Pm.
> Eqs. (12), (13)
16:
Calculate average LOO contribution 8(m) Dea! Compute dynamic weights wm using P.. and 8(mm)
17:
Eq. (14)
18:
Adjust encoder weights o using Wm--
Eq. (15)
19:
end
Table 9; Extended performance comparisons the MIntRec and another benchmark dataset.
(a) Comparison with representative imbalanced learning meth- ods on the MintRec (twenty-clas)) dataset, using MAG-BERT
(b) Plug-and-play validation of ARI on other state tatt--of-the-add art multimodal emo- tion and intent recognition models.
the base model.
Method
ACC
FI(WF1) P(WP)
R
Method
ACC2
ACC7
F1
MAG BERT (Baseline)
72.65
68.64
69.08
69.28
CAGC 30
85.7
44.8
85.6
w/ ARL (Ours)
74.38
71.66
71.55
72.89
DMD
47
86.0
45.6
86.0
w/ Curriculum Dropout 44
67.15
66.21
67.02
66.85
DMD w/ ARL
86.8
46.3
86.7
w/ OGM 322
73.31
69.83
69.91
70.21
w/ OPM 4
73.24
69,22
69.36
70.
w/ InfoReg 46
73.55
70.12
70.25
69.58
22


Reviews:
[{"review_id": "3aIq1pCE0f", "reviewer": "Reviewer_A7Ss", "summary": "The paper addresses the robustness in performance of multimodal intention recognition under modality-compromised conditions. It proposes Adaptive recalibration learning to model modality importance. Experiments are conducted on two benchmark intention detection datasets.", "questions": "1) The authors are doing hard masking (Lines 185). Why is it necessary? If the weaker modalities are too weak, which is quite often the case, this might actually hurt the performance. Did the authors consider trying soft masking?\n\n\n2) Leave one out method kind of assumes that modalities are independent of each other, which is not usually the case. How does inter-modality dependence effect the proposed method?\n\n3) The threshold in Eq 10, is it tuned for each modality separately, one for each dataset, or one across all experiments? \n\n4) Same question as 3 for the η sensitivity parameter in Eq 9.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is easy to follow and written well. The motivation is clear and the idea is explained well. The idea is novel, to the best of my knowledge.\n\n\nThe idea is dependent on the base model and may not be able to eradicate the bias involved in the base model.", "was_revised": true, "final_justification": "The authors have addressed all my concerns, and the strengths of the paper generally outweigh the overall weaknesses. Therefore, I am inclined towards the acceptance of this paper.", "rebuttal": "Dear Reviewer A7Ss,\n\nWe are sincerely grateful for your time and for providing such insightful and constructive feedback on our manuscript. Your questions have allowed us to clarify key aspects of our methodology and further strengthen our paper. We are pleased to address each of your points below.\n\n---\n\n**Regarding Weakness 1: The Consideration of Soft Masking.**\n\nWe thank you for this excellent question. Your suggestion prompted us to conduct a direct comparative experiment to validate our design choice of using hard masking in the Contribution-Inverse Sample Calibration (CISC) component.\n\nOur initial motivation for hard masking was to introduce a strong regularization effect to **decisively break the model's learned over-reliance on dominant modalities**. To empirically test this against a less aggressive alternative, we implemented a soft masking variant and evaluated both approaches with the MAG-BERT model on the MIntRec dataset. The results are as follows:\n\n| Method | ACC | F1 | P | R |\n| :--- | :--- | :--- | :--- | :--- |\n| MAG_BERT (Baseline) | 72.65 | 68.64 | 69.08 | 69.28 |\n| w/ HardMask (Our ARL) | **74.38** | **71.66** | **71.55** | **72.89** |\n| *Δ Improvement*| *+1.73* | *+3.02* | *+2.47* | *+3.61* |\n| w/ SoftMask | 73.24 | 67.31 | 70.14 | 69.57 |\n| *Δ Improvement*| *+0.59* | *-1.33* | *+1.06* | *-0.71* |\n\nThe results clearly show that our proposed hard masking approach yields significant and consistent improvements across all evaluation metrics. While soft masking provided a modest improvement in Accuracy and Precision, it was detrimental to the F1-score and Recall.\n\nWe hypothesize that this is because soft masking, by only attenuating a dominant modality's features, is an insufficient signal to compel the model to fundamentally alter its strategy. Hard masking, in contrast, creates a necessary \"information vacuum\" by completely ablating the over-relied-on features for that instance. This forces the model to actively learn from underutilized modalities, leading to a more robust and balanced representation, as reflected in the superior F1 and Recall scores.\n\n**Action:** We will add this comparative experiment and analysis to the Ablation Studies section in our revised manuscript. This addition, prompted by your valuable feedback, provides a strong empirical justification for our design choice.\n\n---\n\n**Regarding Weakness 2: The modality Independence.**\n\nThank you for raising this important theoretical point. We wish to clarify that our use of the LOO method does not assume modality independence. Rather, it serves as a computationally efficient, empirical tool to approximate the marginal contribution of a modality to the **final, fused representation** in the specific context of all other available modalities.\n\nThe method inherently accounts for inter-modality dependencies because the contribution is measured by the change in the fused representation upon a modality's removal.\n* If two modalities are **redundant**, removing one will cause only a small change in the fused output, as the other modality compensates. LOO correctly assigns a low marginal contribution score in this case.\n* If two modalities are **complementary**, removing one will cause a significant change, correctly resulting in a high contribution score.\n\nThus, our LOO valuation implicitly captures the effects of inter-modality dependencies by operating on the joint, fused output.\n\n**Action:** To make this explicit, we will add a clarification in Section 3.2 of our revised manuscript, stating that the LOO method empirically assesses marginal contributions within the context of modality fusion, thereby accounting for their interactions.\n\n---\n\n**Regarding Weakness 3. & 4: The Tuning of Hyperparameters $\\tau$ (Eq. 10) and $\\eta$ (Eq. 9).**\n\nThank you for these questions regarding our experimental methodology. We agree that transparently detailing our hyperparameter tuning is essential for reproducibility.\n\nBoth $\\tau$ and $\\eta$ are tuned as a single scalar value **per dataset**, determined via grid search on the corresponding validation set. We would like to elaborate on why we chose this over a per-modality approach. In our preliminary designs, we explored per-modality thresholds but found this to be less effective. This is because, for a given dataset, one modality is often consistently dominant. The threshold's primary role is to govern the masking rate of this specific modality. Applying a separate threshold to an already weak modality has a negligible impact, as it rarely becomes dominant enough to be masked, yet it significantly increases tuning complexity. A single, data-driven threshold per dataset proved to be the most effective and practical solution.\n\nFor example, the following table shows the sensitivity analysis for $\\tau$ with the MulT model on the MIntRec validation set:\n\n| $\\tau$ | ACC | F1 | P | R |\n| :--- | :--- | :--- | :--- | :--- |\n| 1.0 (Baseline) | 72.52 | 69.25 | 70.25 | 69.24 |\n| 0.9 | 72.58 | 69.15 | 70.04 | 69.15 |\n| 0.7 | 71.24 | 67.98 | 68.70 | 68.11 |\n| **0.5** | **73.71** | **70.82** | **72.06** | **70.45** |\n| 0.3 | 73.26 | 69.94 | 71.23 | 69.64 |\n| 0.1 | 70.79 | 67.16 | 68.27 | 66.52 |\n\nOptimal performance is achieved at $\\tau = 0.5$, and we observed similar robust trends across our other experimental setups. The same rationale and tuning procedure apply to the parameter $\\eta$.\n\n**Action:** In our revised version, we will add an appendix containing a detailed justification for our per-dataset tuning strategy, along with the complete sensitivity analysis tables for both $\\tau$ and $\\eta$ for each dataset.\n\n---\n\nOnce again, we thank you for your thoughtful and rigorous review. We hope that our responses and planned revisions have fully addressed your concerns. We believe the resulting manuscript will be significantly improved."}, {"review_id": "Fa6JCTh4BS", "reviewer": "Reviewer_foUd", "summary": "This paper focuses on the task of Multimodal Intention Recognition (MIR). The authors solve the modality imbalance issue in MIR, where models tend to over-rely on dominant modalities, thereby limiting generalization and robustness. Specifically, the authors propose Adaptive Re-calibration Learning (ARL), a novel dual-path framework that models modality importance from both sample-wise and structural perspectives. There are two key mechanisms in ARL: Contribution-Inverse Sample Calibration (CISC) and Weighted Encoder Calibration (WEC), which dynamically masks overly dominant modalities at the sample level and adjusts encoder weights based on global modality contributions to prevent overfitting, respectively. Experimental results on multiple MIR benchmarks demonstrate the effectiveness of the proposed approach.", "questions": "Overall, the paper has clear motivation and technically sound solution. Thus, my initial rating is boardline accept. I would like to see more author responses in terms of the overall running efficiency and SOTA comparisons. If well addressed, I would like to raise my rating.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is well written and organized, which is easy to follow;\n- The proposed approach is technically sound, especially for the Contribution-Inverse Sample Calibration (CISC), which is interesting;\n- Experiments are conducted on several mainstream benchmarks for verification;\n\nWeakness:\n- Missing overall running efficiency analysis. While applying ARL to off-the-shelf approaches leads to performance improvements as shown in Tables 4 and 5, it remains unclear whether the proposed approach may cause significant training efficiency burden. In Fig. 3, the authors made Computational Cost comparison between the proposed LOO and Shapley-based solution, this is helpful, but it does not clearly show the overall computational cost introduced by the proposed ARL when applied to off-the-shelf approaches., e.g., MAG_BERT and MulT. The authors are suggested to add more detailed training efficiency analysis to clarify whether their approach is a lightweight solution, which could be easily integrated into existing approaches w/o causing significant training cost. \n- The state-of-the-art comparisons (e.g., CMU-MOSI) is somewhat weak. I acknowledged that the authors aim to show their approach can generally improve existing approaches, it is still encouraged to include more SOTA approaches for comparison (just for comparison, w/o apply ARL on them), e.g., [1, 2] on CMU-MOSI, or maybe recently published one [3].\n- Missing references: [4, 5]. In [4], the authors proposed to drop spatial tokens in order to make the model more focus on the temporal tokens, which is similar to the proposed LOO here; [5] is also working on balancing multiple modalities. Please add these references in the related work for discussion;\n- Minor issues: Some typos, e.g., L37 and L39, missing empty spaces after the citation; \n\n[1] Contextual Augmented Global Contrast for Multimodal Intent Recognition. CVPR24.\n\n[2] Decoupled multimodal distilling for emotion recognition. CVPR23.\n\n[3] EMOE: Modality-Specific Enhanced Dynamic Emotion Experts. CVPR25.\n\n[4] DropMAE: Masked Autoencoders with Spatial-Attention Dropout for Tracking Tasks. CVPR23.\n\n[5] Wasserstein Modality Alignment Makes Your Multimodal Transformer More Robust. TMLR25.", "was_revised": true, "final_justification": "I appreciate the detailed rebuttal provided by the authors, including the additional Overall Running Efficiency  analysis, new state-of-the-art comparisons, comprehensive comparison w/ more imbalanced learning. The rebuttal well addresses my concerns, thus I improve my rating to Accept. For completeness, please include all these experiments and explanation in the final version.", "rebuttal": "To reviewer foUd:\n\nThank you for your detailed and valuable feedback. We are pleased to report that we have conducted new experiments and analyses based on your suggestions, and we believe the results significantly strengthen our paper. Below are our detailed responses.\n\n---\n\n**Regarding Weakness 1: Overall Running Efficiency and Practicality**\n\nWe appreciate your concern regarding the computational cost. To provide a clear and fair assessment, we have benchmarked ARL's training and inference time against the baseline, as well as against other state-of-the-art modality balancing methods, including the Shapley value-based approach **[1]** and InfoReg **[2]**.\n\nThe results, summarized in the tables below, demonstrate that ARL offers a highly competitive balance of efficiency and performance.\n\n**Table A: Efficiency and Performance on MOSI (3 modalities)**\n| Method | Training Convergence Time | Inference Time | Acc. |\n| :--- | :---: | :---: | :---: |\n| MAG_BERT | 17.8 mins | 1.5 mins | 72.65 |\n| **w/ Our ARL** | **25.1 mins** | **1.5 mins** | **74.38** |\n| w/ Shapley [1] | 40.4 mins | 1.5 mins | 72.81 |\n| w/ InfoReg [2] | 26.4 mins | 1.5 mins | 73.55 |\n\n**Table B: Efficiency and Performance on MOSI (2 modalities: {a, v})**\n| Method | Training Convergence Time | Inference Time | Acc.(7) |\n| :--- | :---: | :---: | :---: |\n| MAG_BERT | 12.1 mins | 1.1 mins | 17.78 |\n| **w/ Our ARL** | **18.5 mins** | **1.1 mins** | **19.30** |\n| w/ Shapley [1] | 17.3 mins | 1.1 mins | 18.15 |\n| w/ InfoReg [2] | 14.4 mins | 1.1 mins | 17.52 |\n\nOur key takeaways are:\n* **Reasonable and Competitive Overhead:** ARL's training overhead is moderate and its total convergence time is highly competitive with (and even slightly faster than) other SOTA methods like InfoReg **[2]**.\n* **Superior Efficiency to Shapley-based Methods:** As theoretically motivated, ARL is significantly more efficient than Shapley value-based methods **[1]**, confirming the practical advantage of our LOO approach.\n* **Zero Inference Cost:** Crucially, ARL introduces **no additional overhead during inference**, making it highly practical for real-world deployment.\n* **Robust Performance Gains:** In both three-modality and two-modality settings, the modest training cost translates into consistent and superior performance gains, highlighting a favorable trade-off between efficiency and robustness.\n\nWe will add this detailed analysis to the revised paper to assure readers of ARL's practicality.\n\n[1] Y. Wei, et al. \"Enhancing multimodal cooperation via sample-level modality valuation.\" in CVPR 24.\n\n[2] C. Huang, Y. Wei, et al. \"Adaptive Unimodal Regulation\nfor Balanced Multimodal Information Acquisition\" in CVPR 2025\n\n---\n\n**Regarding Weakness 2: State-of-the-Art (SOTA) Comparisons**\n\nThank you for encouraging us to broaden our SOTA comparison. We have followed your suggestion and included results from recent SOTA methods **[3, 4, 5]**.\n\nFurthermore, to unequivocally demonstrate the generalizability and utility of our framework, we went a step further and applied ARL to some of these SOTA models. The results on the MOSI dataset are highly compelling:\n\n**Table C: ARL Boosts Performance of SOTA Models on MOSI**\n| Method | ACC2 | ACC7 | F1 |\n| :--- | :---: | :---: | :---: |\n| CAGC [3] | 85.7 | 44.8 | 85.6|\n| DMD [4] | 86.0 | 45.6 | 86.0 |\n| **DMD w/ ARL** | **86.8** | **46.3** | **86.7** |\n| EMOE [5]| 85.4 | 47.7 | 85.4 |\n| **EMOE w/ ARL** | **86.1** | **48.9** | **86.3** |\n\nAs shown, ARL consistently improves the performance of strong SOTA models like DMD and EMOE. This powerfully demonstrates the core value of our work: ARL is not just another competing model, but a **versatile and orthogonal plug-and-play module** that effectively addresses the fundamental problem of modality imbalance, thereby empowering a wide range of multimodal architectures. We will incorporate this table and discussion into our main paper.\n\n[3] Contextual Augmented Global Contrast for Multimodal Intent Recognition. CVPR24.\n\n[4] Decoupled multimodal distilling for emotion recognition. CVPR23.\n\n[5] EMOE: Modality-Specific Enhanced Dynamic Emotion Experts. CVPR25.\n\n---\n\n**Regarding Weakness 3 & 4: Missing References and Minor Issues**\n\nWe are grateful for the suggested references and your careful proofreading. We will add discussions of the suggested papers to our Related Work section and have corrected the typos.\n\nTo further strengthen our paper and provide a more comprehensive comparison against the broader field of imbalanced learning (a point also raised by Reviewer G2eC), we have conducted new experiments comparing ARL with several representative imbalanced learning methods by applying them to the MAG-BERT baseline on the MIntRec dataset. The results are presented below. We will add this table and corresponding analysis to the appendix in our revised manuscript.\n\n\n| Method | ACC | F1(WF1) | P(WP) | R |\n| :--- | :--- | :--- | :--- | :--- |\n| MAG_BERT | 72.65 | 68.64 | 69.08 | 69.28 |\n| **w/ ARL** | **74.38** | **71.66** | **71.55** | **72.89** |\n| w/ Curriculum Dropout [6] | 67.15 | 66.21 | 67.02 | 66.85 |\n| w/ OGM [7] | 73.31 | 69.83 | 69.91 | 70.21 |\n| w/ OPM [8] | 73.24 | 69.22 | 69.36 | 70.13 |\n| w/ InfoReg [9] | 73.55 | 70.12 | 70.25 | 69.58 |\n\n\n\nAs shown, our proposed ARL outperforms these strong baselines for imbalanced learning, which further validates its effectiveness.\n\n\n\n[6] P. Morerio, et al, “Curriculum dropout,” in ICCV 2017\n\n[7] X. Peng, Y. Wei, et al, “Balanced multimodal learning via on-the-fly gradient modulation,” in CVPR 2022\n\n[8] Y. Wei, et al. \"On-the-fly modulation for balanced multimodal learning.\" IEEE TPAMI 2024\n\n[9] C. Huang, Y. Wei, et al. \"Adaptive Unimodal Regulation for Balanced Multimodal Information Acquisition\" in CVPR 2025\n\n---\n\nIn conclusion, we have strived to addres your concerns with new experiments and detailed analysis. We believe these revisions have substantially strengthened the paper. Thank you once again for your constructive guidance."}, {"review_id": "sApiLGWrju", "reviewer": "Reviewer_G2eC", "summary": "This paper proposes an adaptive recalibration learning (ARL) framework to solve the modality imbalance problem in multimodal intent recognition through a dual-path calibration mechanism. The main innovations include:  1) proposed a contribution inverse sample calibration (CISC) mechanism to dynamically mask the dominant mode at the sample level.   2) designed a weighted encoder calibration (WEC) mechanism to adjust the encoder weights at the architecture level,  3)developed an efficient leave-one-out (LOO) modality evaluation method to replace the calculation of complex Shapley values", "questions": "see the weakness", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength.\n 1. The paper introduces a novel Adaptive Re-calibration Learning (ARL) framework that uniquely addresses modality imbalance through complementary sample-level (CISC) and architectural-level (WEC) calibration mechanisms.\n\n\n2. Experimental results across multiple benchmarks show consistent improvements over strong baselines,  particularly under challenging noisy/missing-modality conditions.\n\nWeakness.\n\n1. As listed in the related work, the imbalanced multimodal learning is a widely studied problem. However, the experiments do not take the existing method into comparison.\n\n2. The time cost increases linearly with the number of modes. It would be better to design a more efficient method to measure the modality contribution at the sample level.\n\n3. The model performance depends on several key hyperparameters, such as the mask threshold τ in CISC, λ (stability influence weight), and α (contribution influence weight) in WEC. Experiments show that the value of α has a significant impact on the results (Tab. 3), but the optimal range of other hyperparameters under different data sets or modality configurations is not systematically analyzed.\n\n4. This paper claims that ARL can address the imbalanced learning at the sample level. However, no experimental evidence is provided to demonstrate this.", "was_revised": true, "final_justification": "The authors have addressed  my problems. However, the additional revisions on experiments are too heavy; thus, I will increase the score, but it isn't easy to give it acceptance.", "rebuttal": "Dear Reviewer G2eC,\n\nWe would like to extend our sincerest gratitude for your time and insightful feedback on our manuscript. Your comments are invaluable and have helped us identify areas for significant improvement. We have carefully considered each of your points and provide our responses below. We hope our clarifications and additional results will address your concerns.\n\n---\n\n**Regarding Weakness 1: Comparison with existing imbalanced learning methods.**\n\nWe thank the reviewer for this valuable suggestion. We would like to begin by clarifying our experimental motivation. As reviewer foUd noted, ARL is designed as a **\"plug-and-play\" module** to enhance existing SOTA models. Our primary focus is on addressing modality imbalance within the specific domains of intention and sentiment analysis. Therefore, our initial step was to demonstrate ARL's ability to consistently improve strong, representative baselines from this domain (i.e., MAG-BERT, MulT, TCL-MAP), as shown in Tables 1 and 2 of our paper.\n\nTo further address your concern directly, we have now conducted **new experiments comparing ARL with several representative imbalanced learning methods** by applying them to the MAG-BERT baseline on the MIntRec dataset. The results are presented below. We will add this table and corresponding analysis to the appendix in our revised manuscript.\n\n| Method | ACC | F1(WF1) | P(WP) | R |\n| :--- | :--- | :--- | :--- | :--- |\n| MAG_BERT | 72.65 | 68.64 | 69.08 | 69.28 |\n| **w/ ARL** | **74.38** | **71.66** | **71.55** | **72.89** |\n| w/ Curriculum Dropout [1] | 67.15 | 66.21 | 67.02 | 66.85 |\n| w/ OGM [2] | 73.31 | 69.83 | 69.91 | 70.21 |\n| w/ OPM [3]| 73.24 | 69.22 | 69.36 | 70.13 |\n| w/ InfoReg [4] | 73.55 | 70.12 | 70.25 | 69.58 |\n\nAs shown, our proposed ARL outperforms these strong baselines for imbalanced learning, which further validates its effectiveness.\n\n[1] P. Morerio, et al, “Curriculum dropout,” in ICCV 2017\n\n[2] X. Peng, Y. Wei, et al, “Balanced multimodal learning via on-the-fly gradient modulation,” in CVPR 2022\n\n[3] Y. Wei, et al. \"On-the-fly modulation for balanced multimodal learning.\" IEEE TPAMI 2024\n\n[4] C. Huang, Y. Wei, et al. \"Adaptive Unimodal Regulation for Balanced Multimodal Information Acquisition\" in CVPR 2025\n\n---\n\n**Regarding Weakness 2: Efficiency of the LOO method.**\n\nWe appreciate your concern regarding the computational cost of our Leave-One-Out (LOO) modality valuation method. However, we believe there may have been a misunderstanding of LOO's core contribution to efficiency.\n\nThe primary challenge in sample-level modality valuation lies in the **exponential complexity ($O(n2^n)$)** of theoretically-grounded methods like Shapley values, which renders them impractical for real-world MIR systems with multiple modalities. The central contribution of our LOO method is precisely in overcoming this critical bottleneck by reducing the complexity from **exponential to linear ($O(n)$)**.\n\nWe have provided a detailed analysis of this in **Section 4.2** and visualized it in **Figure 3** of our paper. These results clearly demonstrate that LOO is dramatically more scalable and efficient than the Shapley-based approaches. Therefore, achieving linear complexity is not a weakness but a key strength of our work, making dynamic, sample-specific modality calibration computationally feasible for the first time in this context.\n\n---\n\n**Regarding Weakness 3: Insufficient analysis of hyperparameters.**\n\nThank you for pointing out the need for a more thorough hyperparameter analysis. We did perform extensive tuning during our experiments, and we agree that including these results strengthens the paper. Due to space constraints in the initial submission, this analysis was omitted.\n\nTo address your comment, we now provide the sensitivity analysis for the masking threshold $\\tau$ using the MulT model on MIntRec. We will include this in the appendix of the revised manuscript.\n\n| $\\tau$ | ACC | F1 | P | R |\n| :--- | :--- | :--- | :--- | :--- |\n| 1.0 (Baseline) | 72.52 | 69.25 | 70.25 | 69.24 |\n| 0.9 | 72.58 | 69.15 | 70.04 | 69.15 |\n| 0.7 | 71.24 | 67.98 | 68.70 | 68.11 |\n| **0.5** | **73.71** | **70.82** | **72.06** | **70.45** |\n| 0.3 | 73.26 | 69.94 | 71.23 | 69.64 |\n| 0.1 | 70.79 | 67.16 | 68.27 | 66.52 |\n\nOptimal performance is achieved at $\\tau = 0.5$, and we observed similar robust trends across our other experimental setups. This analysis demonstrates that while the hyperparameter is important, its optimal value can be robustly identified via standard tuning procedures.\n\nIn our revised version, we will add an appendix containing a detailed justification for our per-dataset tuning strategy, along with the complete sensitivity analysis tables for these parameters.\n\n---\n\n**Regarding Weakness 4: Lack of experimental evidence for sample-level calibration.**\n\nWe sincerely thank the reviewer for this insightful and constructive comment. We agree that providing direct experimental evidence to demonstrate ARL's sample-level calibration mechanism is crucial and will significantly strengthen our paper. The reviewer’s point is well-taken; while our method is designed to be sample-centric, our original experiments only showed its aggregated effects on the entire test set.\n\nWe would also like to respectfully note that our sample-level focus aligns with a significant research direction in the community. For instance, the recent work by Wei et al. [5] (CVPR 2024) also emphasizes valuing modality contributions on a per-sample basis, underscoring the growing recognition that modality importance is context-dependent rather than static.\n\n[5] Wei et al., \"Enhancing multimodal cooperation via sample-level modality valuation\" (CVPR 2024)\n\nTherefore, the reviewer’s suggestion to provide more direct, sample-centric evidence is particularly relevant. To that end, we have prepared the following quantitative analysis, which we will add to the revised manuscript to compellingly illustrate the effectiveness of ARL's sample-level adaptation.\n\n**Quantitative Grouped Analysis: Statistical Validation**\n\nTo provide broad statistical evidence, we partitioned the MIntRec test set based on a **Sample Imbalance Score**, which we defined as the variance of the three LOO modality contribution scores for each sample. A higher score signifies greater imbalance. We then evaluated the performance of the baseline MAG-BERT and our ARL-enhanced model on these groups.\n\n\n| Sample Group | Baseline Accuracy (MAG-BERT) | ARL-enhanced Accuracy | Performance Improvement |\n| :--- | :--- | :--- | :--- |\n| High-Imbalance Samples (Top 30%) | 64.2% | 73.7% | **+9.5%** |\n| Low-Imbalance Samples (Bottom 70%) | 78.1% | 79.9% | **+1.8%** |\n \nAs the results clearly show, the performance gain from ARL is substantially larger **(+9.5%)** on samples with high modality imbalance. This quantitatively demonstrates that our training-time calibration strategy produces a final model that is specifically more robust to the challenging sample-level imbalances it was designed to address.\n\nWe are confident that these new results provide the direct evidence requested by the reviewer. We thank the reviewer again for pushing us to make this important addition to our work.\n\n---\n\nOnce again, we thank you for your rigorous review. We believe that with the proposed clarifications and additional experiments, our manuscript has been substantially strengthened. We hope that our responses have adequately addressed your concerns."}, {"review_id": "EHK13geJa1", "reviewer": "Reviewer_fBNu", "summary": "Adaptive Re-calibration Learning (ARL) is a plug-and-play framework that remedies modality imbalance in multimodal intention-recognition (MIR) and sentiment-analysis pipelines. The method first quantifies per-sample modality importance through a Leave-One-Out (LOO) valuation, whose complexity grows only linearly with the number of modalities . These scores drive two complementary calibrations: Contribution-Inverse Sample Calibration (CISC), which temporarily masks dominant modalities at the feature level, and Weighted Encoder Calibration (WEC), which periodically re-weights modality-specific encoders using a blend of stability and contribution signals . When attached to three strong baselines (MAG-BERT, MulT, TCL-MAP), ARL boosts twenty-class MIntRec accuracy by up to +1.73 (pp) and MOSI sentiment accuracy by up to +1.60 (pp) while retaining robustness under missing-modality conditions .", "questions": "Below are key questions and suggestions for the authors, focusing on actionable points that could clarify or strengthen the paper:\nQ1: Can you provide a formal definition and measurement of “modality imbalance”?\n- Why it matters: This is the core problem addressed by the paper. A formal definition would help readers understand the scope and assumptions of the method.\n- Suggestion: Introduce a mathematical formulation or statistical measure (e.g., KL divergence, entropy) to quantify modality imbalance at both the sample and dataset levels. You may consider using the LOO contribution scores to define an imbalance index.\nQ2: Could you provide a pseudocode or flowchart of the full ARL training process?\n- Why it matters: The integration of CISC and WEC into the training loop is not fully transparent, limiting understanding of how the method interacts with standard training procedures.\n- Suggestion: Include a pseudocode or visual flowchart showing the order and frequency of operations (e.g., when WEC is updated, how often CISC is applied per batch/epoch), and how gradients are computed and combined.", "limitations": "Yes, the authors have adequately discussed the limitations of their work in Section 5 and Appendix A.2. They acknowledge that:\n- ARL's performance depends on the expressiveness of the base model.\n- There is increased computational overhead due to the LOO evaluation and dual calibration mechanisms.\n- Privacy concerns and potential misuse are considered in broader impacts.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths：\n1. Well-Motivated and Effective Solution to Modality Imbalance\nThe paper addresses a critical challenge in multimodal intention recognition—modality imbalance—by proposing a novel and practical solution. The Adaptive Re-calibration Learning (ARL) framework introduces two complementary mechanisms: Contribution-Inverse Sample Calibration (CISC) and Weighted Encoder Calibration (WEC), which jointly optimize modality usage at both the sample and model levels. This dual-path strategy outperforms existing methods by dynamically balancing modality importance, especially under noisy or missing modality conditions.\n2. Technical Novelty and Computational Efficiency\nARL leverages a Leave-One-Out (LOO) method for modality valuation, offering a computationally efficient alternative to Shapley values with linear complexity. This makes the approach scalable and suitable for real-world applications. The synergy between CISC and WEC provides a more holistic and adaptive calibration mechanism compared to prior techniques that focus only on either gradient modulation or dropout.\n3. Strong Empirical Validation and Practical Applicability\nExtensive experiments across multiple datasets and architectures (e.g., MAG-BERT, MulT, TCL-MAP) demonstrate consistent performance improvements in both accuracy and robustness. ARL shows particular strength in handling degraded or missing modalities, making it highly valuable for deployment in real-world systems such as robotics, intelligent assistants, and autonomous systems.\n\nWeaknesses\nDespite its strengths, the paper has several areas that could benefit from improvement:\n1. Lack of Formal Definition of Modality Imbalance: The central problem of the paper is not formally defined, nor is there a quantitative metric provided to measure modality imbalance. This limits the theoretical grounding of the approach.\n2. Limited Training Procedure Description: The paper lacks a clear step-by-step explanation of how ARL integrates with the training process of base models. For example, the frequency of WEC updates and interaction between CISC and WEC during optimization are not clearly outlined.", "was_revised": true, "final_justification": "The response has address all my concerns, I will keep my rating.", "rebuttal": "We are deeply grateful for your insightful and positive review of our work. We sincerely appreciate your recognition of our paper's strengths, including the well-motivated and effective solution to modality imbalance, the technical novelty and computational efficiency of the Leave-One-Out (LOO) valuation method, and the strong empirical validation. \n\nWe also thank you for your constructive feedback on the areas for improvement. Your suggestions regarding the formal definition of modality imbalance and the clarity of the training procedure are highly valuable. We agree that addressing these points will significantly strengthen the paper's theoretical grounding and reproducibility. Below, we address your questions in detail.\n\n---\n\n**Regarding Weakness 1: Can you provide a formal definition and measurement of “modality imbalance”?**\n\nThank you for this excellent suggestion. We agree that a formal definition and a quantitative metric for modality imbalance would greatly enhance the rigor of our paper. We will incorporate this into the revised manuscript.\n\n**1. Formal Definition:** We will formally define **modality imbalance** as: *\"The degree of non-uniformity in the contributions of different modalities to the final prediction for a given multimodal sample.\"*\n\n**2. Quantitative Metric (Modality Imbalance Index):** As you astutely suggested, we can leverage our Leave-One-Out (LOO) contribution scores to create a quantitative metric. For a sample $i$ with $M$ modalities, we have a vector of contribution scores $\\Delta_i = \\{\\delta_i^{(m_1)}, \\delta_i^{(m_2)}, ..., \\delta_i^{(m_M)}\\}$, where $\\sum_{m} \\delta_i^{(m)} = 1$. An ideal balance implies that all modalities contribute equally (i.e., $\\delta_i^{(m)} = 1/M$ for all $m$).\n\nTo quantify the deviation from this ideal state, we will introduce the **Modality Imbalance Index (MII)**, defined as the variance of these contribution scores:\n\n$$\\text{MII}(i) = \\text{Var}(\\Delta_i) = \\frac{1}{M}\\sum_{m=1}^{M} \\left(\\delta_i^{(m)} - \\frac{1}{M}\\right)^2$$\n\nA higher MII value indicates a greater imbalance for sample $i$, signifying that one or a few modalities are dominating the prediction.\n\n**3. Connection to ARL:** We will further clarify that our proposed ARL framework is designed to implicitly minimize this imbalance.\n* **Contribution-Inverse Sample Calibration (CISC)** acts as a reactive, sample-level mechanism. It directly targets samples with high MII by masking dominant modalities whose contribution $\\delta_i^{(m)}$ exceeds a threshold $\\tau$, thereby encouraging the use of underutilized modalities.\n* **Weighted Encoder Calibration (WEC)** serves as a proactive, architectural-level mechanism. It adjusts encoder weights based on global contribution statistics ($\\bar{\\delta}^{(m)}$) and stability, aiming to reduce the average modality imbalance across the entire dataset over the long term.\n\nWe will add this formal definition and the MII metric to **Section 3.1 (Problem Formulation)** in our revised paper. We believe this will provide a clearer theoretical foundation for the problem we are solving.\n\n---\n\n**Regarding Weakness 2: Could you provide a pseudocode or flowchart of the full ARL training process?**\n\nWe thank you for pointing out the need for a clearer description of our training procedure. We agree that a step-by-step algorithm is essential for understanding and reproducibility. In the revised manuscript, we will add a detailed pseudocode of the ARL training process. Below is a draft of the algorithm we plan to include.\n\n**Algorithm 1: Adaptive Re-calibration Learning (ARL) Training Procedure**\n\n---\n```\nRequire: Training set $\\mathcal{D}_{train}$, Validation set $\\mathcal{D}_{val}$, Base model $f_{\\theta}$\nRequire: ARL hyperparameters: $\\tau, T, \\lambda, \\alpha$\n\nInitialize model parameters $\\theta$\n\nfor epoch = 1 to max_epochs do\n    for each batch in $\\mathcal{D}_{train}$ do\n        // Calculate sample-wise modality contribution (Sec. 3.2)\n        Compute full-modality features $F_i$ and masked features $F_i^{(-m)}$ for each sample.\n        Calculate contribution scores $\\delta_i^{(m)}$ from the feature similarities.\n\n        // Calibrate features with CISC (Sec. 3.3)\n        Calibrate features $\\tilde{x}_i^{(m)}$ by masking if a sample's contribution $\\delta_i^{(m)} \\ge \\tau$.\n\n        // Optimize model using calibrated features\n        Compute final logits from the calibrated features $\\tilde{x}_i$.\n        Calculate prediction loss $\\mathcal{L}$.\n        Update model parameters $\\theta$ via backpropagation.\n    end for\n\n    // Periodically calibrate encoders with WEC (Sec. 3.4)\n    if epoch % T == 0 then\n        Calculate stability score $\\mathcal{P}_m$ on $\\mathcal{D}_{train}$ and $\\mathcal{D}_{val}$.\n        Calculate average contribution score $\\bar{\\delta}^{(m)}$ on $\\mathcal{D}_{val}$.\n        Compute dynamic weights $w_m$ from $\\mathcal{P}_m$ and $\\bar{\\delta}^{(m)}$.\n        Adjust encoder weights $\\theta^{(m)}$ using the computed weights $w_m$.\n    end if\nend for\n```\n---\n\nWe will place this algorithm in the **Methodology** section of our revised paper, accompanied by explanatory text. We are confident this will fully clarify the operational flow of ARL and its integration with base models.\n\nOnce again, we thank you for your time and constructive suggestions. We believe these revisions will make our paper stronger, clearer, and more impactful."}]

Author rebuttal:

Meta-review: 1) Paper Summary

This paper introduces a new framework called Adaptive Re-calibration Learning (ARL) to tackle the issue of modality imbalance in multimodal intention recognition. The core idea is to improve model robustness by understanding the importance of different modalities. ARL works through a dual-path mechanism: one part dynamically masks the dominant modalities at the sample level, and the other adjusts the model's encoder weights based on global modality contributions. To do this efficiently, the authors developed a lightweight "Leave-One-Out" method. Experiments on two standard datasets show that ARL consistently boosts the performance of existing models, especially when some modalities are noisy or missing.

2) Strengths of the Paper

Reviewers highlighted three main strengths. 
* The idea is novel and technically sound. Reviewers A7Ss and fBNu pointed out that the ARL framework, which combines both sample-level and architectural-level adjustments, was an innovative and practical solution. 
* The empirical evidence is strong. Reviewers foUd and fBNu noted that experiments across multiple datasets and with different baseline models consistently demonstrated that ARL improves performance and robustness. 
* The paper is well-written and clear. Reviewers A7Ss and foUd both stated that the paper was easy to follow, with a clear motivation and well-explained idea.

3) Weaknesses of the Paper

Reviewers identified three key weaknesses. 
* The paper lacked a formal definition for the central problem of "modality imbalance" and did not provide a detailed analysis of the computational cost. Reviewer fBNu brought up the former, while reviewers foUd and G2eC pointed out the latter. 
* The model's performance was noted to be dependent on several key hyperparameters, but a systematic analysis of how to tune them was missing, a point raised by reviewers G2eC and A7Ss. 
* The experiments did not include comparisons against other existing imbalanced learning methods or the latest state-of-the-art models, a point made by reviewers G2eC and foUd.

4) Justification for Acceptance

The paper is technically solid and offers a well-motivated solution to a significant problem. The ARL framework is both effective and practical. Although the initial submission had some shortcomings, the authors' detailed and comprehensive rebuttal successfully addressed all of these concerns. They provided a formal definition of the problem, added new experiments to analyze computational cost and compare with other methods, and clarified their hyperparameter tuning strategy. This thorough response resolved the reviewers' concerns and ultimately led to a consensus for acceptance.

5) Rebuttal Summary and Changes

During the rebuttal period, reviewers raised several specific points, and the authors provided extensive responses.

*  Computational Cost: Reviewer foUd asked about the overall training efficiency. The authors provided new experiments showing that the computational overhead is acceptable and that their method is lightweight enough to be easily integrated into existing models.

*  Formal Definitions and Pseudocode: Reviewer fBNu requested a formal definition of "modality imbalance" and step-by-step pseudocode. The authors provided a formal definition and included pseudocode, which improved the paper's clarity and reproducibility.

*  Masking and Dependencies: Reviewer A7Ss questioned the use of hard masking and how the method handles inter-modality dependencies. The authors offered detailed explanations and additional experiments that satisfied the reviewer's concerns.

* Hyperparameter Tuning: Reviewers A7Ss and G2eC inquired about the tuning process for key hyperparameters. The authors clarified that these were tuned for each dataset. While reviewer A7Ss still saw this as a minor limitation for real-world use, they agreed that the paper's overall strengths outweighed this issue.

*  Missing Comparisons: Reviewers G2eC and foUd pointed out the lack of comparisons with other methods. The authors added new experiments in their rebuttal, demonstrating that their approach is effective when compared to these other baselines, which ultimately addressed this concern.

The authors' thorough and effective rebuttal, which included new experimental results and clear explanations, successfully resolved the  main reviewers' significant concerns.
Decision: Accept (poster)