Title: CMoB: Modality Valuation via Causal Effect for Balanced Multimodal Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multimodal Learning

Causal Effect

Modality Valuation
Abstract: Existing early and late fusion frameworks in multimodal learning are confronted with the fundamental challenge of modality imbalance, wherein disparities in representational capacities induce inter-modal competition during training. Current research methodologies primarily rely on modality-level contribution assessments to measure gaps in representational capabilities and enhance poorly learned modalities, overlooking the dynamic variations of modality contributions across individual samples. To address this, we propose a Causal-aware Modality valuation approach for Balanced multimodal learning (CMoB). We define a benefit function based on Shannon's theory of informational uncertainty to evaluate the changes in the importance of samples across different stages of multimodal training. Inspired by human cognitive science, we propose a causal-aware modality contribution quantification method from a causal perspective to capture fine-grained changes in modality contribution degrees within samples. In the iterative training of multimodal learning, we develop targeted modal enhancement strategies that dynamically select and optimize modalities based on real-time evaluation of their contribution variations across training samples. Our method enhances the discriminative ability of key modalities and the learning capacity of weak modalities while achieving fine-grained balance in multimodal learning. Extensive experiments on benchmark multimodal datasets and multimodal frameworks demonstrate the superiority of our CMoB approach for balanced multimodal learning.


Peer reviews:
comments: SUMMARY:
This paper introduces Causal Modality Evaluation (CMoB) to address modality imbalance in multimodal learning. CMoB uses Shannon information uncertainty theory for a gain function, combines causal learning to measure fine - grained modality contributions within samples, and applies dynamic optimization to strengthen weak modalities.

STRENGTHS AND WEAKNESSES:
Pros
1. Combining causal learning with information theory to dynamically evaluate modality contributions at the sample level breaks through the limitations of traditional modality - level analysis.
2. The benefit function is constructed based on Shannon information theory, and the causal effect derivation is complete.
3. Compared with more than 10 baseline methods on 5 cross-modal datasets, the results show the best performance in all cases.

Cons
1. The global nature of ITE is mentioned in the paper, but the consistency under different modality combinations is not sufficiently verified.
2. Lacking ablation experiments to verify the effectiveness of each module.
3. The specific implementation of the Mask(·) function in Equation 5 is not explicitly described.
4. It is recommended to add efficiency tests on large-scale datasets and cross-modal number extension experiments.

QUESTIONS:
Can the quantification of causal effects explain the synergistic/competitive relationships between modalities? For example, the complementarity of audio-video modalities in emotion recognition.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-26

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the CMoB framework to address the \"modal imbalance\" problem in multimodal learning: first use the entropy-inspired sample benefit function to measure the confidence gain of each sample after adding different modalities, then treat each modality as a \"process\" to calculate the individual treatment effect (ITE), and finally dynamically shield and enhance the weak modality. Experiments on five datasets show that CMoB generally outperforms or approaches the latest balancing methods in terms of accuracy/F1, and its improvement effect on weak modalities is verified by t-SNE and Grad-CAM analysis.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n* This work is the first to incorporate causal Individual Treatment Effect (ITE) estimation into sample-level modality valuation, enabling a fine-grained quantification of each modality’s contribution. The introduction of causal inference (ITE estimation) into the modality rebalancing task is novel and insightful. It enriches the modality contribution evaluation beyond simple gradient or attention-based metrics.\n* The shift from global, modality-level contribution estimation to per-sample analysis allows CMoB to dynamically capture fine-grained imbalance and perform more targeted optimizations.\n* Extensive experiments across multiple benchmarks substantiate the effectiveness and robustness of the proposed approach.\n\n**Weaknesses:**\n* Per-sample ablation and dynamic masking are likely to incur substantial additional training time and computational overhead.\n* The paper lacks an ablation study that disentangles the respective contributions of the benefit function, the ITE computation, and the dynamic masking mechanism.\n* It is not recommended to share code via GitHub during the review process, as it may compromise the double-blind review principle. Instead, it is advisable to use an anonymous code-sharing platform such as https://anonymous.4open.science/.\nprocess, as it may compromise the double-blind review principle. Instead, it is advisable to use an anonymous code-sharing platform such as https://anonymous.4open.science/.\n* Typo: \"modality-spcifice\" should be \"modality-specific\" in line 157.\n\nQUESTIONS:\n* How do the benefit‐function values and the final performance figures change if temperature-scaled logits or negative log-likelihood are used instead of the default soft-max confidence?\n* For each benchmark, what is the percentage increase in wall-clock training time and peak GPU memory consumption relative to the strongest baseline?\n* Please report an ablation study that quantifies the individual gains contributed by each module of the proposed framework.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a causal-aware modality valuation approach (CMoB) for addressing the problem of modality imbalance in multimodal learning. This method allows for the balancing of modality contributions at the sample level, helping to improve the performance of weak modalities while mitigating modality imbalance. The paper demonstrates the effectiveness of CMoB through experiments on several multimodal datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. The method focuses an important issue in multimodal learning, modality imbalance, and provides a potential solution to improve the integration of diverse modalities in complex datasets.\n2. The paper validates its approach with experiments on multiple multimodal datasets, showcasing the effectiveness of the proposed method.\n\nWeakness:\n\n1. The motivation behind the paper is not clear. The authors mention being inspired by the human nervous system but then shift to a causal learning perspective without clearly explaining the connection between these two concepts. The introduction section lacks logical coherence in presenting the proposed approach.\n2. The proposed method claims to solve modality imbalance from a causal learning perspective. However, the actual operation involves comparing the effects of including or excluding a modality, which is conceptually similar to existing methods like Shapley value. What are advantages of this approach over Shapley value or other similar techniques?\n3. The paper's writing style needs improvement, especially in the introduction, which spends excessive time discussing related work but fails to clearly explain the motivation behind the proposed method. In addition, Section 3.1 is too detailed and spends considerable space describing concepts like cross-entropy, which do not directly contribute to the core argument of the paper. This section could be more concise and focused.\n4. In Section 4.3, the authors claim to replicate the experiment from [45] on the \"scarcely informative modality\" case but using the original CREMA-D dataset. However, [45] created such a case by adding significant noise to the audio modality of the CREMA-D dataset. The original CREMA-D dataset does not inherently contain \"scarcely informative modality\" cases. \n5. The quality of the images in the paper, particularly Figures 2 and 3, is poor. The legends in these figures are difficult to read, which diminishes the clarity of the data presented.\n\nQUESTIONS:\nPlease refer to the above section.\n\nLIMITATIONS:\nPlease refer to the above section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors discuss the limitations in existing modality rebalancing methods and point out they neglect the dynamic variations of modality contributions at the sample level during training. They propose a causal-aware modality validation approach for balanced multimodal learning. An intervention method is introduced to evaluate the causal effect, quantifying changes in modality contributions at the sample level. The fine-grained evaluation approach enables targeted optimizations across modalities at the sample level, effectively mitigating the issue of multimodal imbalance. Experimental results on multiple datasets show the effectiveness of the proposed algorithm.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe authors propose a causal-aware modality valuation method to evaluate the sample-level modality contribution in multimodal training. \nThe authors design an optimization strategy for modality selection at the sample-level according to the contribution degree of each modality in order to mitigate the modality imbalance problem. \nThe authors validate the effectiveness of the proposed method by comparison and ablation experiments on publicly available data.\n\nWeaknesses:\n1. Some important references are missing. Some related works about sample-level modality valuation and balanced learning, such as PDF (Cao, et al., ICML 2024), SMLS (Zhou, et al., Information Fusion 2025), and ARL (Wei, et al., ICML 2025), should be discussed in the related works.\n\n2. The authors should compare one or some relatively new algorithms, such as “Zhou, et al.: Dataset-aware Utopia modality contribution for imbalanced multimodal learning. Information Fusion 2025,” “Ma, et al.: Improving Multimodal Learning Balance and Sufficiency through Data Remixing. ICML 2025,” and “Zhou, et al.: Sample-level Self-paced Learning to Tackle Multimodal Imbalance Problem. ICASSP 2025.”\n\n3. The language quality of the submission is not very good. Some presentation issues should be addressed:\n(1) For example, “We” should be “we” in Line 143.\n(2) “we define a benefit function” should be “We define a benefit function” in Line 8 of the Abstract.\n(3) “we addresses the above problem” should be “we address the above problem” in Line 69 and Line 70.\n(4) “modality-specifice” should be “modality-specific,” which is used many times in this paper, such as in Line 157, Line 161, and Line 170.\n\nIn addition to the above issues, there are some confusing descriptions, such as “modality data”, I guess it should be “modality-specific data.” I truly suggest the authors carefully check the whole paper and improve the overall presentation quality.\n\nQUESTIONS:\nThe authors should compare their work with some relatively new algorithms.\n\nLIMITATIONS:\nno", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}