Title: Federated Dialogue-Semantic Diffusion for Emotion Recognition under Incomplete Modalities
Venue: NeurIPS 2025
Accepted: yes
Keywords: federated learning; multimodal emotion recognition; diffusion model
Abstract: Multimodal Emotion Recognition in Conversations (MERC) enhances emotional understanding through the fusion of multimodal signals. However, unpredictable modality absence in real-world scenarios significantly degrades the performance of existing methods. Conventional missing-modality recovery approaches, which depend on training with complete multimodal data, often suffer from semantic distortion under extreme data distributions, such as fixed-modality absence. To address this, we propose the Federated Dialogue-guided and Semantic-Consistent Diffusion (FedDISC) framework, pioneering the integration of federated learning into missing-modality recovery. By federated aggregation of modality-specific diffusion models trained on clients and broadcasting them to clients missing corresponding modalities, FedDISC overcomes single-client reliance on modality completeness. Additionally, the DISC-Diffusion module ensures consistency in context, speaker identity, and semantics between recovered and available modalities, using a Dialogue Graph Network to capture conversational dependencies and a Semantic Conditioning Network to enforce semantic alignment. We further introduce a novel Alternating Frozen Aggregation strategy, which cyclically freezes recovery and classifier modules to facilitate collaborative optimization. Extensive experiments on the IEMOCAP, CMUMOSI, and CMUMOSEI datasets demonstrate that FedDISC achieves superior emotion classification performance across diverse missing modality patterns, outperforming existing approaches.


Peer reviews:
comments: SUMMARY:
1.Using a federated learning framework to address the problem of missing modalities also ensures data privacy.
2. The Federated Dialogue-guided and Semantic-Consistent Diffusion (FedDISC) framework uses a diffusion model to ‌recover missing modality features.

STRENGTHS AND WEAKNESSES:
Strengths:
1.This paper offers a fresh perspective on emotion recognition under incomplete modalities.
2.The appendix includes extensive supplementary content to help readers better understand the material presented in the main text.
3.Get a good results across multiple datasets.

Weaknesses:
1.Lack a comparison with state-of-the-art methods, e.g., CIF-MMIN[1], MoMKE[2], CM-ARR[3] and Flex-MoE[4].
2.Information regarding the data used for pre-training DGN and SCN is missing.
3.An ablation study comparing the contributions of DGN and SCN is missing.
4.Given that federated learning is the main innovation, the paper does not explore the impact of the number of clients on performance.
5.There is a significant performance disparity between the two diffusion models (DDPM and DDIM), but the paper lacks an in-depth investigation into why this occurs.
6.How are the clients simulated in the experiments? Are they distinct machines, or is it a simulated distributed environment on the two L40S GPUs?

QUESTIONS:
1. Add some comparative experiments with recent state-of-the-art methods, such as CIF-MMIN[1], MoMKE[2], CM-ARR[3] and Flex-MoE[4] to demonstrate the superiority of your methed.
2.The proposed framework is quite complex; it should include more ablation studies to demonstrate the effectiveness of each module within it.
3. A comparative experiment for the full-modality case should be included in the test set evaluation.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-24

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes FedDISC, a novel framework for Multimodal Emotion Recognition in Conversations (MERC) that addresses the challenge of missing modalities in real-world scenarios. Unlike conventional methods that rely on complete multimodal data, FedDISC integrates federated learning with diffusion models to recover missing modalities across distributed clients. The DISC-Diffusion module ensures semantic consistency between recovered and available modalities through dialogue-aware modeling and semantic conditioning. An Alternating Frozen Aggregation strategy enables collaborative training of recovery and classification modules. Experiments on IEMOCAP, CMUMOSI, and CMUMOSEI datasets show that FedDISC outperforms existing approaches under various missing-modality conditions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nFedDISC integrates federated learning with diffusion models to enable collaborative missing-modality recovery while preserving data privacy.\nDISC-Diffusion ensures context, speaker, and semantic consistency between recovered and existing modalities using dialogue-aware and semantic conditioning networks.\n\nWeaknesses:\nThe integration of federated learning with diffusion models increases computational complexity and may lead to higher training costs and latency.\n\nQUESTIONS:\n1. The authors are encouraged to better clarify the distinctions and advantages of their proposed method compared to previous approaches in the related work section. In particular, a more detailed discussion of existing federated learning methods in multimodal learning is needed to better position the novelty and contributions of FedDISC.\n2. Under the fixed missing protocol, where one modality is completely absent from all samples, why the model still performs modality generation instead of simply relying on the available modalities?\n3. Why does FedDISC perform worse than SDR-GNN on IEMOCAP6 under the fixed missing setting when the textual modality is present?\n4. How do the DGN and SCN modules compare to previous methods that capture context, speaker dependencies, or cross-modal semantics?\n\nLIMITATIONS:\nThe integration of federated learning with diffusion models increases computational complexity and may lead to higher training costs and latency.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces FedDISC, a federated learning method for multimodal emotion recognition in conversations (MERC) under missing modality scenarios. The method integrates generative modality recovery via diffusion models with federated model aggregation. Key components include (1) DISC-Diffusion modules leveraging a Dialogue Graph Network (DGN) and Semantic Conditioning Network (SCN) for context and semantic consistency, and (2) an Alternating Frozen Strategy (AFS) to mitigate optimization conflicts during federated updates. Experiments on IEMOCAP, CMU-MOSI, and CMU-MOSEI demonstrate competitive performance under random and fixed missing modality protocols.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Well-motivated, addressing real-world problem of multimodal data incompleteness with privacy constraints.\n2. Federated generative recovery method is a conceptually interesting direction for MERC.\n3. The DISC-Diffusion module design (DGN+SCN) is methodologically reasonable for enforcing multidimensional consistency in recovered modalities.\n\nWeaknesses:\n1. Most technical components (e.g., R-GCNs, conditional diffusion, federated averaging) are direct adaptations of known methods. Originality lies primarily in system-level integration, rather than algorithmic innovation.\n2. Exact encoder architectures, training configurations are not specified in the main paper.\n3. No details on the number of denoizing steps during inference are provided.\n4. There are ambiguities in notation.\n5. Missing explicit clarification as to whether standard splits (e.g., Session 5 as a test for IEMOCAP) were followed or randomized in the cross-validation process.\n6. No ablation for computational cost or real-time feasibility.\n7. Claims about generalization and practical deployability are premature. No cross-corpus transfer experiments or asynchronous client tests, despite acknowledging this as a limitation.\n8. Typo: \"spealer-aware\" to \"speaker-aware\" and \"to recovery the feature\" to \"to recover the feature\".\n\nQUESTIONS:\n1. Can you report the inference time and total communication cost for diffusion models per round? Is real-time deployment feasible?\n2. How does FedDISC scale with the number of clients? Is there any empirical or theoretical evidence for this?\n3.  Were standard data splits used in the evaluation, such as fixed Session 5 for IEMOCAP? If not, comparison fairness is undermined.\n4. Can you provide effect sizes or statistical significance tests for the main results?\n5. Are there plans to extend FedDISC to asynchronous federated learning or real-world settings with heterogeneous clients?\n\nLIMITATIONS:\nThe authors partially address limitations in Section 5. Acknowledging the sampling inefficiency of diffusion models and the lack of asynchronous training support is a good practice. However, practical constraints such as computational cost, deployment feasibility, and generalization to unseen domains are insufficiently discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of missing modalities in multimodal dialogue emotion recognition by proposing the novel FedDISC framework, which pioneers the integration of federated learning into missing-modality recovery. During recovery, it leverages pretrained Dialogue Graph Network (DGN) and Semantic Conditioning Network (SCN) to enforce consistency between recovered and available modalities along the three dimensions of context, speaker identity, and semantics. Moreover, it introduces an Alternating Frozen Strategy to resolve cross-modal optimization conflicts. The manuscript is logically coherent and rigorously organized, and its advantages are validated through comprehensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths: \n1: The paper is highly innovative, successfully introducing federated learning into missing-modality recovery and offering a new approach for multimodal dialogue emotion recognition under incomplete modalities. \n\n2:  The manuscript is logically rigorous in structure, and demonstrates the method’s strengths through experiments with different missing-modality configurations, varying missing rates, and comprehensive ablation studies.\n\n### Weaknesses:\n1:  The framework is complex, and its computational cost relative to other methods remains unclear. \n\n2:  After presenting the methodology in Section 3, a concise overview of the end-to-end workflows shown in Figures 1 and 2 would further aid reader comprehension.\n\n3:  In the ablation study, only 120 samples were selected for evaluation. Could this sample size be too small and lack sufficient randomness? Furthermore, when designing the ablation experiments for DGN and SCN, is it possible to validate their individual contributions separately?\n\nQUESTIONS:\n1:  The framework is complex—how much higher is its computational cost compared to other architectures? If this overhead remains within an acceptable range, I believe the framework’s advantages are indeed significant.\n\n2:  Provide a concise, end-to-end overview of Figures 1 and 2; I am confident this will help readers grasp the workflow more clearly.\n\n3:  In the ablation study, only 120 samples were selected for evaluation. Could this sample size be too small and lack sufficient randomness? Furthermore, when designing the ablation experiments for DGN and SCN, is it possible to validate their individual contributions separately?\n\n4:  After recovering the missing modalities, have the authors considered the potential issue of information redundancy?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}