# A Modality-Enhanced Multi-Channel Attention Network for Multi-Modal Dialogue Summarization
## 4 Experiment
### Discussion  
MEMA outperforms previous dialogue summarization models in all metrics. The average performance improvement across ROUGE and BLEU scores is substantial, demonstrating that integrating multi-modal information significantly enhances the semantic coverage of the reference summaries. Specifically, the model's ability to harmonize textual and visual cues appears to be a key factor contributing to its improved performance. The fusion mechanism within MEMA allows for a more nuanced understanding of the dialogue context, which is crucial for generating accurate and coherent summaries. However, adding multi-modal data to conventional abstractive summarization models does not guarantee improved performance. If models fail to reconcile conflicts between different modalities, lower performance may be the result. Traditional multi-modal summarization models often need to implement more than purely text-based models, underscoring the complexity of multi-modal dialogue summarization. Models like HOW2 and VMSMO, which use separate components for modal interaction, highlight the fact that disjointed modal interactions are inadequate for this task. These findings emphasize the uniqueness \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **ROUGE-1** & **ROUGE-2** & **ROUGE-L** & **BLEU-1** & **BLEU-2** & **BLEU-3** & **BLEU-4** \\ \hline S2S [25] & 21.14 & 6.99 & 18.22 & 7.31 & 3.73 & 2.51 & 0.37 \\ PGN [26] & 20.97 & 4.42 & 18.46 & 10.84 & 4.24 & 1.42 & 0.19 \\ Transformer [27] & 38.18 & 17.73 & 32.42 & 24.02 & 16.70 & 11.81 & 4.93 \\ T5 [28] & 40.69 & 18.62 & 37.22 & 12.04 & 8.39 & 6.10 & 3.44 \\ MDaiBART [29] & 26.27 & 8.69 & 20.41 & 11.64 & 7.34 & 4.94 & 2.41 \\ ConDigSum [30] & 37.36 & 17.77 & 29.35 & 24.11 & 17.11 & 11.59 & 3.97 \\ HOW2 [1] & 20.71 & 3.85 & 18.04 & 10.38 & 3.71 & 1.14 & 0.14 \\ VMSMO [6] & 15.79 & 4.25 & 13.24 & 6.18 & 2.70 & 1.37 & 0.13 \\ MEMA & **43.55** & **30.58** & **41.11** & **28.37** & **20.98** & **15.20** & **11.95** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: ROUGE scores and BLEU scores of summarization baselines on MDS. and challenges of multi-modal dialogue summarization, which differs significantly from traditional summarization tasks. Previous models need help with modality conflicts and often yield suboptimal results. Our experiments also reveal that using separate modal interaction components is less effective than a unified approach. Despite the advantages typically gained from scaling up models, non-pre-trained dialogue summarization model ConDigSum demonstrates a surprising capability in generating summaries, outperforming the pre-trained MDialBART. The result indicates the challenges of applying pre-trained models' prior knowledge to multi-modal dialogue summarization. In contrast, MEMA leverages the strengths of a pre-trained model and effectively handles modality interactions. The ablation study described in Section 4.4 further confirms that the fusion of multi-modal content enhances ROUGE and BLEU scores, affirming MEMA's superior multi-modal modeling capabilities. However, it is important to acknowledge the limitations of MEMA. For instance, the model may underperform in scenarios where the quality of the input modalities is low, such as under condition of noisy audio or blurry video. Additionally, MEMA's reliance on a pre-trained model might limit its adaptability to domains with significantly different linguistic and visual characteristics relative to the pre-training data. Furthermore, the computational cost of processing multi-modal data can be a barrier to widespread adoption, especially in resource-constrained environments. Addressing these limitations in future work will be crucial for the continued advancement of multi-modal dialogue summarization techniques. In addition, regarding the problem of modality conflicts, future work could focus on enhancing MEMA's ability to identify and reconcile modality conflicts.