# Cmdar: Cross-Modality Dual-Attention Fusion Network for Multimodal Sentiment Analysis
## 3 Methodology
### Cross-Modality Dual-Attention Module  
The main goal of MSA focuses on rich modal fusion representations from multimodal features. As shown in Figure 2, we propose the cross-modality multi-head dual-attention module, which comprises both multi-head dual-attention and self-attention components. It addresses the heterogeneity problem in multimodal data and captures the intra-modality and inter-modal correlations. The module contains N layers, enabling the network to perform multi-level feature fusion. Specifically, primary multimodal feature fusion is  
Figure 1: The overall architecture of the CMDAF. For the raw data, the network obtains unimodal features by pre-training feature extractors. The architecture comprises the following three essential components: the cross-modality dual-attention fusion module, the feature projection module, and the multi-loss learning module.  
conducted in the shallow layers, whereas advanced multimodal fusion takes place in the deeper layers. Since textual modality is more instructive for multimodal sentiment analysis [1; 11], we do not apply the dual-attention mechanism equally to the three modalities but mainly use textual modal features to guide the other modality's features for cross-modality interaction.  
In the first stage, the multi-head cross-modal dual-attention encoders focus on capturing and integrating inter-modal interactions across different modalities. This mechanism enables the model to identify complex dependencies among multimodal data. It provides a more comprehensive understanding of the data environment. This process can be computed as follows:  
\[X_{i}^{T,V} = MHCA(X_{i-1}^{T},X_{i-1}^{V},X_{i-1}^{V}), \tag{1}\] \[X_{i}^{T,A} = MHCA(X_{i-1}^{T},X_{i-1}^{A},X_{i-1}^{A}),\] (2) \[X_{i}^{T} = CA(X_{i}^{T,V},X_{i}^{T,A})+X_{i-1}^{T},\] (3) \[X_{i}^{T} = LN(Proj(X_{i}^{T}), \tag{4}\]  
where the \(MHCA\) (\(\cdot\)) denotes the multi-head dual-attention network, \(CA(\cdot)\) denotes the concatenation operation, \(Proj(\cdot)\) denotes the projection layer, and \(LN(\cdot)\) denotes the layer normalization operation.  
Figure 2: The network structure of the cross-modality dual-attention module.  
\[X_{i}^{V,T} = MHCA(X_{i-1}^{V},X_{i-1}^{T},X_{i-1}^{T}), \tag{5}\] \[X_{i}^{A,T} = MHCA(X_{i-1}^{A},X_{i-1}^{T},X_{i-1}^{T}),\] (6) \[X_{i}^{V} = LN(Proj(X_{i}^{V,T}),\] (7) \[X_{i}^{A} = LN(Proj(X_{i}^{A,T}), \tag{8}\]  
where the \(MHCA(\cdot)\) denotes the multi-head dual-attention network, \(CA(\cdot)\) denotes the concatention operation, \(Proj(\cdot)\) denotes the projection layer, \(LN(\cdot)\) denotes the layer normalization operation.  
In the second stage, the multi-head self-attention encoders focus on digging deeper and extracting the rich interaction information within each modality. This process helps the model better understand and represent the intrinsic structure of single-modality data. The stage can be formalized as follows:  
\[X_{is}^{m}=MHSA(X_{i}^{m},X_{i}^{m},X_{i}^{m}), \tag{9}\]  
\[X_{is}^{m}=LN(X_{is}^{m}+X_{i}^{m}), \tag{10}\]  
\[X_{is}^{m}=LN(PFN(X_{is}^{m})+X_{is}^{m}), \tag{11}\]  
where \(MHSA(\cdot)\) denotes the multi-head self-attention network, \(PFN(\cdot)\) denotes the pointwise feed-forward layer, \(m\in\{t,v,a\}\).  
The first phase of intermodal information interaction involves four main MHCA (multi-head cross-attention) networks. These networks are designed to handle the varying tendencies between different modalities when there are significant differences in their characteristics.