# Dynamic Tuning and Multi-Task Learning-Based Model for Multimodal Sentiment Analysis
## 3 Methodology
### Multi-Task Learning
The specific formulaic expression is as follows:  
\[y_{MSC}=Classifier([T_{s};T_{d};A_{s};A_{d};V_{s};V_{d}]) \tag{16}\]\[L_{\text{MSC}}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C}y_{i,c}\log(y_{MSC}^{i,c}) \tag{17}\]  
where \(N\) denotes the number of samples and \(C\) represents the number of categories. \(L_{\text{{Unimal}}}\)**-Unimodal Sentiment Analysis Loss**. The purpose of this task is to guide the model in delving into the sentiment information embedded within each modality. Here, to ensure consistent treatment of modal features, we feed the similarity features (\(T_{s}\), \(V_{s}\), and \(A_{s}\)) and dissimilarity features (\(T_{d}\), \(V_{d}\), and \(A_{d}\)) of each modality separately into a weight-sharing MLP layer. The MLP layer outputs six sentiment predictions, \(u^{*}=MLP(\left[T_{s}/T_{d}/A_{s}/A_{d}/V_{s}/V_{d}\right])\), with the similarity features used to infer the multimodal sentiment label \(y\) and the dissimilarity features used to predict the corresponding unimodal sentiment labels \(y^{t/v/a}\). In the absence of unimodal labels, the dissimilarity feature prediction task adjusts to instead predict the multimodal label \(y\), maintaining the coherence of model training. Finally, a smooth L1 loss function is employed for each prediction to measure the loss between the prediction and the respective ground-truth label \(u=\left[y/y/y^{t}/y^{p}/y^{a}\right]\). The specific formulaic expression is as follows:  
\[L_{\text{{{{{L}}}}}}=\left\{ \begin{array}{l}{0.5*(\frac{{u^{*}-u}}{\varphi})^ {2},if(\frac{{u^{*}-u}}{\varphi})<1}\\ {({u^{*}-u})-0.5\varphi,otherwise}\end{array} \right\} \tag{18}\]  
\(L_{\text{{{{CL}}}}}\)**-Contrastive Learning Loss**. Contrastive learning tasks help models learn the commonalities among positive examples and the differences from negative examples by comparing positive and negative samples. Through this task, models can extract high-quality similarity and dissimilarity features, thereby paying attention to both unimodal and multimodal sentiment information. Here, considering that text modality data often assume a dominant role in MSA tasks, with other modalities providing auxiliary information to enhance prediction accuracy, we opt to use text data as the reference anchor for constructing positive and negative sample pairs [12]. The specific configuration is as follows:  
\[N=\left\{(T_{s},T_{d}),(T_{s},V_{d}),(T_{s},A_{d})\right\} \tag{19}\]  
\[P=\left\{(T_{s},V_{s}),(T_{s},A_{s})\right\} \tag{20}\]  
Subsequently, we employ our proposed Multi-NT-Xent loss to guide the model in maximizing the similarity between positive sample pairs while minimizing the similarity between negative sample pairs. The calculation formula for the Multi-NT-Xent loss is as follows:  
\[L_{MNTX}=-\log\frac{\sum_{(a,p)\in P}\exp(sim(a,p)/\tau_{m})}{\sum_{(a,k)\in N \cup P}\exp(sim(a,k)/\tau_{m})} \tag{21}\]  
where \(\tau_{m}\) is the temperature coefficient that controls the similarity distribution; \((a,p)\) and \((a,k)\) denote positive and negative sample pairs, respectively; \(N\) represents the set of negative pairs; and \(P\) signifies the set of positive pairs.