# Dynamic Tuning and Multi-Task Learning-Based Model for Multimodal Sentiment Analysis
## 3 Methodology
### Multi-Task Learning  
In this section, we focus on the setup of multi-task learning and how we mitigate negative transfer through dynamic tuning. As shown in Figure 3, we employ a hard parameter sharing method for multi-task learning. In this approach, all tasks share the hidden layer parameters of the neural network, and each task has its own output layer. This method not only reduces the risk of overfitting but also encourages the model to find a general representation that covers all tasks. In terms of task setup, we first draw inspiration from previous studies and introduce a unimodal sentiment classification task to help the model better analyze the sentiment of the information [5]. Given that the MSA task requires the model to predict a value from a wide range, which is quite challenging, we further add a coarse-grained sentiment analysis task to help the model narrow down the prediction range and thus improve prediction accuracy. Moreover, to assist the model in better distinguishing between similarity and dissimilarity features, we also incorporate a contrastive learning task. Additionally, by leveraging the collaborative effect of the unimodal sentiment analysis and contrastive learning tasks, we enable the model to accurately extract similarity and dissimilarity features from the three information modalities. This, in turn, enhances the model's effective utilization of the intrinsic relevance and distinctive emotional inclinations embedded in the sentiment signals of different modalities. The final multi-task learning objective function is given by  
\[L_{MSA}=L_{MSR}+L_{MSC}+\lambda_{UIni}L_{Uni}+\lambda_{CL}L_{CL} \tag{13}\]  
where \(\lambda\) represents the weights generated by the GDWG method, \(L_{MSR}\) is the multimodal sentiment regression loss, \(L_{MSC}\) is the loss related to multimodal sentiment classification, \(L_{Uni}\) is the unimodal sentiment analysis loss, and \(L_{CL}\) is the contrastive learning loss. \(L_{MSR}\)**--Multimodal Sentiment Regression Loss**. The purpose of this task is to guide the model in integrating signals from different modalities to estimate the sentiment intensity of samples accurately. Herein, we feed the fused decomposed similarity and dissimilarity features into a multimodal MLP for sentiment intensity prediction, associating its output with the given multimodal sentiment intensity labels via a smooth L1 loss function to derive this loss. The formulaic expression is as follows:  
\[y^{*}=MLP([T_{s};T_{d};A_{s};A_{d};V_{s};V_{d}]) \tag{14}\]  
\[L_{mml}=\left\{ \begin{array}{l}0.5*(\frac{y^{*}-y}{\varphi})^{2},if(\frac{y ^{*}-y}{\varphi})<1\\ (y^{*}-y)-0.5\varphi,otherwise\end{array} \right\} \tag{15}\]  
where \(y^{*}\) represents the predicted result, \";\" represents concatenation, \(y\) represents the multimodal sentiment label, and \(\varphi\) controls the smoothness. \(L_{MSC}\)**--Multimodal Sentiment Classification Loss**. The purpose of this task is to direct the model toward the coarse-grained classification of sentiment, thus narrowing down the prediction scope. Unlike the fine-grained sentiment analysis task, this task does not require the model to precisely predict the exact intensity of sentiment, but rather to predict the range interval to which the sentiment intensity belongs. Initially, we map the sentiment intensity labels of samples to preset sentiment polarity classifications (such as positive, negative, and neutral) in accordance with predefined regulations, thereby creating a sentiment polarity label collection. Next, the disassembled multimodal features are efficiently combined and fed into a sentiment classifier as input, generating the probability distribution of each sample among different sentiment polarities. Eventually, the predicted probability distribution of the classifier is compared with the actual assigned sentiment polarity labels, and the cross-entropy loss function is utilized to measure the loss between the two.