# Dynamic Tuning and Multi-Task Learning-Based Model for Multimodal Sentiment Analysis
## 1 Introduction  
Multimodal sentiment analysis (MSA) involves integrating and synergistically parsing diverse heterogeneous data modalities [1, 2, 3], encompassing many information forms such as text, visual, auditory, and even biometric markers [4]. With the evolution of social media ecosystems and the proliferation of multimedia content, information presentation has evolved from pure text to richly illustrated content, culminating in today's prevalent video-based information [5, 6]. Traditional unimodal sentiment analysis is confined mainly to the textual domain [7]. In contrast, MSA encompasses a comprehensive interpretation of multiple perceptual channels, including visual cues (e.g., facial expressions, scene color, and body movement) and audio characteristics (e.g., pitch amplitude, frequency distribution, and speech tempo) [8, 9]. MSA has garnered significant attention in recent research. On the one hand, human emotional expression inherently possesses cross-modal properties, with text, speech, and even haptic cues intricately interwoven to form sentiments, rendering a single modality insufficient to fully unveil the complexity of these sentiments [10, 11]. On the other hand, through their deep fusion of multiple signal sources, MSA technologies significantly enhance the accuracy of sentiment recognition and understanding, fulfilling the high-precision sentimentintelligence demands in domains such as intelligent customer service, VR/AR experience optimization, and mental health assessment [12; 13]. Existing deep learning-based MSA models typically consist of three core components: a single-modality feature extraction module, a multimodal feature fusion module, and a classification head [14]. The use of pre-trained models as the single-modality feature extraction module has been widely recognized as an effective strategy. Consequently, current researchers often focus on optimizing multimodal feature fusion methods and model training processes to further enhance the overall performance of the models. Multimodal fusion has emerged as a core technique for understanding video contexts, demonstrating its value across numerous downstream tasks [15; 16; 17]. Prior research has proposed a series of fusion techniques for MSA. For instance, [PERSON] et al. [18] employed a self-supervised joint learning strategy called Self-MM, which integrates discriminative information learned from individual unimodal tasks with shared similarity information from the multimodal task during the late fusion stage, thereby enhancing model performance. While multimodal fusion techniques are crucial for models [19; 20], setting optimization objectives is equally indispensable in model construction [5]. Suitable optimization objectives effectively guide the model toward continuous performance optimization throughout training [5]. Moreover, as shown in Figure 1, the setting of optimization objectives and model structure optimization focus on different modules, complementing each other. Early researchers typically used only the cross-entropy loss function to supervise the training of MSA models [14]. However, relying solely on single-task training makes it difficult to fully exploit the correlations between different modalities [5]. Therefore, subsequent researchers attempted to use multi-task learning methods, allowing MSA models to complete multiple related tasks simultaneously during training, thereby enhancing the utilization of different modality information. In this approach, the selection of related tasks is crucial, as different tasks can influence each other. Only by choosing the right combination of tasks can the model achieve better performance in multi-task training. As shown in Figure 2, the sentiment intensity derived solely from analyzing text data is \(+1\). When analyzing audio data alone, the sentiment intensity is \(-0.2\). Similarly, when considering only visual information, the sentiment intensity is \(-1\).