Cross-lingual Multimodal Sentiment Analysis for Low-Resource Languages via Language Family Disentanglement and Rethinking Transfer (2025)

Abstract: Existing multimodal sentiment analysis (MSA) methods have achieved significant success, leveraging cross-modal large-scale models (LLMs) and extensive pre-training data. However, these methods struggle to handle MSA tasks in low-resource languages. While multilingual LLMs enable cross-lingual transfer, they are limited to textual data and cannot address multimodal scenarios. To achieve MSA in low-resource languages, we propose a novel transfer learning framework named Language Family Disentanglement and Rethinking Transfer (LFD-RT). During pre-training, we establish cross-lingual and cross-modal alignments, followed by a language family disentanglement module that enhances the sharing of language universals within families while reducing noise from cross-family alignments. We propose a rethinking strategy for unsupervised fine-tuning that adapts the pre-trained model to MSA tasks in low-resource languages. Experimental results demonstrate the superiority of our method and its strong language-transfer capability on target low-resource languages. We commit to making our code and data publicly available, and the access link will be provided here.

## introduction sentiment analysis aims to detect the sentiment orientations of opinion data. it can be applied to various domains, including market analysis, social media monitoring, and finance and investment. current advanced social platforms offer diverse multimodal opinion data, thereby extending sentiment analysis beyond its traditional reliance on textual data to encompass multimodal sentiment analysis (msa). early research in multimodal sentiment analysis primarily centers on the fusion of multimodal representations, with the progression ranging from basic feature-level fusion (po- † equal contribution * corresponding author @xcite to more sophisticated decision-level (chen and yang, 2020) and hybrid approaches @xcite . later, the research roadmap was dispersed into various fine-grained tasks such as multimodal pre-training models @xcite , modality-invariant representation learning @xcite , sentiment analysis under missing uncertain modalities @xcite and few-shot learning @xcite . however, none of these works can be utilized for multimodal sentiment analysis in lowresource languages. multilingual models can handle this problem. early studies focused on leveraging multilingual llms (e.g., mbert @xcite , xlm @xcite ) and adversarial training @xcite for cross-lingual transfer. subsequently, multi-view-based methods @xcite and contrastive learning @xcite have emerged for msa. however, these multilingual methods were designed solely for textual data. @xcite first attempted to conduct the multimodal and multilingual sentiment analysis task. nonetheless, the proposed model is overly simplistic and fails to account for the cross-linguistic alignment and cross-lingual noise resistance. in summary, the key problem we aim to address in this paper is that current multimodal methods struggle to adapt to data in low-resource languages due to their dependence on large-scale training data that is typically available from high-resource languages. multilingual models can address the challenge of data scarcity; however, they are primarily applied to single-modal textual data. inspired by the disentanglement technique in @xcite and the language family concept in linguistics, we propose a transfer learning framework called language family disentanglement and rethinking transfer. during the pre-training phase, we conduct cross-lingual and cross-modal alignments. for aligned cross-lingual features, we develop a module for language family disentanglement to share language universals within the same family, while reducing the noise generated by cross-family alignments. then, we use a fusion operation to perform a linguistic-to-modal transfer. we further establish a masked language task to recalibrate the fused representation and conduct the sentiment semantic learning task using double classifiers. during the fine-tuning stage, we propose a novel rethinking strategy to achieve unsupervised fine-tuning of the pre-trained classifiers. the two classifiers conduct a double check on the predictions from the input data. if they reach an agreement, the predicted label serves as a pseudo-label for prediction and backpropagation. if there is inconsistency, the data are sent back as input to conduct a masked prediction task and update the model parameters. in this manner, we finally achieve unsupervised fine-tuning, enabling the pre-trained model to adapt to downstream msa in low-resource languages. the experimental results demonstrate the efficacy of our methodology and its robust cross-linguistic transferability to lowresource target languages. ## related works as most of the existing work has been reviewed in the introduction, we briefly supplement some of the latest representative research. ## conclusion in this paper, we propose a multilingual multimodal sentiment analysis framework leveraging language family disentanglement and rethinking transfer. during pre-training, we first perform