# Multimodal Framework for Long-Tailed Recognition  
[PERSON]  
1 College of Coastal Defense Force, Naval Aviation University, Yantai 264001, China; [EMAIL_ADDRESS] (J.C.); team2 [EMAIL_ADDRESS] (J.G. ); [EMAIL_ADDRESS] (Y.Q. ); [EMAIL_ADDRESS] (H.J.) [PERSON]  
Correspondence: [EMAIL_ADDRESS] (J.  
a representation that is less biased toward head classes, followed by training a classifier with re-weighting schemes. This decoupling improves performance by focusing on the specific challenges of imbalanced classification. However, they do not fully exploit the rich semantic relationships present in multimodal data. Multi-branch Models: Multi-branch architectures, such as RIDE [8] and LFME [9], employ multiple expert models to address class imbalance. Each expert specializes in different portions of the data distribution, allowing for more tailored learning. These methods have shown promising results, particularly in handling tail categories. However, they often involve significant computational complexity, making them less practical for large-scale applications. Additionally, few-shot learning methods also aim to solve the problem of recognizing classes with limited data, which bears some similarity to long-tailed classification. For example, [PERSON] et al. proposed a Hellinger Distance-Attention-based Feature Aggregation Network (HELA-VFA) for few-shot classification [10]. Similarly, [PERSON] et al. introduced a few-shot learning method, Felmi, based on hard mixup [11]. These methods offer valuable insights for addressing the issue of tail classes in long-tailed data. In recent years, the problem of long-tailed recognition has garnered increasing attention and research [12; 13; 14; 15; 16; 17; 18]. Since real-world data often exhibit long-tailed distributions (e.g., Pareto distribution [19]), traditional visual models are prone to class imbalance issues. The abundance of head class data leads models to focus on learning head features, while the scarcity of tail class data results in poor generalization capabilities. Although collecting more data can alleviate this issue to some extent, this approach is often costly and unsustainable. Thus, developing more efficient algorithms to handle long-tailed distributions has become crucial. Despite the effectiveness of existing approaches, such as category rebalancing and network structure improvements, in visual modalities, they often overlook the potential semantic information in multimodal data, such as the semantic features embedded in label texts. These features hold significant potential in enhancing model generalization and improving the recognition of tail classes. Therefore, this paper aims to explore multimodal solutions to the long-tailed recognition problem by integrating visual and textual information to optimize and enhance existing methods. Contrastive visual-semantic models such as CLIP [20] have shown great capabilities in recent years. Such models are mainly pre-trained on a large number of image-text pairs collected from the Internet, aligning visual representation and language representation with contrastive loss. A model trained by this method shows strong classification performance in zero-shot cases. In the long-tail problem setting, if the information contained in the text label can be effectively used, it is expected to improve the recognition accuracy on long-tailed data. CLIP utilizes large-scale pre-trained models to align visual and textual representations, showing exceptional performance on various zero-shot tasks. However, these models have not been extensively applied to long-tailed data, where leveraging both image and text modalities could further enhance classification performance, particularly for tail categories. The motivation behind this work stems from the observation that most existing methods focus solely on visual data, overlooking the valuable information present in textual descriptions.