# Multimodal Framework for Long-Tailed Recognition
## 4 Conclusions  
This paper proposes a new multimodal long-tail recognition method that effectively improves classification performance on tail categories by introducing visual-semantic contrastive learning and a two-stage training framework. Experimental results show that the proposed method outperforms traditional methods on several imbalanced datasets (such as CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist2018), with a significant improvement in recognizing tail categories. Compared to existing unimodal methods, our framework takes advantage of pre-trained visual-semantic models, reducing the reliance on complex resampling strategies and improving model generalization. However, despite the many advantages, our approach has certain limitations. First, the method relies on pre-trained models such as CLIP, and the final performance of the model is somewhat dependent on the quality and suitability of the pre-trained model. In addition, our method involves two stages of training, with the first stage, particularly visual-semantic contrastive learning, requiring simultaneous processing of image and text features, which may result in higher computational costs. Although the performance gains are substantial, this computational complexity may pose challenges in certain practical applications. To mitigate these limitations, several directions for future work can be explored. First, introducing model distillation techniques could simplify the inference phase while maintaining performance. Second, for different application scenarios, developing smaller pre-trained models or reducing the dimensionality of text features could help optimize computational resources. Lastly, further research could explore the efficient application of multimodal models in distributed training frameworks to reduce training time and resource consumption.  
In conclusion, the proposed method demonstrates competitive performance in long-tail recognition tasks. Although its computational complexity is higher, this can be addressed through reasonable optimization strategies in real-world applications. We look forward to future research further enhancing the practicality and scalability of multimodal approaches.  
This research received no external funding.  
Not applicable.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Experiments**} & \multirow{2}{*}{
\begin{tabular}{c} **Contrastive** \\ **Learning** \\ \end{tabular} } & \multirow{2}{*}{**Text Feature Filtering**} & \multicolumn{3}{c}{**Multi-Model**} & \multicolumn{2}{c}{**Top-1 Accuracy**} \\  & & & & **Image** & **Brunch** & **Image–Text Brunch** & **ImageNet-LT** & **iNaturalist2018** \\ \hline
1 & ✓ & & & & 46.36 & 47.69 \\
2 & ✓ & ✓ & & & 46.72 & 48.02 \\
3 & ✓ & ✓ & ✓ & & 66.06 & 73.98 \\
4 & ✓ & & & ✓ & 69.58 & 72.86 \\
5 & ✓ & ✓ & ✓ & ✓ & **70.13** & **74.16** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Ablation experiments on ImageNet-LT and iNaturalist 2018 datasets.  
Data Availability Statement: The data presented in this study are openly available in the CIFAR-10 and CIFAR-100-LT datasets at [[https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)]([https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)), ImageNet-LT dataset at [[https://tensorflow.google.cn/datasets/catalog/imagenet_lt](https://tensorflow.google.cn/datasets/catalog/imagenet_lt)]([https://tensorflow.google.cn/datasets/catalog/imagenet_lt](https://tensorflow.google.cn/datasets/catalog/imagenet_lt)), and iNaturalist 2018 dataset at [[https://tensorflow.google.cn/datasets/catalog/i_naturalist2018](https://tensorflow.google.cn/datasets/catalog/i_naturalist2018)]([https://tensorflow.google.cn/datasets/catalog/i_naturalist2018](https://tensorflow.google.cn/datasets/catalog/i_naturalist2018)) (accessed on 2 October 2024).