# Thangka Image--Text Matching Based on Adaptive Pooling Layer and Improved Transformer
## 5 Conclusions  
In this paper, we propose a Thangka image-text matching method that combines an adaptive pooling layer and an improved Transformer. For the former, we build a Thangka image-text feature extraction model, where named entity recognition and the Faster-RCNN model are used to extract features for the text and image modalities, respectively. We fully extract modality-specific feature information and introduce an adaptive pooling layer that combines the features from token and embedding dimensions. This allows the visual and textual encoders to learn the best way to combine their features and generate fixed-sized embeddings. For the latter, we propose a masked attention mechanism and a Bidirectional Residual model to improve the traditional Transformer, enhancing its focus on local feature information in the Thangka domain. We evaluated the model on both public datasets and Thangka datasets, comparing it with traditional VSE baseline models and large-scale pre-trained models. The results showed that the adaptive pooling layer improved feature extraction capabilities, and the improved Transformer outperformed the traditional Transformer model in matching similarity calculation. Our proposed model achieves good performance while maintaining good computational efficiency.  
Figure 6: Top-5 image match on Thangka images. The ground-truth results are marked with red, and the wrong results are indicated in green.  
Figure 7: Top-5 text match on Thangka datasets. The ground-truth results are marked with red, and the wrong results are indicated in green.  
However, there is still room for further improvement. For example, the traditional Faster-RCNN model performs poorly in processing complex Thangka images, such as those with multiple figures, due to the high similarity between different characters and the similarity between some characters and the background color. Exploring the integration of more effective target detection models, such as YOLO, into the feature processing for image-text matching will be a future research direction. In addition, since the available graphic data related to Thangka is still incomplete, this paper adopts graphic data augmentation to expand the original data. However, this makes the model more sensitive to noise and outliers during the training process. To address these issues, our future plans include exploring new algorithms and methods to improve the accuracy and robustness of the model, and attempting to apply the model to a wider range of datasets and application scenarios. Furthermore, we will supplement and enrich the Thangka image and text dataset to further enhance the performance of the model.  
Conceptualization, [PERSON]; methodology, [PERSON]; software, [PERSON]; validation, [PERSON]; formal analysis, [PERSON]; investigation, [PERSON]; resources, [PERSON]; data curation, [PERSON]; writing--original draft preparation, [PERSON]; writing--review and editing, [PERSON]; supervision, [PERSON]; funding acquisition, T.W. and X.G. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the 2023 Basic Research Operating Expenses of Central Universities (No. 31920230175), National Natural Science Foundation of China (No. 62166035), Natural Science Foundation of Gansu Province (No. 21 JR7 RA163) and Northwest University for Nationalities' Talent Recruitment Scientific Research Project (No. xbmuyjrc2023007).  
Not applicable.  
The data presented in this study are partially available on request from the corresponding author. The data are not publicly available due to their current restricted access.  
The authors declare no conflict of interest.