YNU - HPCC at S em E val-2025 Task 5: Contrastive Learning for GND Subject Tagging with Multilingual Sentence- BERT (2025)

Abstract: This paper describes YNU-HPCC(Alias JH) team’s participation in the sub-task 2 of the SemEval-2025 Task 5, which requires fine-tuning language models to align subject tags with the TIBKAT collection. The task presents three key challenges: cross-disciplinary document coverage, bilingual (English-German) processing requirements, and extreme classification over 200,000 GND Subjects. To address these challenges, we apply a contrastive learning framework using multilingual Sentence-BERT models, implementing two innovative training strategies: mixed-negative multi-label sampling, and single-label sampling with random negative selection. Our best-performing model achieves significant improvements of 28.6% in average recall, reaching 0.2252 on the core-test set and 0.1677 on the all-test set. Notably, we reveal model architecture-dependent response patterns: MiniLM-series models benefit from multi-label training (+33.5% zero-shot recall), while mpnet variants excel with single-label approaches (+230.3% zero-shot recall). The study further demonstrates the effectiveness of contrastive learning for multilingual semantic alignment in low-resource scenarios, providing insights for extreme classification tasks.

## introduction this task emerges in response to the challenges associated with manually tagging library bibliographic records @xcite . our team is primarily involved in task 2, which is dedicated to aligning gnd subjects with tibkat records. this task involves two datasets: the core dataset, which contains 79,427 subject labels, and the all dataset, comprising 204,739 subject labels. the data for the entire task is presented in two languages, german and english, necessitating the consideration of multilingual characteristics during the modeling process. data analysis highlights two critical characteristics that have significant implications for the task. firstly, there is severe label underutilization, with only 31.9% (25,371 out of 79,427) of the core dataset labels and 16.5% (33,898 out of 204,739) of the all dataset labels appearing in the training and development subsets. secondly, there is extreme subject sparsity, as over 50% of the topics contain two or fewer documents. this situation creates high-dimensional semantic space challenges. given the large number of subjects to classify in this task, traditional topic-classification approaches like latent dirichlet allocation (lda, @xcite ) struggle due to complex, non-parallelizable probability calculations that result in low computational efficiency. although bert @xcite @xcite can capture sentence semantics effectively, its generated sentence vectors suffer from anisotropy, being unevenly distributed in the vector space and concentrated in a narrow cone, leading to generally high calculated vector similarities. most transformer-based pre-trained models face this issue in the learned sentencevector space @xcite . researchers @xcite have developed sentence-vector encoders using a "dual-tower" structure with sentence-task training datasets. with the advent of sentence-bert @xcite , sentence vector representation has advanced significantly @xcite @xcite . it modifies bert's structure and fine-tunes it via supervised tasks @xcite , overcoming bert's limitations in sentencevector representation. subsequently, @xcite proposed contrastive learning for sentencerelated tasks, often used to fine-tune sentence-bert. @xcite found that the loss function during sentence-bert training differs from that during prediction, causing poor performance in similarity determination. (nielsen and hansen, 2023) identified a hubness problem in sentence-bert's semantic space, which may also exist in this task. future research could improve the loss function and optimize the training set to address these issues. the pre-trained models used here are based on sentence-bert. the following sections detail our work and results. ## related work to address this task, we carried out a series of investigations. first, training datasets were created using three methods: the multi-label sample scheme, the single-label sample scheme with mixed negative samples, and the multi-label sample scheme with mixed negative samples. the prepared datasets were then utilized. simultaneously, we explored and experimented with several pre-trained models from sentence-transformers ( https://www.sbert. net/ ) and, prior to fine-tuning, tested them according to the official metrics (4.3). the test results are presented in subsequently, we trained several sentencetransformers models using contrastive learning and reported the training results along with their analysis. eventually, the average recall scores on the alltest and core-test datasets were 16.8% and 22.5% respectively (see furthermore, we proposed leveraging a translation model @xcite to conduct bidirectional translation between german and english, thus achieving data augmentation, and performing fine-tuning verification on the augmented data. lastly, we concluded the experiment, analyzed its limitations, and proposed subsequent improvement plans. ## methodology our architecture combines sentence-bert with contrastive learning to address the extreme classification challenge. as shown in figure 1 , the system processes tibkat-gnd pairs through dual encoders with shared parameters. semantic encoding for a tibkat record d and gnd subject s, we compute their embeddings: ## conclusion in this study, we constructed training datasets using both multi-label and single-label sample methods, mixed them with negative samples at a 1:1 ratio, and then fine-tuned several pre-trained models based on the sentence-bert architecture using contrastive learning. ultimately, we implemented and validated a solution for the subtask 2 of the semeval-2025 task 5 under low gpu memory conditions. our approach enables efficient and rapid topic retrieval with limited computational resources. the best results of our model submitted for official evaluation achieved an average recall rate of 0.1677 on the all-test dataset and 0.2252 on the core-test dataset (performance demonstrated only in pre-trained models with fewer than 300m parameters). for future research, bidirectional translation can be employed for data augmentation to address the issue of insufficient sample numbers in certain subjects. during inference, a two-stage approach combined with llms can be adopted. detailed steps for these subsequent tasks are provided in the results and analysis section.