LA ² I ² F at S em E val-2025 Task 5: Reasoning in Embedding Space – Fusing Analogical and Ontology-based Reasoning for Document Subject Tagging (2025)

Abstract: The LLMs4Subjects shared task invited system contributions that leverage a technical library’s tagged document corpus to learn document subject tagging, i.e., proposing adequate subjects given a document’s title and abstract. To address the imbalance of this training corpus, team LA²I²F devised a semantic retrieval-based system fusing the results of ontological and analogical reasoning in embedding vector space. Our results outperformed a naive baseline of prompting a llama 3.1-based model, whilst being computationally more efficient and competitive with the state of the art.

## introduction motivation. to assist human librarians in cataloging documents with suitable subject tags from modern libraries' comprehensive subject indices (turvey and letarte, 2014), the semeval-2025 shared task "llms4subjects" @xcite explores the feasibility of large language model (llm)-based automated subject tagging. the systems should exploit a human-tagged document corpus from the leibniz university's technical library (tibkat) to learn subject tagging @xcite : given a document's title and abstract as input, the system should propose the top-k best-matching subjects from the ontology gemeinsame normdatei 1 (gnd) for both english (en) and german (de) documents. ## experimental setup to estimate our models' generalization error and perform model selection, we created an internal validation split by sampling 10% of the provided training data, utilizing the remaining 90% for creating our models. this also allowed us to conduct ablation studies to measure the impact of the individual reasoning branches and the fusion approach. for producing our test set submission, we combined the provided training and development split for "training" our models, which in our approach only means embedding these corpora with all-mpnet-base-v2, utilized via the sentence transformers/sbert framework @xcite . evaluation metrics comprise precision, recall, and f1 measure assessed for dif-ferent cut-offs of top-k subjects 4 . in addition to the quantitative evaluation on existing ground truth data (which typically includes few ground truth subjects per document, usually 5-7 subjects), task organizers sampled 122 test documents across 14 subject classifications for the qualitative evaluation. the generated top-20 subject labels output by the participant systems then were marked by human subject matter experts as either correct, technically correct but irrelevant, or incorrect subject labels. 5 results ablation study. fig. 3 compares the individual performance of our two reasoning branches and the fusion approach on our own validation set. analogical reasoning outperforms the naive baseline, and also the ontology-based retrieval branch by a large margin. fusing the results of both branches -analogical reasoning and ontology-based retrieval -improves the metrics slightly further, suggesting that both branches extract complementary information, which is also backed by empirical evidence: example document 2 (document id: 3a1649002734) shows one output from our if system with subject predictions contributed from both reasoning branches. this highlights the feasibility of fusing the outputs of both ontological and analogical reasoning, which may retrieve complementary information. the curve of the fusion approach also surpasses the naive llama3.1:8b-based model, which presumably has more advanced nlu capabilities but does not incorporate the information from the ontology and the training data in its reasoning. thus, fusing the outputs of simpler, domainspecific models -geared to the problem domain and focused on one specific reasoning approach at a time -is capable of outperforming a larger, generic and computationally far more costly model. foot_3 ## conclusion our proposed system explores innovative solutions to the challenging problem of document subject tagging: by operating in embedding space for identifying the semantically most similar documents and subjects, the corpus imbalance on the different subjects is not an issue: no matter whether a cluster/manifold is densely or sparsely populated, only the distances to the closest training documents and subjects are factored in. the relative frequency of documents per subject hence does not dominate or bias the "learning" process, unlike with training a neural network-based model with gradient descent or related ml classifiers. if the semantically closest document has similar contents and matching subject tags, those will be consistently proposed as the top-ranked subjects. hence, our approach also corresponds to an interpretable and transparent strategy in terms of explainable ai (xai) @xcite . since our model "training" only requires the embedding of the gnd subjects and labeled training corpus, our approach is computationally efficient and suitable for "online learning" -if new labeled documents should be incorporated in the model, those only need to be embedded into the vector space, no "retraining" is required for dynamically growing the training corpus. from the shared task's evaluations, we conclude that our semantic retrieval-based if system is competitive with state-of-the-art xmlc approaches such as annif, as comparatively evaluated in @xcite . for future work, we aim to enhance our fusion by improving re-ranking, for instance by up-ranking results identified by both reasoning strategies.