Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval (2025)

Abstract: Automatic Term Extraction (ATE) identifies domain-specific expressions that are crucial for downstream tasks such as machine translation and information retrieval. Although large language models (LLMs) have significantly advanced various NLP tasks, their potential for ATE has scarcely been examined. We propose a retrieval-based prompting strategy that, in the few-shot setting, selects demonstrations according to syntactic rather than semantic similarity. This syntactic retrieval method is domain-agnostic and provides more reliable guidance for capturing term boundaries. We evaluate the approach in both in-domain and cross-domain settings, analyzing how lexical overlap between the query sentence and its retrieved examples affects performance. Experiments on three specialized ATE benchmarks show that syntactic retrieval improves F1-score. These findings highlight the importance of syntactic cues when adapting LLMs to terminology-extraction tasks.

## introduction automatic term extraction (ate) identifies domain-specific terms essential for tasks such as machine translation, information retrieval, and content curation @xcite . despite its importance, ate remains underexplored compared to other information extraction (ie) tasks, particularly in low-resource and specialized domains @xcite . large language models (llms) offer new possibilities for ie through in-context learning, yet prior studies @xcite @xcite @xcite show they often underperform compared to task-specific pretrained language models (plms), struggling with domain precision and boundary detection. while strategies like prompt engineering † corresponding author and retrieval-based demonstrations have improved ie in general, their application to ate remains largely unexplored. we address two key challenges in applying llms to ate: (1) dataset scarcity and domain diversity-ate lacks diverse datasets beyond the biomedical field @xcite , limiting cross-domain effectiveness. we propose a retrieval method that generalizes across domains. (2) boundary identification-llms struggle to extract precise term spans @xcite , a critical issue given the annotation-intensive nature of ate @xcite . to address this, we propose a syntactic retrieval method that selects structurally aligned demonstrations. in both in-domain and cross-domain settings, this approach consistently improves ate performance by enhancing annotation consistency and extraction accuracy. ## motivation and methodology in this section, we formulate ate within the existing in-context learning framework for llm-based information extraction @xcite , and introduce our proposed methodology. ## ablation study in this section, we conduct ablation studies to examine: (1) how the performance of each retrieval method scales with the number of demonstrations, and (2) how our in-context llm approach compares to strong plm baselines across selected dataset model p r f1 cross-domain acter roberta-large 69.3 ±1.3 54.7 ±1.2 61.2 ±1.1 bart-large 66.1 ±1.2 48.2 ±1.2 55.8 ±1.2 gemma-2 fastkassim 64.3 ±1.1 56.6 ±1.1 60.2 ±1.0 in-domain aclr2 roberta-large 85.8 ±2.5 88.9 ±2.1 87.4 ±2.1 bart-large 81.3 ±2.6 84.8 ±2.3 83.0 ±2.2 gemma-2 bm25 77.3 ±3.0 84.4 ±2.2 80.7 ±2.2 bcgm roberta-large 88.0 ±0.8 89.0 ±0.8 88.5 ±0.8 bart-large 79.4 ±1.0 77.1 ±1.1 78.2 ±1.0 mistral fastkassim 50.8 ±1.4 57.2 ±1.2 53.8 ±1.1 ## conclusion we explored the use of llms for ate and proposed a syntactic retrieval method to address two key challenges: dataset scarcity and term boundary identification. experiments on acter, aclr2, and bcgm showed that syntactic similarity-based retrieval improves ate performance across both in-domain and cross-domain settings. we also introduced the term overlap ratio to analyze how different retrieval strategies depend on the presence of gold terms in the demonstration corpus. our results indicate that syntactic retrieval relies less on such overlap compared to semantic or lexical methods, highlighting its robustness in low-resource scenarios.