Inferring Semantic Relations Between Terms with Large Language Models (2025)

Abstract: The purpose of this paper is to investigate the ability of Large Language Models (LLMs) to identify relations among terms, with the goal of facilitating and accelerating the construction of thesauri and terminological resources. We investigate whether the use of LLMs in this context can provide a valuable initial set of relations, serving as a basis upon which professional terminologists can build, validate, and enrich domain-specific knowledge representations.

## introduction the identification and formalization of semantic relations among terms constitute a fundamental task in the development and refinement of thesauri and terminological resources. in any specialized knowledge domain, terms do not exist in isolation but form a complex net of semantic relations. these relationships enable the structuring of domain knowledge, facilitate precise communication, and support tasks such as indexing, searching, and reasoning. early foundational work in terminology science, such as by @xcite , emphasized the importance of defining clear hierarchical relations (broader-narrower) and associative relations to support knowledge organization and retrieval. recent theoretical developments, such as frame-based terminology @xcite , further highlight the central role of semantic relations in the organization of specialized knowledge. systematic terminological standards, including iso 25964-1:2011 and iso 704:2022 for thesauri, codify relations among terms into three main types: formal models such as skos (simple knowledge organization system) have standardized the representation of semantic relations in thesauri, providing an interoperable framework for semantic web applications @xcite . this work aims to analyze and compare the outof-the-box performance of two llms (chatgpt and gemini) in identifying terminological semantic relations, with the aim of supporting the development of thesauri and other terminological resources. the case study and the gold standard dataset is in the domain of building materials and constructions. one of the first attempts to automatically identify hierarchical relations was proposed by @xcite , who introduced lexico-syntactic patterns (e.g., "x such as y", "y is a type of x") as indicators of hyponymy. one influential approach in this area is the use of knowledge patterns, as introduced by @xcite , which have been instrumental in the extraction and identification of semantic relations. automatic systems based on this approach have been developed to support the extraction of semantic relations from corpora. notable examples include the ecolexicon semantic sketch grammar (essg) @xcite , which applies knowledge patterns within a frame-based approach to identify and organize conceptual relations in domain-specific texts, and corpógrafo @xcite , a tool designed to facilitate corpus-based terminological analysis by detecting and visualizing term relationships and cooccurrence patterns. studies by @xcite ; @xcite ; @xcite extended this by combining pattern-based extraction with distributional semantics, enabling more nuanced identification of various semantic relations beyond simple hierarchies. in parallel, graph-based methods have gained traction for modeling and identifying term relations @xcite . tools like wordnet @xcite exemplify a rich network of lexicalsemantic relations structured as a graph, which has influenced the design of domain-specific thesauri. identifying these relations often requires domain expertise combined with semi-automated methods, such as supervised learning approaches trained on annotated corpora. studies such as @xcite have shown that llms can retrieve encyclopedic facts through cloze tasks (e.g., "paris is the capital of ___"), but the ability to infer more abstract conceptual relations (e.g., hypernymy, meronymy) remains underexplored. recently, some works have analyzed the implicit presence of structured semantic knowledge in llms. for example, @xcite investigated the extent to which llms can correctly answer questions about relations between entities. however, these studies almost always rely on rich contextual input (sentences, definitions, knowledge triples), whereas our work focuses on a minimal setting in which the model receives only a list of terms and must infer possible relations. trained on massive text corpora, llms have demonstrated a strong capacity to understand and generate human-like text. they are also capable of handling a wide range of linguistic tasks with little need for explicit guidance. as a result, recent stud-ies have increasingly explored their effectiveness across a variety of nlp tasks. these capabilities raise a compelling question: can llms effectively support or even partially automate the process of finding relations among terms for terminological resource construction purposes? in this regard, despite a growing interest in the use of llms in automatic terminology extraction tasks for the creation of resources, glossaries, and thesaurus @xcite , there is still no systematic evaluation of their ability to infer semantic relations from simple term lists, without context, examples, or explicitly provided background knowledge. ## case study our study explores the zero-shot capabilities of llms in inferring relations between terms, without access to external texts or the use of fine-tuning techniques. we analyze the llms ability in this task by comparing their output against a thesaurus about construction materials of monuments, buildings, and structures taken as gold standard reference. no examples are provided in the prompt; the model relies only on prior knowledge. ## results and evaluation the following examples showcase representative outputs produced by each model, offering qualitative insight into their ability to identify semantic relations among the listed terms. the comparative evaluation of chatgpt and gemini highlighted a significant difference in their ability to capture hierarchical depth in conceptual structures. the recall is slightly lower than precision across models, indicating a tendency to be conservative in identifying relations, often abstaining when unsure. in addition to the quantitative metrics, a qualitative evaluation was carried out to understand the nature of the errors and the strengths of each model. while both llms performed almost adequately in detecting first-level (direct) conceptual relations, such as hypernymy (e.g., 'granite' or 'limestone' is a type of 'stone'), none of the models showed great performance in capturing deeper taxonomic structures and inferring multi-step hierarchical relations, which can in some cases be very complex and reach deep levels, as in figure1. generally speaking, gemini retains and operationalizes more granular domain knowledge, possibly due to a more extensive training dataset or improved alignment mechanisms. conversely, chat-gpt tended to limit more often its inferences to surface-level relations and classifications (e.g., @xmath0, particularly when terms were more technical or less frequent. these findings suggest that gemini may have stronger capabilities in ontology-oriented processing, making it better suited for tasks involving the structuring or expansion of specialized terminologies. both llms, however, may still provide valuable support to the terminographer, especially in the identification of first-level semantic relations-such as broader term (bt), narrower term (nt), and related term (rt) links-across wellrepresented conceptual domains. while gemini may offer more depth in modeling nested hierarchies and domain-specific nuances, chat-gpt can nonetheless assist in generating baseline classifications or verifying established semantic patterns, particularly when supplemented with domain-specific prompts or curated input examples. ## conclusion and future works this study investigated the ability of llms to infer semantic relations between terms in a minimalinput, zero-shot setting, without access to external context, training data, or fine-tuning. our results suggest that, while llms demonstrate promising capabilities in identifying firstlevel types of conceptual relations, they still struggle with a deeper level (second and third level) of analysis. one of the key contributions of this work is to highlight that llms encode a certain degree of structured semantic knowledge that can be activated even in the absence of linguistic context or definitional cues. at the same time, our findings underline the limits of this implicit competence, especially when models are required to infer secondlevel hierarchies. from a terminological practice perspective, these insights suggest that llms could serve as lightweight tools for semi-automatic relation extraction, particularly in the early stages of resource construction or when dealing with low-resource domains. however, their use should be complemented with expert validation or human-in-the-loop, given the high degree of variability and wrong hierarchy structuring. as future works, several directions emerge from this initial study: further analysis could take into account different prompting strategies (e.g., fewshot, chain-of-thought) to improve the quality and explainability of relational inferences. moreover, extending the range of relation types (e.g., madeof, causes, used-for) would allow for a broader assessment of the models' semantic competence. future experiments may also focus on reproducing this experiment on different domain-specific term lists (e.g., in medicine, law, or engineering) as well as on other languages in order to generalize the output results and evaluation.