Inferring Adjective Hypernyms with Language Models to Increase the Connectivity of O pen E nglish W ordnet (2025)

Abstract: Open English Wordnet is a key resource published in OntoLex-lemon as part of the linguistic linked open data cloud. There are, however, many links missing in the resource, and in this paper, we look at how we can establish hyper-ymy between adjectives. We present a theoretical discussion of the hypernymy relation and how it differs for adjectives in contrast to nouns and verbs. We develop a new resource for adjective hypernymy and fine-tune large language models to predict adjective hypernymy, showing that the methodology of TaxoLLaMa can be adapted to this task.

## introduction open english wordnet @xcite is an open-source fork of the princeton wordnet @xcite , which is modelled as and released as ontolex data @xcite . this work aims to continue the work of maintaining the resource, as well as providing a central resource for linked data resources to connect to. however, oewn is not itself a completely connected graph, as recent work on the verb hierarchy has shown @xcite . for adjectives and adverbs, there are a very large number of synsets 1 that are not linked and this reduces the effectiveness of this resource as a central resource in linguistic linked open data (llod) cloud @xcite . recent advances in large language models have significantly reshaped the field of lexical semantics @xcite . these models, trained on huge corpora, have demonstrated an emerging ability to infer nuanced lexical relations, and in tasks of taxonomy extraction and hypernymy discovery @xcite . our goal * work completed at insight research ireland centre for data analytics, university of galway. 1 'synonym set' or 'synset' for short are equivalent to lexical concepts in the ontolex model is to use this to fully connect the graph of oewn 2024, however, the performance of these models has been predominantly evaluated on nouns and verbs, leaving open the question of whether they can meaningfully capture a complex relation such as hypernymy among adjectives, which is underrepresented in existing datasets and lexical resources. hypernymy has been extracted from large text corpora and modelled in widely used benchmarks for lexical relations classification such as bless @xcite , evalution @xcite and hyperlex @xcite . yet, its application to adjectives remains considerably underexplored, despite its potential theoretical and practical relevance. unlike noun and verb hypernymy, which often relies on well-defined hierarchical taxonomies, hypernymy between adjectives is inherently more fluid and context-dependent, influenced by issues such as polysemy, gradability, and contextual ambiguity @xcite , which are often investigated by existing resources only from other relations' points of view @xcite . understanding hypernymy among adjectives is then critical and in order to explore its issues and despite their acknowledged complexity, our aim is to establish a clearer theory of adjective hypernymy that can offer insights into the broader organization of lexical meaning and support the development of more semantically aware language models. in this work, we try to address the theoretical and practical gaps surrounding hypernymy between adjectives, with two main objectives. first, we propose a theoretical framework for interpreting hypernymy among adjectives and construct a goldstandard english dataset of hyponym-hypernym pairs reflecting this relation, which is released in rdf using the ontolex model. 2 starting from existing lexical resources for other languages such as polish wordnet 3.0 @xcite and open dutch wordnet @xcite , we perform careful human annotation and validation to address issues of semantic ambiguity and ensure the reliability of the dataset for english by leveraging the oewn. second, we explore the ability of language models to recognise and interpret adjective hypernymy. we evaluate their performance both before and after training them on our gold standard, analysing how well they can capture this complex lexical relation 3 . examining models' performance on fine-grained and subjective semantic relations can explore and unveil their underlying linguistic competence and biases, contributing to the ongoing effort to interpret and refine their behaviour. in order for the models to be better in understanding language dynamics, we think that the development of specialized evaluation datasets is a foundational step towards benchmarking and improving their ability to disentangle complex semantic phenomena @xcite . in addition to its theoretical contribution that tries to address the lack of hierarchical structure of adjectives in english, our dataset offers several practical benefits: it fills a structural gap in lexical resources, enhancing their utility in knowledge representation and reasoning tasks; it supports improved performances in nlp tasks such as lexical entailment and word sense disambiguation; it serves as a novel benchmark for evaluating llms understanding of adjective semantics; it creates a foundation for possible multilingual extensions, towards better integration in the llod cloud. the rest of this paper is structured as follows. in section 2, we discuss related work on hypernymy, adjective semantics, and language model evaluation, and in section 3, we introduce the open english wordnet. section 4 presents our theoretical framework, while section 5 describes the construction of our adjective hypernymy dataset. section 6 focuses on the evaluation of language models on this task, including both zero-shot testing and finetuning experiments. section 7 presents the results and a summary of the findings. finally, section 8 concludes the paper and suggests directions for future research, followed by the limitations of this study. moving the focus to the task of lexical entailment and the specific relation of hypernymy, we face more ambiguity. traditional definitions of hypernymy, such as the proposal that one term a is a hypernym of another term b if a's meaning covers the meaning of b or much broader @xcite , are more straightforward if applied to nouns and verbs, whose hierarchical relations and structures are clearer. however, adjectives introduce challenges related to gradability, scalar structure, and context dependency @xcite . unlike nouns, which can be organized in chains where each hyponym naturally inherits properties from its hypernyms @xcite , adjectives resist simple hierarchical arrangement and prefer more compact and less vertical scales. notably, hypernymy relations between adjectives are absent from foundational lexical resources such as princeton wordnet @xcite . instead, adjectives are organized via antonymy (e.g., wet-dry), semantic similarity (e.g., dry-arid), and pertainymy (e.g., crime-criminal), with an absence of explicit modelling of hierarchical relations that could lead to a conflation between synonymy and hypernymy in adjectival semantics @xcite . the question is now around the existence of hypernymy for adjectives. recent works suggest that lexical relations should be conceptualized not in binary terms, but rather as graded semantic phe-nomena, with terms represented along a continuous scale according to principles of category membership @xcite . however, given the fact that most of the existing lexical resources and datasets created for lexical semantics tasks derive from the structure of the princeton wordnetwhich does not incorporate adjectival hypernymythe representation and formalization of this relation remains underdeveloped. some lexical resources for other languages have taken steps towards modelling adjectival hypernymy. for example, the germanet @xcite abandons the cluster-based approach of princeton wordnet, adopting instead a hierarchical structuring for adjectives similar to that of nouns and verbs, where, for example, "gut" ("good") is the hypernym of "toll" ("great"). similar hierarchical mappings are observed in the polish wordnet, where the original "dumbbell model" of princeton wordnet has been transformed into a vertical hyponymy structure @xcite . in the open dutch wordnet, although antonymy remains the dominant relation for adjectives, instances of hierarchical hypernymy are also introduced, such as "knotsgek, stapelgek, krankjorum, knettergek" ("very mad") as hyponyms of "gek, dwaas" ("mad") @xcite . in the context of the nlp community, the hypernymy relation has been explored in many works, from its automatic extraction from text corpora @xcite to investigations around language models' knowledge and classification capabilities @xcite . various evaluation studies have been conducted on language models' performances, but they overwhelmingly focus on nouns @xcite , verbs @xcite , or general natural language inference tasks @xcite . specific techniques such as promptingbased evaluation of hypernymy knowledge @xcite , dataset augmentation and benchmarking strategies @xcite have been proposed, but still none of them are dedicated specifically to adjective hypernymy. thus, while significant advances have been made in understanding lexical entailment for other parts of speech, an investigation specifically tailored to adjectives remains an unexplored field. ## open english wordnet the open english wordnet @xcite is a comprehensive, open-source lexical resource for the english language, derived from the original princeton wordnet @xcite , and developed using the w3c ontolex-lemon model @xcite . as an ontolex resource, oewn publishes lexical data on the web in accordance with linked data principles, thereby facilitating interoperability across linguistic and semantic resources. oewn plays a central role in the linguistic linked open data (llod) cloud @xcite , acting as a foundational resource for representing lexical semantics, enabling cross-linguistic comparison, and linking to other language resources. it uses the ontolex-lemon model as a standardized vocabulary for representing lexical information as rdf. the global wordnet association has developed formats for the representation of wordnets @xcite , which extend the core principles of ontolex. these formats support multiple serializations in xml, json(-ld), and rdf/turtle-providing interoperability with other ontolex resources. these formats have been adopted by oewn and other wordnets in the open multilingual wordnet (omw) to ensure compatibility and extensibility. in rdf, the gwa format directly encodes lexicalentry, lexicalsense, and lexicalconcept using ontolex classes, while wordnet-specific metadata and relations (e.g., sense keys, synset ordering) are modelled via a dedicated ontology at https: //globalwordnet.github.io/schemas/wn . ## theoretical proposal even though it is true that hypernymy for adjectives is difficult to define and has unclear boundaries, we think that it is a necessary relation that could help in the semantic organization of a language by introducing a hierarchy of meaning and usage that is not limited to a simple categorization in broad classes or to fuzzy and often confused horizontal and opposition relations. the distinction with synonymy is in fact very narrow, but we think that in the scope of the same semantic field, there are adjectives with a broader meaning than others, and those need to be distinguished (e.g., cognitive -mental -immaterial; displeasing -unpleasant -negative). the princeton wordnet has 117,659 synsets and 84,428 hypernym-hyponym relations, but those are only for nouns and verbs, and not for adjectives and adverbs. furthermore, there is currently no reliable dataset for the english language that represents this relation for adjectives. facing the limited availability of high-quality training data, one of the main objectives of this study was thus the creation of a reliable initial dataset containing adjective pairs and their definitions, drawn from the oewn, in order to address the problem of word-sense disambiguation. throughout the annotation and reliability evaluation of hyponym-hypernym pairs extracted from existing lexical resources, a principle of substitution was followed (see @xcite for other applications of lexical substitution): the hypernym should be substitutable in place of the hyponym in context, such that the resulting sentence preserves the original meaning at an acceptable general level, without contradiction or need for forced interpretation. while a semantic broadening is both expected and required, the sense of the hyponym needs to be included within the possible interpretations of the hypernym without ambiguity. thus, hypernymy is characterized by inclusion of meaning, but not equivalence, distinguishing itself from synonymy. in the annotation process, if the connection between the two adjectives of the candidate pair was not perceived as direct and intuitive by the annotators, the pair was discarded, as we do not want dubious inference or multiple plausible readings. given that hypernymy is inherently difficult to define -being vague and often subject to flexible and debatable boundaries -only those pairs deemed most reliable were retained (even though, undoubtedly, some may still be debated, highlighting once again the subjective nature of such a relation). one of the main challenges in identifying a hypernymy relation between two adjectives lies in the polysemous nature of many of them. a first example is "cold", which can refer either to the semantic field of temperature or to aspects of human behaviour and personality. another case is "hard", which in one sense refers to a physically solid material, and in another to the more abstract concept of difficulty. adjectives' meaning and their relations are therefore defined not only in isolation but, more importantly, depending on the associated noun and the semantic domain they evoke: "cold" can describe both an environment and a person; "hard" can qualify both a material and a problem. in this regard, context plays a crucial role. while this is already a challenge for human annotators, it becomes even more complex for language models that have to deal with semantic disambiguation (see section 5). moreover, many english words can assume more than one part of speech depending on context and usage, without undergoing inflectional changes, as english has a very limited morphological richness. a first example is the word "clean", which can function as both a verb and an adjective. a broader and more frequent case involves present and past participles, which are at times interpreted as verbs and at other times as adjectives. this ambiguity can be extended to nouns as well (the above mentioned word "cold" can have multiple adjectival meanings, but it can also be a noun: "a mild viral infection involving the nose and respiratory passages"), and given the pervasiveness of this phenomenon in english, it was deemed necessary to incorporate oewn definitions for both hyponyms and hypernyms into the construction of the gold standard dataset (see also the experiments of @xcite for why definitions are crucial). in contrast to the more straightforward case of nominal hypernymy, we think that adjectival hypernymy needs to be grounded in sense disambiguation, operationalized through substitution-based inclusion tests, and evaluated in a context. our gold-standard dataset leverages those principles and aims to provide an initial but reliable resource for further studies of adjectives' semantics and its modelling in computational systems. ## dataset creation the construction of the gold-standard dataset was based on pre-existing lexical resources, leveraging wordnets from languages that explicitly encode adjectival hypernymy relations: the open dutch wordnet (odwn) and the polish wordnet (plwn). these two resources were chosen because, unlike the princeton wordnet, they organize adjectives hierarchically, making them suitable starting points for our purposes. hyponym-hypernym pairs of adjectives were automatically extracted from those two resources by using the wn python library (goodman and bond, 2021), and an initial pool of 450 pairs was randomly picked from the total (166 from odwn and 284 from plwn). only adjectives were considered, and both child hyponym and father hy-pernym nodes were included. each extracted pair was then manually translated into english using bilingual online dictionaries (wiktionary and the cambridge dictionary), consistently selecting the first suggested translation to minimize subjective bias and arbitrary choice. after this, each pair was reviewed individually by two annotators. if both adjectives of the pair appeared in the same synset within the oewn -indicating synonymy rather than a hierarchical relation -the pair was discarded and the original relation was kept: for example, "difficult" and "hard", which are hypernym-hyponym in plwn, were not included because they are synonyms in oewn ("not easy; requiring great physical or mental effort to accomplish or comprehend or endure"). when multiple hypernyms were available for a hyponym, if they belonged to the same oewn synset they were kept; otherwise, only the most reliable one was chosen and agreed upon by the annotators, while the others were discarded. so, starting from the initial 450 pairs, 148 were discarded, and the final gold standard for english comprises 302 adjective pairs: 92 sourced from odwn, 170 from plwn, and 40 derived from either of the two (18 odwn, 22 plwn) but for which a more reliable alternative hypernym was proposed and approved by the annotators (e.g., the plwn had "effective" as hypernym for "deft", but "skillful" was suggested and approved as an alternative and kept in the dataset). acceptance rates were similar across resources, with a retention ratio of 0.55 for odwn (92/166) and 0.60 for plwn (170/284). furthermore, as already mentioned in section 4, for each adjective included in the gold standard, the corresponding english definition was included after retrieving it from the oewn. this was essential to ensure that the annotated relations reflected the intended senses of the adjectives rather than possible polysemous variants. the annotation process began with an initial small dataset of 50 adjective pairs, annotated by two annotators using a three-label classification system (yes, no, maybe). all cases of negative agreement (no-no), strong disagreement (yes-no), weaker disagreement (no-maybe) and shared doubt (maybe-maybe) were discarded, while complete agreement (yes-yes) cases were retained and partial agreement (yes-maybe) ones were further discussed (examples are shown in annotation hypo hyper yes-no intelligent rational yes-maybe limitless vast yes-yes lucid aware no-maybe multiple plural maybe-maybe unneeded useless no-no productive rich given the difficulty of semantic relations tasks which introduce a high level of subjectivity, interannotator agreement levels were moderate, but still acceptable, consistent and comparable across the different samples, with most disagreements involving "maybe" labels rather than direct contradictions (15 occurrences of the "maybe" label in the first sample, 24 in the second and 93 in the third). in addition to the main gold standard where each hyponym is associated to one single exact hypernym, a second version of the dataset was later developed for model fine-tuning and evaluation (see section 6.1). here, synonyms of each hypernym were added based on synset membership in oewn, in order to account for cases where multiple semantically correct hypernyms exist (see ## methodology from the definition of hypernymy for adjectives to the issues in creating a benchmark dataset, it is clear that the task of lexical entailment already presents many issues for human understanding. as for the capabilities of language models, there have been attempts to assess their reliability following either a binary classification approach (choosing the correct hypernym) or a generation one (given a hyponym, predicting its most probable hypernym). in this study, we use two models: taxollama and smollm-360m-instruct. we first test their dataset hyponym-lemma hypo_definition hypernym-lemma hyper_definition single relaxed without strain or anxiety calm not agitated; without losing self-possession multiple relaxed without strain or anxiety calm, serene, tranquil, unagitated not agitated; without losing self-possession taxollama @xcite is the finetuned version of the llama-2-7b model @xcite , which was trained on the wordnet dataset for 16 taxonomy-related tasks and reached sota results on hypernymy discovery in different domains and languages. as it was neither trained nor tested on adjective examples, however, its performance in predicting them was expected to be lower. smollm-360m-instruct is part of smollm, a family of state-of-the-art small models foot_1 . it is optimized for instruction-following tasks through supervised fine-tuning on multiple instruction datasets. even though it is much smaller than taxollama, its size makes it ideal for rapid finetuning and evaluation on specialized tasks such as adjective hypernymy discovery, enabling us to explore if smaller models can acquire semantic relations when provided with limited data. while recent models offer significantly improved performances across many tasks, our focus was on small to mid-sized models that could be fine-tuned efficiently, guided by high-quality and focused supervision. this emphasizes practical accessibility and cost-efficiency, but future work could investigate whether larger models perform significantly better in zero-shot or few-shot settings, or whether fine-tuning continues to provide a clear advantage. ## results the original base taxollama model reached sota results and scored an mrr (mean reciprocal rank) of 54.39 for english, but this was trained only on verbs and nouns sampled from the wordnet-3.0 graph. so, its performance on hypernymy discovery for adjectives was expected to be lower. in order to assess this, we test it in a zeroshot setting on our test set (91 pairs, 30% of the total gold standard), and we record an mrr of 9.4 when the model is not prompted with the definition of the input hyponym, and an increase to 25.8 when the definition is given. after the fine-tuning on the multiple dataset with the synonyms, those scores improve, respectively, to 23.6 and 33.3. the difficulty in treating adjectives and distinguishing them from other parts of speech was a major one with the base taxollama: when evaluated on the test set in a zero-shot setting without the definition, 58% of times it gives as output only nouns, 21% only adjectives, and 21% both (by providing it with the definition, the numbers change respectively to 14, 44 and 42%). after the fine-tuning on the single dataset, the amount of predicted adjectives significantly increases: 95% without the definition and 100% with the definition (both 100% when fine-tuned on the multiple dataset). this improvement is achieved by both taxollama and smollm-360m-instruct, as after the fine-tuning, the models were tested in two settings against the two different datasets, in order to first evaluate their capability of inferring the exact correct hypernym of a given hyponym (against the single dataset), and then introducing also the possibility of giving synonyms in output (against the multiple dataset). this variability and influence due to definitions was also one of the motivations to introduce synonyms into the multiple gold standard ("real" and "existent" are in the same synset in oewn). by allowing synonyms, we can consider acceptable also cases where the model predicts a semantically correct hypernym which is not exactly the one we expected. additionally, it is important to note that the base taxollama is trained to output a list of hypernyms, whereas taxollama-ft-single produces only a single prediction. this structural difference increases the chances of including a correct hypernym in the output for the base model (resulting in a higher f1-score of 0.39 against 0.31). however, when evaluating only the first-ranked hypernym from the base taxollama's output list, its performance drops, with only 7 out of 14 correct hypernyms appearing in the first position. the results shown in ## conclusion in this paper, we explore the understudied relation of adjective hypernymy, both from a theoretical and a computational perspective. we propose a definition for it, grounded in semantic inclusion and contextual substitutability, distinguishing it from synonymy and other relations. making use of this framework, we construct a two-version goldstandard english dataset for adjective hypernymy by adapting the lexical information stored in the polish, dutch and open english wordnets. our dataset was validated through human annotation and synset-based disambiguation, offering a reliable, small and initial benchmark for future research. we then evaluate the capabilities of language models -the large-scale taxollama and the smaller smollm-360m-instruct -on the task of hypernymy discovery, both in zero-shot and finetuned settings. our results show that the models initially struggle with adjective hypernymy, particularly with issues of pos ambiguity and semantic polysemy. however, after fine-tuning on our dataset, especially the synonym-augmented variant, their performances improve, highlighting the value of task-specific training data. we also found that providing the models with explicit definitions of the input adjectives improves their ability to identify correct hypernyms. this underscores the central role of word sense disambiguation in first identifying and then modelling adjectival meaning. as future work, we would like to a) expand the gold-standard dataset to reach a higher coverage and more generalizability, and allow for a better theoretical description and more reliability in training and evaluating language models, b) model adjective hypernymy in oewn, identifying and representing hypernymy relations for all adjectives in the resource, c) support the interoperability between the ontolex and nlp communities, promoting the use of adjective hypernymy in downstream applications, d) extend language models evaluation to other semantic relations between adjectives to explore how they differ theoretically and how well they are captured computationally.