W ord N et Is All You Need: A Surprisingly Effective Unsupervised Method for Graded Lexical Entailment (2023)

Abstract: We propose a simple unsupervised approach which exclusively relies on WordNet (Miller,1995) for predicting graded lexical entailment (GLE) in English. Inspired by the seminal work of Resnik (1995), our method models GLE as the sum of two information-theoretic scores: a symmetric semantic similarity score and an asymmetric specificity loss score, both exploiting the hierarchical synset structure of WordNet. Our approach also includes a simple disambiguation mechanism to handle polysemy in a given word pair. Despite its simplicity, our method achieves performance above the state of the art (Spearman \rho = 0.75) on HyperLex (Vulic et al., 2017), the largest GLE dataset, outperforming all previous methods, including specialized word embeddings approaches that use WordNet as weak supervision.

## introduction a crucial aspect of language understanding is the ability to draw inferences between sentences. in many cases, these inferences are directly licensed by the semantics of words: e.g., the sentence a duck is in the room entails an animal is in the room simply because the concept of duck entails that of animal. these cases of (taxonomic) lexical entailment (le) hold for words whose extensional denotations form a partial order: that is, the set of ducks is included in the set of birds which is itself included in the set of animals. an intriguing research question is whether existing hand-crafted lexical hierarchies like wordnet are indeed able to capture gle. preliminary experiments by @xcite ; @xcite report largely negative results: their best wordnet-only based system achieves a mere 0.234 spearman correlation score with human judgments from hyperlex. these poor performance results are blamed on the binary coding of the hypernymhyponym relation in wordnet. yet lear @xcite , the best gle system to date achieving a 0.682 spearman correlation score, uses wordnet as a source of constraints for specializing static word embedding models to the task. as static word embeddings alone achieve poor performance for gle, the question of the contribution of word-net in the lear improved performance remains open. in this paper, we propose a simple method that directly and solely exploits the internal structure of wordnet to predict gle. our approach relies on information content (ic), a continuous informationtheoretic measure introduced in @xcite to model semantic similarity in wordnet. specifically, we propose to model gle as a trade-off between a symmetric semantic similarity score and an asymmetric specificity loss score, both of which are defined in terms of ic. our method is completed with a disambiguation mechanism to address the fact that (g)le is sense, rather than word specific, and is therefore sensitive to polysemy, an issue that has been largely overlooked in previous work: e.g., the noun plant entails building only in its working plant sense, and not in its botanical sense. this simple method achieves a 0.744 spearman correlation score with human judgements, outperforming all previous systems, including specialized word embeddings methods and supervised models, as well as systems based on contextual language models. to sum-up, our main contributions are threefold. first, we show that the internal structure of word-net, as revealed by information-theoretic measures and completed by a disambiguation mechanism, is a reliable predictor of the graded nature of le. second, our simple wordnet-only based approach provides a new state-of-the-art for gle, outperforming previous methods that specialize word embeddings to the le task using wordnet as weak supervision. third, we provide a detailed analysis of our method showing the role of the two information-theoretic terms and the importance of sense disambiguation. we also present a simplified version of our scoring function without any frequency information in the computation of ic, which further improves the correlation score (0.753), thus emphasizing the singular importance of wordnet hierarchical structure for gle. ## proposed method given a(n) (ordered) pair of words (x, y ), instantiating a pair of latent (i.e., unknown) concepts (s x , s y ), we aim to predict a score gle(x, y ) indicating to what degree s x entails s y . specifically, we propose to compute the score gle(x, y ) as the sum of two terms: information content (ic) at first glance, word-net might appear inadequate to model gle because it encodes the hypernym-hyponym relation as a binary relation. but this claim is oblivious of two main facts. first, wordnet has some built-in gradedness as it models the hypernym-hyponym relation as a transitive relation. second, the binary nature of the taxonomic links in wordnet can be easily bypassed by resorting to the notion of ic. this information-theoretical notion provides a continuous value for synsets by fully exploiting the tree structure associated with the hypernym-hyponym relation. following @xcite , @xcite proposes to quantify the information content (aka self-entropy) of each lexical concept s as the log of its inverse probability by ic(s) = log(1/p (s)). while one can simply estimate p (s) via the word frequencies associated with s in a large text corpus, the crucial innovation of @xcite was to use the taxonomic tree structure of wordnet in this estimation. specifically, p (s) is estimated as similarity we define sim as the ic-based similarity measure introduced in @xcite . the similarity between two synsets ŝx and ŝy is defined as the ratio between the information shared by the two concepts, modeled by the ic value of their least common subsumer node (denoted as lcs below), and the information needed to fully describe the two concepts, modeled as the sum of their ics, leading to ## experiments this section presents our experimental framework and results of our approach against various baselines and competing systems. foot_2 ## analysis this section analyses the different components of our approach via several targeted ablation studies. similarity without specificity or vice-versa we present correlations for the individual similarity and relative specificity functions compared to that of their sum in ## conclusion we presented a new state of the art method for predicting gle between word pairs, which solely relies on the structure of wordnet accessed through information-theoretic measures. this new result contradicts previous results arguing that the binary nature of wordnet relations prevented to use word-net for graded le. it also shows that a direct use of wordnet performs better than using static word embeddings specialized with wordnet-extracted constraints, suggesting that such methods have not been able to fully leverage the rich structural information of wordnet. our work also emphasizes the importance of polysemy in this task, an issue being largely ignored in previous work. we tested different clm-based methods but so far, to the best of our knowledge, no clm-based method is able to solve the gle task, thus raising the question whether (enhanced) clms are able to model the hierarchical structure of concepts inherent to human semantic memory.