SANDW i CH : Semantical Analysis of Neighbours for Disambiguating Words in Context ad Hoc (2025)

Abstract: The rise of generative chat-based Large Language Models (LLMs) over the past two years has spurred a race to develop systems that promise near-human conversational and reasoning experiences. However, recent studies indicate that the language understanding offered by these models remains limited and far from human-like performance, particularly in grasping the contextual meanings of words—an essential aspect of reasoning. In this paper, we present a simple yet computationally efficient framework for multilingual Word Sense Disambiguation (WSD). Our approach reframes the WSD task as a cluster discrimination analysis over a semantic network refined from BabelNet using group algebra. We validate our methodology across multiple WSD benchmarks, achieving a new state of the art for all languages and tasks, as well as in individual assessments by part of speech. Notably, our model significantly surpasses the performance of current alternatives, even in low-resource languages, while reducing the parameter count by 72%.

## introduction in 2022, openai fine-tunned their previously released gpt-3 @xcite model using reinforcement learning from human feedback (rlhf), resulting in the instructgpt model @xcite . using this model and a massive dataset as a base, in november of the same year, openai released a sibling model, the now famous chatgpt. the release served as the starting pistol for the ongoing race of chat-based large language models. during the last two years, we have seen a consistent improvement in capabilities on different tasks @xcite from bigger and newer models like llama @xcite , palm-2 @xcite falcon @xcite , mistral @xcite or @xcite . however, recent studies @xcite @xcite suggest that these models struggle in logic reasoning tasks when the data is out of distribution from their train corpus and fail to match the performance of previously introduced specialized solutions. the difference in performance is particularly noticeable in the tasks requiring to assess the meaning in which words are used in a sentence, where we observe that recent chat-based models lag behind much smaller fine-tuned architectures @xcite @xcite . the word sense disambiguation (wsd) task consists in identifying the sense in which a word is used in some given context from a pool of possible senses @xcite , e.g., in the sentence "the crane was lifting a concrete block.", a crane refers to a lifting machine used in construction rather than a large, long-necked bird. far from the massive chat-based language models that we can find today, the state-of-the-art models for this task are considerably smaller and were introduced during the last five years (e.g. @xcite @xcite ) . in general, these models address the wsd problem by assessing the semantic similarity between the target word or its neighbouring context and the candidate definitions, using a fine-tuned encoder-based model @xcite @xcite or external resources @xcite . although these approaches generally provide remarkable results, they show a significant decrease in performance for verbs compared to other parts of speech, rare glosses, and underrepresented languages @xcite @xcite . these problematic cases suggest that current solutions struggle to accurately model word senses in low-resource settings, as verbs often have multiple possible senses that are unevenly distributed across different contexts, rare glosses are typically endemic to specialized domains, and training data for underrepresented languages is generally scarce. addressing these deficiencies is crucial for improving generalization across out-of-distribution domains, where the traditional approach of training on large batches of annotated general-domain examples often fails to produce satisfying results @xcite and bridging the wsd performance gap between english and low-resource languages. ideally, a general solution to the wsd problem should reduce the dependency of performance on the frequency of senses and contexts present in the training data @xcite , while addressing the issue at a structural level shared by all languages. under this premise, we hypothesize that reframing the wsd task as a cluster discrimination task over a semantic network (e.g., babelnet @xcite ) could address the aforementioned challenges. in this work, we introduce sandwich 1 , a word disambiguation framework that leverages the close relationship between a candidate sense and its neighbors in a semantic network to shift the task from discriminating individual senses to discriminating semantically-close clusters. to this end, sandwich incorporates additional elements into the two-level framework introduced in @xcite , which consists of coarse sense retrieval followed by a fine-tuned encoder-based model. specifically, we introduce the processing of the semantic network to ensure it is sense-separated through extensive experimentation on the english all-words wsd task @xcite , we establish a new state of the art, achieving a 8% improvement in f1 score across all datasets, consistently outperforming existing solutions in every subset, including those defined by individual datasets and parts of speech. we further evaluate our framework on the more challenging dataset introduced by @xcite , achieving an improvement over the previous state of the art ranging between 10-30% depending on the dataset. addi1 we release all the code for reproducing the paper results in https: www.github.com/danielguzmanolivares/sandwich tionally, on the multilingual dataset @xcite , we improve state-of-the-art results for all languages, with particularly notable gains in underrepresented ones. therefore, the key contributions of this work are as follows: our system achieves a 8% improvement in f1 score on the english all-words wsd task, consistently surpassing the state of the art across all datasets and parts of speech, including the challenging dataset introduced by @xcite . ## related work historically, the wsd problem was introduced in the second half of the twentieth century as part of machine translation efforts @xcite @xcite , later evolving into a standalone problem. early successful approaches primarily relied on rule-based algorithms, statistical methods, and unsupervised techniques @xcite @xcite @xcite . the development of large-scale structured language resources like wikipedia and babelnet @xcite enabled models to use gloss similarity heuristics and graph proximity metrics to address the wsd problem @xcite @xcite . the introduction of the first word embedding algorithms (e.g., word2vec @xcite , fasttext @xcite , or glove @xcite ) significantly advanced wsd performance by leveraging seq-to-seq supervised approaches @xcite @xcite . the use of word embeddings as a foundation for neural approaches led to substantial performance gains, which became even more pronounced with the adoption of dynamic embeddings from encoder models (e.g., bert @xcite , roberta @xcite , or deberta @xcite ), derived from the transformer architecture @xcite . fine-tuning an encoder model has since then become the cornerstone of top-performing systems, which can be broadly categorized into two variants based on their approach to the wsd problem. the first variant comprises purely transformerbased architectures that leverage the representational power of large encoder models. these models often frame the problem by jointly encoding all candidate definitions alongside the given sentence to extract the correct sense of a target word @xcite @xcite . notable examples include consec @xcite , the previous state-of-the-art, which encodes not only the candidate senses of the target word but also non-ambiguous or already disambiguated words in the sentence; bem @xcite , which separates the encoding of glosses and context to compute similarity through a dot product; and esc @xcite , which redefines the wsd problem as a span extraction task, analyzing the concatenation of all possible senses to determine the start and end indices of the correct sense. the second variant includes transformer models that integrate external information, usually from a lexical knowledge base or semantic network @xcite @xcite . notable examples include ewise @xcite and its improved version ewiser @xcite ), which incorporate wordnet information into the neural model; dhfm @xcite , which enriches pretrained embeddings with graph encodings of senses; and @xcite , which use synonyms and hypernyms from word-net to train an encoder via a triplet loss over semantically related glosses. recent approaches have started exploring parallel alternatives, such as @xcite , which propose using neurosymbolic embeddings that reach 90% f1 for target senses with an explicit class structure (about 70% of the raganato et al. (2017a) dataset), and @xcite , which use a representation based on superposition states to eliminate dependency on training set size and improve accuracy for rare senses. although both variants produce competitive results, to the best of our knowledge, no system has surpassed the 82% f1 score on the unified benchmark @xcite . additionally, @xcite highlighted that generalization to outof-domain tasks remains a challenge, complicating the ability of current solutions to scale to specialized domains. moreover, most modern systems rely heavily on encoder models predominantly trained in english, limiting their applicability to underrepresented languages @xcite . to address these challenges, we propose reframing the wsd problem as a semantic cluster discrimination task within a semantic network (babelnet) and, in the next section, introduce the sandwich framework as a comprehensive solution to the multilingual word sense disambiguation problem. ## experimentation in this section, we present and discuss the results of our experiments to evaluate the sandwich framework against existing alternatives. in section 4.1, we first assess our model's performance on the english all-words benchmark @xcite , breaking down results by individual datasets and parts of speech. following this, in section 4.2, we examine how well sandwich generalizes to previously unseen domains and rare senses, using the more challenging dataset from @xcite , and compare it to the current state of the art. we then perform an ablation study in section 4.3 to evaluate the individual contribution of each system component. in section 4.4, we investigate the framework's adaptability to other languages. finally, in section 4.5, we explore alternative backbone models for the cross-encoders and analyze the trade-off between model size and performance. ## results analysis the performance of the proposed sandwich framework across existing datasets demonstrates that reframing the wsd problem as a discrimination task over semantically related clusters effectively addresses the limitations of current solutions, confirming our initial hypothesis. specifically, in the all-words english benchmark, we surpass the previous state of the art across each dataset and in the combined total (all). this improvement extends to rare senses and out-of-domain data, as shown by results on the @xcite dataset, where sandwich significantly outperforms prior solutions. this success includes cases where words have a large number of possible senses (see appendix b), indicating that our approach mitigates challenges in such scenarios. additionally, we analyze the individual contribu-tions of each component in the architecture, concluding that the key to sandwich's success is the creation of separable clusters over the semantic network. furthermore, separating cross-encoders by pos leads to considerable performance gains. we also extend the framework to multiple languages, outperforming all existing alternatives and making significant strides in low-resource language disambiguation. notably, we demonstrate that sand-wich achieves these results with only 28% of the parameters used by the previous state-of-theart, proving the robustness of the sense-cluster approach. ## conclusion in this paper, we introduced the sandwich framework, a novel approach to the wsd problem, which arose from the hypothesis that reframing the disambiguation task as sense cluster discrimination over a semantic network could address the challenges faced by previous state-of-the-art solutions when generalizing to low-resource languages and domains. through extensive experimentation, we confirmed our hypothesis, surpassing the state of the art across all benchmarks, including rare senses and multiple languages. furthermore, we evaluated various alternatives for the backbone model and demonstrated the efficiency of our architecture, achieving a 72% reduction in model size while still surpassing the state of the art. in future work, it would be valuable to explore alternative methods for creating sense clusters, extend our approach to additional languages, and investigate whether sandwich's disambiguation capabilities can serve as a baseline or be combined with existing solutions for translation into low-resource languages and specialized text analysis.