FOOL ME IF YOU CAN ! An Adversarial Dataset to Investigate the Robustness of LM s in Word Sense Disambiguation (2024)

Abstract: Word sense disambiguation (WSD) is a key task in natural language processing and lexical semantics. Pre-trained language models with contextualized word embeddings have significantly improved performance in regular WSD tasks. However, these models still struggle with recognizing semantic boundaries and often misclassify homonyms in adversarial context. Therefore, we propose FOOL: FOur-fold Obscure Lexical, a new coarse-grained WSD dataset, which includes four different test sets designed to assess the robustness of language models in WSD tasks. Two sets feature typical WSD scenarios, while the other two include sentences with opposing contexts to challenge the models further.We tested two types of models on the proposed dataset: models with encoders, such as the BERT and T5 series of varying sizes by probing their embeddings, and state-of-the-art large decoder models like GPT-4o and the LlaMA3 family, using zero shot prompting. Across different state-of-the-art language models, we observed a decrease in performance in the latter two sets compared to the first two, with some models being affected more than others. We show interesting findings where small models like T5-large and BERT-large performed better than GPT-4o on Set 3 of the dataset. This indicates that, despite excelling in regular WSD tasks, these models still struggle to correctly disambiguate homonyms in artificial (Set 3) or realistic adversarial contexts (Set 4).

## introduction the task of word sense disambiguation (wsd) is a fundamental challenge in natural language processing (nlp). homonyms, which are formally identical words with completely independent meanings @xcite , present a challenge in * these authors contributed equally to this work tasks like machine translation, text annotation, and question answering @xcite . in order to comprehend the intended meaning of homonyms, it is necessary to consider the context, in which they are used. consequently, the accurate disambiguation of homonyms provides evidence of the model's comprehension of the context and, in turn, of language. contextualized language models, such as bert @xcite , produce word embeddings that reflect the word's meaning based on its context @xcite . this has led to significant improvements in wsd performance in both fine-grained or coarse-grained wsd @xcite @xcite . while fine-grained wsd addresses the nuanced senses a word can have, coarse-grained wsd focuses on broader, unrelated word meanings @xcite . the emergence of context-based language models suggests that the challenge of regular wsd has largely been resolved. however, it is still unclear if these models can understand context well enough to disambiguate homonyms effectively. let us consider the following sentence: ## related work word sense disambiguation (wsd) is a wellstudied task in natural language processing, focusing on fine-grained polysemy disambiguation. the majority of standard wsd benchmarks, such as the unified evaluation framework by @xcite , heavily rely on wordnet @xcite . this dependence on wordnet, known for its finegrained classification, poses a challenge even for humans to distinguish all possible senses. to tackle this issue, @xcite introduced the dataset "coarsewsd-20", which extracts sentences from wikipedia articles to create a coarse-grained sense inventory wsd dataset. the performance of pre-trained language models has been tested on both fine-grained and coarsegrained datasets. especially, bert @xcite achieved overall good results with over 94% accuracy in coarse-grained wsd @xcite . for example, @xcite fine-tuned bert on a wsd task and tested it on a variety of different fine-grained wsd benchmarks @xcite @xcite @xcite , achieving promising results with accuracies ranging from 74% to 78%. additionally, without fine-tuning @xcite and @xcite showed that bert can effectively perform fine-grained wsd by combining its contextualized word embeddings with a knn classification algorithm. moreover, @xcite employed a knn bert classifier and reported human-like performance on their coarse-grained noun wsd dataset, with over 94% accuracy. more recently, @xcite tested different bert-based models, including bert @xcite and roberta @xcite on coarse-grained wsd. they clustered wordnet senses to match coarse-grained homonym sense distinction and found that bert's accuracy is as high as 95%. in our work, we conduct an extensive analysis on two types of models. on the one hand, we tested models that include encoder, such as the bert and t5 family @xcite , by probing their word embeddings. on the other hand, we analyzed state-of-the-art language models including gpt-3.5 turbo (openai, 2022), gpt-4 turbo (openai, 2023) , and gpt-4o (openai, 2024), llama3-8b, llama3-70b (meta, 2024) and mixtral-8x7b @xcite using prompts in zero shot settings. while @xcite investigated gpt-3.5's performance on wsd among other tasks, our work significantly differs in that we have created a new coarse-grained adversarial dataset and tested various models from different families. to the best of our knowledge, this paper is the first to conduct an extensive analysis comparing models like bert and t5 with models like gpt and llama on adversarial wsd tasks. furthermore, it is evident that there is no existing dataset that aligns with the one proposed in this paper. despite this, there have been some attempts to test models on adversarial sentences. for example, @xcite considered adversarial attacks in wsd. they changed adjectives in sentences in front of homonyms and checked the performance in a machine translation task. these changes lead to translation errors in lstm @xcite , transformer @xcite and convs2s @xcite . inspired by @xcite approach, we adopted the idea of modifying adjectives in order to test the resilience of more recent pre-trained language models based on their contextualized word embeddings. moreover, @xcite incorporated opposing context words in their study. in their paper, the authors analyzed the performance of pre-trained language models, primarily bert, on semcor @xcite , a fine-grained sense dataset. while they succeeded in this task, they also combined two sentences with distinct meanings of a homonym to create sentences with opposing contexts. thereby they found a higher number of classification errors than in normal conditions. this test was done using fine-grained senses of words. although this represents a promising initial step, there is a need to further extend this idea. we analyze coarse-grained wsd performances of different state-of-the-art models beyond bert and have developed an entire human-made test set to evaluate our approach. ## dataset in this section, we introduce our dataset fool, a coarse-grained wsd dataset that is designed to differentiate between four different categories of context changes. this design allows us to test both regular homonym disambiguation settings and adversarial context settings. therefore, this dataset serves as a tool to evaluate the robustness of large language models against different context changes and their ability to discern between various coarsegrained homonym senses. ## discussion in the following we discuss the main findings and open questions that remain after our analysis. set 1/2 vs. set 3/4 one of the main findings from the analysis above is that there is a major performance gap between set 1 and set 2 compared to set 3 or set 4. the significant decline in perfor-mance observed between set 1 and set 3 in the wsd test, despite the only change being the replacement of one adjective, appears to be out of proportion. also the performance decrease in set 4 is disproportionate. adding opposing yet realistic context while still remaining the overall meaning of the homonym can lead to a decrease in the f1score to up to 30% even for advanced models like llama3-8b and gpt-3.5 turbo. one explanation for the changing results could be that contextualised language models do not pay attention to semantic boundaries like @xcite mentioned in their paper about bert. this could be extended by the findings of @xcite who state that language models do not learn which context words are useful and pay attention mostly to the homonym itself. unimportant context words, which humans can successfully sort out, have a major impact on the word embedding produced by language models. this could be one factor language models have to improve in order to achieve human-like results also in smaller model sizes. model size another finding is the correlation between model sizes and wsd performance in all four sets. the results indicate a positive correlation between model size and f1-score. larger models with more parameters store more training data information and have bigger embedding vectors that capture extensive contextual details, improving disambiguation. furthermore, a decline in performance is observed in sets 3 and 4, with smaller models experiencing a larger drop than larger models. this supports the hypothesis that larger models are more robust to adversarial attacks. this robustness is likely due to larger models' ability to recall more information and recognize different contexts. t5 vs. bert the best overall performance is seen in the encoder of flan-t5-xxl. in general, the t5-based models show the best overall results not least because of the bigger model sizes. even in the base size flan-t5 surpasses bert-large which has more model parameters than flan-t5. this may suggest that t5-based language models are an optimal choice for the task of word sense disambiguation. one potential explanation for the enhanced performance is that t5 employs a distinct masking approach distinct from bert. while bert can only mask one word at a time, t5 masks multiple words at the same time. additionally, t5 was trained on a larger data corpus than bert which could also improve the performance in wsd since more knowledge about words in different usages is collected. ## conclusion in this paper, we introduce fool, a new coarsegrained wsd dataset featuring various types of contexts, which serves as both a benchmark for assessing model performance on wsd tasks and a tool for evaluating context comprehension by models. our experiments using this dataset demonstrate that sota language models still struggle to understand context and disambiguate homonyms in the presence of opposing contexts, compared to their performance in regular wsd tasks. this effect is most prominent not only in smaller models like bert-base and t5-base but also in larger models like llama3 and gpt-3.5 turbo. among the series of models that include an encoder, our results show that t5, especially flan-t5 is a better alternative to bert-based models. with more than 99% score in set 1, flan-t5-xxl shows human-like disambiguation skills. furthermore, we showed that models incorporating an encoder are less vulnerable to adversarial addition of context (set 3) with the best performing model being flan-t5xxl, which outperforms gpt-4o and gpt-4 turbo. interestingly, small models like bert-large and flan-t5-base outperform gpt-4o on the same set. however, these small models struggle with set 4, which includes realistic opposing context usage of words, which we believe requires a deeper understanding of language and some degree of reasoning abilities. in the future, we plan to extend the fool dataset to include sentences with finegrained homonyms to investigate how language models perform on them. additional adversarial settings could also be added to further challenge the models, potentially exposing new weaknesses in their contextual understanding and disambiguation capabilities. this will provide further insights into the limitations of current language models and guide the development of more robust systems.