Now, It’s Personal : The Need for Personalized Word Sense Disambiguation (2021)

Abstract: Authors of text tend to predominantly use a single sense for a lemma that can differ among different authors. This might not be captured with an author-agnostic word sense disambiguation (WSD) model that was trained on multiple authors. Our work finds that WordNet’s first senses, the predominant senses of our dataset’s genre, and the predominant senses of an author can all be different and therefore, author-agnostic models could perform well over the entire dataset, but poorly on individual authors. In this work, we explore methods for personalizing WSD models by tailoring existing state-of-the-art models toward an individual by exploiting the author’s sense distributions. We propose a novel WSD dataset and show that personalizing a WSD system with knowledge of an author’s sense distributions or predominant senses can greatly increase its performance.

## introduction authors of text tend to predominantly use a single sense for a lemma that can differ among different authors @xcite . this might not be captured with an author-agnostic word sense disambiguation (wsd) model that is trained on multiple authors. our work finds that wordnet's first senses, the predominant senses of our dataset's genre, and the predominant senses of an author can all be different and therefore, author-agnostic models could perform well over an entire dataset, but poorly on individual authors. ideally, each author would have access to a personalized wsd model, which is a model that was tailored toward that individual. in this work, we explore methods for personalizing wsd models by tailoring existing state-of-the-art models toward an individual by exploiting the author's sense distributions. we evaluate our models on our proposed dataset which contains 1586 sense-annotated instances for 11 lemmas across 36 authors. our evaluation includes metrics that focus on the performance of models with respect to the authors that they perform poorly on, which highlights the potential gain for individual authors. the most similar work to this is presented in @xcite , which created a dataset that contains sense-annotated instances of tweets for a list of authors. we differ from them by including more annotated instances from a single author and by using text from blog posts, which do not have a limitation on the length of text, unlike text from tweets. we evaluate different wsd models, including a state-of-the-art model (sensembert) @xcite , which we extend with personalization techniques to achieve our best scores. we personalize sensembert with knowledge of an author's sense distributions or predominant senses, which outperform author-agnostic wsd systems. our work also shows that the use of author-specific sense distributions outperforms the use of genrespecific senses. in this first work on personalized wsd, we do not automatically learn the sense distributions of an author, but instead, we use the author's true sense distributions to demonstrate the importance of learning author-level sense distributions when considering personalized wsd. ## related work two common frameworks for wsd systems include knowledge-based, which utilizes information contained in a sense inventory, and supervised, which involves training on annotated instances. in this work, we focus on knowledge-based models because they do not require annotated instances and they are able to work well on less frequent lemmas and senses @xcite . many modern knowledge-based methods extend the simplified lesk method @xcite , which involves classifying a token with the sense that contains the most overlapping words in its defi-nition with the context of the word being classified. wsd methods that involved a lesk-style approach are @xcite , which included looking at the definition of words that are similar to the token being classified according to word-net and @xcite , which looked at the definition of similar words but added a vector representation of the target token -generated by a topic model -into their similarity calculations. topic modeling is a probabilistic model that views documents as a distribution over topics and topics as a distribution over types @xcite . topic modeling was also applied to wsd in @xcite and @xcite . the current knowledge-based model, that achieves state-of-the-art performance on the unified wsd evaluation framework @xcite , creates an embedding for each sense of a word by using bert @xcite to embed both the sentences in wikipedia articles related to that sense -determined by words in the sense's wordnet synset -and the gloss of the sense @xcite . the two vectors are then concatenated to make the final sense embedding. at runtime, the context of the target word is embedded using bert and is compared to the previously described sense embeddings to determine which sense is assigned. sensembert works well on rare lemmas and senses @xcite , which is important to consider for our experiments that contain author-specific text since authors tend to favour a single sense for a word @xcite , which might not be the predominant sense of a domain. therefore, if a model performs poorly on rare senses according to the sense inventory, then it might perform poorly on the favoured sense of the author. likewise, if an author frequently uses rare words -according to the sense inventorythen a model that performs poorly on rare words would perform poorly for the author. the ability to perform well on all authors and demographics is a way to evaluate the fairness of a model @xcite . in this work, we consider the fairness of our models by focusing on authors that they perform poorly on. ## data statement in this section, we discuss the properties of our dataset, while following the proposed schema from @xcite . ## experimental results in this section, we evaluate our different models. we first explore the tuning of the override rank for predom-based methods. we then evaluate our models using average accuracy across all authors and for individual authors. ## conclusions in this work, we proposed a novel dataset for personalized wsd and showed that sense distributions and predominant senses of an author can be used to personalize an existing knowledge-based wsd model (sensembert). our experiments consistently show that models that consider authorspecific sense distributions (sebert pers) or predominant senses (sebert pers predom) can outperform models that do not consider any knowledge of sense distributions (sensembert). furthermore, we show that models that use authorlevel sense distributions or predominant senses outperform models that use genre-level sense distributions (sebert dataset) or predominant senses (sebert dataset predom). sensem-bert achieved the highest accuracy across all authors with an absolute improvement of 0.127 above sensembert. we further explore the fairness of our models by evaluating their accuracy on authors that they perform poorly on and showed that the lowest accuracy achieved by sebert pers on a single author is 0.21 above sensembert's lowest accuracy. this finding demonstrates that se-bert pers is more fair than sensembert, which indicates that personalization can produce more fair wsd systems. our work shows the importance of learning sense distributions of individual authors for wsd and therefore, we plan on developing methods for learning an author's sense distributions in future work similar to @xcite and @xcite . our personalized models could be learning topic-related content from the author to assist with their classification, therefore, an extension of this work could further explore this dataset with a focus on topic-related features.