U o R - NCL at S em E val-2023 Task 1: Learning Word-Sense and Image Embeddings for Word Sense Disambiguation (2023)

Abstract: In SemEval-2023 Task 1, a task of applying Word Sense Disambiguation in an image retrieval system was introduced. To resolve this task, this work proposes three approaches: (1) an unsupervised approach considering similarities between word senses and image captions, (2) a supervised approach using a Siamese neural network, and (3) a self-supervised approach using a Bayesian personalized ranking framework. According to the results, both supervised and self-supervised approaches outperformed the unsupervised approach. They can effectively identify correct images of ambiguous words in the dataset provided in this task.

## introduction in several natural languages, there are ambiguous words that denote different meanings depending on their contexts. for example, the word "bank" in english may have different senses including "financial institution" and "riverside". to identify the exact sense of an ambiguous word for a particular context, a natural language processing (nlp) task called word sense disambiguation (wsd) was introduced @xcite . despite the fact that wsd has been a long-standing task, it is still challenging to apply wsd methods to downstream nlp applications (lopez de lacalle and agirre, 2015; @xcite . therefore, the organizers of semeval-2023 task 1 proposed a task of using wsd in an information retrieval system @xcite . specifically, given a word and some limited textual context, the task is to select among a set of candidate images the one corresponding to the given word's intended meaning. according to the goal, two challenges are raised. the first challenge is how to disambiguate a given word based on a given context. normally, a human brain differentiates multiple senses of an ambiguous word based on prior knowledge and experience. similarly, in the case of machines, external knowledge is required to map an ambiguous word to its actual meaning. many knowledge-based wsd methods have been relying on external lexical resources such as wordnet @xcite . these resources contain word senses which can be extracted for disambiguation @xcite . the second challenge is how to match a word with an image. in many image retrieval systems, word representations/embeddings and image representations/embeddings are learned in a way that allows word-image association @xcite . this work adopts this approach to learn word-sense and image embeddings in order to associate word senses with their corresponding images. we propose three approaches utilizing wordnet to disambiguate words and learn word-sense embeddings and image embeddings for image retrieval. these approaches are (1) an unsupervised approach using word senses and image captions, (2) a supervised approach using a siamese neural network to learn word-sense and image embeddings, and (3) a selfsupervised contrastive learning approach using a bayesian personalized ranking framework. ## related work there are a number of methods that have been researched in wsd. the first method is knowledge based method that relies on lexical resources and dictionaries @xcite . the second approach is supervised machine learning methods that use manually sense-annotated corpora, e.g, train-o-matic @xcite , which is a language-independent that generates senseannotated training instances for senses of words. the third approach involves semi-supervised methods that use both labeled and unlabeled data and therefore the amount of annotated data need not to be large. @xcite proposed a method that uses word embeddings such that words are in a continuous space. these word em-beddings come from unlabeled data and therefore this method becomes semi-supervised. the fourth approach is unsupervised machine learning methods that use clusters to identify the different occurrences of word senses. for instance, @xcite presented an unsupervised learning algorithm trained on unannotated text. supervised methods have outperformed all the other methods. however, due to the lack of a flexible knowledge base, they might be replaced by neural networks that use sequence learning for word disambiguation. in this work, we use wordnet, a lexical database, to disambiguate the context phrases to get the different senses of words that describe the gold image. ## conclusions to solve the task of using wsd in an image retrieval system, this work proposes three approaches: