A lex U - AUX - BERT at S em E val-2020 Task 3: Improving BERT Contextual Similarity Using Multiple Auxiliary Contexts (2020)

Abstract: This paper describes the system we built for SemEval-2020 task 3. That is predicting the scores of similarity for a pair of words within two different contexts. Our system is based on both BERT embeddings and WordNet. We simply use cosine similarity to find the closest synset of the target words. Our results show that using this simple approach greatly improves the system behavior. Our model is ranked 3rd in subtask-2 for SemEval-2020 task 3.

## introduction a word meaning can be affected by its context. for polysemous words, the change in meaning is usually clear and evident. but even for words that are not necessarily considered polysemous, there can be subtle changes in their meaning. in semeval-2020 task 3 @xcite , one goal is to predict how similar two words are in a given context, specifically given two different shared contexts for each pair of target words. identifying the meaning of words in context is known as word sense disambiguation (wsd). it is a core task of natural language processing (nlp) and has many potential applications. in @xcite , the authors presented the motivations for solving the ambiguity of words. example 1 context 1 small arms include handguns, rifles, machine guns, etc. ## system overview figure 2 illustrates how our system works, and the pipeline is described below: t 1 .@xmath0.synsets(t 1)} 3. for each target word, we select the top k synsets. we use the definition included in the synset to create an auxiliary context for the target word, as shown below. target word: solution definition: the successful action of solving a problem auxiliary context: solution is the successful action of solving a problem 4. we then run the created auxiliary context through bert to get an embedding for the target word. for the selected k senses, we get k embeddings. we select the sense that has minimum cosine distance with the embedding from the original context. ## experimental setup we use pre-trained bert base embeddings from pypi foot_1 . to get word senses, we use the wordnet interface from nltk foot_2 . as an evaluation metric, we use the harmonic mean of the pearson and spearman correlations against the gold scores by human annotators. we evaluate our technique on semeval-2020 task3 evaluation data and the stanford contextual word similarity (scws) dataset @xcite . we used the (scws) dataset to evaluate our different models since the practice data for the task was very small in size. we evaluated our method on english data only. semeval-2020 task3 evaluation data covers three other languages; croatian, finnish, and slovenian. unfortunately, the tool we used to access the wordnets does not support these languages. we can use other tools to access the wordnets foot_3 . it would be interesting to apply our method to other languages and see the performance. ## results as shown in ## conclusion we presented a system that requires no training and uses a simple method to generate word embeddings that represent the word sense. we improved bert embeddings by using information about the word senses. we varied the number of word senses from which we choose the closest sense and found the optimum number that reduces the error. we experimented with different aggregation methods. by using our technique, the score increased from 0.573 when we use the bert embedding baseline to 0.723 when evaluated on semeval dataset.