hub at S em E val-2021 Task 2: Word Meaning Similarity Prediction Model Based on R o BERT a and Word Frequency (2021)

Abstract: This paper introduces the system description of the hub team, which explains the related work and experimental results of our team’s participation in SemEval 2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC). The data of this shared task is mainly some cross-language or multi-language sentence pair corpus. The languages covered in the corpus include English, Chinese, French, Russian, and Arabic. The task goal is to judge whether the same words in these sentence pairs have the same meaning in the sentence. This can be seen as a task of binary classification of sentence pairs. What we need to do is to use our method to determine as accurately as possible the meaning of the words in a sentence pair are the same or different. The model used by our team is mainly composed of RoBERTa and Tf-Idf algorithms. The result evaluation index of task submission is the F1 score. We only participated in the English language task. The final score of the test set prediction results submitted by our team was 84.60.

## introduction and background with the continuous development of science and technology, we are now in an era of massive data. we cannot use manual methods in the processing and retrieval of text data. especially in the work of comparing and calculating the semantic difference at the word level in the text. in this type of work, automatic processing of text data with machines has become a new choice. the research on the detection method @xcite and evaluation method @xcite of semantic similarity has become a subject of wide discussion. specific application scenarios have been produced in some fields of natural language processing and information retrieval. such as sentiment analysis @xcite , 2019), medical disease similarity query @xcite , text question and answer @xcite etc. similar to humans' strategies for detecting the meaning of words in different sentences, machines and algorithms also need to predict the results based on the context. therefore, the method of generating vectors based on each word is not suitable for such tasks. for example word2vec @xcite . based on the characteristics of text serialization, extracting contextual information in the text as the input of the model will provide the model with richer and more accurate information. for example, in dealing with the problem of polysemous and synonymous words. the elmo @xcite ) method based on lstm @xcite overcomes the difficulty that the model cannot learn the context. elmo can dynamically adjust word embedding according to the context, so it can solve the problem of ambiguity. however, the use of a bidirectional lstm as a feature extractor makes its training time and feature extraction effect unsatisfactory. in the follow-up work, the appearance of transformer @xcite introduces new and better feature extractors for the model. the bert @xcite model based on transformer encoder @xcite achieved the best results in many nlp tasks. we participated in semeval-2021 task 2: multilingual and cross-lingual word-in-context disambiguation (mcl-wic) english task. this task is to predict whether a word with the same part of speech has the same meaning in a sentence pair @xcite . we are inspired by the work of chen, weilong and others on the task of predicting the influence of context on word similarity @xcite , and use methods based on roberta @xcite and tf-idf @xcite to complete the task. at the same time, we also tried to combine albert @xcite with bert @xcite and tf-idf to observe their performance on the english data set. we introduce our methods and experiments in detail in sections 2 and 3. our model code can provide reference 1 . ## data and methods in this section, we will introduce the data we use in the task and the models and methods we use. ## experiment and results in this section, we will introduce the data preprocessing methods and experimental settings we used in the task and the final results. ## results the final result score evaluation index uses the f1 score. therefore, the effects of the different models we used in the experimental phase are all using f1 scores to determine which model is better. we use the same validation set data to evaluate the performance of different models. comparing the result score obtained by the combination of albert, bert and tf-idf with the score obtained by the combination of roberta and tf-idf, it can be seen that the combination strategy of roberta can get a better f1 score. compared with the f1 score obtained by using roberta alone, the f1 score obtained by roberta+tf-idf is better. this also verifies the feasibility and effectiveness of our method. we sort the results according to the prediction result of the english test set we finally submitted is predicted by roberta+tf-idf. compared with the f1 scores obtained by the top three teams in the english data, there is still a certain gap. our f1 score ranks middle among all result scores. our final ranking is 49th. we sort the results according to ## conclusion this paper proposes a model that combines roberta and tf-idf to calculate whether the target words in english sentence pairs are similar. we introduced our analysis of the data, the methods used in the experiment, and the results of the experiment in sections 3 and 4. we compared the effects of different models of albert, bert, roberta and the combination of tf-idf. the experimental results also prove that roberta+tf-idf can get better results in our method. in future work, we will improve our methods to get better results. for example, other types of word embedding vectors can be introduced into our model, and the method of weighting and vector fusion can also be improved.