CSECU - DSG at S em E val-2022 Task 3: Investigating the Taxonomic Relationship Between Two Arguments using Fusion of Multilingual Transformer Models (2022)

Abstract: Recognizing lexical relationships between words is one of the formidable tasks in computational linguistics. It plays a vital role in the improvement of various NLP tasks. However, the diversity of word semantics, sentence structure as well as word order information make it challenging to distill the relationship effectively. To address these challenges, SemEval-2022 Task 3 introduced a shared task PreTENS focusing on semantic competence to determine the taxonomic relations between two nominal arguments. This paper presents our participation in this task where we proposed an approach through exploiting an ensemble of multilingual transformer methods. We employed two fine-tuned multilingual transformer models including XLM-RoBERTa and mBERT to train our model. To enhance the performance of individual models, we fuse the predicted probability score of these two models using weighted arithmetic mean to generate a unified probability score. The experimental results showed that our proposed method achieved competitive performance among the participants’ methods.

## introduction lexical relations are regarded as the most important semantic relations to infer the meanings of words effectively @xcite . therefore, identifying such relations is beneficial to understand the semantic competence and distill the underlying context of the textual expression. if we consider the nlp applications, it has a significant impact on several tasks including semantic search, automatic question/answering, story generation, relation extraction, and machine translation @xcite . however, most of the prior works focused mainly on syntactic @xcite and contex-**the first two authors have equal contributions. ## proposed framework in this section, we describe our proposed approach for the pretens shared task. our goal is to determine the semantic competence between two arguments focusing on the taxonomic relations. the task is articulated into both the binary classification subtask and the regression subtask. we depict the overview of our proposed framework in figure 1 . given an input sentence containing two nominal arguments, we employ two transformer models including xlm-roberta and multilingual bert (mbert) to extract the diverse contextual features. later, a feed-forward linear layer is employed in each model to estimate the probability score of each class. in figure 2 , we illustrate an overview of the setup of mbert transformer model. finally, we fuse these models' predictions by taking the weighted arithmetic mean of these scores to determine the final label.