Hitachi at S em E val-2022 Task 2: On the Effectiveness of Span-based Classification Approaches for Multilingual Idiomaticity Detection (2022)

Abstract: In this paper, we describe our system for SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding. The task aims at detecting idiomaticity in an input sequence (Subtask A) and modeling representation of sentences that contain potential idiomatic multiword expressions (MWEs) (Subtask B) in three languages. We focus on the zero-shot setting of Subtask A and propose two span-based idiomaticity classification methods: MWE span-based classification and idiomatic MWE span prediction-based classification. We use several cross-lingual pre-trained language models (InfoXLM, XLM-R, and others) as our backbone network. Our best-performing system, fine-tuned with the span-based idiomaticity classification, ranked fifth in the zero-shot setting of Subtask A and exhibited a macro F1 score of 0.7466.

## introduction semeval-2022 task 2 @xcite involves detecting idiomaticity in a given sentence (subtask a) and learning effective representations of sentences that may contain idiomatic multiword expressions (mwes) (subtask b) in three languages: english, portuguese, and galician. processing idiomaticity in a sequence correctly is an essential task in natural language processing (nlp), as idiomatic expressions are a key component of natural languages. its high performance will contribute to various downstream tasks, such as sentiment analysis, information retrieval, and machine translation @xcite . in this work, we propose two different approaches for multilingual idiomaticity detection (subtask a) that take advantage of mwe spanbased features. we use several cross-lingual pretrained language models (infoxlm @xcite , xlm-r @xcite , and others) and exploit their mwe span representations for classification, instead of using a special classification token ([cls]), which typically corresponds to the first input token. our concept is that these models should be able to focus more on the idiomaticity of an mwe in a given sequence by using its span representation rather than using the @xcite token for classification, potentially resulting in a better detection performance. our main findings in the shared task are in threefold. consequently, our best-performing system, using the span-based classification, ranked fifth among 20 systems in the zero-shot setting of subtask a and showed a macro f 1 score of 0.7466 on the test set. ## task description we briefly describe a multilingual idiomaticity detection task (subtask a). given a sentence composed of n words s @xmath0, . . . , w n ] that contains an m-word mwe @xmath1, . . . , w mwe m ], s's preceding sentence s prev , and succeeding sentence s next , the task is to classify if w is idiomatic (0) or not (1). the task dataset is based on tayyar @xcite and contains galician in addition to english and portuguese. each sample consists of @xmath2; s target ; s next ], w, a language type @xmath3, "pt", "gl"}, and an idiomaticity label y @xmath4, 1}. in the zero-shot setting, participants do not have any training samples for galician and are only allowed to use the officially provided training and development sets for training. they must also use the same approach for all samples except language and can submit up to five systems for evaluation. ## system overview our system relies on cross-lingual pre-trained language models (infoxlm and xlm-r and others) and classifies samples using either span-based classification or span prediction-based classification. we fine-tune several pre-trained language models and obtain final predictions by using an ensemble method. figure 1 visualizes our approach for multilingual idiomaticity detection. foot_0 ## experimental setup models we mainly utilized infoxlm and xlm-r for our system submission, but we also tested several other cross-lingual pre-trained language ## results ensemble we utilized ensemble methods in four out of five submissions, of which two use stacked 6 for the detailed configurations of each model, please see appendix d. ## conclusion in this paper, we have proposed two approaches for detecting idiomaticity in a given sequence: spanbased classification and span prediction-based classification. while the performance of the latter was almost on par with that of the well-known standard sequence classification approach using a @xcite hidden representation, the former outperformed it and showed the best macro f 1 score of 0.7466, which ranked fifth in the zero-shot setting of subtask a. we also found that it is essential to guide a model on which tokens to look at when no training data is available for a particular language. in future work, we will investigate a more effective idiomaticity detection approach against unseen language data.