ITNLP 2022 at S em E val-2022 Task 8: Pre-trained Model with Data Augmentation and Voting for Multilingual News Similarity (2022)

Abstract: This article introduces a system to solve the SemEval 2022 Task 8: Multilingual News Article Similarity. The task focuses on the consistency of events reported in two news articles. The system consists of a pre-trained model(e.g., INFOXLM and XLM-RoBERTa) to extract multilingual news features, following fully-connected networks to measure the similarity. In addition, data augmentation and Ten Fold Voting are used to enhance the model. Our final submitted model is an ensemble of three base models, with a Pearson value of 0.784 on the test dataset.

## introduction the task @xcite aims to design a system that can find the similarities of multi-language news articles. the task focuses on the consistency of actual events reported in two news articles, but not the subjective factors such as writing style or political factors. geographical location, time, common entity, and common narrative were used to judge similarity. consistency is measured by a float value between @xcite . the lower the value is, the more likely the two news are reporting the same thing. the metric is the pearson coefficient of the predicted and ground truth values. challenge of this task: 1. need to understand every aspect of a news event: what happened, where and when, who was involved, and why and how it happened 2. make the writing style and common phrases clear because some unnecessary content can be misleading. 3. some information about the event is hard to get, such as the time, location, and description of the event. we use the pre-trained language model xlm-roberta and infoxlm to fine-tune. specifically, we splice the available information of two news articles (exact method will be described in chapter foot_0 *these authors contributed equally. our code is available on github 1 . ## experiment all experiments were run on two gpus: nvidia geforce rtx 3080 ti and nvidia geforce rtx 3090. for optimizer selection, we use adamw optimizer with weight decay taken as 0.01 and set 1e-5 as learning rate for the pre-trained layer and 1e-4 for downstream layers with the batchsize 8.if not specified, we use infoxlm-large as the pre-trained model from hugging face foot_3 .the weight of the loss entity , loss narrative and loss overall in the loss multitask are set to 0.3, 0.7 and 1.0 respectively.the weight of the loss al is set to 0.1.