HHU at S em E val-2023 Task 3: An Adapter-based Approach for News Genre Classification (2023)

Abstract: This paper describes our approach for Subtask 1 of Task 3 at SemEval-2023. In this subtask, task participants were asked to classify multilingual news articles for one of three classes: Reporting, Opinion Piece or Satire. By training an AdapterFusion layer composing the task-adapters from different languages, we successfully combine the language-exclusive knowledge and show that this improves the results in nearly all cases, including in zero-shot scenarios.

## introduction in semeval-2023 task-3 @xcite , the authors intend to find out what makes a news article persuasive. to this end, they designed three subtasks: news genre categorisation, framing detection and persuasion techniques detection. in this paper, we present our results for the first subtask, which aims to classify the genre of a news article into one of three classes; reporting, opinion piece or satire. the news articles are available in different languages, meaning this is a multilingual problem. furthermore there are a few languages without training data in order to test the performance of the models in a zero-shot scenario. the results for the different languages were evaluated separately while taking the f1-macro score as the evaluation metric. in the following, we present our approach for english, italian, german and russian, as well as the zero-shot languages spanish and greek. we use an adapter-based setup @xcite , which enhances bert-type architectures @xcite with modular weights. we train a task-adapter for each of the languages before using adapterfusion @xcite to combine what the language-specific task adapters learned. we show that using adapterfusion largely improves the performances of the single-language task-adapters, which is a behavior that has already been observed for low-resource tasks. with an f 1macro score of 0.5942, we achieved 4th place out of 27 teams for the english dataset. in addition, for the zero-shot language greek, we came in third place out of 15 teams with an f 1 -macro score of 0.7504. for the other languages, we ranked near 10th place, meaning that our approach still shows some instabilities. ## background the trainig data for subtask-1 consists of whole texts in six different languages: english, italian, german, french, polish and russian. as we noted above, we only submitted results for english, italian, german and russian. we will elaborate on the reason for this in subsection 3.4. however, we still used the french and polish datasets to augment our data, as we explain in the next section. the main challenge of this subtask is presented by the limited availability, as well as the imbalance of the training data. here, the opinion label is over-represented (76% of the labels) while the satire label is the clear minority class (5.8% of the labels). on top of that, the texts are often longer than the 512 token-limit bert-type systems can handle. consequently, utilizing all of the available data requires preprocessing the training data to address these issues. since the dataset for each language is fairly small but we have data for multiple languages, it makes sense to try and combine the distributed knowledge that can be extracted from the training data. to fully exploit the information contained in all languages, we use adapterfusion @xcite , which combines task-specific representations using a task-composition layer. we will elaborate more on adapters and adapterfusion in the next section. ## system overview as mentioned, we use an adapter-based model to classify the news data. adapter-based frameworks have shown to be very useful in dealing with multilingual problems, as their modular structure makes it possible to adapt task-specific representations to different languages @xcite . before we get into the details of our experimental setup, we want to elaborate on different measures we took in order to augment our data, as well as introduce the concepts of adapters and adapter fusion. ## results in ## conclusion we successfully investigate multiple approaches to determine the genre of news articles. we find that using a configuration which utilizes adapterfusion to combine the knowledge learned from datasets in various languages delivers the best results for most languages. by comparing the attentions of the adapterfusion between a zero-shot language and a trained language, we see that it effectively composes the learned knowledge when used to predict for an unseen language. further work could try to improve the setup of the adapterfusion, in order to make it possible to train it with multilingual data at once. in addition, including more languages (for example french and polish, after training language-adapters for them), might improve the results, as more knowledge would be available.