Team T he S yllogist at S em E val-2023 Task 3: Language-Agnostic Framing Detection in Multi-Lingual Online News: A Zero-Shot Transfer Approach (2023)

Abstract: We describe our system for SemEval-2022 Task 3 subtask 2 which on detecting the frames used in a news article in a multi-lingual setup. We propose a multi-lingual approach based on machine translation of the input, followed by an English prediction model. Our system demonstrated good zero-shot transfer capability, achieving micro-F1 scores of 53% for Greek (4th on the leaderboard) and 56.1% for Georgian (3rd on the leaderboard), without any prior training on translated data for these languages. Moreover, our system achieved comparable performance on seven other languages, including German, English, French, Russian, Italian, Polish, and Spanish. Our results demonstrate the feasibility of creating a language-agnostic model for automatic framing detection in online news.

## introduction framing is an advanced form of discourse through which the communicator attempts to generate a cognitive bias by consistently associating a topic with a specific context, commonly referred to as a frame. it serves as a commonly used, albeit powerful, political tool and is widely credited as a significant factor for influencing the attitude of the audience towards important policy matters @xcite . in the era of digitalization, as media articles wield more power than ever to sway and to mould public opinion, we believe it is worth an attempt to foray into how multilingual news articles frame prominent topics for consumption by their audiences. in essence, in the media context, frame is the perspective under which a piece of news is presented. how a piece of information is framed is critical for determining how the information will be interpreted by the audience. analyzing the frames used by the media and their impact on public perception is challenging due to language dynamics and the nuances of wording. the goal of semeval-2023 task 3 subtask 2 @xcite is to predict the frames used in a given news article, drawing from a set of 14 possible frames from @xcite . this is a multi-label classification task at the article level. our hypothesis is that despite linguistic differences, different languages use similar framing in news articles, and that framingremains consistent despite information loss during translation. our results provide evidence in support of this hypothesis, in the form of zero-shot transfer for greek and georgian. this suggests that the frames used in online news articles remain intact, despite information loss during translation, and that these techniques may be similar across languages. our system first translates the input article to english, and then applies an english model on the translated input. on the official test set, it achieved micro-d1 scores of 53% for greek and 56.1% for georgian, demonstrating zero-shot transfer and effectiveness for detecting frames in multilingual online news. we achieved similar performance for the five other languages. ## related work in recent years, the wide use of the internet has led to an increase in biased content. previous work such as @xcite focused on identifying framing at varying levels of granularity, with a focus on sentence-level analysis. furthermore, their dataset and frames were tailored to their specific dataset and they also worked on detecting the polarity of a sentence. in contrast, our work examined articles at a more comprehensive level, and our categories were more generalized, encompassing data from various topics. their work focused on feature engineering, which meant that one of their limitations was the lack of feature extraction tools in languages other than english. here, we overcome these issues. @xcite also did framing detection at the sentence level. other work @xcite @xcite focused specifically on news headlines rather than news articles. this unfortunately has a very limited scope because only specific frames can be extracted just from the headline. @xcite did word-to-word translation of specific words of the headline and then used multilingual bert for classification on code switched headlines. ## system description we developed our multilingual system for detecting frames in news articles using a machine translation module, which translates every non-english article to english. the output of this module is fed into a fine-tuned bert-based model for multi-label classification. ## conclusion and future work we described our system and official submissions to semeval-2022 task 3 subtask 2, which aimed to detect framings in multi-lingual online news. our primary aim was not to beat languagespecific models, but to provide wider coverage across all nine languages that were featured in this challenge. we adopted a multi-lingual approach, which combined machine translation and fine-tuning bert for multi-label classification. our results demonstrated the feasibility of creating a language-agnostic model for framing detection in online news, with zero-shot transfer capability. the micro-f1 scores of 53% for greek and 56.1% for georgian, without any prior training on translated data, are a testament to the effectiveness of our system. moreover, we achieved comparable performance on seven other languages including german, english, french, russian, italian, polish, and spanish. this research contributes to the field of natural language processing and provides insights into the potential of creating language-agnostic models for similar tasks. in future work, we hypothesize that our model's overall scores can be significantly improved if we incorporate the additional data from the dev set into our training process. additionally, we are intrigued by the potential benefits that multilingual models can provide. therefore, we plan to experiment with an ensemble model that combines our languageagnostic module with a multilingual bert-type model to achieve more robust and accurate results. we further plan to apply this model to related multilabel tasks such as propaganda techniques detection @xcite .