DSH acker at S em E val-2023 Task 3: Genres and Persuasion Techniques Detection with Multilingual Data Augmentation through Machine Translation and Text Generation (2023)

Abstract: In our article, we present the systems developed for SemEval-2023 Task 3, which aimed to evaluate the ability of Natural Language Processing (NLP) systems to detect genres and persuasion techniques in multiple languages. We experimented with several data augmentation techniques, including machine translation (MT) and text generation. For genre detection, synthetic texts for each class were created using the OpenAI GPT-3 Davinci language model. In contrast, to detect persuasion techniques, we relied on augmenting the dataset through text translation using the DeepL translator. Fine-tuning the models using augmented data resulted in a top-ten ranking across all languages, indicating the effectiveness of the approach. The models for genre detection demonstrated excellent performance, securing the first, second, and third positions in Spanish, German, and Italian, respectively. Moreover, one of the models for persuasion techniques’ detection secured the third position in Polish. Our contribution constitutes the system architecture that utilizes DeepL and GPT-3 for data augmentation for the purpose of detecting both genre and persuasion techniques.

## introduction the ability to detect genres foot_0 and persuasion techniques in the text is crucial in nlp, as it allows to better evaluate the quality of information and categorize it. the semeval-2023 foot_1 task 3 is a shared task that aims to evaluate the performance of systems in detecting genres, framing, and persuasion techniques in multiple languages. this paper presents our systems for detecting genre and per-suasion techniques foot_2 , which was prepared for that workshop. our methods involve machine translation and text generation techniques to augment the original training data in all participating languages. specifically, we augmented the genre detection datasets in these languages by creating synthetic texts for each class with the openai gpt-3 davinci language model. afterward, we fine-tuned a single multilingual xlm-roberta-large model for genre detection for all languages. our approach to building systems to detect different persuasion techniques involved a combination of feature engineering and deep learning models. we primarily utilized the deepl 4 translator for text translation to augment the dataset. our final systems performed well in the semeval-2023 task 3, ranking among the top 10 participants across all languages. we structured our paper as follows: in section 2, we give a summary of prior work in the field of detecting genres and persuasion techniques. sections 3 and 4 describe the methods we used for data augmentation and our systems architecture. sections 5 and 6 present our experimental setup and the results we obtained. section 7 presents a multilingual persuasion detection system that outperforms all other systems in that task, submitted after the original deadline. finally, sections 8 and 9 present our conclusions, limitations of our approach, and potential avenues for future research. ## dataset this section provides a brief overview of the data available for task 3 and the methods of data augmentation we used for the st1 and st3 in which we participated. as outlined by @xcite , our task involved training and development datasets, which had known labels and a test dataset without labels. all of the datasets contained news articles. the training and development datasets comprised english, french, german, italian, polish, and russian articles. the test dataset included articles in these languages and three additional languages, spanish, greek, and georgian. the test dataset was used to make final predictions. for a more thorough description of the available data, refer to @xcite . ## results each result presented for genre categorization, and persuasion techniques detection is the average of 5 different runs on random seeds. ## post deadline multilingual persuasion techniques detection due to constraints on available computing power, we were compelled to decide for which task to use larger and multilingual pre-trained models. we assumed that the classification of persuasion techniques would be more language-specific. as a result, in the st3, we used different models for each language. however, we proceeded with experiments following the final call for submissions and fine-tuned xlm-roberta-large. this approach produced superior results to our final solution to this competition for each language. ## conclusion in this paper, we present our solutions for two se-meval subtasks: news genre categorization and persuasion techniques' detection. we developed a single multilingual system utilizing the pre-trained roberta model to classify online news genres. to improve the performance of the system, we added to the training dataset texts generated by the gpt3-davinci language model. regarding the persuasion techniques' detection task, we provided individual solutions for each language with available training data. for the polish language we created a system that employs ensemble learning of three classical machine learning models: lightgbm, xgboost, and logistic regression. for that system, we generated two types of features from data, specifically, herbertbased embeddings and fully interpretable linguistic features. all other systems for persuasion techniques detection employ bert-based pre-trained models and use summarization applied to longer paragraphs in online news articles. most final systems were trained on augmented data obtained through various techniques, including gpt-3-based text generation, machine translation utilizing the deepl api. our systems consistently ranked among the top 10 solutions, exceeding the baseline the organizers set. in future work it would be interesting to further develop the multilingual approach to detect persuasion techniques.