Kb at S em E val-2023 Task 3: On Multitask Hierarchical BERT Base Neural Network for Multi-label Persuasion Techniques Detection (2023)

Abstract: This paper presents a solution for Semeval 2023 subtask3 of task 3: persuasion techniques in paragraphs detection. The aim of this task is to identify all persuasion techniques in each paragraph of a given news article. We use hierarchical multitask neural networks combined with transformers. Span detection is an auxiliary task that helps in the main task: identifying propaganda techniques. Our experiments show that if we change the index of BERT embedding from the first token of the whole input to the first token of the identified span, it can improve performance. Span and label detection can be performed using one network, so we save data and, when data is limited, we can use more of it for training.

## introduction news articles are expected to present reliable information, however quite often contain some kind of manipulation. unconscious reader can be unable to spot all kind of persuasion that he is exposed to. reading such news can influence a reader's point of view especially if the reader has a low level of political knowledge @xcite . automatic tools for detection of persuasion can help in media analysis and creating more objective news. the task prepared by the organisers @xcite covers three topics, as follows: category of news article identification, framing classification and paragraph-level persuasion techniques recognition. our team focused on the third topic and created solution tested on all languages that have training data available: english, french, german, italian, polish and russian and for additional languages with only test data: spanish, greek and georgian. our system combines multitask learning and hierarchical neural networks. the system identifies span where the persuasion techniques appear and then passes this information to the next module that classifies it to one or more categories of persuasion techniques. we used pre-trained and fine-tuned bert embedding as the shared input layer. we discover that adding an auxiliary task such as span identification as the first step of training neural network may give better results than simply classifying the whole paragraph. moreover, using one neural network for that approach makes solving the task possible even for limited data. the code of our solution is available at https://github.com/katarzynaa/ persuasion_detection ## background the persuasion technique identification problem has been addressed before as a problem of span and technique classification. there exist several works on solving the persuasion techniques detection problem, for example: as a problem of multimodal image and text input @xcite or as problem of span and technique classification @xcite . in our case it is the multi-label classification task, the authors of the task are expected to classify all persuasion techniques that appear in every paragraph of a news article. the input data is a single paragraph, the output is a list of detected techniques. the data is provided in three folders: "train", "dev" and "test". each folder contains files with articles. each article is divided into paragraphs separated by an additional empty line. for each paragraph of each article in "train" and "dev" sets organisers provide the lists of labels of persuasion techniques in separate files. labels of test data are unknown. as an additional information authors provide for each paragraph the list of spans, start and end character index, and its class. there are 23 classes with a distribution that is not balanced (far from the uniform one). these classes are presented in ## system overview our system proceeds in the following general steps: one approach is used for all languages that contain train and dev set but models are trained separately. for languages that do not have train and dev set we translate the articles from polish to the required language. for these languages we use simple bert model for sequence classification. we do not use our model as we believe it is hard to find exact spans after translation. ## experimental setup for final evaluation we use train file for training and test file for evaluation. after choosing the right parameters we train the whole model on both datasets. we used random search for hyper-parameter tuning. for final model we used hyper-parameters as follows. maximum length of a sequence is 256, as it speeds up the computation and we did not see much improvement when using longer sequences. train batch size is 8 as it is mostly limited by the machines that were available. maximum number of epochs is 30 and we saved the best model on dev set according to the micro-f1 metric. learning rate was set to 1e-05 and max gradient norm was set to 10. dropout before both linear layers is set to 0.1. the network is trained using adamw optimization algorithm with weight decay equal to 0.01. ## results the results on the official test set are presented in ## conclusion we discovered that simple change of the index in bert embedding may help to improve the persuasion classification. moreover, we are able to identify spans and perform classification on limited data using the described networks. our system works better than classic bert for sequence classification but still needs some improvements. the next step would be to check another language models, that may be more powerful for this task. one could improve so that all the detected spans are included in classification. if more data is available, we can use separate network for spans and for classification. as a future work we should also extend the error analysis and find an approach for finding classes that are poorly recognized. we believe that persuasion techniques are strongly diverse and hard to be detected by one model. we think that trying ensemble models could improve the results.