DAAI at CASE 2021 Task 1: Transformer-based Multilingual Socio-political and Crisis Event Detection (2021)

Abstract: Automatic socio-political and crisis event detection has been a challenge for natural language processing as well as social and political science communities, due to the diversity and nuance in such events and high accuracy requirements. In this paper, we propose an approach which can handle both document and cross-sentence level event detection in a multilingual setting using pretrained transformer models. Our approach became the winning solution in document level predictions and secured the 3rd place in cross-sentence level predictions for the English language. We could also achieve competitive results for other languages to prove the effectiveness and universality of our approach.

## introduction with technological advancements, today, we have access to a vast amount of data related to social and political factors. these data may contain information on a wide range of events such as political violence, environmental catastrophes and economic crises which are important to prevent or resolve conflicts, improve the quality of life and protect citizens. however, with the increasing data volume, manual efforts for event detection have become too expensive making the requirement of automated and accurate methods crucial @xcite . considering this timely requirement, case 2021 task 1: multilingual protest news detection is designed @xcite . this task is composed of four subtasks targeting different data levels. subtask 1 is to identify documents which contain event information. similarly, subtask 2 is to identify event described sentences. subtask 3 targets the cross-sentence level to group sentences which describe the same event. the final subtask is to identify the event trigger and its arguments at the entity level. since a news article can contain one or more events and a single event can be described together with some previous or relevant details, it is important to focus on different data levels to obtain more accurate and complete information. this paper describes our approach for document and cross-sentence level event detection including an experimental study. our approach is mainly based on pretrained transformer models. we use improved model architectures, different learning strategies and unsupervised algorithms to make effective predictions. to facilitate the effortless generalisation across the languages, we do not use any language-specific processing or additional resources. our submissions achieved the 1 st place in document level predictions and 3 rd place in crosssentence level predictions for the english language. demonstrating the universality of our approach, we could obtain competitive results for other languages too. the remainder of this paper is organised as follows. section 2 describes the related work done in the field of socio-political event detection. details of the task and datasets are provided in section 3. section 4 describes the proposed approaches. the experimental setup is described in section 5 followed by results and evaluation in section 6. finally, section 7 concludes the paper. additionally, we provide our code to the community which will be freely available to everyone interested in working in this area using the same methodology foot_0 . ## related work in early work, the majority of event detection approaches were data-driven and knowledge-driven @xcite . since the data-driven approaches are only based on the statistics of the underlying corpus, they missed the important semantical relationships. the knowledge-driven or rule-based approaches were proposed to tackle this limitation, but they highly rely on the targeted domains or languages @xcite . later, there was a more focus on traditional machine learning-based models (e.g. support vector machines, decision trees) including different feature extraction techniques (e.g. natural language parsing, word vectorisation) @xcite . also, there was a tendency to apply deep learning-based approaches (e.g. cnn, ffnn) too following their success in many information retrieval and natural language processing (nlp) tasks @xcite . however, these approaches are less expandable to low-resource languages, due to the lack of training data to fine-tune the models. targeting this major limitation, in this paper we propose an approach which is based on pretrained transformer models. due to the usage of general knowledge available with the pretrained models and their multilingual capabilities, our approach can easily support event detection in multiple languages including low-resource languages. ## subtasks and data case 2021 task 1: multilingual protest news detection is composed of four subtasks targeting event information at document, sentence, cross-sentence and token levels @xcite . mainly the socio-political and crisis events which are in the scope of contentious politics and characterised by riots and social movements are focused. among these subtasks, we participated in subtask 1 and subtask 3 which are further described below. subtask 1: document classification subtask 1 is designed as a document classification task. participants need to predict a binary label of '1' if the news article contains information about a past or ongoing event and '0' otherwise. to preserve the multilinguality of the task, four different languages english, spanish, portuguese and hindi have been considered for data preparation. comparatively, a high number of training instances were provided with english than spanish and portuguese. no training data were provided for the hindi language. for final evaluations, test data were provided without labels. the data split sizes in each language are summarised in the main motivation behind the proposed approaches for event document identification and event sentence coreference identification is the recent success gained by transformer-based architectures in various nlp and information retrieval tasks such as language detection @xcite question answering @xcite and offensive language detection @xcite . apart from providing strong results compared to rnn based architectures, transformer models like bert @xcite and xlm-r @xcite provide pretrained language models that support more than 100 languages which is a huge benefit when it comes to multilingual research. the available models have been trained on general tasks like language modelling and then can be fine-tuned for downstream tasks like text classification @xcite . depending on the nature of the targeted subtask, we involved different transformer models along with different learning strategies to extract event information as mentioned below. ## experimental setup this section describes the learning configurations, transformer models and hyper-parameters used for the experiments. ## results and evaluation in this section, we report the conducted experiments and their results.