E vent G raph at CASE 2021 Task 1: A General Graph-based Approach to Protest Event Extraction (2022)

Abstract: This paper presents our submission to the 2022 edition of the CASE 2021 shared task 1, subtask 4. The EventGraph system adapts an end-to-end, graph-based semantic parser to the task of Protest Event Extraction and more specifically subtask 4 on event trigger and argument extraction. We experiment with various graphs, encoding the events as either “labeled-edge” or “node-centric” graphs. We show that the “node-centric” approach yields best results overall, performing well across the three languages of the task, namely English, Spanish, and Portuguese. EventGraph is ranked 3rd for English and Portuguese, and 4th for Spanish.

## introduction the automated extraction of socio-political event information from text constitutes an important nlp task, with a number of application areas for social scientists, policy makers, etc. the task involves analysis at different levels of granularity: document-level, sentence-level, and the finegrained extraction of event triggers and arguments within a sentence. the case 2022 shared task 1 on multilingual protest event detection extends the 2021 shared task @xcite with additional data in the evaluation phase and features four subtasks: (i) document classification, (ii) sentence classification, (iii) event sentence coreference, and (iv) event extraction. the task of event extraction involves the detection of explicit event triggers and corresponding arguments in text. current classification-based approaches to the task typically model the task as a pipeline of classifiers @xcite @xcite @xcite or using joint modeling approaches @xcite @xcite @xcite . in this paper, we present the eventgraph system and its application to task 1 subtask 4 in the 2022 edition of the case 2021 shared task. event-graph is a joint framework for event extraction, which encodes events as graphs and solves event extraction as semantic graph parsing. we show that it is beneficial to model the relation between event triggers and arguments and approach event extraction via structured prediction instead of sequence labelling. our system performs well on the three languages, achieving competitive results and consistently ranked among the top four systems. in the following, we briefly describe the data supplied by the shared task organizers and present subtask 4 in some more detail. we then go on to present an overview of the eventgraph system focusing on the encoding of the data to semantic graphs and the model architecture. we experiment with several different graph encodings and provide a more detailed analysis of the results. ## data and task our contribution is to subtask 4, which falls under shared task 1 -the detection and extraction of sociopolitical and crisis events. while most subtasks of shared task 1 have sentence-level annotations, subtask 4 has been annotated at the token-level while providing the annotators the document-level contexts. subtask 4 focuses on the extraction of event triggers and event arguments related to contentious politics and riots @xcite . this subtask has been previously approached as a sequence labeling problem combining various methods of fine-tuning pre-trained language models @xcite . the data supplied for subtask 4 is identical to that of the 2021 edition of the task, as presented in @xcite . the data is part of the multilingual extension of the glocon dataset @xcite with data from english, portuguese, and spanish. the source of the data is protest event coverage in news articles from specific countries: china and south africa (english), brazil (portuguese), and argentina (spanish). the data has been doubly annotated by graduate students in political science with token-level information regarding event triggers and arguments. @xcite reports the token level inter-annotator agreement to be between 0.35 and 0.60. disagreements between annotators were subsequently resolved by an annotation supervisor. ## system overview we use our system, eventgraph, that adapts an end-to-end graph-based semantic parser to solve the task of extracting socio-political events. in what follows, we give more details about the graph representation and the model architecture of our system. ## experimental setup data we use all the official training data to train our final model, without using any additional data. during development time, we set aside about 10 percent of the training data for development. a breakdown of the number of articles and sentences in train and dev are provided in joint training we train our model on the training data of all three languages and test on the official test data. as shown in ## results and discussion we summarize the results of our systems on the official test data in ## conclusion in this paper we have presented the eventgraph system for event extraction and its application to the case 2022 shared task on multilingual protest event detection. eventgraph solves the task as a graph parsing problem hence we experiment with different ways of encoding the event data as general graphs, contrasting a so-called "labeled-edge" and "node-centric" approach. our results indicate that the "node-centric" approach is beneficial for this task and furthermore that the separation in the graph of nodes belonging to different events in the same sentence proves useful. a more detailed analysis of the development results indicates that our system performs well in trigger identification, however struggles in the identification of target and fname arguments.