FKIE _itf_2021 at CASE 2021 Task 1: Using Small Densely Fully Connected Neural Nets for Event Detection and Clustering (2021)

Abstract: In this paper we present multiple approaches for event detection on document and sentence level, as well as a technique for event sentence co-reference resolution. The advantage of our co-reference resolution approach, which handles the task as a clustering problem, is that we use a single neural net to solve the task, which stands in contrast to other clustering algorithms that often are build on more complex models. This means that we can set our focus on the optimization of a single neural network instead of having to optimize numerous different parameters. We use small densely connected neural networks and pre-trained multilingual transformer embeddings in all subtasks. We use either document or sentence embeddings, depending on the task, and refrain from using word embeddings, so that the implementation of complicated network structures and unfolding of RNNs, which can deal with input of different sizes, is not necessary. We achieved an average macro F1 of 0.65 in subtask 1 (i.e., document level classification), and a macro F1 of 0.70 in subtask 2 (i.e., sentence level classification). For the co-reference resolution subtask, we achieved an average CoNLL-2012 score across all languages of 0.83.

## introduction gathering information about current and past events is quite important since such information can help to detect, analyze, prevent and forecast dangerous social and political situations. an accumulation of protest events in a certain region may indicate massive discrepancies between two or more parties. such situations can escalate and result in violence. using modern systems and data including for example news articles, violent events can be forecast @xcite . today, caused by a globally connected world, there exists an endless stream of news and information. to conquer this flood of data, much human effort is needed. therefore, automation of information analysis can help to reduce the workload. one task in this area is the detection of events in texts consisting of natural language, for example newspaper articles. it is an easy task for humans to read, understand and identify such events. for computers it is more difficult to process natural language and detect event mentions. in this paper, we present our approaches for event detection in articles and sentences based on simple densely connected neural networks as part of task 1 @xcite of the shared task on socio-political and crisis events detection at case @ acl-ijcnlp 2021. the first task is split up into four different subtasks. we participated in the first three. for the first subtask, we used an accumulation of trained neural nets with majority voting, where each net is a densely connected net consisting of only six layers including the in-and output layer. for the second subtask, we used a single net with the same specifications as in the first subtask. the third subtask aims at co-reference resolution of event sentences. we see this subtask as a typical clustering task. therefore, we use a comparison based algorithm, which reduces the clustering problem mainly to the optimization of a single neural net. co-reference resolution in our case, is based on the comparison of sentence pairs and will be described later in more detail. all code used in this paper is publicly available 1 . the paper will proceed as follows: first, related work will be introduced. after that, the subtasks the we participated in will be described. the next chapter presents our methodology, including data preparation and system descriptions for all subtasks. then, the results are depicted. in the end, we come to a conclusion and give an outlook for future work. ## related work since this workshop is a follow up event of the clef protestnews 2019 and aespen at lrec 2020 shared task, many approaches were already made as mentioned by @xcite and @xcite . aside from these approaches, a variety of other experiments trying to solve the task of event detection can be found in the literature. in earlier years, pattern matching approaches as described by @xcite were common and successful for the detection of events, but often required much human effort and domain knowledge for pattern construction. this lead to the idea propagated by @xcite of the automatic construction of such patterns. with the rise of available and affordable computing power, these techniques were replaced by modern machine learning techniques and especially artificial neural networks. state of the art systems for event detection, see for example @xcite , use a combination of different kinds of neural nets, like bidirectional lstms and modified graph convolutional networks. other models, as presented by @xcite , use convolutional neural networks and reduce the task to a multi class labeling problem. event detection can also be seen as a question answering task, where one could ask if an event exists in the given text or not, as done by @xcite . what all of the systems have in common is that they need a representation of text that is understandable for a computer. @xcite showed that modern transformer embeddings are the best choice by comparing them to classic word embeddings and achieving superior results with them. based on these findings, we decided to make use of them in our work too. for subtask 3, common clustering algorithms could be used for co-reference resolution, when using suitable metrics. co-reference resolution using mention pair models, such as those proposed by @xcite and radford (2020), could also be implemented. ## task description the first task of the workshop consists of four different subtasks. the different subtasks build upon each other, starting at document level (subtask 1) and go on to gradually focus on smaller instances (sentence level, word level). we provide three different models for the first three subtasks. the data for all three subtasks is provided in a json format. ## conclusion we presented three different approaches for the three different subtasks. the accumulation of several neural nets used in subtask 1 improved the results of the model just very slightly in comparison to a single densely connected neural net. in general, we can see that working on word level is not mandatory. sentence and document embeddings in combination with simple dense nets can lead to good results. this decreases the complexity of the task immensely. the results on the sentence level improve in comparison to the ones achieved on the document level, with exception of the results for the english data. the clear difference between the results obtained on the self-compiled test set and the test set of the organizers with regard to english serves as a good starting point for future work. for subtask 3, we presented a simple solution for event sentence co-reference resolution, focusing on the optimization of a function for comparison by using a multi input neural network. using this approach, we were able to solve the task in a way that does not require metrics, thresholds and other hyperparameters, which are often needed in clus-tering, and thus save time during the clustering process. for future work it would be interesting to use bidirectional lstms and other techniques to improve the results for co-reference resolution further.