Where “where” Matters : Event Location Disambiguation with a BERT Language Model (2023)

Abstract: The method method presented in this paper uses a BERT model for classifying location mentions in event reporting news texts into two classes: a place of an event, called main location, or another location mention, called here secondary location. Our evaluation on articles, reporting protests, shows promising results and demonstrates the feasibility of our approach and the event geolocation task in general. We evaluate our method against a simple baseline and state of the art ML models and we achieve a significant improvement in all cases by using the BERT model. In contrast to other location classification approaches, we completelly avoid lingusitic pre processing and feature engineering, which is a pre-requisite for all multi-domain and multilingual applications.

## introduction detecting event location from online text sources is a key area of research since the advent of social networks @xcite , @xcite . applications have been developed in fields as diverse as disaster management @xcite , @xcite , tracking disease outbreaks @xcite , or fight against crime @xcite . detecting socio-political events (and in particular, protests) emerged as an important use case @xcite , which relies on comprehensive, timely and high-quality data that is sometimes not available or it is difficult to be obtained. recently, the case (challenges and application of automatic extraction of socio-political events from news) series of workshops @xcite have introduced a set of event detection shared tasks and an annotated corpora of protest events, which contains annotations of event places among the other arguments. the case initiative significantly boosted the work in the area of socio-political event analysis and gave birth to shared tasks and research works with focus on event location identification, @xcite and @xcite . formally, geographical place recognition is a sub-category of named entity recognition (ner) @xcite . however, it has many particular features: first, geographic names are in the range of millions and unlike names of people and organizations, there are no reliable rules for recognizing these entities by their textual form. therefore, the first level in recognizing geographic entities is by searching for them in big geographical dictionaries, called gazetteers. second, geographic names can be mismatched with names of people: as an example, let's consider place names like washington, georgia, alexandria and many others. third, identifying the place names in text is just the first step in recognizing them: disambiguating locations (i.e. which of the many paris is it) and identifying their precise coordinates is even more challenging task @xcite . the fourth problem, related to location analysis is recognizing the semantic role of the location mentions. currently, very little work is dedicated to this important problem: our paper aims at filling this gap, by applying the latest advances in natural language processing @xcite , leveraging large language models and the knowledge encoded in them, to recognize locations, where events happen, distinguishing them from other location mentions. our approach is designed to be used as an integral part of an automated process for event extraction: in particular, we aim at linking protest events from news articles to the locations where they took place. a classical problem of such location identification is the fact that apart from the locations of the main events reported in the news, here called main locations, many more places are usually referred to in the text, called here secondary locations. typically, a news article focuses on one main event, which however is related to various reported real or possible happenings, which took place before or after the main one. each event also has an elaborated structure and may feature different semantic arguments, among them places, as well as sub-events and larger events, which encompass it. conversely, locations may be used to define the places of the events, as well as to address the origins and affiliations of people and organizations ("refugees from syria", "the mayor of brussels"), to answer the event location detection challenge, we proposed an approach which uses a bert @xcite model for text classification; it classifies each location as the place of an event on which the news article is focused (main location) or as a secondary location mention (places of secondary events or location mentions, which are not event places). our model uses only lexical context and the position of the sentence in the article. however, our approach makes use of the implicit semantic knowledge about the similarities of the words and their relations, encoded in the bert model. in this way, we avoid using semantic features and other text pre-processing, relying entirely on the semantic knowledge, encoded in bert. moreover, recent research @xcite aims at transferring bert models across languages , potentially bringing making our our approach multilingual. ## related work earlier work on event extraction, such as @xcite ) and the rees system @xcite use syntactic patterns for detecting locations and other event arguments. similarly, one of the first disease outbreak systems, pulse @xcite , makes use of syntactic clues and proximity to essential event arguments, such as disease names, to select the outbreak locations. some recent approaches for event location detection like @xcite also makes use of proximity of the location to specific event arguments. these linguistic approaches, although having a reasonable precision, are limited in their application, since syntactic patterns and clues require a significant amount of expert knowledge and efforts and strongly depend on the event classes, which are being considered in the event extraction system. moreover, linguistic approaches cannot efficiently exploit big data repositories, i.e. corpora and public event data bases, such as acled @xcite , global terrorism database (lafree and dugan, 2007), and others, which has recently emerged. in contrast, machine learning (ml) models can significantly benefit from such data: a recent ml work on event geolocation, based on an existing event data set @xcite ) is presented in @xcite . their work is similar to the approach presented in this paper. however, they use semantic pre-processing of the text by annotating each event-specific keyword and expression: event trigger verbs and nouns (e,g, breaking into), actors (e,g, ukraininan soldiers), temporal expressions and others. this work reports 75% accuracy for detecting the main event location in a protest event data set. they use support vector machines (svm), neural network and random forests, all methods delivering similar results. another work which relies on training a classifier for event location detection is presented in @xcite . they use svm classifier and word embeddings for identifying the sentences likely to contain the main event location. then they extract from them the most frequent location. similartly, @xcite proposes a convolutional neural network, which finds the main event location. they have manually created a data set of 8'000 sentences, containing information about military offensives in the syrian war. the event geolocation accuracy they achieve is around 84%. ## protest events and their locations protest are socio-political events, which include rallies, protests, marches, strikes, riots, violent disorders and civil unrest. each socio-political event assumes action by a large group of people. in particular, the protest events express disapproval and oppose to concrete actions or policies of governments, administrators, parties, institutions or companies. the definition of protest event, given in @xcite is: a protest event is open to the public, politically motivated and not institutionalised as opposed to e.g. elections. in some cases protest actions pose concrete demands, e.g. lowering taxes or raising wages. on certain occasions, these events attempt to focus the public attention on causes, such as minority rights, peace in war zones, environmental problems, etc. these events may include spontaneous violent actions, mass violence against people, vehicles and infrastructure. such actions may manifest characteristics of crime or event small-scale armed conflicts, when armed opposition to the police takes place. in order to better understand the dynamics of the protest events and their relation with various geographic locations, we have manually analyzed a small set of news, identifying the main and the secondary events and their related locations. our analysis found four basic types of location mentions: let's consider as an example, a news article fragment describing a protest in oslo: "dozens of activists, including greta thunberg of neighboring sweden, blocked the entrance to the energy ministry in oslo monday to protest a wind farm they say hinders the rights of the sami indigenous people to raise reindeer in arctic norway" in this fragment three locations are mentioned, while only one, i.e. oslo, is the main location. the other locations mentions , (sweden and arctic norway), are secondary ones. the first relates to the origin of one of the prominent protesters (greta thunberg) and the second is the place, where the cause for the protest is located: a wind farm in arctic norway. our analysis shows that the complexity of the events, described in the news, not only the sociopolitical ones, has its impact on the location references: one happening can trigger mentions of multiple related events and people, and the corresponding locations, related to them. in the case of protest actions, the event which is a cause for them is frequently mentioned along with its place. moreover, the effects of the protest on the people and the urban environment: blocked traffic, police actions and similar, bring in the text additional locations. in some sense, this is in agreement with davidson's view on the event semantics @xcite , for whom the cause and effect constitute important characteristics of the event phenomena. ## approach the approach we propose for geolocating events belongs to the class of machine learning approaches, it is similar in spirit to the work of @xcite . we, however, chose to use a bert classification model, since it provides the necessary level of abstraction by encoding the texts into a semantic space, trained on millions of documents. in this way, we avoided the feature abstraction phase, which is part of all the other ml approaches for geocoding, cited so far. in order to train and evaluate our approach, we used a corpus of protest news, reporting various types of protests in india and china. @xcite . in this corpus the annotated event locations are main locations. moreover, we have additionally annotated the secondary location mentions, which were not annotated in the corpus, using the mordecai open source software @xcite . in this way, we obtained a corpus with main and secondary locations. a sample annotated sentence from the corpus is shown below: ## experimental set up in our experiments we used the corpus of protest events with already annotated locations, enriched with automatic location identification from mordecai, as explained in the previous section. following the procedure for extraction of location windows ( from this data we have performed a crossvalidation, generating 10 random train/test data splits, each containing 66% location windows for training and 34% for test. we fine-tuned the fastbert model on the training set of each data split and evaluated the performance of the model on the test set. in order to evaluate the difficulty of the location classification task, we introduced also a simple baseline first sentence, which considers a location ## evaluation we have calculated precision, recall, the f1 measure and the accuracy of the fastbert, the two svm models, bag-of-words (bow) and google news word2vec (w2v), and the first sentence baseline on the test set of each of the 10 data splits. in clearly, fastbert significantly outperforms the baseline first sentence, especially as a recall, f1 measure and accuracy. notably, the recall of fast-bert is more than twice the recall of the baseline: this shows the importance of the model for identifying main event locations, which can frequently be mentioned after the first sentence. compared to the svm bow and svm w2v, our method showed significantly better accuracy with respect to the two svm models: 0.73 vs 0.64 for svm bow and 0.65 for svm w2v. the f 1 measure of bert and the svm models are comparable, still bert outperforms the two svm models with 0.02 and 0.03. the standard deviation of the f1 of fastbert across all the 10 splits is @xmath0.03. this shows that our evaluation was reliable and the results do not depend strongly on the data split. our evaluation shows that bert outperforms two state-of-the-art machine learning models and a baseline for detecting event locations. although not directly comparable, the results we achieved are similar in terms of accuracy to the results reported by @xcite on a different data set of protest events: the best accuracy they achieve is 0.75, using svm. their approach, however, uses a significant amount of semantic and morphological pre-processing. in contrast, we entirely relied on the semantic knowledge encoded in the bert model. this is a clue that the bert models could decrease the need of extensive feature engineering and provide a basis for non complex identification of event arguments. ## conclusions the objective of this paper was to validate an approach, based on the use of a large language model (fastbert) to leverage context and semantics in the task of detecting primary (main) event locations. in this process we completely avoided complex feature engineering and linguistic pre-processing. we achieved encouraging results, outperforming an heuristic baseline and svm classifiers based on bag of words and word embedding vectors. in this work we focused on protest events, since they are important measure for the level of political discontent in the society and provide a basis for conflict prediction. other socio-political events, such as armed conflicts, manifest similar problems when analysing their spacial dynamics. in this line of thought, locations are important parameters for each news report. moreover, distinguishing main location mentions from secondary ones is an important and challenging task. therefore, our work has larger scope and applicability which goes beyond the protest events. the question of performance of such approach for less resourced languages should be tackled. being multilingual by design is of paramount importance for many automatic event detection applications. the promise of last generation models to transpose learning efficiently from one language to another is in this view a strong incentive to further invest in their use. in this perspective training, testing and evaluating the latest large language models with multi-lingual annotated event location corpora is a relevant research direction in the context of automated location analysis in news and social media streams.