Improving Zero-Shot Event Extraction via Sentence Simplification (2022)

Abstract: The success of sites such as ACLED and Our World in Data have demonstrated the massive utility of extracting events in structured formats from large volumes of textual data in the formof news, social media, blogs and discussion forums. Event extraction can provide a window into ongoing geopolitical crises and yield actionable intelligence. In this work, we cast socio-political event extraction as a machine reading comprehension (MRC) task. % With the proliferation of large pretrained language models Machine Reading Comprehension (MRC) has emerged as a new paradigm for event extraction in recent times. In this approach, extraction of social-political actors and targets from a sentence is framed as an extractive question-answering problem conditioned on an event type. There are several advantages of using MRC for this task including the ability to leverage large pretrained multilingual language models and their ability to perform zero-shot extraction. Moreover, we find that the problem of long-range dependencies, i.e., large lexical distance between trigger and argument words and the difficulty of processing syntactically complex sentences plague MRC-based approaches. To address this, we present a general approach to improve the performance of MRC-based event extraction by performing unsupervised sentence simplification guided by the MRC model itself. We evaluate our approach on the ICEWS geopolitical event extraction dataset, with specific attention to ‘Actor’ and ‘Target’ argument roles. We show how such context simplification can improve the performance of MRC-based event extraction by more than 5% for actor extraction and more than 10% for target extraction.

## introduction with the proliferation of social media, microblogs and online news, we are able to gain a real-time understanding of events happening around the world. ## methodology given that an event has been detected in a sentence, we focus on the problem of identifying the arguments of the detected event. for instance, in fig. 1 the task is to identify the arguments 'actor' and 'target' of the event 'bring lawsuit against'. corresponding to each event type, we first generate qa pairs corresponding to actor and target arguments. the qa generation procedure for the dataset used in this paper for evaluation is outlined in 4. reading comprehension models can be brittle to subtle changes in context. they can be thrownoff by syntactic complexity, especially when the questions are not specific and do not include words overlapping with the context. moreover, long range dependencies between the trigger/predicate and the argument are a leading source of error for mrc models applied to event extraction as described in section 1. for this purpose, we propose an mrcguided unsupervised sentence simplification algorithm (russ), that iteratively performs deletions and extractions from the context in search for a higher-scoring candidate. the score function incorporates components that ensure sentence fluency, information preservation and the confidence of the target mrc model. fig. 2 gives an overview of the proposed approach. sentence a businessman detained for his links to disgraced army general xu caihou has been sued by his former employees. ## datasets and metrics we evaluate russ on the icews event dataset foot_2 @xcite ) from years 2013 to 2016. in this dataset, event data consists of coded interactions between socio-political actors (i.e., cooperative or hostile actions between individuals, groups, sectors and nation states) mapped to the cameo foot_3 ontology. we preprocess the icews data to extract event triples consisting of a source actor, an event type (according to the cameo taxonomy of events), and a target actor. an icews record contains an event sentence, source and target names (actor and target) and event text amongst other metadata. however these source and target names are normalized, i.e. the exact source and target spans might not occur in event sentence. for e.g. a source name in an icews record is "north atlantic treaty organization" however, the event sentence contains its abbreviation "nato". to retrieve the exact source and target names corresponding to spans that occur in the event sentence we perform denormalization by using the icews actors and agents dictionaries foot_4 that contain aliases of different source and target entities. for the "nato" example above, the actor dictionary contains the following aliases "north atlantic treaty organization, nato, north atlantic treaty organisation". we resolve the source name to the alias that occurs within the sentence, which in this example is "nato". we also remove country name from paranthesis of source and target names: citizen(iraq) → citizen because of the format in which they occur in the dictionaries. after deduplication and cleaning of icews data we obtain actor, event, target tuples for each event sentence. the next step is generating qa pairs for each tuple depending on the event type. ## qa dataset generation we first grouped the preprocessed icews event records by event type. for each event type we identified a list of most common predicates (triggers) for that event type using a heuristic approach since trigger labels are not available in the icews dataset. using this approach we obtained a list of common predicates corresponding to event types and their cameo codes as shown in ## results & discussion next, we increase the coefficients of actor and target roles from 1 to 3. the reason why we choose an odd number for this exponent is because sometimes for bad candidates the rc scores can be negative and since all the scores are combined in a multiplicative way, raising a negative score to an even power would reverse the desired effect. observing the results in rows 5 & 6 of ## conclusion & future work in this work, we motivated the need for mrc-based socio-political/conflict event extraction paradigm especially for zero-shot scenarios( § 1). next, we discussed the long-range dependency problem faced by event extraction systems. we proposed a simplification algorithm to reduce the syntactic complexity of the context aided by mrc-system feedback to address the problem( § 2). our results indicate that simplification can not only aid mrc systems in a zero-shot setting( § 5.1-5.3) but also when they're finetuned on in-domain data( § 5.4). in future work, we plan to scale our qa generation approach to improve coverage over more event types and languages. we can also make russ simplification more efficient by generating parallel training data for simplification using the russ method offline and train a simplification model using the generated data. in this way we can obtain guided simplifications via inference over a model. reproducibility: we release our code foot_7 .