NUS - IDS at CASE 2021 Task 1: Improving Multilingual Event Sentence Coreference Identification With Linguistic Information (2021)

Abstract: Event Sentence Coreference Identification (ESCI) aims to cluster event sentences that refer to the same event together for information extraction. We describe our ESCI solution developed for the ACL-CASE 2021 shared tasks on the detection and classification of socio-political and crisis event information in a multilingual setting. For a given article, our proposed pipeline comprises of an accurate sentence pair classifier that identifies coreferent sentence pairs and subsequently uses these predicted probabilities to cluster sentences into groups. Sentence pair representations are constructed from fine-tuned BERT embeddings plus POS embeddings fed through a BiLSTM model, and combined with linguistic-based lexical and semantic similarities between sentences. Our best models ranked 2nd, 1st and 2nd and obtained CoNLL F1 scores of 81.20%, 93.03%, 83.15% for the English, Portuguese and Spanish test sets respectively in the ACL-CASE 2021 competition.

## introduction the ability to automatically extract sentences that refer to the same event from any given document is useful for downstream information extraction tasks like event extraction and summarization, timeline extraction or cause and effect extraction @xcite . event sentence coreference identification (esci) aims to cluster sentences with event mentions such that each cluster comprises of sentences that refer to the same specific event. we address esci for news articles referring to socio-political and crisis event information in a multilingual setting, introduced as one of the acl-case 2021's shared tasks @xcite . given that news articles comprise of multiple events spread across a few sentences, and the syntax referring to the same event differs in different contexts, esci for news articles is a challenging nlp problem @xcite . furthermore, considering the availability of news in various languages, esci techniques that are applicable beyond english and robust across different languages are desirable. the esci task is illustrated using an example article shown in figure contributions: we propose a two-step solution for the esci task. in step-1, we obtain sentence pair embeddings by fine-tuning bidirectional encoder representations from transformers (bert) @xcite embeddings combined with parts-of-speech (pos) embeddings that are fed through a bi-directional long short-term memory (bilstm) model. next, these sentence pair embeddings are combined with novel features based on lexical and semantic similarities to train a classifier that predicts if the sentence pair is coreferent. step-2 involves the clustering of sentences using sentence pair probabilities predicted from step-1. we apply the clustering algorithm from örs et al. (2020) to obtain a variable number of clusters for each article. we illustrate the effectiveness of our proposed solution via detailed validation experiments on the training datasets from acl-case 2021. we show that our features are effective on documents from all three languages studied in the competition, viz, english, portuguese, and spanish. indeed, on the acl-case 2021 shared task 1 subtask 3, our best-performing models ranked 2 nd , 1 st and 2 nd and obtained conll f 1 scores of 81.20%, 93.03%, 83.15% for the english, portuguese and spanish test sets respectively. organization: in the next section, we present closely related work on esci. subsequently, section 3 introduces our features and classification model while section 4 discusses our dataset, experimental setup, results, and findings. in section 5, we conclude the paper with some future directions. ## related work most end-to-end event coreference systems approach the task in a two-stage manner: (1) to detect the mention or event of interest, and (2) to resolve the given mentions or events and cluster if coreferent @xcite . in our work, we focus only on latter task of coreference resolution and have direct access to identified event sentences. early works of esci adopted linguistic @xcite or template-based features @xcite . subsequently, neural network methods to encode textual events and contexts became increasingly popular @xcite . the combination of the two methods have also proved to be effective in recent works @xcite . in the previous run of esci by the same organ-isers @xcite , the bestperforming team ( örs et al., 2020) deconstructed the task into two steps: (1) to predict if a sentence pair is coreferent or not, and (2) use the predictions as scores for clustering. this approach is common amongst other event coreference resolution methods too @xcite . we employ this general approach and focus on enriching the feature space using linguistic-based similarity measures along with richer text embeddings based on bert with pos embeddings. ## our approach figure 2 summarizes our proposed pipeline. in this section, we describe our approach in detail. 1 let a be an article containing m sentences with event mentions {s 1 , s 2 , ..., s m }. to produce a list of c clusters that group these m sentences, we adopt the approach of örs et al. (2020) and first extract sentence pairs from a. next, binary classification is performed to identify if a given pair is coreferent. let (h, t) represent a sentence pair, with h and t referring to the lower and higher sentence numbers in a, respectively. the features employed for training a binary classifier that identifies coreferent sentences are described next. ## conclusions and future work we presented a two-step solution for the esci task of acl-case 2021. our solution based on sentence pair classification effectively harnesses bert and pos embeddings and combines them with linguistic similarity features for accurate sentence coreference resolution across languages. indeed, our models ranked 2 nd , 1 st and 2 nd obtaining conll f 1 scores of 81.20%, 93.03%, 83.15% for the english, portuguese and spanish test sets, respectively, in the competition. in this paper, we focused on within-document coreference sentences. it is common for coreference resolution tasks to also focus on crossdocument settings (i.e. identify coreferent event mentions across multiple documents) @xcite as such models can better aid downstream tasks like contradiction detection or identification of "fake news". in future, we hope to extend our models to work across documents. additionally, multiple events might be presented in a sentence. the shared task focuses on hard clustering (i.e. each sentence can only belong to one cluster). however, we believe it is valuable to also investigate cases where the event clusters overlap.