Dynamically Updating Event Representations for Temporal Relation Classification with Multi-category Learning (2020)

Abstract: Temporal relation classification is the pair-wise task for identifying the relation of a temporal link (TLINKs) between two mentions, i.e. event, time and document creation time (DCT). It leads to two crucial limits: 1) Two TLINKs involving a common mention do not share information. 2) Existing models with independent classifiers for each TLINK category (E2E, E2T and E2D) hinder from using the whole data. This paper presents an event centric model that allows to manage dynamic event representations across multiple TLINKs. Our model deals with three TLINK categories with multi-task learning to leverage the full size of data. The experimental results show that our proposal outperforms state-of-the-art models and two strong transfer learning baselines on both the English and Japanese data.

## introduction reasoning over temporal relations relevant to an event mentioned in the document can help us understand when the event begins, how long it lasts, how frequent it is, and etc. starting with the time-bank @xcite corpus, a series of temporal competitions (tempeval-1,2,3) @xcite @xcite are attracting growing research efforts. temporal relation classification (trc) is the task to predict a temporal relation (after, before, includes, etc.) of a tlink from a source mention to a target mention. less effort has been paid to explore the sharing information across 'local' pairs and tlink categories. in recent years, a variety of dense annotation schemas are proposed to overcome the 'sparse' annotation in the original timebank. a typical one is the timebank-dense (td) corpus @xcite , which performs a compulsory dense annotation with the complete graph of tlinks for the mentions located in two neighbouring sentences. such dense annotation increases the chance of pairs sharing common events and demands of managing 'global' event representations across pairs among tlink categories. however, globally managing event representations of a whole document takes an extremely heavy load for the dense corpora. timebank-dense contains around 10,000 tlinks in only 36 documents and is 7 times denser than the original timebank. thus, we propose a simplified scenario called source event centric tlink (sect) chain. for each event e i in a document, we group all tlinks containing the common source event e i into the e i centric tlink chain and align them with the chronological order of the target mentions appearing in the document. we assume that our system is capable of learning dynamic representations of the centric event e i along the sect chain via a 'global' recurrent neural network (rnn). ## experiments and results we conduct the experiments of applying the sec model on both the english td and japanese bt corpora. juman++ @xcite foot_3 is adopted to do morphological analysis for japanese text. td annotation adopts a 6-relation set (after, before, simultaneous, includes, is included and vague). we follow the 'train/dev/test' data split foot_4 of the previous work. for bt, we follow a merged 6-relation set as @xcite . we perform the document-level 5-fold cross-validation. in each split, we randomly select 15% documents as the dev set from the training set. the tlinks statistics of the two corpora are listed in we adopt the english and japanese pre-trained 'base' bert foot_5 and empirically set rnn hidden size equal to bert hidden, 4 sect chains per batch, 20 epochs, and adamw (@xmath0. the other hyperparameters are selected based on the dev micro-f1. all the results are 5-run average. for the lack of comparable transfer learning approaches, we build two bert baselines as follows (fine-tuning 5 epochs, batch size is 16): ## conclusion this paper presents a novel transfer learning based model to boost the performance of temporal information extraction task especially for densely annotated dataset. our model can dynamically update event representations across multiple tlinks in a source event centric chain scope. our model exploits a multi-category learning framework to leverage the total data of three tlink categories. the empirical results show that our proposal outperforms the state-of-the-art systems and the ablation tests suggest the effectiveness of two main proposals. the non-english experiments support the feasibility of our system on the japanese data.