RSGT : Relational Structure Guided Temporal Relation Extraction (2022)

Abstract: Temporal relation extraction aims to extract temporal relations between event pairs, which is crucial for natural language understanding. Few efforts have been devoted to capturing the global features. In this paper, we propose RSGT: Relational Structure Guided Temporal Relation Extraction to extract the relational structure features that can fit for both inter-sentence and intra-sentence relations. Specifically, we construct a syntactic-and-semantic-based graph to extract relational structures. Then we present a graph neural network based model to learn the representation of this graph. After that, an auxiliary temporal neighbor prediction task is used to fine-tune the encoder to get more comprehensive node representations. Finally, we apply a conflict detection and correction algorithm to adjust the wrongly predicted labels. Experiments on two well-known datasets, MATRES and TB-Dense, demonstrate the superiority of our method (2.3% F1 improvement on MATRES, 3.5% F1 improvement on TB-Dense).

## introduction temporal relation extraction (tre) is crucial for natural language understanding and can facilitate various downstream applications such as summarization @xcite , question answering @xcite , and clinical diagnosis @xcite . as shown in figure 1 , the goal of tre is to determine the temporal order between an event pair (before, after, etc.). most early methods were based on statistical machine learning @xcite . in recent years, neural network based methods and large-scale pre-trained language models such as bert @xcite have contributed to a substantial increase in the performance of tre task @xcite . there are four events in these sentences. the graph below shows the pair-wise event temporal relationships. however, these methods may ignore the global structure features which carry non-consecutive and long-distance semantics @xcite . this shortcoming is obvious in dealing with an event pair that the two events belong to different sentences (inter-sentences event pair), such @xmath0, e 4 > in figure 1 . few previous works differentiate inter-sentence event pairs from intrasentence ones (where the two events appear in the same sentence). thus, the performance may be impacted. for example, we test that there is a 5 accuracy points gap between intra-sentences and inter-sentence event pairs with the recent state-ofthe-art method @xcite . to fill this gap, we aim to develop a structural features method that captures temporal semantic relations for both the inter-sentence and intra-sentence event pairs. specifically, we adopt graph neural networks (gnns), which have been proved to be effective in preserving global structure information of a graph in graph embeddings @xcite , to bridge the temporal relations. based on the above analysis, we present rsgt: relational structure guided temporal relation extraction. to enable our model to learn more ef-fective representation for relational structures, we take the following strategies: first, to obtain more relational information, we create different types of connections for the graph nodes based on their syntactic and semantic information. such connections are combined together to generate a rich relational graph. in particular, the node embeddings are obtained with the ggnn algorithm @xcite . to avoid graph over-smoothing, roberta @xcite embeddings are concatenated with ggnn embeddings to make the final prediction. second, unlike most previous graph-based models which directly use the pre-trained language model as the node encoder, we present a task called temporal event neighbor prediction to fine-tune the encoder. this task aims to predict the neighbor node of event mentions from the relational graph. the fine-tuned encoder can help rsgt better understand the correlation between the relational structure and raw text. ablation studies demonstrate that it can significantly boost efficiency. finally, we present a conflict detection and correction algorithm based on the transitivity rule of temporal relations to promote performance. experiments on two popular benchmarks, ma-tres @xcite and tb-dense @xcite , show that rsgt outperforms the stateof-the-art methods (2.3% f1 points improvement on matres, 3.5% f1 points improvement on tb-dense). meanwhile, we improve the accuracy of inter-sentence relations to the same level as intrasentence relations. ## method we formulate the tre problem as a multi-class classification task. for a document d with n sentences (s 1 , s 2 , ..., s n ), it can have multiple event mentions @xmath1, e 2 , ..., e m ). the goal of tre is to predict the temporal relation type between event pairs. for an event @xmath2, e j >, the input of our model is the sentence they belong to. in particular, if two events belong to different sentences (we call it inter-sentence event pair), two @xmath3, s @xmath4. our work rsgt involves five major parts: algorithm to revise temporal errors. ## end this algorithm is concise and efficient, and it can be well adapted to the correction work of various datasets without training. ## related work earlier efforts on tre (temporal relation extraction) use statistical machine learning techniques (support vector machine, max entropy) and handcraft features (e.g @xcite and @xcite ). recently, neural methods and large-scale pre-training language models have also achieved promising improvement @xcite @xcite . the early feature-based methods for tre have explored different features and resources to improve the performance, including syntactic patterns and lexical features @xcite . unlike previous works, our approach rsgt takes account of relational structure features to induce more accurate representations. a wave of research at the intersection of deep learning on graphs has influenced a variety of nlp tasks, including event extraction @xcite , relation extraction @xcite and event argument extraction @xcite . these graph-structured data can encode complicated relations between event pairs to infer temporal order. our model is different from such related works in that we designed a relational structure guided graphs that are tailored to our tre task. in addition, we introduce a novel temporal event neighbor prediction task for the fine-tuning of the node encoder. ## conclusion this paper proposes rsgt to capture relational structure information for the temporal relation extraction task. the experimental results well demonstrate our model's effectiveness and superiority in both the overall datasets and the inter-sentence event pairs. ablation experiments show that the relational graph model and temporal event neighbor prediction contribute greatly to rsgt's performance. our future work will focus on how to apply temporal event neighbor prediction, and conflict detect and correct algorithm to other tasks with rich relations such as casual relations @xcite . we believe these methods are promising in processing relational structure information from other relational extraction tasks.