Understand before Answer: Improve Temporal Reading Comprehension via Precise Question Understanding (2022)

Abstract: This work studies temporal reading comprehension (TRC), which reads a free-text passage and answers temporal ordering questions. Precise question understanding is critical for temporal reading comprehension. For example, the question “What happened before the victory” and “What happened after the victory” share almost all words except one, while their answers are totally different. Moreover, even if two questions query about similar temporal relations, different varieties might also lead to various answers. For example, although both the question “What usually happened during the press release?” and “What might happen during the press release” query events which happen after “the press release”, they convey divergent semantics. To this end, we propose a novel reading comprehension approach with precise question understanding. Specifically, a temporal ordering question is embedded into two vectors to capture the referred event and the temporal relation. Then we evaluate the temporal relation between candidate events and the referred event based on that. Such fine-grained representations offer two benefits. First, it enables a better understanding of the question by focusing on different elements of a question. Second, it provides good interpretability when evaluating temporal relations. Furthermore, we also harness an auxiliary contrastive loss for representation learning of temporal relations, which aims to distinguish relations with subtle but critical changes. The proposed approach outperforms strong baselines and achieves state-of-the-art performance on the TORQUE dataset. It also increases the accuracy of four pre-trained language models (BERT base, BERT large, RoBERTa base, and RoBETRa large), demonstrating its generic effectiveness on divergent models.

## introduction understanding temporal relationships between events in a passage is essential for natural language understanding @xcite . temporal reading comprehension (trc) @xcite is a natural way to study temporal relations since natural language questions are flexible to capture divergent temporal relations @xcite . figure 1 shows several examples of temporal reading comprehension, where given a free-text passage, a system is required to answer temporal questions like "what usually happened during the press release?". a natural solution for temporal ordering understanding is to compare each candidate answer and the referred event in the question and classify their temporal relation into several pre-defined cate-gories, e.g., @xcite defines 13 possible relations such as after, ends, equal to. nonetheless, since temporal relationships vary greatly, it is almost impossible to enumerate all possible relationships. figure 1 shows several divergent varieties of temporal relations: one might query about plain after in q1, negated after in q2, constrained after in q3, etc. similarly, a question might query about usually happen in q4, might happen, or other relations. moreover, creating sufficient labels for all such relations is costly and poses great challenges for real-world applications. therefore, the classification-based approach is incompetent to handle the flexible relations in temporal reading comprehension. another paradigm is to formulate it as a reading comprehension problem and directly predict the answer to a question. with the help of large pre-trained language models (e.g., bert @xcite and roberta @xcite ), such approaches have achieved relatively good performance. however, they still struggle for the temporal reading comprehension task due to the lack of precise question understanding. for example, given the same passage, the bert model fine-tuned on squad @xcite predicts the same answer to the two questions @xcite , "what happened before a woman was trapped" and "what happened after a woman was trapped". in this case, although the two questions share almost the same words, the only different one between before and after leads to completely opposite intentions. moreover, even if two questions query about similar relations, different varieties might also lead to various answers. take the question q4 "what usually happened during the press release?" and "what might happen during the press release?" in figure 1 as an example. although they both query about events occurring after the press release, the slight difference conveys divergent semantics and leads to different answers. to tackle these challenges, we propose a novel question answering approach with precise question understanding. intuitively, temporal ordering questions consist of two elements, referred events, and concerned temporal relations. for example, the question "what usually happened during the press release?" can be decomposed into the referred event the press release and the concerned relation usually happen during. inspired by this observation, we first encode such questions into two representations, the event vector h c and the relation vector h r . then we evaluate how well each candidate answer matches the relation h r compared to h c with a separate mlp module. such fine-grained representations enable a better understanding of questions by focusing on different elements with different vectors and further provides good interpretability about the reasoning process. more importantly, it empowers the model to capture the semantics of divergent variants of temporal relations. specifically, we harness an auxiliary contrastive loss that aims to distinguish relations with subtle but critical changes. we evaluate the proposed approach on the torque dataset and achieve state-of-the-art performance compared to strong baselines. we further testify its effectiveness based on four different models (i.e., bert base, bert large, roberta base, roberta large) and demonstrate that precise question understanding can improve the qa accuracy for all models. ablation study shows that both question representation learning and contrastive loss play a critical role in the approach. ## related work temporal machine comprehension is closely related to two areas of works, i.e., machine reading comprehension and temporal ordering reasoning. ## our approach we first introduce the definition of temporal reading comprehension (trc) and then describe the model architecture consisting of contextual encoder, question understanding, and event relation assessment. finally, we provide details for the learning and inference process. ## experiments this section describes an empirical evaluation of our proposed approach. we also provide analysis, ablation studies, and case analysis to demonstrate its effectiveness. ## conclusion and future work temporal reading comprehension plays a critical role in natural language understanding. in this paper, we propose a precise question understanding method to tackle the trc problem. specifically, we encode temporal ordering questions into repre-sentations of referred events and concerned temporal relations, based on which candidate answers are evaluated in terms of their temporal relations to the referred events. in addition, a contrastive loss is employed to empower the model to capture essential differences among temporal relations. experimental results based on four pre-trained models verify the effectiveness of our proposed approach. in the future, we will investigate general approaches to handle more diverse temporal relation understanding problems and improve the passage understanding capability for temporal reading comprehension.