Time-Aware Representation Learning for Time-Sensitive Question Answering (2023)

Abstract: Time is one of the crucial factors in real-world question answering (QA) problems. However, language models have difficulty understanding the relationships between time specifiers, such as ‘after’ and ‘before’, and numbers, since existing QA datasets do not include sufficient time expressions. To address this issue, we propose a Time-Context aware Question Answering (TCQA) framework. We suggest a Time-Context dependent Span Extraction (TCSE) task, and build a time-context dependent data generation framework for model training. Moreover, we present a metric to evaluate the time awareness of the QA model using TCSE. The TCSE task consists of a question and four sentence candidates classified as correct or incorrect based on time and context. The model is trained to extract the answer span from the sentence that is both correct in time and context. The model trained with TCQA outperforms baseline models up to 8.5 of the F1-score in the TimeQA dataset. Our dataset and code are available at https://github.com/sonjbin/TCQA

## introduction question answering (qa) models @xcite have achieved significant success in recent years. however, most existing qa models fail to understand time @xcite since most qa datasets @xcite lack temporal information. ignoring temporal constraints when answering questions can lead to inaccurate or unreliable results @xcite . for instance, as shown in figure 1 , neglecting the time while extracting the answer may lead to the selection of an incorrect entity, 'katie'. to overcome this limitation, language models must be able to incorporate temporal information into their comprehension of the context in which a question is asked. this requires the model to recognize temporal expressions within the text and understand the relationship between the time specifiers and numerical values. for example, asking about anything that happened 'after 2020' and 'before 2020' are entirely different, even though they include the same number. therefore, models must be capable of comprehending the connection between time specifiers and numbers beyond simple numerical comparisons. this study aims to investigate methods for enhancing the performance of qa models in timesensitive tasks. specifically, we aim to develop a model that can process temporal information and utilize it to answer time-sensitive questions precisely. injecting time awareness and numeracy into qa models is challenging since there are many possible temporal expressions, and the model must consider time information as an independent part of the context. therefore, we propose a time-context aware question answering (tcqa) framework to achieve this issue. we train the model through time-context dependent span extraction (tcse) task and contrastive time representation learning. in this paper, our contributions are: ## related work several previous works have addressed the issue of temporal reasoning in question answering using knowledge graphs. @xcite proposed a novel framework for handling complex temporal questions that involve time ordering. @xcite jointly train the model using text with timestamps. however, these approaches may not be sufficient for time-sensitive qa tasks, as temporal knowledge graphs typically handle only structured time information such as (barack obama, position held, president of usa, @xcite ). despite these efforts, there remains a gap in research regarding handling various time expressions and numerical reasoning in time-sensitive qa tasks. @xcite attempted to address this gap by constructing a dataset containing time-sensitive question-passage pairs. their analysis revealed that existing language models often fail to adequately consider temporal constraints in such tasks, resulting in significantly lower performance than humans. ## method we present an approach to improve the performance of models in time-sensitive qa tasks by proposing a time-context aware qa (tcqa) framework. ## conclusion in this paper, we demonstrated that existing qa models are inadequate in understanding time expressions. to address this problem, we proposed tcqa, which enables models to learn time expressions while maintaining their understanding of context. we constructed question-context templates to generate time-context dependent data for tcse and contrastive learning, and jointly trained the model. our experimental results showed that tcqa improves the performance of qa models on timeqa. additionally, we proposed a new evaluation metric, tc-score, and showed a gap in performance between models in terms of time and contextual understanding. future research should focus on advancing temporal reasoning capabilities beyond the comprehension of simple temporal expressions.