Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models (2023)

Abstract: Reasoning about time is of fundamental importance. Many facts are time-dependent. For example, athletes change teams from time to time, and different government officials are elected periodically. Previous time-dependent question answering (QA) datasets tend to be biased in either their coverage of time spans or question types. In this paper, we introduce a comprehensive probing dataset TempReason to evaluate the temporal reasoning capability of large language models. Our dataset includes questions of three temporal reasoning levels. In addition, we also propose a novel learning framework to improve the temporal reasoning capability of large language models, based on temporal span extraction and time-sensitive reinforcement learning. We conducted experiments in closed book QA, open book QA, and reasoning QA settings and demonstrated the effectiveness of our approach.

## introduction in recent years, large language models (llms) have achieved significant success in many natural language processing (nlp) tasks, such as natural language understanding (nlu) @xcite , information extraction (ie) @xcite , and question answering (qa) @xcite . many facts and answers are dependent on their related time scopes, such as 'what soccer club was lionel messi playing for? @xcite has pointed out around 48% of the qualifiers in the widely-used knowledge base wikidata @xcite are time-related. that is, a significant number of the knowledge triples in the wikidata kb have their expiry dates. correct understanding of temporal concepts is crucial for language models to be successful in real-world applications. to examine the temporal reasoning capabilities of llms, the time-sensitive question answering (tsqa) task has been proposed and several evaluation datasets were published for research purposes. the time-sensitive qa dataset @xcite and the templama dataset @xcite were constructed based on the wikidata temporal kb. streamingqa @xcite was constructed by news article collections in english wmt challenges from 2007 to 2020. one consensus of prior work is that time-sensitive qa is a challenging task and its performance is still far below human performance. however, they did not provide a systematic analysis of lm's temporal reasoning capability. in this paper, we aim to systematically analyze such capability and identify the strengths and weaknesses of lms on temporal reasoning. as shown in figure 1 , humans' understanding of temporal reasoning could be broken down into three levels: time-time (l1) relation, time-event (l2) relation, and event-event (l3) relation. for the understanding of time-time relations, humans can easily determine the relation between two timestamps t 1 and t 2 on the time axis. for example, when humans are asked 'what is the year after 2020?', they are able to answer this question without any external information. this level of temporal understanding could be regarded as a set of logic rules and is highly generalizable across different times, while this type of reasoning was overlooked in prior tsqa research @xcite @xcite . for time-event relations, the reasoning process requires grounding events to their specific time ranges. in this paper, the concept of events includes time-dependent facts. humans either memorize a large number of timeevent pairs or need to rely on relevant contexts to deduce such relations. an example question is 'what soccer club was lionel messi playing for in level -1 time-time relation abstractive concept, can be viewed as a set of logic rules. once human understands the concepts, it is easily generalizable. example question: what is the year after 2010? target: 2011 ## preliminaries we aim to examine the capability of lms for simple year prediction. we first design a set of question templates that reflects the basic concepts of tempo-ral prediction, as shown in the second lm is flan-t5-large @xcite model. this model is instruction-tuned on data of more than 60 nlp tasks. the fine-tuned model demonstrated competitive zero-shot reasoning capability, and achieved strong performance on many natural language understanding and generation tasks. the third model is the popular chatgpt @xcite model. to ensure that the predictions are consistent, we used the gpt-3.5-0301 version of chatgpt. we aim to evaluate the temporal reasoning capability of the three language models. we evaluate the answers using the following three metrics: (1) exact match (em), which is a standard metric for qa. besides, since the expected answers are numeric, we also evaluate the answers by (2) mean absolute error (mae) and (3) trend accuracy (trend acc). trend accuracy is calculated by whether the predicted year is before or after the reference year. if the trend is correct, the prediction is deemed to be correct. the experimental results on year prediction are shown in in the pre-training corpora. since many large lms are trained and released after 2018, the pre-training corpora may contain more year expressions that are closer to that date. in contrast, the first two lms perform significantly worse in the past and the future (2020-2040) years. ## improving temporal reasoning in order to improve the temporal reasoning capabilities, we propose a temporal training framework for sequence-to-sequence language models. firstly, we pre-train the language model with a temporal span extraction task to encourage the model to pay more attention to the temporal and entity spans. we then fine-tune the model on task-specific data in tempreason. finally, we further fine-tune the language model by time-sensitive reinforcement learning with our novel reward function. ## conclusions and future work in this paper, we tackled the under-explored temporal reasoning problem for large language models. we found that large language models are highly susceptible to biases of time, and their temporal reasoning capability varies depending on the specific time given in the question. besides, we proposed a comprehensive time-sensitive qa dataset temp-reason to evaluate lms' temporal reasoning capability in diverse settings. lastly, we proposed a novel training paradigm to improve language models' reasoning capability by temporal span extraction pre-training and time-sensitive reinforcement learning. we conducted extensive experiments and demonstrated that our proposed model consistently outperformed strong baselines.