TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in
  Real-World Scenarios
Authors: name: Shaohang Wei; name: Wei Li; name: Feifan Song; name: Wen Luo; name: Tianyi Zhuang; name: Haochen Tan; name: Zhijiang Guo; name: Houfeng Wang
Published: 2025-05-19 09:22:02

Abstract: Temporal reasoning is pivotal for Large Language Models (LLMs) to comprehend
the real world. However, existing works neglect the real-world challenges for
temporal reasoning: (1) intensive temporal information, (2) fast-changing event
dynamics, and (3) complex temporal dependencies in social interactions. To
bridge this gap, we propose a multi-level benchmark TIME, designed for temporal
reasoning in real-world scenarios. TIME consists of 38,522 QA pairs, covering 3
levels with 11 fine-grained sub-tasks. This benchmark encompasses 3
sub-datasets reflecting different real-world challenges: TIME-Wiki, TIME-News,
and TIME-Dial. We conduct extensive experiments on reasoning models and
non-reasoning models. And we conducted an in-depth analysis of temporal
reasoning performance across diverse real-world scenarios and tasks, and
summarized the impact of test-time scaling on temporal reasoning capabilities.
Additionally, we release TIME-Lite, a human-annotated subset to foster future
research and standardized evaluation in temporal reasoning. The code is
available at https://github.com/sylvain-wei/TIME , and the dataset is available
at https://huggingface.co/datasets/SylvainWei/TIME .