D ate L ogic QA : Benchmarking Temporal Biases in Large Language Models (2025)

Abstract: We introduce DateLogicQA, a human-curated benchmark of 190 questions specifically designed to understand temporal bias in Large Language Models (LLMs). Covering seven date formats across past, present, and future contexts, DateLogicQA examines four reasoning types: commonsense, factual, conceptual, and numerical. Through human-led evaluations of 12 state-of-the-art LLMs, we identify Representation-Level Bias, arising from suboptimal embeddings that distort date semantics, and Logical-Level Bias, manifesting when correct date tokens yield flawed temporal reasoning. Our findings underscore persistent challenges in handling various date formats and temporal contexts, revealing the need for more robust pretraining data, targeted post-training methods, and precise tokenization strategies. By illuminating these biases, we provide actionable insights to guide the development of LLMs for accurate temporal reasoning across diverse real-world applications.

## introduction accurate temporal reasoning is essential for realworld applications like event planning and historical questions. however, biases in large language models (llms) can lead to misinterpretations or errors in date-related tasks. understanding these biases is essential for precisely handling numerical structures and contextual meanings, making temporal reasoning ideal for identifying and analysing biases in tokenization, representation, and logical reasoning. a significant source of these biases originates from the tokenization process. while tokenizers divide the text into subword units, inconsistencies in tokenizing dates can disrupt reasoning tasks. this can lead to two types of biases: representation-level bias, caused by inconsistencies in embeddings affecting semantic structures of dates, and logical-level bias, where correct tokens do not yield accurate outputs due to misaligned internal processing. together, these biases highlight the challenges llms face in preserving the integrity and interpretability of temporal data across diverse formats and contexts. this paper makes two significant contributions to understanding temporal biases in llms. (1) we introduce datelogicqa, a dataset of 190 curated questions for evaluating temporal reasoning across various date formats, contexts (past, present, future), and reasoning types (commonsense, factual, conceptual, numerical). (2) we conduct human evaluations of model responses to analyse tokenization accuracy and reasoning quality, providing insights beyond automated metrics. we have organised the paper as follows: section 2 reviews related works, summarising the impact of tokenization on llm performance and past temporal reasoning approaches. section 3 details the creation of the datelogicqa dataset, including its design principles and examples. section 4 outlines methods for temporal reasoning, and biases. section 5 presents experiment results, followed by a discussion of findings and bias mitigation in section 6. lastly, section 7 summarises our contributions. ## datelogicqa we introduce datelogicqa, a dataset designed to explore how llms handle dates in various formats and contexts to tokenize, interpret, and reason with them. it consists of 190 questions divided into four categories: commonsense, factual, conceptual, and numerical. each category features one of seven date formats across three temporal contexts: past, present, and future. this systematic variation allows for an in-depth analysis of llms' performance with temporal information. objective and purpose the dataset aims to assess llms' tokenization and understanding of dates, as errors can lead to interpretative biases. by embedding dates within questions, we evaluate context-rich date interpretation, simulate realworld scenarios where dates carry contextual significance, and test llms' ability to extract and interpret date information accurately. ## discussion this study highlights the need for targeted strategies to address temporal biases in large language models (llms). a key step is to enhance pretraining datasets to ensure temporal diversity, incorporating historical, contemporary, and futuristic contexts. while resources like redpajama @xcite and @xcite are open source, researchers should develop data focused on temporal reasoning with varied formats and cultural contexts. post-training methods, such as direct preference optimization (dpo) @xcite , offer a promising avenue for fine-tuning models using curated datasets specifically designed to improve their logical temporal reasoning capabilities @xcite . these approaches can help align the models' outputs with humanpreferred logical reasoning patterns, addressing specific shortcomings in temporal tasks. additionally, retrieval-augmented generation (rag) @xcite enhances llms by integrating external knowledge dynamically during inference, allowing the models to access up-to-date or context-specific temporal information beyond their static training data. moreover, prompting techniques such as chain of thought (cot) prompting @xcite enable models to break down complex temporal reasoning tasks into incremental steps, improving interpretability and logical coherence @xcite . however, while these post-training methods significantly mitigate biases in temporal reasoning and improve model performance, they are not sufficient to completely eliminate inherent biases. factors such as the limitations of pre-trained embeddings, the static nature of foundational knowledge, and the variability in task-specific datasets mean that biases are likely to persist at some level. thus, post-training approaches should be viewed as an important step toward reducing biases. ## conclusion our paper addresses the challenges of temporal biases in large language models (llms) and proposes a structured approach to analyse their performance with temporal data. we introduced the date-logicqa dataset and the semantic integrity metric to evaluate the impact of diverse date formats and contexts on tokenization and reasoning. our findings highlighted representation-level biases, where temporal contexts are inconsistently encoded, and logical-level biases, evident in varying outputs for similar prompts. we suggest mitigation strategies, such as temporally balanced pretraining datasets, post training and prompting methods.