T ic T ac: Time-aware Supervised Fine-tuning for Automatic Text Dating (2025)

Abstract: Pre-trained langauge models have achieved success in many natural language processing tasks, whereas they are trapped by the time-agnostic setting, impacting the performance in automatic text dating. This paper introduces TicTac, a supervised fine-tuning model for automatic text dating. Unlike the existing models that always ignore the temporal relatedness of documents, TicTac has the ability to learn temporal semantic information, which is helpful for capturing the temporal implications over long-time span corpora. As a fine-tuning framework, TicTac employs a contrastive learning-based approach to model two types of temporal relations of diachronic documents. TicTac also adopts a metric learning approach, where the temporal distance between a historical text and its category label is estimated, which benefits to learn temporal semantic information on texts with temporal ordering. Experiments on two diachronic corpora show that our model effectively captures the temporal semantic information and outperforms state-of-the-art baselines.

## introduction the temporal dimension of texts is essential to many time-sensitive tasks, such as question answering @xcite , natural language inference @xcite , event detection @xcite , and text generation @xcite . time-based reasoning in these tasks benefits from timestamps or temporal metadata of documents, which, however, are not always available @xcite . one way to tackle this issue is automatic text dating (atd), which refers to the task of identifying when a document was written according to its content @xcite . a straightforward way of atd is to find temporal clues within texts. for example, an article with a sentence the novel was published in 2021 suggests that it was written no earlier than 2021. however, it is more challenging when explicit temporal mentions do not appear. to address it, research on atd focuses on representation learning, trying to find temporal clues from diachronic changes in language. this work has a wide range of application scenarios in the digital humanities @xcite . one example is to identify the dates of historical documents in digital libraries where temporal metadata are missing, which is also known as historical text dating @xcite . current studies on historical text dating usually adopt pre-trained language models (plms) to representation learning @xcite . although such models have achieved success in many natural language processing tasks, they are trapped by the time-agnostic setting, impacting the performance in time-sensitive downstream tasks, including text dating @xcite . to address it, recent efforts on diachronic text modeling either train large language models on diachronic corpora @xcite , trying to improve their performances on time-sensitive tasks, or employ sophisticated learning models to capture lexical semantic changes over time @xcite . however, the prominent limitation of the former is the limited task data, high training costs, and the risk of catastrophic forgetting, while the latter ignores the temporal relatedness of diachronic documents, which may be helpful for capturing temporal semantic variations. besides, temporal categories in most studies are viewed as independent from each other rather than sequentially related, impacting the ability of temporalbased text modeling. in this paper, we propose tictac, a supervised fine-tuning model for atd. this work attempts to tackle three aforementioned issues in current studies: 1) how to develop a method to learn temporal semantic information at low training costs; 2) how to capture language evolution features based on temporal-related documents and; 3) how to model the sequential relatedness of temporal categories for atd. to this end, our model firstly adopts a supervised fine-tuning framework to learn from diachronic documents, which avoids high computational costs caused by training from scratch and offers boarder applicability to downstream tasks as well. secondly, we propose a contrastive learningbased diachronic document modeling approach, trying to capture temporal implications in temporalrelated documents. for example, two documents discussing different technological advances of spinning machines and the internet suggest that they may be written in different eras. specifically, we define two types of temporal relations: one is relative temporal relations between documents, and the other is absolute temporal relations between documents and temporal category labels. this idea contributes to a better understanding of the relationship between time and language evolution. thirdly, we treat atd as an ordinal classification task and propose a corresponding approach to improve the performances by appropriately correcting prediction errors of different degrees. in this approach, the ordinal relationship between temporal categories is considered, which helps to promote evaluation accuracy and thus benefits the modeling of diachronic documents with temporal ordering. our contributions are summarized as follows: ## conclusion in this paper, we propose tictac, a novel approach to the atd task that integrates time-aware contrastive learning and time-aware ordinal classification to jointly model ordinally relative and absolute temporal relationships in long time-span texts. experiments on two diachronic corpora show that tic-tac outperforms state-of-the-art baselines, demonstrating its effectiveness in capturing temporal implications. tictac also shows the limitations on short time-span texts, and further evaluation on multilingual datasets is needed to assess its generalization ability.