Temporal Information Retrieval via Time-Specifier Model Merging (2025)

Abstract: The rapid expansion of digital information and knowledge across structured and unstructured sources has heightened the importance of Information Retrieval (IR). While dense retrieval methods have substantially improved semantic matching for general queries, they consistently underperform on queries with explicit temporal constraints–often those containing numerical expressions and time specifiers such as “in 2015.” Existing approaches to Temporal Information Retrieval (TIR) improve temporal reasoning but often suffer from catastrophic forgetting, leading to reduced performance on non-temporal queries. To address this, we propose Time-Specifier Model Merging (TSM), a novel method that enhances temporal retrieval while preserving accuracy on non-temporal queries. TSM trains specialized retrievers for individual time specifiers and merges them into a unified model, enabling precise handling of temporal constraints without compromising non-temporal retrieval. Extensive experiments on both temporal and non-temporal datasets demonstrate that TSM significantly improves performance on temporally constrained queries while maintaining strong results on non-temporal queries, consistently outperforming other training methods. Our code is available at https://github.com/seungyoonee/TSM.

## introduction in the contemporary era of digital information, information retrieval (ir)-the process of finding and ranking documents from a large collection that are most relevant to a search query-has become increasingly important as information and knowledge rapidly expand across both structured sources (e.g., knowledge bases) @xcite and unstructured sources (e.g., wikipedia, web documents) @xcite . this significance is more amplified in the era of * corresponding author large language models (llms), where ir is a crucial component of retrieval-augmented generation (rag) @xcite pipelines. as the importance of ir continues to grow, there have been significant advances in retrieval methods, notably the development of dense retrieval methods @xcite . dense retrieval leverages neural models to encode both queries and documents into dense embeddings to capture semantic similarity, substantially improving retrieval effectiveness for generaldomain queries. however, these models exhibit attention bias, where their embeddings are optimized primarily for semantic similarity and topical relevance, making them less effective at capturing temporal expressions in queries @xcite . as a result, dense retrievers struggle with queries containing temporal expressions (e.g., "in 2015," "between 2010 and 2012") @xcite . to address these challenges, the field of temporal information retrieval (tir) has emerged, focusing on improving retrieval accuracy for temporal queries by enhancing temporal understanding capabilities of retrievers @xcite . recent research has attempted to increase the time-awareness of dense models from the pre-training process using different temporal information masking @xcite @xcite , fine-tuning process @xcite @xcite . by incorporating temporal awareness, tir aims to enhance the accuracy and relevance of retrieved documents for temporal queries. previous studies have primarily focused on improving retrieval performance for temporal queries, often overlooking the resulting performance drop on non-temporal queries. however, while enhancing temporal retrieval capabilities is important, it is equally crucial to maintain robust performance on non-temporal queries. this is because both tem-poral and non-temporal queries are fundamentally part of general-domain information retrieval and do not require domain-specific knowledge. unlike domain-specific retrieval tasks that target specialized topics, temporal queries remain general in scope, with their distinction based solely on the presence of explicit time constraints-typically signaled by time specifiers such as "in," "after," or "between." accordingly, this paper treats temporal queries as a subset of general queries with explicit time constraints, while non-temporal queries lack such time specifiers. this distinction highlights the need for retrieval models that can flexibly and effectively handle both query types without sacrificing overall performance. despite this need for balanced retrieval capabilities, fine-tuning dense models to improve accuracy on temporal queries often comes at a significant cost: a noticeable decline in performance on general, non-temporal queries, primarily due to catastrophic forgetting @xcite . for instance, as illustrated in figure 1 , fine-tuning contriever @xcite on timeqa @xcite enhances temporal retrieval but substantially reduces performance on the general-domain dataset natural questions (nq) @xcite . to address this issue, @xcite and @xcite proposed a routing-based method that directs temporal queries to a temporally fine-tuned retriever and non-temporal queries to a vanilla retriever, which helps mitigate catastrophic forgetting. however, this approach requires maintaining and operating two separate dense retrievers models, resulting in an increased memory usage, which can be resource-intensive in practical deployments. furthermore, while this method helps preserve performance across both query types, it heavily relies on accurate classification of queries as temporal or non-temporal, which can result in suboptimal retrieval accuracy, as shown in to address the challenge of handling both temporal and non-temporal queries, we propose time-specifier model merging (tsm), a novel temporal fine-tuning method. tsm involves separately training specialized retrievers on data subsets corresponding to specific time specifiers (e.g., "in," "after," "between") for temporal queries with explicit expressions. each retriever develops expertise in a particular temporal constraint. we then merge these specialized models by simply averaging their parameters, allowing the unified retriever to inherit the specialized performance of each time-specifierspecific model. this merging process is effective at mitigating catastrophic forgetting because it results in lowermagnitude weight changes-preserving knowledge from both temporal and non-temporal data, rather than overwriting it as in standard fine-tuning @xcite . as a result, the merged model can more effectively encode temporal relevance associated with each time specifier while still maintaining strong performance on non-temporal queries. extensive experiments on both temporal and non-temporal datasets demonstrate that tsm significantly improves performance on temporal queries while preserving performance on non-temporal datasets. tsm consistently outperforms alternative temporally-aware training methods, including full fine-tuning, regularization, lora, routing, and ensembling. to summarize, our contributions are threefold: another approach is ensemble learning, which combines the predictions of multiple models-each specialized for different tasks or domains -to achieve balanced performance @xcite @xcite . however, this approach requires running multiple models simultaneously, increasing both memory usage and inference costs. routing-based methods have also been proposed, dynamically directing queries to either a fine-tuned or the vanilla model based on the query type @xcite . while routing leverages the strengths of both specialized and general models, its effectiveness depends on accurate query classification and still requires maintaining multiple models, making it resource-intensive in practice. model merging has recently emerged as a simple and effective approach to mitigating catastrophic forgetting by flattening high-magnitude weight changes during adaptation, resulting in more stable and higher-quality parameter updates @xcite . motivated by these findings, we adopt model merging in this study and propose a novel temporal fine-tuning method. our method fine-tunes specialized retrievers for individual time specifiers and merges them into a unified model, enabling effective retrieval for both temporal and non-temporal queries. ## method we define the temporal and non-temporal retrieval problem and introduce out method, time-specifier model merging (tsm). ## main results on temporal datasets, tsm achieves the strongest performance across all metrics for both contriever and dpr. for example, on timeqa, tsm with contriever achieves substantial improvements over the vanilla retriever. similarly, on the nobel prize dataset-which serves as an out-ofdomain temporal test set-tsm achieves the best performance for both unsupervised and supervised retrievers, confirming its strong generalization to unseen temporal data. although ensembling yields a marginally higher ndcg@5 on nobel prize, tsm remains the most robust performer overall. on non-temporal datasets, tsm also maintains competitive performance, achieving the strongest results across most metrics with both contriever and dpr. on nq, where dpr is trained in-domain, vanilla dpr achieves the highest recall and ndcg. however, dpr-based tsm performs most closely to vanilla dpr on recall@5/20 and ndcg@5/20, while outperforming ft, lora, and ensembling. on contriever, which is not trained in-domain, tsm significantly improves retrieval effectiveness. on ms marco, which is out-of-domain for both contriever and dpr, tsm achieves highly competitive performance. for contriever, it matches or exceeds other baselines on recall@5 and ndcg@5, and trails slightly behind vanilla and router on recall@20 and ndcg@20. similarly, for dpr, tsm outperforms all other methods across all retrieval metrics. this competitive performance on non-temporal datasets can be attributed to tsm's model merging approach, which reduces the magnitude of weight changes during fine-tuning and helps to preserve non-temporal retrieval capabilities while integrating temporal expertise. overall, the average results for both contrieverbased and dpr-based tsm show that tsm consistently outperforms other baselines. these results demonstrate that tsm significantly improves temporal retrieval performance without sacrificing effectiveness on non-temporal queries. ## analyses in this section, we systematically examine the effectiveness and underlying mechanisms of our proposed approach. 40 50 60 70 80 90 recall@20 from to in between after before in early in late time specifier sampled timeqa dataset contriever ft + reg ensembling tsm (ours) ft ft + reg lora ensembling tsm methods 0 50 100 150 200 250 300 magnitude total parameter weight change contriever dpr figure 3: left: recall@20 for each time specifier on the timeqa test set, comparing vanilla contriever, ft + reg, ensembling, and tsm (ours). right: total parameter weight change after fine-tuning for each method, showing how much all network weights are updated. lower values indicate more stable parameter adaptation. ## conclusion this work addresses the challenge of balancing temporal and non-temporal information retrieval by introducing time-specifier model merging (tsm), a method designed to address attention bias and catastrophic forgetting. tsm trains specialized retrievers for each time specifier and merges them into a unified model. experiments on both temporal and non-temporal datasets demonstrate that tsm substantially improves performance on temporally constrained queries while maintaining strong performance on non-temporal queries. our analysis further show that tsm effectively integrates temporal and non-temporal information, mitigating attention bias and outperforming other baselines. these results establish tsm as a robust and efficient solution for diverse information retrieval tasks.