MRAG : A Modular Retrieval Framework for Time-Sensitive Question Answering (2025)

Abstract: Understanding temporal concepts and answering time-sensitive questions is crucial yet a challenging task for question-answering systems powered by large language models (LLMs). Existing approaches either update the parametric knowledge of LLMs with new facts, which is resource-intensive and often impractical, or integrate LLMs with external knowledge retrieval (i.e., retrieval-augmented generation). However, off-the-shelf retrievers often struggle to identify relevant documents that require intensive temporal reasoning. To systematically study time-sensitive question answering, we introduce the TempRAGEval benchmark, which repurposes existing datasets by incorporating complex temporal perturbations and gold evidence labels. As anticipated, all existing retrieval methods struggle with these temporal reasoning-intensive questions. We further propose Modular Retrieval (MRAG), a trainless framework that includes three modules: (1) Question Processing that decomposes question into a main content and a temporal constraint; (2) Retrieval and Summarization that retrieves, splits, and summarize evidence passages based on the main content; (3) Semantic-Temporal Hybrid Ranking that scores semantic and temporal relevance separately for each fine-grained evidence. On TempRAGEval, MRAG significantly outperforms baseline retrievers in retrieval performance, leading to further improvements in final answer accuracy.

## introduction facts are constantly evolving in our ever-changing world. this dynamic nature highlights the need for natural language processing (nlp) systems capable of updating information @xcite @xcite and providing accurate responses to time-sensitive questions (chen et @xcite . for instance, a common query like "who is the uk prime minister?" sees the answer transition from "boris johnson" to "rishi sunak" in 2022 (figure 1 ). with developments of large language models (llms), existing approaches rely on the parametric knowledge of llms to answer time-sensitive questions directly, and constantly update the parametric knowledge on new facts @xcite @xcite . however, updating llm parameters are often resource-intensive. an alternative line of research explores retrieval-augmented generation (rag), which integrates llms with external knowledge (e.g., wikipedia) through information retrieval @xcite . while rag allows for the incorporation of new facts with minimal effort, its performance heavily relies on off-the-shelf retrieval systems, which are often limited to keywords or semantic matching. time-sensitive questions, however, often require intensive temporal reasoning to identify relevant documents, i.e., reasoning-intensive retrieval @xcite . for example, in figure 1 , retrievers should infer the date "24 july 2019" as relevant to the constraint "as of 6 may 2021", rather than only match with date like "may 2021". although temporal qa is a well-established domain, there remains a lack of research for temporal reasoningintensive retrieval systems. we fill that lacuna. we begin by conducting a diagnostic evaluation of existing retrieval approaches for temporal reasoning-intensive retrieval. following the idea of systematic evaluation with contrast set @xcite , we repurpose two existing datasets, timeqa @xcite and situ-atedqa @xcite , to introduce the temporal qa for rag evaluation benchmark (temprageval). we manually augment the test questions with complex temporal perturbations (e.g., modifying the time period to avoid textual overlap). in addition, we annotate gold evidence on wikipedia for more accurate retrieval evaluation. we observe remarkable degradation of retrieval performance, which points out a serious robustness issue for existing retrievers. to address this issue, we propose a training-free modular retrieval framework (mrag) to enhance temporal reasoning-intensive retrieval. complementary to recent agentic rag methods such as r1-searcher and search-o1 @xcite , mrag provides an improved retrieval component and reduces the number of invocations for retrieval when handling complex temporal questions. specifically, mrag contains three key modules: (1) question processing, which decomposes each question into a main content and a temporal constraint; (2) retrieval and summarization, which utilizes off-the-shelf retrievers to find evidence passages based on the main content, segments them into independent sentences, and guides llms to condense the most relevant passages into query-specific sentences.(3) semantic-temporal hybrid ranking, which ranks each evidence sentence using a combination of a semantic score measuring semantic similarity, and a temporal score, a novel symbolic component that assesses temporal relevance to the query's temporal constraint. on temprageval, our proposed mrag framework achieves substantial improvements in performance, with 9.3% top-1 answer recall and 11% top-1 evidence recall. we also incorporate state-of-the-art (sota) answer generators @xcite @xcite , and demonstrate that the improvements in retrieval from mrag propagate to enhanced final qa accuracy, with 4.5% for both exact match and f1. detailed case studies further confirms mrag's robustness to temporal perturbations qualitatively. our contributions can be summarized as follows. ## background in this section, we first define the time-sensitive question answering task ( §2.1), and then introduce the baseline retrieval-augmented generation qa based systems ( §2.2). ## temprageval benchmark in this section, we first present existing timesensitive qa datasets ( §3.1), then introduce our diagnostic benchmark dataset temprageval ( §3.2), finally we evaluate existing retrieval approaches on temprageval ( §3.3). ## mrag: modular retrieval motivated by the performance degradation of the existing retrieval methods in the preliminary evaluation, we propose a modular retrieval (mrag) framework (as shown in figure 3 ) to enhance temporal reasoning-intensive retrieval. at a high level, mrag disentangles relevance-based retrieval from temporal reasoning, leveraging a dense embedding model for semantic scoring and a set of symbolic heuristics for temporal scoring. specifically, mrag has three key modules: question processing, retrieval and summarization, and semantictemporal hybrid ranking. question processing. we prompt llms to decompose each time-sensitive question into a main content (mc) and a temporal constraint (tc). this approach disentangles temporal relevance from semantic relevance: mc measures the semantic relevance of the evidence, while tc determines its temporal relevance. ## experiments in this section, we evaluate mrag and baseline systems on temprageval. ## analysis this section presents a detailed analysis of the results and the contribution of each mrag module. ## conclusion this study focuses on time-sensitive qa, a task that challenges llm based qa systems. we first present temprageval, a diagnostic benchmark featuring natural questions, evidence annotations, and temporal complexity. we further propose a training-free mrag framework, which disentangles relevance-based retrieval from temporal reasoning and introduces a symbolic temporal scoring mechanism. while existing systems struggle on temprageval due to limited temporal reasoning capacities in retrieval, mrag shows significant improvements. we hope this work advances future research on reasoning-intensive retrieval.