Think Right, Not More: Test-Time Scaling for Numerical Claim Verification (2025)

Abstract: Fact-checking real-world claims, particularly numerical claims, is inherently complex that require multistep reasoning and numerical reasoning for verifying diverse aspects of the claim. Although large language models (LLMs) including reasoning models have made tremendous advances, they still fall short on fact-checking real-world claims that require a combination of compositional and numerical reasoning. They are unable to understand nuance of numerical aspects, and are also susceptible to the reasoning drift issue, where the model is unable to contextualize diverse information resulting in misinterpretation and backtracking of reasoning process. In this work, we systematically explore scaling test-time compute (TTS) for LLMs on the task of fact-checking complex numerical claims, which entails eliciting multiple reasoning paths from an LLM. We train a verifier model (VERIFIERFC) to navigate this space of possible reasoning paths and select one that could lead to the correct verdict. We observe that TTS helps mitigate the reasoning drift issue, leading to significant performance gains for fact-checking numerical claims. To improve compute efficiency in TTS, we introduce an adaptive mechanism that performs TTS selectively based on the perceived complexity of the claim. This approach achieves 1.8x higher efficiency than standard TTS, while delivering a notable 18.8% performance improvement over single-shot claim verification methods. Our code and data can be found at https://github.com/VenkteshV/VerifierFC

## introduction automated verification of claims is critical to mitigate misinformation and disinformation. while many approaches focus on synthetically generated claims from wikipedia @xcite ; thorne * both authors contributed equally to this work mitigating reasoning drift for numerical fc claim: diaspora remittances "now stand at an all-time high of over ksh400 billion in 2021". real-world claims contain numerical (quantitative and temporal) elements @xcite and verifying such claims necessitates systematically comparing numerical aspects within the claim against corresponding evidence provided. consider the example in figure 1 , where verifying the claim about diaspora remittances requires ensuring that the claimed amount (ksh400 billion) and the specified year (2021) align correctly with the evidence, even when the evidence presents the amount in a different currency (usd). unlike binary verification tasks, real-world claims often contain contradictory or partially correct components, making verification more complex. standard automated fact-checking approaches employ a multistage pipeline involving fine-tuned nli models. recently, large language models (llms) have emerged as promising alternatives to nli-based methods for claim verification @xcite . however, these approaches have been evaluated mainly in synthetic claims, and recent studies highlight significant shortcomings when applied to complex claims in the real world that involve numerical data @xcite @xcite . although prior work has acknowledged these limitations, they have not thoroughly analyzed why llms fail in realistic scenarios. to address this gap, we analyze failure modes in both open-source and proprietary models of different sizes. beyond noise in retrieval, we identify a critical reasoning challenge even in the presence of relevant evidence introduced specifically due to multi-aspect or inability to contextualize diverse information from evidence sources, termed reasoning drift due to issues in its ability for accurate analysis of claims and evidence. this differs fundamentally from the "overthinking" phenomenon observed in simpler q&a contexts, where models unnecessarily elaborate on obvious details @xcite . while it can impact simple claims, reasoning drift is especially pronounced in complex cases with multiple aspects and diverse evidences, leading to unfocused reasoning that strays from the core verification task. our analysis reveals that approximately 34% of claims in the quantemp dataset suffer from this reasoning problem. in the context of fact-checking, reasoning drift stems from llms focusing on unimportant aspects of the claim or misinterpretation of information in diverse evidences as part of its reasoning process. for instance, in the example in figure 1 the claim mentions an amount of 400 billion ksh (kenyan shillings). while one of the evidences confirm this figure, the other evidence shown in the example quotes an amount of 3718 mil. usd. though this is equivalent to 400 billion ksh, the llm interprets this quantity as not being equivalent to the one mentioned in the claim and falls short by billions, predicting its verdict as "conflicting". to our knowledge, our work is the first to study the prob-lem reasoning drift occuring due to overanalysis and explore test-time scaling as a solution to the same in the context of fact checking of real-world claims. more formally reasoning drift due to overanalysis can be defined as the phenomenon in which: we introduce and investigate test-time scaling (tts) @xcite , an approach that allocates additional computational resources at inference time, enabling llms to explore multiple reasoning pathways to mitigate reasoning drift. to our knowledge, this is the first study to examine test-time scaling specifically for verifying complex factual claims. to effectively navigate and select from diverse reasoning paths, we train a dedicated verifier model (verifierfc). our experiments demonstrate that employing this verifier with a best-of-n (bon) strategy significantly outperforms standard majority-voting @xcite methods. since real-world claims are diverse, applying a fixed uniform computational effort across all claims is inefficient: simpler claims may consume unnecessary resources, and complex claims may not receive sufficient computational attention, resulting in suboptimal verification. to address this issue, we propose a novel adaptive tts strategy that dynamically adjusts computational resources based on claim complexity, determined by analyzing layerwise latent representations within the llms. this adaptive method boosts verification performance by 18.8% on complex numerical claims (quantemp) and 21.27% on compositional claims (claimdecomp) compared to single-shot techniques, while also delivering 1.8x greater computational efficiency than the standard tts approach. in this paper, we answer the following research questions through empirical validation: rq1: to what extent llms suffer from reasoning drift issue for complex numerical fact-checking tasks? rq2: how does test-time scaling impact the reasoning performance of llms for fact-checking complex numerical claims? rq3: how does test-time scaling compare to current strategies for fact-checking complex claims? ## methodology in this section, we present a test-time scaling (tts) strategy aimed at mitigating reasoning drift, a reasoning failure wherein the model fails to prioritize relevant information or misconstrues aspects of the claim during inference. our approach involves training a process verifier (section 3.1) to select more accurate reasoning chains from multiple candidates. to further reduce unnecessary compute on straightforward claims, we explore an adaptive tts mechanism that predicts claim complexity using layer-wise latent representations from the base llm (section 3.2). we adopt a standard factchecking pipeline of bm25 retrieval + re-ranking to extract evidences for claim and employ llms for claim verification instead of standard nli models. ## experimental setup datasets. we evaluate our methods on @xcite , which is the first benchmark for complex numerical claims. it comprises 9935 training, 3084 validation and 2495 test claims. we employ their training set to train the verifier model and create representations for the complexity classes, which are used for adaptive tts. claimdecomp @xcite is another benchmark dataset that focuses on compositional real-world claims. we use its evaluation set (200 claims) to test the generalization capabilities of our verifier (trained on quantemp) to other types of complex claims. fact-checking pipeline: we use a standard factchecking pipeline as adopted in @xcite which comprises an initial retrieval of top-100 results for each claim using bm25 (elasticsearch). we then re-rank using paraphrase-minilm-l6-v2 @xcite to select top-3 evidence snippets for each claim. metrics: we use per-class f1 scores, macro and weighted f1 score as evaluation metrics. llms used for claim verification: we use llama 3.1 (8b) and gpt-4o-mini as llms for claim verification stage and deepseek-r1 which internally does tts as baseline. we employ a temperature of 0.45 for generating diverse reasoning paths in tts setting. we use few-shot chain of thought (cot) prompting (in figure 10 ) for eliciting reasoning for all baselines and our approach. we experiment with different number of reasoning paths (@xmath0,3,5,10) as elaborated in section 5. verifier model for tts: the verifier model used for best-of-n in tts setting is based on the llama-3.2-3b model. it is fine-tuned for classification using lora adapters @xcite with a rank of 8 and a scaling factor (α) of 16. the model is fine-tuned for 3 epochs with a batch size of 32, using the adamw optimizer with ϵ = 1 × 10 -8 and an initial learning rate of 1 × 10 -3 . prompts: all prompts for claim verification, decomposition and llm based annotations can be found in appendix c. ## conclusion we identify and study the reasoning drift issue in llm reasoning for complex and numerical claims. to address this, we propose a best-of-n (bon) test-time scaling strategy, which outperforms self-consistency and top-1 decoding. for improved compute-efficiency, we introduce adaptive bon (verifierfc). verifierfc achieves better performance than bon while reducing computational cost. in future work, we aim to incorporate stronger verifier models and further optimize verifierfc for performance and efficiency.