V erifi A gent: a Unified Verification Agent in Language Model Reasoning (2025)

Abstract: Large language models demonstrate remarkable reasoning capabilities but often produce unreliable or incorrect responses. Existing verification methods are typically model-specific or domain-restricted, requiring significant computational resources and lacking scalability across diverse reasoning tasks. To address these limitations, we propose VerifiAgent, a unified verification agent that integrates two levels of verification: meta-verification, which assesses completeness and consistency in model responses, and tool-based adaptive verification, where VerifiAgent autonomously selects appropriate verification tools based on the reasoning type, including mathematical, logical, or commonsense reasoning. This adaptive approach ensures both efficiency and robustness across different verification scenarios. Experimental results show that VerifiAgent outperforms baseline verification methods (e.g., deductive verifier, backward verifier) among all reasoning tasks. Additionally, it can further enhance reasoning accuracy by leveraging feedback from verification results. VerifiAgent can also be effectively applied to inference scaling, achieving better results with fewer generated samples and costs compared to existing process reward models in the mathematical reasoning domain.

## introduction large language models (llms) have demonstrated significant capabilities in natural language reasoning tasks, exhibiting potential to solve complex problems across diverse domains @xcite @xcite . however, despite their advanced reasoning abilities, these models often produce responses that are unreliable or incorrect, which poses substantial challenges for practical applications that require high precision and trustworthiness @xcite . to address this critical issue, several verification methods have been proposed, ranging from task-specific verifiers to generalised verification methods leveraging prompting techniques. training a task-specific verifier to verify the output of llm requires specific training data. for instance, @xcite train a verification model that judges language-to-code outputs based on both program text and execution results. @xcite design verifiers trained on outputs from multiple reasoning paradigms, using correctness signals for improvement. @xcite train a lightweight verifier for improving semantic graph generation in text-to-graph tasks. @xcite train a verifier that corrects potential syntactic and semantic first-order logic translation errors. nevertheless, these existing solutions typically face limitations such as domain restriction, computational inefficiency, and lack of scalability when handling varied reasoning tasks. in this paper, we propose verifiagent, a unified verification agent designed specifically to overcome these limitations by offering a generalisable and efficient verification framework. unlike prior methods, verifiagent adopts a two-layer verification mechanism, comprising meta-verification and tool-based adaptive verification. the metaverification layer ensures completeness and logical consistency of responses, while the tool-based adaptive verification autonomously selects appropriate external tools (e.g., python interpreters, symbolic solvers, search engines) to deal with different reasoning types, including mathematical, logical, commonsense, and hybrid reasoning tasks. our approach not only achieves superior verification accuracy compared to existing baseline methods, such as deductive verifier @xcite , backward verifier @xcite , but also enhances reasoning accuracy by integrating method training-free generalised tool-based fine-grained feedback diverse @xcite × × × × pive @xcite × × × ✓ math/code-rev @xcite × × ✓ × lever @xcite × × ✓ × cove @xcite ✓ × × ✓ csv @xcite ✓ × ✓ ✓ deductive verifier @xcite ✓ ✓ × × backward verifier @xcite ✓ ✓ × × verifiagent (ours) ✓ ✓ ✓ ✓ ## verfiagent verifiagent is a plug-and-play verification framework that empowers frozen llms to utilise external mechanisms to verify the correctness of solutions of diverse reasoning tasks. as illustrated in figure 1 , verifiagent adopts a two-layer verification mechanism, which contains two levels of verification. the first level is to do a meta verification, and the second level is to do a tool-based adaptive verification. the solution will be evaluated sequentially through the two layers. the secondlevel verification can further validate the results from meta-verification, enhancing the accuracy of the verification results. with this two-layer verification mechanism, the verifiagent can provide fine-grained feedback of the verification process. ## analysis verification score visualisation. figure 3 visualises the v score for correct and incorrect solutions across three datasets. as illustrated, the mean v score for correct solutions is slightly higher than for incorrect ones on all the reasoning tasks. since v score represents the confidence of the verification result, this indicates that the verifiagent is more confident when identifying correct solutions compared to incorrect ones. additionally, the consistently lower variance in v score among correct solutions further supports the reliability of the agent in verifying correct responses. tool usage analysis. verifiagent autonomously determines the reasoning type of a task and selects the appropriate tool for verification. figure 4 illustrates tool usage across four reasoning task types. for the math dataset (mathematical), the python interpreter is predominantly used (98.6%), with minimal reliance on the symbolic solver (1.4%), reflecting the computational nature of the task. strategyqa (commonsense) exclusively relies on the search engine (100%), highlighting its dependence on external knowledge for the verification. proverqa (logical) solely utilises the symbolic solver (100%), aligning with its need for logical and symbolic reasoning. rewild (hybrid) shows a more balanced tool distribution, primarily using the python interpreter (84.5%), supplemented by the symbolic solver (11.2%) and the search engine (4.3%). the results demonstrate that the verifiagent effectively selects appropriate external tools based on the nature of the reasoning task. error analysis. to further investigate the capability of verifiagent, we conducted an error analysis on different types of questions. the math dataset contains seven types of math problems: al- we used the number of decomposed sub-questions (ranging from 1 to 5) as an indicator of question difficulty, classifying them into five levels (level 1 through level 5). figure 5 illustrates the distribution of question types among cases where verifiagent provided incorrect verifications. in the math dataset, precalculus and geometry questions accounted for the highest proportion of errors, suggesting these question types pose greater verification challenges for verifiagent. errors in counting & probability, prealgebra, intermediate algebra, and algebra occurred at similar rates, while verifiagent performed best on number theory problems. this trend is in line with the capability of the backbone llms. see appendix b for the error distributions on each type of question. for proverqa, verifiagent's verification accuracy correlated clearly with question difficulty, making the highest number of errors on hard questions and the fewest on easy questions. conversely, no clear error pattern emerged for strategyqa. since verification relies mainly on search engines to retrieve factual knowledge, verifiagent appears capable of accessing sufficient information irrespective of question difficulty, indicating that the complexity of questions in strategyqa has minimal impact on verification performance. ## conclusion in this paper, we introduced verifiagent, a unified verification agent that verifies and improves outputs from llms across mathematical, logical, commonsense, and hybrid reasoning tasks. ver-ifiagent employs a two-layer verification framework combining meta-verification, which assesses completeness and consistency, and adaptive toolbased verification tailored to each reasoning type. experimental results demonstrate that verifiagent consistently outperforms baseline methods in verification accuracy. additionally, verifiagent can be integrated with inference scaling approaches, achieving improved performance with fewer samples than prms. overall, verifiagent provides an efficient and scalable solution, enhancing the reliability and trustworthiness of large language model reasoning. verifiagent heavily relies on the instructionfollowing capabilities of the backbone llm, meaning that only models proficient at accurately interpreting and executing instructions can serve effectively as the backbone. this reliance indicates the importance of selecting suitable backbone llms to ensure optimal performance. verifiagent currently supports only three verification tools (python interpreter, search engine, and symbolic solver). expanding its capabilities by integrating additional verification tools could further enhance verifiagent's adaptability and effectiveness across a broader range of reasoning scenarios.