NeurIPS 2025

Incentivizing LLMs to Self-Verify Their Answers
Fuxiang Zhang1,2
Jiacheng Xu1,2
Chaojie Wang2
Ce Cui2
Yang Liu2
Bo An1,2∗
1 Nanyang Technological University, Singapore
2 Skywork AI
Abstract
Large Language Models (LLMs) have demonstrated remarkable progress in com-
plex reasoning tasks through both post-training and test-time scaling laws. While
prevalent test-time scaling approaches are often realized by using external reward
models to guide the model generation process, we find that only marginal gains
can be acquired when scaling a model post-trained on specific reasoning tasks.
We identify that the limited improvement stems from distribution discrepancies
between the specific post-trained generator and the general reward model. To
address this, we propose a framework that incentivizes LLMs to self-verify their
own answers. By unifying answer generation and verification within a single
reinforcement learning (RL) process, we train models that can effectively assess
the correctness of their own solutions. The trained model can further scale its
performance at inference time by verifying its generations, without the need for
external verifiers. We train our self-verification models based on Qwen2.5-Math-
7B and DeepSeek-R1-Distill-Qwen-1.5B, demonstrating their capabilities across
varying reasoning context lengths. Experiments on multiple mathematical rea-
soning benchmarks show that our models can not only improve post-training
performance but also enable effective test-time scaling. Our code is available at
https://github.com/mansicer/self-verification.
MATH500 Score (avg@1)
Base Model: Qwen2.5-Math-7B
AIME24 Score (avg@10)
Base Model: DeepSeek-R1-Distill-Qwen-1.5B
Figure 1: Average performance of post-trained models (dotted lines) and test-time scaling methods
(solid lines) on the MATH500 and AIME24 benchmarks. Our self-verification framework not only
enhances post-training performance with RL on both problem-solving and verification, but also
enables effective t...