Title: Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations
Primary area: foundation or frontier models, including LLMs
Keywords: hallucination, factuality, reinforcement learning, retrieval-augmented generation

Abstract: Language models frequently generate factually incorrect information with high confidence, a phenomenon known as extrinsic hallucination. Existing approaches for improving factuality often come at the cost of diminished performance on other downstream tasks, limiting their practical deployment.
We propose a novel on-policy reinforcement learning (RL) approach that uses binary retrieval-augmented rewards (RAR) to address this challenge. Our binary reward scheme assigns a reward of zero whenever any factual error is detected and one otherwise. 
We evaluate our method through continual RL from Qwen3 models across multiple tasks. For open-ended generation, binary RAR achieves a 39.3\% reduction in hallucination rates, significantly outperforming supervised training or online RL with dense reward. 
In short-form settings, models learn calibrated abstention, answering ``I don’t know'' when parametric knowledge is insufficient, leading to 44.4\% and 21.7\% fewer incorrect answers on PopQA and GPQA, respectively. 
Crucially, these factuality gains come without performance degradation on instruction following, math, or code, whereas dense-reward RL, despite improving factuality, induces quality regressions.