Title: Med-REFL: Enhancing Complex Reasoning via Fine-grained Self-Correction
Primary area: foundation or frontier models, including LLMs
Keywords: Large Language Models, Self-Correction, Reflection Learning, Medical Reasoning, LLM Reasoning, Reasoning, Tree of Thoughts, healthcare applications, medical QA

Abstract: Large reasoning models excel in domains like mathematics where intermediate reasoning is straightforward to verify, but struggle to self-correct in medicine fields where evaluating intermediate reasoning is cumbersome and expensive. This verification bottleneck hinders the development of reliable AI reasoners for high-stakes application. Here we propose Med-REFL, a novel framework that learns fine-grained reflection without human labels or model distillation. Med-REFL introduces a deterministic structural assessment of the reasoning space to automatically generate preference data for reflection. By globally evaluating all explored reasoning paths in a tree-of-thoughts, our method quantifies the value of corrective actions, enabling the automated construction of  direct preference optimization pairs. This trains the model to recognize and amend its own reasoning fallacies.  Extensive experiments show Med-REFL delivers robust gains across diverse models architectures and medical benchmarks, boosting a general-purpose Llama3.1-8B by +5.82\% and the state-of-the-art Huatuo-o1 by +4.13\% on the MedQA benchmark. Our Med-REFL-8B achieves state-of-the-art performance among 7-8B models while even competing with models twice its size. Crucially, targeted ablations prove its success stems from instilling a genuine, prompt-activatable self-correction capability, rather than superficial mimicry. 
Ultimately, our framework provides a scalable solution to the verification bottleneck, paving the way for more reliable AI reasoners in high-stakes domains like medicine. Med-REFL has been made publicly available.

TL;DR: Med-REFL enhances LLM reasoning in domains like medicine where step-verification is costly. It deterministically scores reasoning steps to automatically generate preference data, teaching models to effectively self-correct.