Towards Medical Complex Reasoning with LLM s through Medical Verifiable Problems (2025)

Abstract: The breakthrough of OpenAI o1 highlights the potential of enhancing reasoning to improve LLM. Yet, most research in reasoning has focused on mathematical tasks, leaving domains like medicine underexplored. The medical domain, though distinct from mathematics, also demands robust reasoning to provide reliable answers, given the high standards of healthcare. However, verifying medical reasoning is challenging, unlike those in mathematics. To address this, we propose **Medical Verifiable Problems** with a medical verifier to check the correctness of model outputs. This verifiable nature enables advancements in medical reasoning through **a two-stage approach**: (1) using the verifier to guide the search for a complex reasoning trajectory for fine-tuning LLMs, (2) applying reinforcement learning (RL) with verifier-based rewards to enhance complex reasoning further. Finally, we introduce HuatuoGPT-o1, a medical LLM capable of complex reasoning, which outperforms general and medical-specific baselines using only 40K verifiable problems. Experiments show complex reasoning improves medical problem-solving and benefits more from RL. We hope our approach inspires advancements in reasoning across medical and other specialized domains. Code, datasets, and models are publicly available at https://github.com/FreedomIntelligence/HuatuoGPT-o1.

## introduction the release of openai o1 has marked a significant milestone in large language model (llm) development, showcasing impressive capabilities @xcite @xcite . this breakthrough highlights the potential of scaling chain-of-thought (cot) and reinforcement learning to enhance llm performance @xcite @xcite . while subsequent research efforts attempt to replicate these advancements, they often remain limited to mathematical reasoning tasks @xcite @xcite . the application of o1-like methods to specialized fields, such as medicine, remains largely underexplored. medical tasks often involve deeper reasoning @xcite @xcite . in real-world medical diagnoses or decisions, doctors often deliberate carefully. such a life-critical field necessitates meticulous thinking to ensure more reliable answers @xcite . thus, enabling llms to perform extended reasoning and reflection to provide more reliable medical responses holds significant value for the future applications of llms in healthcare. we term this extended and reflective thinking process as complex reasoning @xcite . moreover, medical reasoning closely resembles real-world applications in domains like finance, law and education, making advancements in this area readily transferable @xcite . however, a key challenge for medical reasoning is verifying the thought process, which often lacks clear steps. inspired by mathematical problems that allow verification through their outcomes, we construct 40k medical verifiable problems reformatted from challenging, closed-set medical exam questions. these verifiable problems are characterized as open-ended with unique, objective groundtruth answers that allow an llm verifier to check solution correctness. such verifiability enable a two-stage method for medical complex reasoning: stage 1: learning from verified reasoning trajectories the verifier feedback can guide llms to search for a long cot with reflection. the llm first initializes a cot and an answer. when the verifier rejects the answer, the llm extends by applying a strategy sampled from backtracking, medical verifiable problem (x): a 30-year-old woman recently traveled to india and now presents with shaking, chills, fevers, headaches, pallor, and jaundiced sclera. vital signs: temp 38.9°c, rr 19/min, bp 120/80 mm hg, pulse 94/min. labs: hct 30%, total bilirubin 2.6 mg/dl, direct bilirubin 0.3 mg/dl. what is the most severe complication of this condition? exploring new paths, verification, and correction to refine its answer until it reaches a correct answer. successful trajectories, involving iterative refinements, are then used to fine-tune the llm with complex reasoning skills. stage 2: reinforcement learning with verification rewards after acquiring complex reasoning skills, reinforcement learning (rl) further enhances this ability using verification-based rewards. the verification feedback enables the model to spontaneously explore optimal long cot strategies without human preset guidance. using this approach, we present huatuogpt-o1, a medical llm that performs extended and reflective thinking before answering. experiments demonstrate that our method (using only 40k data points) yields an 8-point improvement on medical benchmarks with an 8b model. furthermore, our 70b model outperforms other general and medicalspecific llms of similar parameter scale on medical benchmarks. the experiments further reveal the effectiveness and domain compatibility of our method. our contributions are as follows: inspired by this, we introduce medical verifiable problems, which are open-ended yet have with unique, objective ground-truth answers, as illustrated in figure 1 . this brings verifiability to the medical domain, akin to mathematics, enabling a result-driven verification process. ## methodology in this section, we describe the method for training llms to perform medical complex reasoning. complex reasoning refers to longer chains-of-thought (cot) coupled with reflective behaviors. the formal definition of complex reasoning is provided in appendix i. as shown in figure 1 , the method consists of two stages based on the medical verifiable problems. ## conclusion this study advances the medical reasoning capabilities of llms. firstly, we construct the medical verifiable problems and a medical verifier. this enabled a two-stage training process: (1) learning complex reasoning and (2) enhancing it through rl. we developed huatuogpt-o1, a medical llm with thinks-before-it-answers behavior, achieving outstanding performance in medical benchmarks. experiments show that complex reasoning improves medical problem-solving and benefits obviously from rl. additional validation in chinese medical contexts shows the method's adaptability to other fields. we believe our approach can enhance domain-specific reasoning beyond mathematics.