O1 Replication Journey -- Part 3: Inference-time Scaling for Medical
  Reasoning
Authors: name: Zhongzhen Huang, name: Gui Geng, name: Shengyi Hua, name: Zhen Huang, name: Haoyang Zou, name: Shaoting Zhang, name: Pengfei Liu, name: Xiaofan Zhang

Building upon our previous investigations of O1 replication (Part 1: Journey
Learning [Qin et al., 2024] and Part 2: Distillation [Huang et al., 2024]),
this work explores the potential of inference-time scaling in large language
models (LLMs) for medical reasoning tasks, ranging from diagnostic
decision-making to treatment planning. Through extensive experiments on medical
benchmarks of varying complexity (MedQA, Medbullets, and JAMA Clinical
Challenges), our investigation reveals several key insights: (1) Increasing
inference time does lead to improved performance. With a modest training set of
500 samples, our model yields substantial performance improvements of 6%-11%.
(2) Task complexity directly correlates with the required length of reasoning
chains, confirming the necessity of extended thought processes for challenging
problems. (3) The differential diagnoses generated by our model adhere to the
principles of the hypothetico-deductive method, producing a list of potential
conditions that may explain a patient's symptoms and systematically narrowing
these possibilities by evaluating the evidence. These findings demonstrate the
promising synergy between inference-time scaling and journey learning in
advancing LLMs' real-world clinical reasoning capabilities.