Title: Single Answer is Not Enough: On Generating Ranked Lists with Medical Reasoning Models
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: medical, reasoning, list, answer, reinforcement learning, supervised fine-tuning, prompting, chain-of-thought

Abstract: This paper presents a systematic study on enabling *medical* reasoning models (MRMs) to generate **ranked lists** of answers for open-ended questions. Clinical decision-making rarely relies on a single answer but instead considers multiple options, reducing the risks of narrow perspectives. Yet current MRMs are typically trained to produce only one answer, even in open-ended settings. We propose an alternative format: *ranked lists* and investigate two approaches: *prompting* and *fine-tuning*. While prompting is a cost-effective way to steer an MRM's response, not all MRMs generalize well across different answer formats: *choice*, *short text*, and *list answers*. Based on our prompting findings, we train and evaluate MRMs using supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT). SFT teaches a model to imitate annotated  responses, and RFT incentivizes exploration through the responses that maximize a reward. We propose new reward functions targeted at ranked-list answer formats, and conduct ablation studies for RFT. Our results show that while some SFT models generalize to certain answer formats, models trained with RFT are more robust across multiple formats. We also present a case study on a modified MedQA with multiple valid answers, finding that although MRMs might fail to select the benchmark's preferred ground truth, they can recognize valid answers. To the best of our knowledge, this is the first systematic investigation of approaches for enabling MRMs to generate answers as *ranked lists*. We hope this work provides a first step toward developing alternative answer formats that are beneficial beyond single answers in medical domains.

TL;DR: We study how medical reasoning models can produce ranked lists for open-ended questions. Comparing prompting, SFT, and RFT, we find RFT more robust, highlighting ranked lists as a promising alternative to single answers.