InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via
  Rubric-Based Incremental Training
Authors: name: Pengkai Wang; name: Qi Zuo; name: Pengwei Liu; name: Zhijie Sang; name: Congkai Xie; name: Hongxia Yang
Published: 2025-10-17 17:51:28

Abstract: Large Language Models (LLMs) have shown substantial advances through
reinforcement learning (RL), particularly in domains where rewards can be
programmatically verified, such as mathematics and code. In these areas, models
benefit from a well-defined operational base guided by explicit rule-based
objectives. However, this progress reveals a significant limitation: in
open-ended domains where rewards are ambiguous, subjective, or
context-dependent, such as creative writing, scientific reasoning, and notably
medical consultation, robust reward functions are lacking, making these areas
challenging for current RL strategies. To bridge this gap, we introduce ORBIT,
an open-ended rubric-based incremental training framework specifically designed
for high-stakes medical dialogue. ORBIT integrates syn- thetic dialogue
generation with the dynamic creation of rubrics, employing these rubrics to
direct an incremental RL process. In particular, this approach does not depend
on external medical knowledge or manual rules, instead utilizing rubric-guided
feedback to shape learning. When implemented on the Qwen3-4B-Instruct model,
our method can greatly enhance its performance on the HealthBench-Hard
benchmark from 7.0 to 27.2 using only 2k samples, thus achieving
state-of-the-art results for models of this scale. Our analysis confirms that
rubric-driven RL fos-ters consistent performance gains across diverse
consultation scenarios, going beyond simple numerical improvements. These
findings underscore rubric-based feedback as a scalable strategy for advancing
LLMs in intricate, open-ended tasks.