OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks
  with Reinforcement Fine-Tuning
Authors: name: Yuxiang Zhang; name: Yuqi Yang; name: Jiangming Shu; name: Yuhang Wang; name: Jinlin Xiao; name: Jitao Sang
Published: 2024-12-22 04:21:30

Abstract: OpenAI's recent introduction of Reinforcement Fine-Tuning (RFT) showcases the
potential of reasoning foundation model and offers a new paradigm for
fine-tuning beyond simple pattern imitation. This technical report presents
OpenRFT, our attempt to fine-tune generalist reasoning models for
domain-specific tasks under the same settings as RFT. OpenRFT addresses two key
challenges of lacking reasoning step data and the limited quantity of training
samples, by leveraging the domain-specific samples in three ways: question
augmentation, synthesizing reasoning-process data, and few-shot ICL. The
evaluation is conducted on SciKnowEval, where OpenRFT achieves notable
performance gains with only 100 domain-specific samples for each task. More
experimental results will be updated continuously in later versions. Source
codes, datasets, and models are disclosed at:
https://github.com/ADaM-BJTU/OpenRFT