Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement
  Fine-Tuning of Large Language Models
Authors: name: Xuchen Pan; name: Yanxi Chen; name: Yushuo Chen; name: Yuchang Sun; name: Daoyuan Chen; name: Wenhao Zhang; name: Yuexiang Xie; name: Yilun Huang; name: Yilei Zhang; name: Dawei Gao; name: Yaliang Li; name: Bolin Ding; name: Jingren Zhou
Published: 2025-05-23 12:41:09

Abstract: Trinity-RFT is a general-purpose, flexible and scalable framework designed
for reinforcement fine-tuning (RFT) of large language models. It is built with
a decoupled design, consisting of (1) an RFT-core that unifies and generalizes
synchronous/asynchronous, on-policy/off-policy, and online/offline modes of
RFT, (2) seamless integration for agent-environment interaction with high
efficiency and robustness, and (3) systematic data pipelines optimized for RFT.
Trinity-RFT can be easily adapted for diverse application scenarios, and serves
as a unified platform for exploring advanced reinforcement learning paradigms.
This technical report outlines the vision, features, design and implementations
of Trinity-RFT, accompanied by extensive examples demonstrating the utility and
user-friendliness of the proposed framework.