Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement
  Fine-Tuning of Large Language Models
Authors: name: Xuchen Pan, name: Yanxi Chen, name: Yushuo Chen, name: Yuchang Sun, name: Daoyuan Chen, name: Wenhao Zhang, name: Yuexiang Xie, name: Yilun Huang, name: Yilei Zhang, name: Dawei Gao, name: Yaliang Li, name: Bolin Ding, name: Jingren Zhou

Trinity-RFT is a general-purpose, flexible and scalable framework designed
for reinforcement fine-tuning (RFT) of large language models. It is built with
a decoupled design, consisting of (1) an RFT-core that unifies and generalizes
synchronous/asynchronous, on-policy/off-policy, and online/offline modes of
RFT, (2) seamless integration for agent-environment interaction with high
efficiency and robustness, and (3) systematic data pipelines optimized for RFT.
Trinity-RFT can be easily adapted for diverse application scenarios, and serves
as a unified platform for exploring advanced reinforcement learning paradigms.
This technical report outlines the vision, features, design and implementations
of Trinity-RFT, accompanied by extensive examples demonstrating the utility and
user-friendliness of the proposed framework.