INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized
  Reinforcement Learning
Authors: name: Prime Intellect Team, name: Sami Jaghouar, name: Justus Mattern, name: Jack Min Ong, name: Jannik Straube, name: Manveer Basra, name: Aaron Pazdera, name: Kushal Thaman, name: Matthew Di Ferrante, name: Felix Gabriel, name: Fares Obeid, name: Kemal Erdem, name: Michael Keiblinger, name: Johannes Hagemann

We introduce INTELLECT-2, the first globally distributed reinforcement
learning (RL) training run of a 32 billion parameter language model. Unlike
traditional centralized training efforts, INTELLECT-2 trains a reasoning model
using fully asynchronous RL across a dynamic, heterogeneous swarm of
permissionless compute contributors.
  To enable a training run with this unique infrastructure, we built various
components from scratch: we introduce PRIME-RL, our training framework
purpose-built for distributed asynchronous reinforcement learning, based on top
of novel components such as TOPLOC, which verifies rollouts from untrusted
inference workers, and SHARDCAST, which efficiently broadcasts policy weights
from training nodes to inference workers.
  Beyond infrastructure components, we propose modifications to the standard
GRPO training recipe and data filtering techniques that were crucial to achieve
training stability and ensure that our model successfully learned its training
objective, thus improving upon QwQ-32B, the state of the art reasoning model in
the 32B parameter range.
  We open-source INTELLECT-2 along with all of our code and data, hoping to
encourage and enable more open research in the field of decentralized training.