FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid
  Control
Authors: name: Younggyo Seo; name: Carmelo Sferrazza; name: Haoran Geng; name: Michal Nauman; name: Zhao-Heng Yin; name: Pieter Abbeel
Published: 2025-05-28 17:55:26

Abstract: Reinforcement learning (RL) has driven significant progress in robotics, but
its complexity and long training times remain major bottlenecks. In this
report, we introduce FastTD3, a simple, fast, and capable RL algorithm that
significantly speeds up training for humanoid robots in popular suites such as
HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably
simple: we train an off-policy TD3 agent with several modifications -- parallel
simulation, large-batch updates, a distributional critic, and carefully tuned
hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours
on a single A100 GPU, while remaining stable during training. We also provide a
lightweight and easy-to-use implementation of FastTD3 to accelerate RL research
in robotics.