Efficient Multi-turn RL for GUI Agents via Decoupled Training and
  Adaptive Data Curation
Authors: name: Pengxiang Li; name: Zechen Hu; name: Zirui Shang; name: Jingrong Wu; name: Yang Liu; name: Hui Liu; name: Zhi Gao; name: Chenrui Shi; name: Bofei Zhang; name: Zihao Zhang; name: Xiaochuan Shi; name: Zedong YU; name: Yuwei Wu; name: Xinxiao Wu; name: Yunde Jia; name: Liuyu Xiang; name: Zhaofeng He; name: Qing Li
Published: 2025-09-28 13:19:20

Abstract: Vision-language model (VLM) based GUI agents show promise for automating
complex desktop and mobile tasks, but face significant challenges in applying
reinforcement learning (RL): (1) slow multi-turn interactions with GUI
environments for policy rollout, and (2) insufficient high-quality
agent-environment interactions for policy learning. To address these
challenges, we propose DART, a Decoupled Agentic RL Training framework for GUI
agents, which coordinates heterogeneous modules in a highly decoupled manner.
DART separates the training system into four asynchronous modules: environment
cluster, rollout service, data manager, and trainer. This design enables
non-blocking communication, asynchronous training, rollout-wise trajectory
sampling, and per-worker model synchronization, significantly improving the
system efficiency: 1.6*GPU utilization for rollout, 1.9* training throughput,
and 5.5* environment utilization. To facilitate effective learning from
abundant samples, we introduce an adaptive data curation scheme: (1)
pre-collecting successful trajectories for challenging tasks to supplement
sparse success in online sampling; (2) dynamically adjusting rollout numbers
and trajectory lengths based on task difficulty; (3) training selectively on
high-entropy steps to prioritize critical decisions; (4) stabilizing learning
via truncated importance sampling for policy mismatch between policy rollout
and updating. On the OSWorld benchmark, DART-GUI-7B achieves a 42.13% task
success rate, a 14.61% absolute gain over the base model, and 7.34% higher than
open-source SOTA. We will fully open-source our training framework, data, and
model checkpoints via computer-use-agents.github.io/dart-gui, which we believe
is a timely contribution to the open-source community of agentic RL training.