UFO2: The Desktop AgentOS
Authors: name: Chaoyun Zhang, name: He Huang, name: Chiming Ni, name: Jian Mu, name: Si Qin, name: Shilin He, name: Lu Wang, name: Fangkai Yang, name: Pu Zhao, name: Chao Du, name: Liqun Li, name: Yu Kang, name: Zhao Jiang, name: Suzhen Zheng, name: Rujia Wang, name: Jiaxu Qian, name: Minghua Ma, name: Jian-Guang Lou, name: Qingwei Lin, name: Saravan Rajmohan, name: Dongmei Zhang

Recent Computer-Using Agents (CUAs), powered by multimodal large language
models (LLMs), offer a promising direction for automating complex desktop
workflows through natural language. However, most existing CUAs remain
conceptual prototypes, hindered by shallow OS integration, fragile
screenshot-based interaction, and disruptive execution.
  We present UFO2, a multiagent AgentOS for Windows desktops that elevates CUAs
into practical, system-level automation. UFO2 features a centralized HostAgent
for task decomposition and coordination, alongside a collection of
application-specialized AppAgent equipped with native APIs, domain-specific
knowledge, and a unified GUI--API action layer. This architecture enables
robust task execution while preserving modularity and extensibility. A hybrid
control detection pipeline fuses Windows UI Automation (UIA) with vision-based
parsing to support diverse interface styles. Runtime efficiency is further
enhanced through speculative multi-action planning, reducing per-step LLM
overhead. Finally, a Picture-in-Picture (PiP) interface enables automation
within an isolated virtual desktop, allowing agents and users to operate
concurrently without interference.
  We evaluate UFO2 across over 20 real-world Windows applications,
demonstrating substantial improvements in robustness and execution accuracy
over prior CUAs. Our results show that deep OS integration unlocks a scalable
path toward reliable, user-aligned desktop automation.