PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex
  Task Automation on PC
Authors: name: Haowei Liu; name: Xi Zhang; name: Haiyang Xu; name: Yuyang Wanyan; name: Junyang Wang; name: Ming Yan; name: Ji Zhang; name: Chunfeng Yuan; name: Changsheng Xu; name: Weiming Hu; name: Fei Huang
Published: 2025-02-20 05:41:55

Abstract: In the field of MLLM-based GUI agents, compared to smartphones, the PC
scenario not only features a more complex interactive environment, but also
involves more intricate intra- and inter-app workflows. To address these
issues, we propose a hierarchical agent framework named PC-Agent. Specifically,
from the perception perspective, we devise an Active Perception Module (APM) to
overcome the inadequate abilities of current MLLMs in perceiving screenshot
content. From the decision-making perspective, to handle complex user
instructions and interdependent subtasks more effectively, we propose a
hierarchical multi-agent collaboration architecture that decomposes
decision-making processes into Instruction-Subtask-Action levels. Within this
architecture, three agents (i.e., Manager, Progress and Decision) are set up
for instruction decomposition, progress tracking and step-by-step
decision-making respectively. Additionally, a Reflection agent is adopted to
enable timely bottom-up error feedback and adjustment. We also introduce a new
benchmark PC-Eval with 25 real-world complex instructions. Empirical results on
PC-Eval show that our PC-Agent achieves a 32% absolute improvement of task
success rate over previous state-of-the-art methods. The code will be publicly
available.