PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex
  Task Automation on PC
Authors: name: Haowei Liu, name: Xi Zhang, name: Haiyang Xu, name: Yuyang Wanyan, name: Junyang Wang, name: Ming Yan, name: Ji Zhang, name: Chunfeng Yuan, name: Changsheng Xu, name: Weiming Hu, name: Fei Huang

In the field of MLLM-based GUI agents, compared to smartphones, the PC
scenario not only features a more complex interactive environment, but also
involves more intricate intra- and inter-app workflows. To address these
issues, we propose a hierarchical agent framework named PC-Agent. Specifically,
from the perception perspective, we devise an Active Perception Module (APM) to
overcome the inadequate abilities of current MLLMs in perceiving screenshot
content. From the decision-making perspective, to handle complex user
instructions and interdependent subtasks more effectively, we propose a
hierarchical multi-agent collaboration architecture that decomposes
decision-making processes into Instruction-Subtask-Action levels. Within this
architecture, three agents (i.e., Manager, Progress and Decision) are set up
for instruction decomposition, progress tracking and step-by-step
decision-making respectively. Additionally, a Reflection agent is adopted to
enable timely bottom-up error feedback and adjustment. We also introduce a new
benchmark PC-Eval with 25 real-world complex instructions. Empirical results on
PC-Eval show that our PC-Agent achieves a 32% absolute improvement of task
success rate over previous state-of-the-art methods. The code will be publicly
available.