Mano Report
Authors: name: Tianyu Fu; name: Anyang Su; name: Chenxu Zhao; name: Hanning Wang; name: Minghui Wu; name: Zhe Yu; name: Fei Hu; name: Mingjia Shi; name: Wei Dong; name: Jiayao Wang; name: Yuyang Chen; name: Ruiyang Yu; name: Siran Peng; name: Menglin Li; name: Nan Huang; name: Haitian Wei; name: Jiawei Yu; name: Yi Xin; name: Xilin Zhao; name: Kai Gu; name: Ping Jiang; name: Sifan Zhou; name: Shuo Wang
Published: 2025-09-22 03:13:58

Abstract: Graphical user interfaces (GUIs) are the primary medium for human-computer
interaction, yet automating GUI interactions remains challenging due to the
complexity of visual elements, dynamic environments, and the need for
multi-step reasoning. Existing methods based on vision-language models (VLMs)
often suffer from limited resolution, domain mismatch, and insufficient
sequential decisionmaking capability. To address these issues, we propose Mano,
a robust GUI agent built upon a multi-modal foundation model pre-trained on
extensive web and computer system data. Our approach integrates a novel
simulated environment for high-fidelity data generation, a three-stage training
pipeline (supervised fine-tuning, offline reinforcement learning, and online
reinforcement learning), and a verification module for error recovery. Mano
demonstrates state-of-the-art performance on multiple GUI benchmarks, including
Mind2Web and OSWorld, achieving significant improvements in success rate and
operational accuracy. Our work provides new insights into the effective
integration of reinforcement learning with VLMs for practical GUI agent
deployment, highlighting the importance of domain-specific data, iterative
training, and holistic reward design.