Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks
Authors: name: Zhenhailong Wang; name: Haiyang Xu; name: Junyang Wang; name: Xi Zhang; name: Ming Yan; name: Ji Zhang; name: Fei Huang; name: Heng Ji
Published: 2025-01-20 20:35:46

Abstract: Smartphones have become indispensable in modern life, yet navigating complex
tasks on mobile devices often remains frustrating. Recent advancements in large
multimodal model (LMM)-based mobile agents have demonstrated the ability to
perceive and act in mobile environments. However, current approaches face
significant limitations: they fall short in addressing real-world human needs,
struggle with reasoning-intensive and long-horizon tasks, and lack mechanisms
to learn and improve from prior experiences. To overcome these challenges, we
introduce Mobile-Agent-E, a hierarchical multi-agent framework capable of
self-evolution through past experience. By hierarchical, we mean an explicit
separation of high-level planning and low-level action execution. The framework
comprises a Manager, responsible for devising overall plans by breaking down
complex tasks into subgoals, and four subordinate agents--Perceptor, Operator,
Action Reflector, and Notetaker--which handle fine-grained visual perception,
immediate action execution, error verification, and information aggregation,
respectively. Mobile-Agent-E also features a novel self-evolution module which
maintains a persistent long-term memory comprising Tips and Shortcuts. Tips are
general guidance and lessons learned from prior tasks on how to effectively
interact with the environment. Shortcuts are reusable, executable sequences of
atomic operations tailored for specific subroutines. The inclusion of Tips and
Shortcuts facilitates continuous refinement in performance and efficiency.
Alongside this framework, we introduce Mobile-Eval-E, a new benchmark featuring
complex mobile tasks requiring long-horizon, multi-app interactions. Empirical
results show that Mobile-Agent-E achieves a 22% absolute improvement over
previous state-of-the-art approaches across three foundation model backbones.
Project page: https://x-plug.github.io/MobileAgent.