Mobile-Agent-V: Learning Mobile Device Operation Through Video-Guided
  Multi-Agent Collaboration
Authors: name: Junyang Wang, name: Haiyang Xu, name: Xi Zhang, name: Ming Yan, name: Ji Zhang, name: Fei Huang, name: Jitao Sang

The rapid increase in mobile device usage necessitates improved automation
for seamless task management. However, many AI-driven frameworks struggle due
to insufficient operational knowledge. Manually written knowledge helps but is
labor-intensive and inefficient. To address these challenges, we introduce
Mobile-Agent-V, a framework that leverages video guidance to provide rich and
cost-effective operational knowledge for mobile automation. Mobile-Agent-V
enhances task execution capabilities by leveraging video inputs without
requiring specialized sampling or preprocessing. Mobile-Agent-V integrates a
sliding window strategy and incorporates a video agent and deep-reflection
agent to ensure that actions align with user instructions. Through this
innovative approach, users can record task processes with guidance, enabling
the system to autonomously learn and execute tasks efficiently. Experimental
results show that Mobile-Agent-V achieves a 30% performance improvement
compared to existing frameworks.