BTL-UI: Blink-Think-Link Reasoning Model for GUI Agent
Authors: name: Shaojie Zhang; name: Ruoceng Zhang; name: Pei Fu; name: Shaokang Wang; name: Jiahui Yang; name: Xin Du; name: Shiqi Cui; name: Bin Qin; name: Ying Huang; name: Zhenbo Luo; name: Jian Luan
Published: 2025-09-19 04:03:44

Abstract: In the field of AI-driven human-GUI interaction automation, while rapid
advances in multimodal large language models and reinforcement fine-tuning
techniques have yielded remarkable progress, a fundamental challenge persists:
their interaction logic significantly deviates from natural human-GUI
communication patterns. To fill this gap, we propose "Blink-Think-Link" (BTL),
a brain-inspired framework for human-GUI interaction that mimics the human
cognitive process between users and graphical interfaces. The system decomposes
interactions into three biologically plausible phases: (1) Blink - rapid
detection and attention to relevant screen areas, analogous to saccadic eye
movements; (2) Think - higher-level reasoning and decision-making, mirroring
cognitive planning; and (3) Link - generation of executable commands for
precise motor control, emulating human action selection mechanisms.
Additionally, we introduce two key technical innovations for the BTL framework:
(1) Blink Data Generation - an automated annotation pipeline specifically
optimized for blink data, and (2) BTL Reward -- the first rule-based reward
mechanism that enables reinforcement learning driven by both process and
outcome. Building upon this framework, we develop a GUI agent model named
BTL-UI, which demonstrates consistent state-of-the-art performance across both
static GUI understanding and dynamic interaction tasks in comprehensive
benchmarks. These results provide conclusive empirical validation of the
framework's efficacy in developing advanced GUI Agents.