UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents
Authors: Han Xiao
Guozhi Wang
Yuxiang Chai
Zimu Lu
Weifeng Lin
Hao He
Lue Fan
Liuyang Bian
Rui Hu
Liang Liu
Shuai Ren
Yafei Wen
Xiaoxin Chen
Aojun Zhou
Hongsheng Li
Venue: NeurIPS 2025 poster ()
Keywords: Multi-modal Large Language Model
GUI Agent
Primary area: applications

Abstract: In this paper, we introduce UI-Genie, a self-improving framework addressing two key challenges in GUI agents: verification of trajectory outcome is challenging and high-quality training data are not scalable. These challenges are addressed by a reward model and a self-improving pipeline, respectively. The reward model, UI-Genie-RM, features an image-text interleaved architecture that efficiently processes historical context and unifies action-level and task-level rewards. To support the training of UI-Genie-RM, we develop deliberately-designed data generation strategies including rule-based verification, controlled trajectory corruption, and hard negative mining. To address the second challenge, a self-improvement pipeline progressively expands solvable complex GUI tasks by enhancing both the agent and reward models through reward-guided exploration and outcome verification in dynamic environments. For training the model, we generate UI-Genie-RM-517k and UI-Genie-Agent-16k, establishing the first reward-specific dataset for GUI agents while demonstrating high-quality synthetic trajectory generation without manual annotation. Experimental results show that UI-Genie achieves state-of-the-art performance across multiple GUI agent benchmarks with three generations of data-model self-improvement. We open-source our complete framework implementation and generated datasets to facilitate further research in https://github.com/Euphoria16/UI-Genie.

UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents
Han Xiao Guozhi Wang?, Yuxiang Chai Zimu Lu', Weifeng Lin -22, Hao He', Lue Fan", Liuyang Bian", Rui Hu², Liang Liu", Shuai Ren 21se Yafei Wen', Xiaoxin Chen', Aojun Zhou' 18 Hongsheng Li 13,,3 CUHK MMLab vvvv Lab *Shanghai AI Laboratory 4Ace Robotics (11552291230link, hslisee aoounzhou@1ii ink). cuhk. edu. hk shuai. ren@vivo. com "Corresponding author Project lead
Interns at vivo.
Abstract
In this paper, we introduce UI-Genic, self-improving framework addressing two key challenges in GUI agents: verification of trajectory outcome is challenging and high-quality training data are not scalable. These challenges are addressed by a reward model and a seff-improving pipeline, respectively. The reward model, UI-Genie-RM, features an image-text interleaved architecture that efficiently pro- cesses historical context and unifies action-level and task-level rewards. To sup- data genera-
port the training of UI-GenieRRM, we develop deliberately-dessieee tion strategies including rule-based verification, controlled trajectory corruption, and hard negative mining. To address the second challenge, a self-improvement pipeline progressively cxpands solvable complex GUI tasks by enhancing both the agent and reward models through reward-gen gurdddduuddd exploration and outcome verification in dynamic environments. For training the model, Genie-RM- RM-517k and UJ-Genie-Agent-Iee 16k, establishing the first reward-specin dataset for GUI agents while demonstrating high-quality synthetic trajectory gen- eration without manual annotation. Experimental results show that UI-Genie achieves state-of-the-art performance across multiple GUI agent benchmarks with three generations of data-model self-improvement. We open-source our complete framework implementation and generated datasets to facilitate further research in https: / / git.ub....... com/ http///pporiaa//// -ariiiaiii//eeeeei Genie.
generate UI-
Introduction
Large Language Models (LLMs) [23, 24, have demonstrated remarkable potential for powering
autonomous mobile GUI agents capable of completing complex tasks through natural language instructions. The emergence of Multi-modal Large Language Models (MLLMs) [13, 17, represents a significant advancement in this field, introducing enhanced perception and reasoning abilities [46, 8] crucial for interface navigation. By directly perceiving screenshots, identifying functional UI elements, and generating executable action sequences in dision-centria manner, MLLM-basee GUI agents have achieved better performance compared their text-only counterparts.
While recent advancements in general MLLMs have demonstrated the effectiveness of large-scale synthetic data generation [3, 52], creating high-quality synthetic data for GUI agents remains partic- ularly challenging: (/) Accurate Trajectory Outcome Verification. Verifying GUT agent trajectory outcomes presents unique challenges that distinguish it from other multi-modal understanding tasks. tasks where correctness can be directly judged by checking final answer, the completion state of GUI agent tasks is heavily dependent on historical context,
Unlike common
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Table I: Ul-Genic dataset statistics. Ul-Genie-RM-5177 is the first dedicated GUI agent reward dataset, while (GGnne-Agente 16k contains synthetic trajectories without manual annotation.
Owann 516-13
Ur- to Ageniit (Duna)
Reward Dainsels
Size
Manual Aunotation
Source
Data
Posilive
Num
Nogative Num
267
AntrodCrantol
121k
147k
Ul-Gemie-RM-517%
24k
AndrvidLab
AMEX
68k
14k
102K I0K
5%%
Exploration
29k
Sik
Agent Dutasets
Size
Manual Auaotation
Average Steps
Trajectories
Instruct
Task
Android Concrol
68k
5.5
15263
High&Low High High
Andraaa drura
Androiilab
Anarrill Cantral
AMEX
34k
12.8
2946
Figure 1: Performance compari- son between UI-Genie. Qwen2.5-VL. and UI-TARS on three benchmarks.
Androiddaah
6k
8.5
726
U--Genie-Aeent-B 1ok
16k
2208
High&Low
making evaluation substantially more complex. Existing assessment frameworks, including those utilizing proprietary models as judges [25, 43], fail to provide accurate outcome verification and reliable step-level validation of intermediate actions. (2) Scalable Sigh--uulity Training Data. Due to the lack of reliable verification methods, current training approaches still rely on human-annotated operation trajectories [15, 5, 44], which are inherently time-consuming, expensive, and difficult to scale. These manually-created datasets suffer from limited volume and diversity, particularly lacking high-quality demonstrations for complex multi-step tasks.
To address these challenges, we introduce UI-Genie, self improving framework that generates high- quality synthetic trajectories with accurate process supervision. eliminating the need for extensive human annotation. To provide fine-grained trajectory assessment, we build UI-Genie-RM, the first reward model specifically designed for evaluating GUI agents. Our reward model features an image-tex text interleaved architecture that efficiently processes historical screenshots and actions as context while unifying both action-level and task-level rewards. This innovative approach enables both single-step and multi-step trajectory evaluation within the same architecture. To overcome the significant challenge of the absence of training datasets designed for GUI agent reward models, we develop deliberate data generation strategies, including rule-based verification, controlled trajectory corruption, and hard negative mining based on open- open-source training datasets. This comprehensive dataset construction process, combined with UI-Genie-PP RM's long-context processing capabilities and unified reward representation, enables accurate validation at both the action level and task level without requiring manual annotation.
To generate high-quality synthetic trajectories, we leverage UI-Genie-RM, RM for process supervision during agent exploration in dynamic environments. In these rollouts, UI-Genie-RM ranks candidate actions by reward scores and expands only the most promising trajectories. At terminal states, we use outcome verification to assess trajectory success. To address low initial success rates on complex tasks, we introduce a self-improvement framework where both the agent and reward models evolve iteratively. Specifically, we expand training data with successful trajectories to boost performance on increasingly complex tasks, as well as correctly labeled steps from failed trajectories to refine the reward model. Our framework overcomes the limitations of static, aannotty-annottteed datasets and enables continuous improvement in GUI agent performance. Through this process, we generate UI-Genie-RMV RiiieeMM-51177 7k, the first large-scale reward dataset for GUI agents, and Ul-Genie-Agent 16k, high-quality synthetic trajectory dataset. As shown in Tab. 1, our method yields substantially more data without human annotation. Experiments show that UI-Genie achieves of ofte-----heettt results with three celf-improvement cycles, as illustrated in Fig. 1. We release our full open-source framework to support further research in GUT agents.
Our contributions can be summarized as follows:
1. We develop UI-Genie-RM. RMM, a specialized reward model for GUI trajectory assessment with rich interleaved image-text observations, while unifying action-level and task-level rewards.
2. We introduce Ul-Genie, a novel self-improving framework where both the agent model and reward model evolve iteratively. This framework progressively solves complex GUI tasks through reward-guided trajectory exploration, training data expansion and iterative model fine-tuning.
3. We create and open-source two novel datasets (UI-Genie-RM-517k and UI-Genie Agent- along with our complete framework implementation, establishing the first seward-specific dataset for GUI agents and demonstrating synthetic trajectory generation without manual annotation.

---

2 Related Work
Multi-modal GUI Agents. Recent advances in Large Language Models (LLMs) [23, 24, 38] have enabled the development of GUI agents, with Multi-modal LLMs (MLLMs) further enhancing UI information perception [9. 12]. Prior work [1, 36, 37, 50] leveraging commercial MLLMs has shown promising results across mobile and desktop environments. Meanwhile, open-source approaches have focused on specialized architectures and training methods. OS-Atlas [42] adopts a unified action space for different UI grounding tasks. UI-TARS [27] introduces comprehensive pretraining element grounding and reasoning. [20, 19] apply rule-based Reinforcement Learning [11] and GRPO [30] to achieve competitive performance. Despite these advances, significant challenge persists: most approaches remain dependent on 34] incorporate synthetic trajectory generation, they still rely on human effort or closed API models for verification, limiting scalability. In contrast, our work enables fully automatic trajectory synthesis and evaluation with an effective self-immprvemment pipeline.
trajectories. While some methods [44, 33, 27, 22,
Reward Models. Reward models have been widely used in reasoning intensive tasks to assist in test- time scaling [32, 16, 47], expand solution paths [10], and facilitate model fine-tuning [35, 39, 48, 14]. Prior studies such as [16] and [35] demonstrate that, compared to outcome-supervisio reward models (ORMs). process-supervision reward models (PRMs) can better perceive the correctness of intermediate steps in reasoning trajectories, thereby providing more reliable feedback. To avoid the high cost of step-level human annotation, various works [21, 39] have proposed methods to automatically obtain process supervision by estimating the potential of intermediate steps to lead to the correct final result. Similar these studies, our work also leverages PRM trained on step-level rewards obtained through a sophisticated rollout in dynamic environments. Recent works [40, 49, 41] have extended the application of PRMs from LLMs to MILMs. [33] employ GPT-40 as trajectory- level reward model to assess the quality of GUI agent trajectories. Similarly, [26] trains an outcome reward model for trajectories, enabling the model learn from both successful and failed attempts. In contrast, our method creates a large-scale reward training dataset from scratch and does not rely on costly proprietary models. Moreover, unlike these outcome-only rewards, our model provides both task-level and action-level rewards and refines both reward and agent models iteratively with the increasingly complex training data.
3 Method
This section introduces UI-Genie, a framework for solving mobile GUI agent challenges through an iterative self-improving approach. We begin by formulating the reward modeling task in Sec. 3.1. Sec. 3.2 presents the specialized reward model. Then, we detail the dataset construction strategies in Sec. 3.3. Finally, Sec. 3.4 explains how the self-improvement pipeline works to continuously enhance both components.
3.1 Task Formulation for GUI Trajectory Reward
Action-level reward. Given a task goal G, current screenshot I f. and historical observations O (I_0,AA00 (_0), A_),-.,,,... (I_t-
1, A_ 1), the objective to evaluate whether candidate action A_t constitutes a correct step toward achieving G. The reward model outputs a positive reward (y 1) for correct actions and negative reward (y 0) otherwise.
Task-level reward. This task evaluates whether an entire trajectory successfully completes the goal G. The input consists of all actions and resulting screenshots in the trajectory, and the output is single reward score indicating overall success or failure.
3.2 UI-Genie-RM: A Unified Reward Model
Previous approaches of trajectory verification [27, 42, 43, 25] rely on human efforts or proprietary models for validation, which are inherently costly and difficult to scale. Moreover, these methods fails to provide validation of intermediate steps, constraining their practical adaptability. To address these limitations, we develop UI-Genie-RM. GUI reward model which builds upon standard MLLM backbone, but incorporates crucial modifications to handle sequential GUI interactions and unify leep- and outeome-level rewards.
3

---

Training Data Construction
Roward Evoluztion
Rule-based Verification
Trajectory Corruption
Hard Negative Mining
Action-level
<<
Type Aigement
Coorainate Accuracy
contacts piaces
(87, 95S)
Senardic Conalstancy
Rejec!
X
Swipe to view more contacts
Task-koval Roward
Complete
X
UI-Genie-M
Task
Historical Context
Current
Candidate Actions
Screemshot
Step
Step 2
Slop 1-2
Step
Swppe Up
Share
with my perscnall
Mirrryy
Open Ply
Books app
Click on the smarch ber on the top of the screen
Citck on the three dals the top right comer of the screon
Click on ahomn option ou the screen
W
(61,
Complete
Figure 2: Overview of UI-Genie-RM model and reward training data construction. The model processes task instruction, historical context, current screenshot, and candidate action as inputs. Outputs are supervised by both action-level and task-level rewards. The training data constructed by rule-based verification. trajectory corruption, and hard negative mining processes.
Model Architecture. UI-Genie-RR RM determines a proposed action A, advances the task goal G. given the current state. Because GUI tasks depend long-term context, accurate action assessment requires understanding more than the current screenshot. For example, in a cross-app task where a user copies headline from News Notes, the correct action in Notes (typing the headline) depends on knowing what was seen earlier in News, not just the present screenshot.
To address this issue, Ul-Genie-Re RM uses image-text interleaved architecture that leverages past interactions to evaluate current actions. As shown in Fig. 2, the model takes the task goal, current screenshot, and candidate actions, plus interleaved sequence of recent sereenshots and action history for context. This helps the model track progress and understand the impact of previous steps.
Processing full histories is costly for complex tasks. We address this by context window using only the five most recent screenshots, while summarizing earlier actions as language descriptions. Our agent predicts these natural language summaries (e.g.. "Access Categories & Budget section") alongside action primitives (e-g., click yy)), providing semantic context even when screenshots missing. This representation balances historical coverage and computational efficiency.
Unified Action-Level and Task-Level Rewards. Our model architecture's ability process histori- cal information provides necessary context for trajectory-level judgments, which enables us to unify step-level and task-level assessment within the same architecture. To achieve this goal, we formulate special action within the action space, allowing us to judge whether this
"task completion terminal action appropriate given past GUI interactions when evaluating task lesel outcomes.
This unified representation allows both action-level and task-level rewards to be handled through the optimization objective, eliminating the need for separate models across different evaluation granularities. Specifically, our reward model assigns binary rewards to both actions and tasks given the task goal G. current screenshot I, and historical context descriptions H. (Ho, H. ..., H.-1)- We define the reward ground truth R, E {0, 1}. where indicates a correct action and 0 indicates an incorrect one. Following previous work [5]], we formulate reward prediction generation task. The model is trained to maximize the likelihood of predicting y* for correct actions and y for incorrect ones. Given dataset D containing positive action examples A, and negative examples A,- our training objective minimizes
next-token
LRM
Σ
log Pvy 1ly++1,20 I. H. Ht)
Σ
log P II, H,
(I)
(AD
(AT M JDD
where yt and y denote the positive and negative action labels, respectively.

---

Reeraivve Fine-
Ul-Genie Agent
Ul-Genie-RM
Iterative Tine-tuning
Trajectory Exploration
Fil
Ul-Gene RRM
Ul-Geno-Agent Training Set
Progress Supervision
Training Set
Dataset Expansion
Dataset
Expansion
Outcome Verification
X
Android System
Falled Trajectories
Successful Trajestones
Figure 3: Self-improvement of agent and reward models for UT-Genie. expands training sets both agent and reward models through reward-guided trajectory exploration and outcome verification, then finetunes both models. This process repeats iteratively to improve capabilities increasingly complex tasks.
3.3 Data Construction for Reward Model Training
To train MLLM into an effective GUI agent reward model, we develop deliberately-deesieee data construction strategies that create a comprehensive training corpus of approximately 517k reward data. The detailed data statistics are presented in Tab. 1. We initially leverage task instructions and corresponding ground-trun -rutt actions from open-source datasets [15, 5, 44] as sources. We then apply three complementary data generation techniques the base data source create positive and negative samples, resulting in 458k synthetic training samples for our initial reward model. This dataset is further enriched with an additional 59k samples acquired through trajectory exploration in dynamic environments as part of our self-improvement pipeline described in Sec. 3.4.
Rule-based Verification. We first train an initial agent model on the base GUI operation trajectory datasets [15, 5, 44]. For each task instruction, we generate candidate actions by sampling prediction from this initial agent given the same task instructions, then validate them against ground truth actions using three criteria: (1) Type alignment-wetthe
predicted and ground truth actions must share the same type; (2) Coordinate accuracy for spatial operations like "click" and "long press". predicted coordinates must fall within ground truth bounding boxes to ensure functional validity; (3) Semantic consistency-foe text-based operations like "typing". generated content must maintain semantic equivalence with ground truth.
Controlled Trajectory Corruption. To create negative trajectory samples, systematically perturb successful trajectories through three mechanisms: (1) Early gruncation-terminattin terminating trajecto- ries at intermediate steps simulate incomplete executions; (2) Cross-task substitution trajectory segments with actions from different tasks within the same application, mimicking goal misinterpretation: (3) Redundant
aatp:/uaaioon-pppniiiini unnecessary steps beyond task completion to model failure in recognizing terminal states.
Hard Negative Mining. After obtaining the initial agent model (trained existing trajectory datasets) and reward model (whose training dataset constructed with rule-based verification and controlled trajectory corruption techniques), we adopt hard-negative mining process to find challenging negative examples in reward dataset. We start with existing annotations from the AMEX dataset [5], which provides target tasks, ground-truth actions and descriptions for other target- unrelated UI elements. We then use an open-source LLM [18] to convert these descriptions into executable actions. All of these actions are regarded as step-level negative actions for the target task. After that, we use the initial reward model to score them. the reward model mistakenly assigns a positive score to one action, this action is identified hard negative.
These complementary strategies yield a diverse, high-quality dataset that enables effective training of UL-Genie- RM across varying difficulty levels and failure modes, establishing a robust foundation for accurate single-step and multi-step assessment.
3.4 Iterative Self-Improvement of Agent and Reward Models
Following the training of initial UI-Genie-RM reward model, we establish self- relf-improvement framework that dynamically generates GUI operation trajectories and progressively enhances both reward and agent models. As illustrated in Fig. 3, this framework operates within dynamic Android
5

---

environment where agent actions directly modify environmental states. We first introduce our approach of reward-guided trajectory exploration and training data expansion, then present our recipe of iterative model fine-tuning with increased task complexity.
Reward-Guided Trajectory Exploration. For GUI trajectory exploration, we find traditional Monte Carlo Tree Search (MCTS) mechanisms [4, 31] are suboptimal due to their computational expense. More importantly, conventional MCTS fails to effectively address the unique characteristics of GUI interaction spaces. Specifically, unlike mathematical problem solving where erroneous steps lead to incorrect answers, GUI interactions often include invalid actions that maintain the last state (e.g.. clicking blank areas or non-functional elements). To address these challenges, we develop reward- guided beam search approach inspired by [7]. At each step, El--GeieeAgent generates ten candidate actions that are evaluated by UI-Genie-RA Rieeie--MMii to produce step-level rewards. We rank these candidates by cumulative rewards along each partial trajectory and retain only the top-5 paths, as indicated by the green circles in the tree in Fig. 3. This exploration efficiently discovers both successful completion paths and informative partial trajectories for our training data expansion process.
Training Data Expansion with Outcome Verification. Our framework iteratively expands training data for both the agent and reward models through a dual labeling approach. To expand the training set of Ul-eenie-Agent, we apply outcome verification using UI-Genie-MM to identify successful completions. These successful trajectories are directly added the agent's training set to enhance its generalization capabilities. To expand the training set of UI-Genie-RM. we employ potential- -bsse labeling approach [39] for step-level reward annotation. For successful trajectories, each constituent step is annotated correct (label y* For unsuccessful trajectories, we determine the viability of each intermediate step through continuation rollouts. From each intermediate step, we sample five continuation paths and evaluate their outcomes. If any of these rollouts successfully completes the task, we classify the original step as correct (y+), indicating it maintains task completion potential despite being part of a failed trajectory. This approach automatically generates fine-grained step-level supervision for refining the reward model.
Iterative Model Fine-tuning. We implement our framework through three iterative rounds with gradually increasing task complexity: Initial generation: We utilize task instructions directly derived from source training data [44, 5], establishing baseline performance. 2. Expansion generation: We introduce novel task instructions generated by prompting an advanced open- source L.LM [18] with app descriptions and example tasks from the first generation. 3. Complex task generation: We combine filtered instructions of failed tasks from previous rounds with manually crafted complex scenarios requiring more than ten steps to complete.
This progressive approach creates a positive feedback loop between framework components: enhanced agent capabilities generate more diverse, successful trajectories for increasingly complex tasks; these trajectories provide richer supervision signals for the reward model; the refined reward model delivers more precise guidance during exploration; and this improved guidance enables the discovery of solutions to even more complex tasks. Each generation thus reinforces both components' capabilities while progressively expanding the frontier of solvable GUI tasks.
4 Experiments
In this section, we evaluate UI-Genie across diverse benchmarks designed to assess both GUI agent capabilities and reward model accuracy. We first describe our implementation details, followed by a comprehensive overview of the evaluation benchmarks and metrics. Ablation studies are provided in Sec. A to validate the design choices of each component.
4.1 Implementation Details
We implement both UI-GenieRMM and Ul-Genie-Agent based on the Qwen2.5-VL family of models due to its strong multimodal understanding capabilities. For the reward model, we adopt Qwen2.5- VL-7B the backbone. For agent models, we experiment with three variants across different model sizes: 3B, 7B, and 72B, enabling analysis of scaling effects on performance.
Reward Model Training. UI-Genie-RM is initialized with Qwen2.5- VLL-7B and trained using binary classification objective. We introduce special tokens (I [1+11 and (I - }} to represent positive and negative class labels, respectively, and formulate reward prediction next-token generation

---

Table 2: Performance comparison on AndroidControl benchmark [15]. The table presents commercial sys- tems (Claude Computer Use and GPT-4o), open-source foundation models, and our approach. Higher values are better for all metries; SR (Success Rate), Type (Action Type Accuracy), and Grounding (UI Element Interaction Accuracy) across both low-level and high-level task settings. Highest values indicated as bold.
AndroidControl-
AndroidConntrol High
Agent
Model Size
Type
Grounding
SR
Type
Grounding
SR
Claude Computer Use
74.3
0.0
19.4
63.7
0.0
12.5
GPT-4o
74.3
0.0
19.4
66.3
0.0
20.8
Aria-UI
3,9B
87.7
67.3
43.2
10.2
UI-R1
3B
94.3
82.6
GUI-R1
38
58.0
56.2
46.6
Qwen2.5-VL
3B
79.3
72.3
90.8
63.7
OS-Atlas
4B
91.9
83.8
80.6
84.7
73.8
67.5
UI-TARS
2B
98.1
87.3
89.3
81.2
78.4
68.9
InfiGUI-RI
3B
96.0
93.2
92.1
82.7
74.4
71.1
Ul-Genie-Agent (Ours)
97.8
94.7
93.8
82.5
82.5
72.9
OS-Genesis
7B
90.7
74.2
66.2
44.5
GUI-RI
7B
71.6
65.6
51.7
SeeClick
9.6B
93.0
73.4
75.0
82.9
62.9
59.1
Qwen2.5-VL
7B
91.4
60.1
Aguvis
80.5
61.5
OS-Atlas
7B
93.6
85.2
78.5
71.2
UI-TARS
7B
98.0
89.3
90.8
83.7
80.5
72.5
UI-Genie-Agent (Ours)
7B
98.1
94.9
94.3
83.5
82.9
74.2
Aguvis
84.4
66.4
V-
72B
67.4
UI- UI-TARS
72B
98.1
89.9
91.3
85.2
81.5
74.7
UI-Genie-Agent (Ours)
72B
98.3
95.4
94.8
84.9
86.3
77.0
task. We first conduct supervised fine-tuning existing datasets. Subsequently, we perform iterative refinement using a total of 59k process-reward samples collected during self improvement cycles. We train the model using the AdamW optimizer with learning rate le-5 and a global batch size 160.
constructed dataset of 458k samples derived from
Agent Model Training. We first establish an initial agent model based on Owen2.5-VL-77. using existing datasets including AndroidControl [15], AMEX [5], and AndroidLab [44]. This initial model performs trajectory exploration in our dynamic Android environment under UI-Genie-MM's supervision. Through three rounds of iterative improvement, we generate Ul-Genie-Agent synthetic dataset containing 16k high-quality trajectories without manual annotation. For variants beyond 7B, we train Il-Genie-Agent-33 and --Genie--Agett the aforementioned open-source datasets for one epoch. All agent models are fine-tuned with learning rate le-5 and batch size 160.
using Ul-Genie- -Genie-Agent-164 and
4.2 Evaluation Benchmarks
4.2.1 Agent Model Evaluation
To comprehensively evaluate UI-Genie, conduct experiments on both static and dynamic bench- marks. Static evaluation assesses the agent's is ability to predict correct actions in static environments where only single frame with ground-truie action history is input, while dynamic evaluation tests the agent's performance in dynamic, interactive settings in an emulator where each action directly affects task progress and completion.
Statie Evaluation. We utilize the AndroidControl [15] benchmark to evaluate UI-Genie-Agent's planning and action-execution capabilities in static mobile environments without actually executing actions on a device. This benchmark compares the agent's predicted actions with ground truth actions, providing controlled and reproducible evaluation framework.
Following previous work [45, 27], we evaluate our agent on two settings: (1) high-level tasks requiring the agent to autonomously plan and execute actions to complete task goal based on screenshots and

---

Table Performance comparison on AndroidLab [44]. The table presents commercial systems (lop section), open-source foundation models and their tine-tuned variants (middle section), and our approach (bottom). The best performance values are highlighted in bold, while underlined values denote the sub-optima results.
Model
Sub-Gool
Reversed Redun-
Reasonable Operation Ratio
Agent
Size
Success Rate
dancy Ratio
Suecess Rate
Gemini-1.0
12.6
72.5
76.7
10.9
Claude-3-Opus
15.1
81.4
83.9
13.0
Gemini-I.5-Pa
18.5
106.0
91.5
16.7
Claudo-3.5-Sonnct
32.7
113.4
81.2
29.0
GPT-4o
35.0
87.3
85.4
31.2
AutoGLM
36.2
LLaMA3,2-1 IB-vsioo
11B
13.0
61.7
87.9
10.1
CogVLM2-ft
19B
16.1
57.4
85.6
11.6
UI-TARS-ft
7B
18.2
64.1
88.0
14.0
Qw
7B
18.7
70.6
76.8
14,9
Qwen2-VL-ft
7B
22.6
65.2
88.3
18.1
Qwen2.5-VI.-ft
24.2
59.5
76.3
20.6
Ul-Genie-Agent (Ours)
35.4
91.3
28.8
Ul-Genie-Agent (Ours)
46.3
92.8
38.7
UI-TARS
72B
12.0
40.8
70.0
9.6
UI-TARS-ft
72B
28.4
81.4
81.6
22.1
Qwen2.5-VI.
72B
26.1
68.7
81.4
23.9
Qwen2-VL.ae
29.3
84.5
24.6
Qwen2.5-VI-fft
72B
30.9
81.3
79.3
25.0
Ul-Genie-Agent (Ours)
72B
48.2
92.1
93.3
41.2
action history; and (2) low-level tasks that provide step-by-stey low-level instructions for the agent to execute predefined actions. For both settings, employ three primary metrics: (1) Success Rate (SR), measuring the percentage of successfully completed actions; (2) Type Accuracy, assessing the agent's ability correctly predict action types (e.g., click, long-press, scroll); and (3) Grounding Accuracy, evaluating the agent's s precision in locating and interacting with the correct UI elements.
Tab. 2 presents comparison of our UI-Genie-Agent against proprietary systems and open-source foundation models on the AndroidControl benchmark [15]. Our approach consistently outperforms existing methods across all model sizes. The 3B variant achieves 93.8% and 72.9% success rates low and high-level tasks respectively, surpassing previous SOTA by 1.7% and 1.8%. The 7B and 72B variants further improve performance, with 72B model reaching 77.0% success rate high-level tasks, exceeding UI-TARS by 2.3%. The consistent improvements across model sizes demonstrate the robust scalability of our UI-Genie framework with the integration of high-quality trajectories during agent training.
Dynamic Evaluation. AndroidLab [44] provides controlled emulator environment where the agent's actions are directly executed, and success depends on its ability to adapt to dynamic states resulting from its interactions. The benchmark comprises 138 tasks across nine categories of frequently used offline static applications (c.g., maps, calendars, books, music players, etc.). These tasks simulate typical daily user interactions such managing events, editing notes, and retrieving information.
We evaluate performance on AndroidLab using the metrics defined in [44], which capture task completion different levels: (1) Sub-Goal Success Rate (Sub-SR), evaluating completion of individual task components; (2) Reversed Redundancy Ratio (RRR). assessing operational efficiency compared to human benchmarks; (3) Reasonable Operation Ratio (ROR), measuring the proportion of effective operations that change the screen state; and (4) Success Rate (SR), measuring the percentage of tasks where all uub-goals are completed. As shown in Tab. 3, our approach demonstrates superior performance compared to both commercial systems and various open-source foundation models and their fine-tuned variants.
Besides, Android Agent Arena (A3) [6] presents more comprehensive and challenging online evaluation environment. It employs an actual Android device emulator with 201 tasks spanning 20 predefined online top-rated applications selected from 18 categories (news, travel, ticketing, etc.). This benchmark includes particularly demanding tasks requiring more than 15 interaction steps and multi-frame information retrieval. These features make A3 highly effective for assessing an agent's ability to perform tasks real-life scenarios execute complex action sequences in response to evolving environments. The metrics detailed in A3 [6] used to evaluate UI-Genie's '

---

Table 4: Evaluation results on benchmark [6]. Fune Task Success Rate by Evaluation Function: LLM
SR: Task Success Rate by Commercial LLM Evaluation; EASR: Essential State Achievement Rate. Highest overall scores are indicated as bold for Func SR, underlined for LLM SR, and in italic for EASR. Op., S.Q. and M.Q. represent operation, single-frame frame query and multi-frame frame query in A3 respectively. Agent
Metric
Easy
Med.
Hard
Op.
S.
M. Q.
Overall
Func SR
11.7
2.6
0.0
8.4
2.0
0.0
6.5
Chaude-3.5-some
LLM SR
13.8
2.6
0.0
9.8
2.0
0.0
8.8
ESAR
23.6
16.7
14.2
22.1
14.7
14.9
17.6
Fune SR
23.4
5.2
0.0
17.5
0.0
0.0
12.9
Qwen2.5 LL-22
LLM
27.7
19.6
0.0
15.9
ESAR
39.8
27.2
29.4
36.4
25.6
28.3
33.2
Func SR
28.7
9.1
0.0
21.0
0.0
16.9
UI-TARS 7B
L.M SR
34,0
15.6
0.0
23.8
16.3
0.0
21.9
ESAR
55.8
40.1
41.8
51.1
36.2
35.7
46.5
Func SR LLM ESAR
38.3
6.5
0.0
28.0
2.1
0.0
20.4
UI-Genie-Agent-78 (Ours)
40.4
30.1
0.0
24.4
61.8
40.4
43.6
57.8
39.5
34,9
51.4
capability. This evaluation incorporates two distinct methods for assessing task completion: (i) function-basee evaluation and (ii) commercial LLMM-based essential states evaluation. Employing these different approaches provides comprehensive understanding of UT-Genie's performance from multiple perspectives.
Tab. 4 demonstrates UI-Genie's superior performance compared to previous SOTA agents such as UI-TARS. Specifically, Ul-Genie achieves an approximate 5% higher success rate function- based evaluation and 3% improvement in commercial LLM-based evaluation. Furthermore, its Essential State Achievement Rate (ESAR) surpasses that UI-TARS by approximately 5% overall, highlighting UI-Genie's ' enhanced capabilities in real-life scenarios.
4.2.2 Reward Model Evaluation
We conduct evaluation of UI-Genie-RM, using custom benchmark. since there is established standard benchmark for GUI agent reward models. Our benchmark derives from test sets of three open- source datasets: AndroidControl [15], AMEX [5], and AndroidLab [44]. For step-level evaluation, we sample 200 distinct ground truth actions as positive examples from each dataset, pairing each with corresponding negative action generated by the agent model and verified through rule-based methods. For outcome-level evaluation, we include 200 ground truth trajectories as positive samples, complemented by equal number of negative trajectories created through controlled trajectory corruption for AndroidControl and AMEX. We further augment this with 100 additional trajectories (50 successful, 50 failed) generated during AndroidLab dynamic testing and validated using pre- defined rules. This creates a comprehensive benchmark containing 1,050 paired items.
We evaluate LeGGenie-RM' accuracy using F1-score for both step-level and outcome-level reward assessment, with detailed performance breakdowns across task complexity categories: easy (under steps), medium (5-10 steps), and hard (over 10 steps). For comparison, establish strong baselines using advanced proprietary MLLMS (GPT-40, Geminil. 1.5pro, Gemini2.5-pro) and open-source models (Qwen2.5-VL--BB72B) As these models lack specific reward modeling training, we carefully craft prompts to adapt them for this purpose. Additionally, we enhance inputs with visual prompts in screenshots compensate for these models' limitations in processing spatial coordinates.
Tab. presents our evaluation results. UI-Genic-RR RM consistently outperforms all baseline models across both evaluation types and all task complexity levels. The performance gap becomes particularly pronounced hard tasks, where UI-Genie- maintains robust performance (68.7% F1-score for step-level, 70.5% for outcome-level) while other models show significant degradation. This demonstrates the effectiveness our specialized architecture and training approach, particularly for complex GUI interactions requiring extensive historical context understanding.
Discussions
We presented UT-Genie, a self-improving framework that addresses key challenges training GUI agents by generating high-quality synthetic trajectories and introducing specialized reward model,

---

Table Performance comparison of reward models on step-level outcome-level evaluation across different
task complexity categories. We report Fl Fl-score (%) for classifying actions and outcomes as positive or negative.
Step Reward
Outcome Reward
Model
Overall
Easy
Medium
Hard
Overall
Easy
Medium
Hard
GPT-4o
68.1
72.5
65.2
54.8
66.7
70.3
58.8
56.8
Geminil.5 p-rpo
65.1
70.5
62.5
51.2
71.5
72.4
67.5
54.5
Gemini2.55pn
72.9
74.1
64,4
55.8
74.3
78.2
73,2
64.9
Qwen2,5-VL-
56.6
60.2
55.1
47.9
58.9
64.1
48.4
41.3
Qwen2.5-VL-72B
66.2
71.5
64.5
50.9
67.6
68.6
65.5
46.4
Ul-Genie-RM (Ours)
79.6
83.5
81.9
68.7
82.1
84.3
81.3
70.5
UI-Genie- R.M. Our framework eliminates reliance on manual annotation through iterative agent- reward model co-evolution, achieving state-of-the-art performance and producing two novel datasets. The results demonstrate the potential of synthetic data generation with process supervision for scalable GUT agent development.
Limitations. While effective overall, our reward model may occasionally generate suboptimal rewards signals, resulting in failed trajectories during training data expansion. Although observe significant performance improvements when applying our synthetic data to enhance agent capabilities, the framework cannot guarantee the production of fully correct trajectories across all GUI tasks.
Broader Impact. This work can help people with disabilities better use mobile devices, making the technology more accessible to broader population. However, the high computation cost of training such a model results significant carbon emission. Moreover, while the Ul-Genie framework aims to advance automation in GUI interactions, we acknowledge the importance of recognizing potential risks associated with fully automated bots. If not properly monitored, these bots could lead to misuse, such conducting financial fraud stealing private data without authorization.
6
Acknowledgment
This study was supported in part by National Key R&D Program of China Project 2022/ID0111 100, in part by the Centre for Perceptual and Interactive Intelligence, CUHK-led InnoCentre under the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Government, in part by NSFC-RGC Project N CUHK498/24, and part by XW).
Guangdong Basic and Applied Basic Research Foundation (No.
10

---

References
[1] Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: compositional generalist-specialis framework for computer use agents, arXiv preprint UrXiv:2504.00906 2025.
[2] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. ArXiv, abs/2308.12966 2023.
[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.1392 2025.
[4] Cameron Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter Cowling. Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and Al in games, 4(1): 1-13, 2012.
[5] Yuxiang Chai, Siyuan Huang. Yazhe Niu, Han Xiao, Liang Liu, Dingyu Zhang, Peng Gao, Shuai Ren, and Hongsheng Li. Amex: Android multi-annotation expo dataset for mobile gui agents. arXiv preprint wxXiv2240..17499 2024.
[6] Yuxiang Chai, Hanhao Li, Jiayu Zhang, Liang Liu, Guangyi Liu, Guozhi Wang, Shuai Ren, Siyuan Huang, and Hongsheng Li. A3: Android agent arena for mobile gui agents. arXiv preprint arXiv:2501.01 149,,1111,,,,0 2025.
[7] Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: process supervision without process. arXiv preprint arXiv:2405.03553, 2024.
[8] Xinyan Chen, Renrui Zhang, Dongzhi Jiang, Aojun Zhou, Shilin Yan, Weifeng Lin, and Hongsheng Li. Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning. arXiv preprint arXiv:2506.0533/, 2025.
[9] Kanzhi Cheng. Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401. 2024.
[10] Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rstar-math: Small lims can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04519 2025.
[11] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501.12948 2025.
[12] Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14281-142999 2024.
[13] Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. arXiv preprint arXiv:2307.12981, 2023.
[14] Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng- Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703 2025.
[15] Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Camphell-Ajala, Divya Tyamagundlu, and Oriana Riva. On the effects of data scale on computer control agents. arXiv e-prints, pages arXiv-2406, 2024.
11

---

[16] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards. Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever. and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
[17] Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprinl 1.Xiv:3311.07555 2023.
[18] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 2xX1:24112.19437, 2024.
[19] Yuhang Liu, Pengxiang Li, Congkai Xie, Xavier Hu, Xiaotian Han, Shengyu Zhang, Hongxia Yang, and Fei Wu. Infigui-rl: Advancing multimodal gui agents from reactive actors to deliberative reasoners. arXiv preprint arXiv:2504. 14239, 2025.
[20] Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hongsheng Li. Ui-rl: Enhancing action prediction of gui agents by reinforcement learning. arXiv preprint arXiv:2503.21620 2025.
[21] Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406. 06592, 2024.
[22] Shikhar Murty, Hao Zhu, Dzmitry Bahdanau, and Christopher D Manning. Nnetnav: Unsuper- vised learning of browser agents through environment interaction in the wild, arXiv preprint arXiv:2410.02907, 2024.
[23] OpenAl. Chatgpt. https: //t::///chaa /cttt.cchtttaaa//// .eeaat.ppea....m com, 2023.
[24] OpenAl. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023.
(25) Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint NXXv:2408.001999 2024.
[26] Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun. Shuntian Yao, al. Webrl: Training I1m web agents via self-evolving online curriculum reinforcement learning. arXiv preprint arXiv:2411.02337 2024.
[27] Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprini urXiv:2501. 122.2.2.............. 2025.
[28] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, al. Language models are unsupervised multitask learners. OpenAl blog, 188)9 2019.
[29] Christopher Rawles, Sarah Clinckemaillic, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Mary- beth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Camphell-Ajala, et al. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint 2XXi:2405.11577. 2024.
[30] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.033: 0000003000 2024.
[31] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre. George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pannecrshelvam, Mare Lanctot, et al. Mas- tering the game of go with deep neural networks and tree search. nature, 529(7587):484 489, 2016.
[32] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling ulm test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arXiv:2408. 03314, 2024.
12

---

[33] Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, et al. Os-genesis: Automating gui agent trajectory construction via reverse task synthesis. arXiv preprint arXiv:2412. 19723, 2024.
[34]
Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu, and Ruslan Salakhutdinov. Insta: Towards internet-scale training for agents, arXiv preprint arXiv:2502.067700 2025.
[35] Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14277. 2022.
[36] Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent--2: Mobile device operation assistant with effective navigation via multi-agent collaboration. arXiv preprint arXiv:2406..1000 2024.
[37] Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan. Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent: Autonomous multi moltll mobile device agent with visual perception. arXiv preprini arXiv:2401.16158. 2024.
[38] Ke Wang, Junting Pan. Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang. Houxing Ren, Mingjie Zhan, and Hongsheng Li. Mathcoder- VL: Bridging vision and code for enhanced multimodal mathematical reasoning. In The 63rd Annual Meeting of the Association for Computational Linguistics, 2025.
[39] Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations. arXiv preprint arXiv:23/2.08995 2023.
[40] Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503. 10291, 2025.
[41] Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation. arXiv preprint Siii:2503.052344 2025.
[42] Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, et al. Os-atlas: A foundation action model for generalist gui agents. arXiv preprint arXiv:2410.23218. 2024.
[43] Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Agentrm: Enhancing agent generalization with reward modeling. arXiv preprint arXiv:2502. 18407, 2025.
[44] Yifan Xu, Xiao Liu, Xueqiao Sun, Siyi Cheng, Hao Yu, Hanyu Lai, Shudan Zhang, Dan Zhang, Jie Tang, and Yuxiao Dong. Androidlab: Training and systematic benchmarking of android autonomous agents. arXiv preprint arXiv:2410. 24024, 2024.
[45] Yiheng Xu, Zekun Wang, Junli Wang, Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint arXiv:2412.04454 2024.
[46] Junyan Ye, Dongzhi Jiang. Jun He, Baichuan Zhou, Zilong Huang, Zhiyuan Yan. Hongsheng Li, Conghui He, and Weijia Li. Blink-twice: You see, but do you observe? a reasoning benchmark on visual perception. arXiv preprint arXiv:2510.09361, 2025.
[47] Fei Yu, Anningzhe Gao, and Benyou Wang. Ovm, outcome--upprriseed value models for planning in mathematical reasoning. arXiv preprini arXiv:2311.09724 2023.
[48] Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels. arXiv preprint arXiv:2412.0198/ 2024.
[49] Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Ziyu Liu, Shengyuan Ding, Shenxi Wu, Yubo Ma, Haodong Duan, Wenwei Zhang, et al. Internlm-xcoopoosee.. 5-reward: A simple yet effective multi-modal reward model. arXiv preprint arXiv:2501.12366 2025.
13

---

[50] Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. Appagent: Multimodal agents as smartphone users. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1-20, 2025.
[51] Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408. 15240, 2024.
[52] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprini arXiv:2504 10479, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The three main conclusions and the scope of our paper are detailed in the final section of the introduction and summarized in the abstract.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Sec. 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
All the assumptions and theorems are clearly stated.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The architecture details are provided in the Methodology and Experiment Section for reproducing the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We open-source our code and data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nipp cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee
/nips. wttpccccpppcccaeeccccccccccccclt /tweeddeddee/oeeeemiiiiiiii ssssssolicyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The training and test details are provided in the Experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We conduct experiments only once and report the accuracy of the best model, and would be too computationally expensive to conduct the pre-training multiple times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //te://eeeips// http://rrrppp...pppiii////////// httpp//ww..ttttth.cssteeeeeeeeee ctt.iieisseillenneccc ines?
Answer: [Yes]
Justification:
Our research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss potential societal impact of the work in Sec. 5.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We cite the original papers or websites that produced the code package or dataset.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our released code constitutes new asset and will be well documented on GitHuu to complement the documentation provided by this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: We use LLMs in three key components of our methodology: (1) converting UT element function descriptions to executable action descriptions, (2) judging whether sampled actions are true negatives or equivalent to positíves during the negative sample construction process, and (3) expanding current task instructions. We present details in Sec. 3.3, Sec. 3.4 and Sec. D.1.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy (https: http://eeurrpppeeeee wwc/oooeeeenne///2002 /et.a//ceee//////1/// for what should or should not be described.
21

---

A
Ablation Study
A.1 Ul-GenieRM for Test-time Scaling.
Beyond serving an effective model for synthetic GUI operation trajectory selection, Ul-Genie-RM, RMM can also enhance agent model performance through a best-of-N sampling strategy. During inference, the agent model generate N candidate actions, which are then ranked by Ul-Genie-RM RM to select the highest-scoring one as the final output.
We evaluate this approach on the AndroidControl benchmark using Qwen2.5- base models. Our experiments measure both step-level success rates (accuracy of predicted single-step actions compared to ground truth) and task-level success rates (successful completion of entire tasks, where each step must align with ground truth). Additionally, we categorize the results by task difficulty: easy (less than 5 steps), medium (5-10 steps), and difficult tasks (more than 10 steps).
As shown in Tab. 6. UI-Genie-RM RM consistently improves performance across both metries and all model sizes. The Qwen2.5-VEL VL-3B model shows substantial gains, with the step-level success rate increasing from 60.3% to 63. 1% and the task-level success rate improving from 7.5% to 10.2% when using best-of- 10 sampling. Similarly, the Qwen2.5-VL.- particularly for medium-diffeeee 12.2%. Importantly, our results demonstrate scaling benefits when expanding the sampling space from N=5 to N=10. This performance improvement indicates that Ul-Genie-RM effectively identifies optimal actions from larger candidate pools, showcasing its ability to provide accurate reward signals for GUT interactions across varying task complexities.
model demonstrates improvements,
tasks where the task-level success rate increases from 10.5% to
A.2 Effectiveness of Ul-Genie-RM Architecture
To evaluate the sufficiency of using five most recent screenshots to process historical context for Ul-Genie-RM, we conduct an ablation study analyzing our model's performance and efficiency across different task complexities.
For simple tasks (<5 steps), the actual input is naturally limited by the number of all available history (c.g., 3 screenshots at step 4). We evaluate preserving at most I, 3, and 5 screenshots in Tab. 7. While using one screenshot is most efficient, it shows limited performance due insufficient historical context, particularly in outcome reward evaluation. Using up to five screenshots achieves the best performance without introducing significant inference overhead, since the actual number of processed images remains low.
For complex tasks (2 8 steps), we evaluate the screenshot number at 5, 8, and 10. We observe that using 5 screenshots achieve comparable performance to screenshots, while increasing to leads to performance degradation, most recent ones may distract the model by less relevant information. Moreover, larger window sizes significantly increase the computational cost. These results demonstrate the effectiveness of our choice of using five most recent screenshots to achieve the best accuracy and efficiency trade-off.
presented in Tab. 7. This suggests that older screenshots beyond the
A.3 Effectiveness of Self-Improvement Approach
To quantitatively evaluate our iterative self-improvement framework, we track the performance enhancement of both the agent and reward models across multiple rounds. Fig. 4 illustrates the progressive performance improvement measured through task success rate on Androidl .ab (for UI- 7enie-Agent-7B) and step reward accuracy (for Ul-e Our results demonstrate substantial and consistent improvements across iterations. Notably, the task success rate of Ul-Genie-Agent- on AndroidLab increases substantially from an initial 18.1% 1% (round 0) to 38. 78.7% (round 3), finally surpassing the state-of-the the-art art AutoGLM method. Concurrently, the reward model accuracy of UI- Genie-RM improves from 68.2% 79.6%, demonstrating the effectiveness of our self-improveemet approach. The most dramatic improvement occurs during the first round, highlighting the strength of our reward-guided exploration for trajectory discovery. Rounds 2 and show continued improvement as the framework progressively incorporates more complex tasks and refines both models. These results validate our self-improvement framework's ability to break through the initial performance limitations complex tasks through mutual enhancement of agent and reward models.
22

---

Table Performance comparison of Qwen2.5-VV models with and without UI-Genie- RM test-time scaling. Results show step-level success rate (Step SR) and task-level success rate (Task SR).
Overall
Easy
Medium
Hard
Model
N
Step SR
Task SR
Step SR
Task SR
Step
Task
Step SR
Task SR
QwenVVL2.5-3B
60.3
7.5
50.4
11.6
63.5
7.1
58.3
0.9
+UI-Genie-RM
61.8
8.3
52.2
13.5
65.7
7.6
58.4
0.9
+UI-Genie-RM,
10
63.1
10.2
53.5
15.5
67.0
9.7
59.6
1.9
QwenV7.2.5-7B
64.9
14.7
64.4
29.7
66.7
10.5
61.1
1.4
+UI-Gemie-RM
64.9
15.1
65.8
31.1
67.6
10.7
58.6
0.9
+UI-Gemie-RM 10
65.2
16.2
65.5
31.7
68.1
12.2
58.9
1.4
Table 7: Reward model performance with different seen images.
Task
Img Step-level Acc(%)
Outcome Acc(%)
Inference Time(s/iter)
Complexity
Num
76.4
74.5
0.59
Simple
3
78.2
79.8
1.05
5
83.5
84.3
1.57
5
72.9
73.5
2,46
Complex
73.4
75.2
5.10
Frentee
10
69.7
71.3
7.81
Figure 4: Performance evolution across iter- ative self-improvement rounds.
A.4 Ablation on Complex Task Generation and Reward Model Updating
To evaluate the effectiveness of each component of our method, we conduct ablations the following
two aspects, as shown in Tab. 8: (1) Removing complex task generation. Without introducing challenging tasks in the final round, the agent's improvement is limited after Round 2 (31.2 to 32.6), highlighting the importance of a curriculum model updates. Updating the reward model during the self- self- imffroemeentt phase provides essential guidance for trajectory exploration, resulting in higher nigher-quality synthetic data generation and enhanced performance of both models.
increasing difficulty. (2) Freezing reward
B
Dataset Analysis
provide the detailed statistics of the Ul-Genie-Agent-16) dataset analyze its diversity.
Action Distribution: As shown in Tab. 9, our dataset contains 8 distinct action types with realistic distribution of GUI interactions. While Click is naturally the most frequent action, the significant presence of Type (9.7%) and Swipe (6.7%) ensures the agent learns to handle complex data entry and navigation tasks.
Table More ablation studies.
Complex Task Reward Model
Reward Accuracy
Agent Success Rate
Generation
Update
Round
Round
Round 2 Round 3 Round 0 Round
Round
Round
X
X
69.2
69.2
69.2
69.2
18.1
29.8
31.2
32,6
69.2
73.6
75.4
77.1
18.1
29.8
33.3
34.1
69.2
69.2
69.2
69.2
18.1
29.8
31.2
35.5
69.2
73.6
75.4
79.6
18.1
29.8
33.3
38.7
23

---

Goal Diversity: We categorize task goals into 7 distinct categories in Tab. 10, showing significant diversity across mobile GUI scenarios. The task goals span wide range of realistic user intents, preventing model overfitting to a narrow set of tasks.
Task Complexity: Crucially,
dataset emphasizes challenging tasks over trivial simple tasks, as presented in Tab. 11. We find that medium (5-10 steps) and hard tasks (>10 steps) comprise over 91 % of the dataset, which is essential for developing highly capable agents.
Table 9: Dataset analysis on action distribution.
Action Type
Open
Click
Swipe Long press
Type
System button
Wait
Terminate
Number
2208
8428
1054
15
1526
218
25
2208
Percentage
14.08%
53.74 74%
6.72%
0.10%
9.73%
1.39%
0.15%
14.08%
Table 10: Dataset analysis goal category.
Goal Category
Numher
Percentage
Configuration & Settings
518
23.46%
Information Retrieval
610
27.63%
Task & Item Management
291
13.18%
Scarch Tasks
194
8.79%
Application Control
178
8.06%
Site Navigation
183
8.29%
Data Entry
234
10.60%
Table 1I: Dataset analysis on task difficulty.
Task Difficulty
Trajectory Length
Number
Percentage
Easy
<5 Steps
182
8.24%
Medium
5-10 Steps
1165
52.76%
Hard
>10 Steps
861
38.99%
More Evaluation Results
C.1
Performance AndroidWorld Benchmark
We evaluate our final agent models on the AndroidWorld benchmark. As shown in the Tab. 12, UI- Genie-Agent-7B achieves success rate of 36.2%. surpassing strong proprietary models like GPT-4o. Our 72B model shows significant performance gains compared to its base model Qwen2.5-VL-72B, outperforming all other methods.
C.2 Out-of-distributiio Test of UI-Genie-RM
We evaluate UI-Genie-RM on an out-of-distribution test set constructed from the AndroidWorld [29] benchmark, using 11 apps that were unseen during Ul-Genie-RM training. As shown in Tab. 13, UI-Genie-RM achieves performance competitive with the powerful proprictary model Gemini2.5-pro and significantly outperforms other open-source MLLMs, demonstrating strong generalization to unseen domain.
D UI-Genie Training
D.1 Definition of Action Space
To facilitate effective cross-dataset training, UI-Genie implements comprehensive unified action space across all datasets including AndroidControl [15], AMEX [5], AndroidLab [44], and our newly- generated synthetic data. While building upon the foundation established by Qwen2.5-VI models [3], we incorporate several critical enhancements to improve operational flexibility and generalization capabilities. As detailed in Tab. 14, our action space comprises eight fundamental action types: open,

---

Table 12: Comparisons on AndroidWorld benchmark.
Models
Agent Success Rate (%)
GPT-4o
34,5
Claude Computer Use
27.9
Qwen2.5-VI.-7B
22.0
UI-TARS-7B
33.0
UT-Gemie-Agent-7B
36.2
Qwen2,5-VL-72B
35,0
Aguvis-72B
26.1
Ul-Genie-Agent-72B
47.4
Table 13: OOD test of Ul-Genie-RM.
Model
Step-level Ace (%)
Outcome Acc (%)
Gemini2.5-pro
71.5
75.7
Qwen2.5-
46.8
64.4
Owen2.5-VL-722
54.2
68.0
U1-Genie-RM
74.5
76.2
click, swipe, long-press, type, system_button, button, wait, and terminate. For each action type, we define specific parameters that ensure precise execution and compatibility with real-world mobile environments.
key advancement over the original Qwen2.5-VV VL implementation is UI-Genie's dual-mode interac- tion capability. Beyond predicting absolute coordinates, our model can process Set-of-Mark (SoM) annotations, where interactive UI elements are labeled with numeric tags in the sereenshot. This enables Ul-Genie to predict element indices rather than exact coordinates when provided Set-of- Mark visual prompts, significantly enhancing gencralization across diverse interface layouts. For each action type, define tailored parameters: the open action specifies app name; click and
use either coordinates or SoM indices to identify targets; swipe includes starting position (coordinate/SoMM), direction (up/down//eef/riiht)) and distance (short/mediumm/long)) wait specifies duration in seconds; and terminate indicates task completion status (success/failure). Additionally, incorporate semantic diction_desc parameter across all action types, offering crucial semantics when incorporated into the historical record of operations. Moreover, in the negative sample construction process, we utilize the sampled action and its action_ _ddsc evaluate the action alignment effectively. We use some pre-defined bise- functions and potentially trigger an LLM judge whether the sampled action is true negative equivalent to positive.
Table 14: Action space and parameter specification for UI-Genie
Action Type
Parameters
open
text; action desc
click swipe
coordinate; som; action_dess
coordinate; som; direction; distance; action_desc
long-press
coordinate; som; action_ text; action_
type
system_ tonton
button; action desc
wait
time; action _esoon__ecceee
terminate
status; action desc
D.2 Training details
We implement both UI-Genie-RM and UT-Genie-Agent based on the Qwen2.5-Vi family of models due to its strong multimodal understanding capabilities. For the reward model, we adopt Qwen2.5- VL-7B as the backbone. For agent models, we experiment with three variants across different model sizes: 3B, 7B, and 72B, enabling analysis of scaling effects on performance.
Reward Model Training. For training Ul-Genie-RM, we adopt Qwen2.5-VIL-7B as the backbone. We introduce special tokens (I+]) and -|) into the original vocabulary of Qwen2.5-VL to represent
25

---

You are expert in evaluating the performance of phone operating agent. The agent designed to help user to complete tusk retrieve information from the phone. Given the user's task instruction, the agent's history of actions and current action, your goal is to decide whether the agent's current action is corrctt or not. Each action the sequence preeeded by corresponding screenshot that captures the context in which the action occurs. you green hollow circle surrounded by red reetangle with "P" in the top-right comerr this is an additional marker I've added to indicate where the action is performed on the sereen.
#HE valuation Criteria
Whether the agent's current action is correct and corresponding the user's task instruction # Istruutttio
Review the lask instruction and the history of actions leading up the Check if the current action logically progresses toward the goal (e.g., opening the right upp, entering correct input, navigating efficiently).
slep.
If the action irrelevant, incorrect, Icads the agent away from the goal, mark it [wrong]. If the action is appropriate and contributes task completion, mark it [correct]. If the agent is stuck loop (repeating the same action without progress), mark as [wrong].
#IMPPRTANT
action always follows corresponding screenshot (even only few are provided). You should whether answer [correct] or [wrong].
goal the task: (lask instruction)
history actions: [formatted__m __t_i_cttos
current action agent: [current action]
Figure 5: Step-level reward evaluation prompt used for comparative baseline models.
positive and negative class labels, respectively, thus formulating reward prediction as a next-token our constructed dataset of 458k samples
generation task. We first conduct supervised derived from existing datasets. Subsequently, we perform iterative refinement using a total of 59k process-reward samples collected during seff-improvement cycles. After each cycle, we train the model using the AdamW optimizer with a learning rate of le-5 and a global batch size of 160 on 20 L.00s machines with 4 GPUs each. The model parameters are fully finetuned, while we keep the vision encoder frozen. The input image is limited to have maximum pixels of 602112.
Agent Model Training. We first establish an initial agent model based on Qwen2.5-VL using existing datasets including AndroidControl [15], AMEX [5], and AndroidLab [44]. This initial model fully finetuned these datasets with only freezing the vision encoder, with learning rate le-5 and batch size 160.
Then we use this initial agent model for trajectory exploration in our dynamic Android environment under UI-Genie-RM's supervision. Through three rounds of iterative improvement, we generate Ul-Genie-Agent-Iee 16k, synthetic dataset containing 16k high-quality trajectories without manual annotation. After each round, we continue fine-tuning this 7B model using newly collected operational trajectories with the same training hyperparameters for one epoch.
Furthermore, we train Ul-Genie-Agent-D and UT-Genie-Agent-72B on UI- Sl-Geeie-Ageet-166 and the aforementioned open-source datasets for one epoch. For the 3B model, it is fully finetuned with vision encoder parameter frozen using the Adam W optimizer with learning rate of le-5 and a global batch size of 160. For the 72B model, we finetune it use lora due to GPU memory limit. Specifically, we use rslora and we add lora modules to the vision encoder, projection layer and the LLM. The lora rank and lora alpha are set to be 256 respectively, and we use PiSSA method to initialize the lora weights. All experiments for training agent models are conducted on 20 L40s machines with 4 GPUs each.
26

---

You expert evaluating the performance phone operating agent.
The agent designed help user complete task retrieve information from phone. Given the user's task instruction, the agent's trajectory of actions and final response, your goal decide whether the agent's execution successful not. Each action screenshot that cuptures context which action uccurs. you see green hollow circle surroumded by rectangle with "P" the lop-right comeer, this is an additional marker I've added to indicate where action performed the screen.
sequence preceded by corresponding
##E valuation Criteria
Whether the agent's trujectory is effective corresponding to the user's task instruction #lInstructions
Review the agent's actions and the corresponding screenshoot step by step. if the agent is stuck boginning, that's failure.
very first login stage, which means fails to log into target website at the
Determine the agent has achieved all the requirements of the task instruction based on the trajectory.
The agent sometimes can't stop after finishing a task and continue doing repeated actions. This should be considered failed us the ugent should output 'finished' the final step.
If the agent is stuck in the loop, which means they don't even get close to the goal before they get stuck in the loop, that's a failure.
If the task is to retrieve information or answer question, the agent can be considered successful only when it provide the correct and necessary information. You should carefully check if the provided answer is correct and sufficient to answer the user query.
#IMPPRTTANT
In the trajectory, an action always follows corresponding screenshot in the images(only several last screenshots are provided, but the trajectory of actions is compactly presented in this context), which shows the observation of the agent.
2. You should w wetther answer [success] or [failure]. #slnppt
The goal of the task: (task instruction)
trajectory of actions: (formatted actions) final response of the agent: (final response)
Figure 6: Outcome-level reward evaluation prompt used for comparative baseline models.
D.3 Implementation Details
We implement both UI-Genie-RM and UI-Genie-Agent based on the Qwen2.5-VV VL model family, leveraging its strong multimodal understanding capabilities. For our experiments, we develop variants across three model sizes (3B, 7B, and 72B) to analyze performance scaling effects. All experiments are conducted on a distributed training setup of L40s machines with GPUs each.
D.3.1 Reward Model Training
For UI-Genie-RM, we adopt Qwen2.5-VI--77 the backbone and introduce special tokens () and (I 1) into the vocabulary to represent positive and negative class labels, effectively formulating reward prediction
next-token generation task. We conduct initial supervised fine-tuning constructed dataset of 458K samples derived from existing GUI operation datasets. Then we perform iterative refinement using 59K process-rewe reward samples collected during self-imp seff-improvement cycles. For the reward model training, we use the AdamW optimizer with a learning rate of le-5 and a global batch size of 160. We fully fine-tune the model parameters while keeping the vision encoder frozen. Input images are constrained to maximum of 602, 12 pixels to maintain computational efficiency.
D.3.2 Agent Model Training
Our agent model training follows progressive enhancement approach:
27

---

Initial model establishment: We first train a 7B baseline agent using existing datasets from Android- Control, AMEX, and AndroidLab. This model is fully fine-tuned with only the vision encoder frozen, using a learning rate of le-5 and batch size of 160.
Iterative self-improvement: Using this initial agent, we conduct three rounds of trajectory explo- ration in dynamic Android environments under UI-Genie-RM. supervision. This process generates Ul-Genie-Agent-Ioo our synthetic dataset containing 16K high-quality trajectories without manual annotation. After each round, we continue fine-tuning the 7B model on newly collected trajectories for one epoch, maintaining the same hyperparameters.
Model scaling: Finally, we train UI-Genie Agent at different parameter scales (3B and 72B) using the combined UI- Ul-Genie-Agent 16K and open-source datasets. The 3B model is fully fine-tuned with frozen vision encoder using A (Ir=le-5, batch size=160) We fine-tune the 72B model using rank-stabilized LoRA (rslora) due GPU memory constraints, with lora modules added to the vision encoder, projection layer, and L.M.. We set both lora rank and lora alpha to 64 and 256 respectively, and initialize Iora weights using the PiSSA method.
This progressive training approach enables us to effectively leverage both existing and synthetic data while efficiently scaling our model across different parameter sizes.
E Prompt for Evaluation
In this section, we provide the detailed prompts used to adapt our comparative baseline models for reward evaluation tasks. Since neither proprietary MLLMs (GPT-4o, 1.5pnil..-prr, Gemini2.5-pro) nor open-source models (Qwen2.5-VL-7BB722) are specifically trained for reward modeling in GUI operations, carefully craft specialized prompts enable fair comparison. Our step-level reward evaluation prompt and outcome-level reward evaluation prompt are presented in Fig. 5 and Fig. respectively. These prompts were designed to provide structured guidance for evaluating both individual action correctness and overall task completion success, leveraging the models' capabilities for processing visual information from UI screenshots.
F
Example of UI-Genie Training Data
F.
Reward Data Example
We present examples of our reward data used for model training. Fig. illustrates action-level reward data samples, which include both positive and negative action examples. These samples are crucial for training our model to distinguish effective interactions from ineffective ones. Fig. showcases trajectory-leve reward data, comprising complete successful trajectories alongside failed ones, which serve as positive and negative examples respectively.
28

---

Open App. Salary
Step1: Open Spendie App.
Step2:
Manually Option
Step3: Income Option Expense Option. Expenses.
Step4: Salary Option
Neres
Walei
Labes
Lowe
Noation
Musrie
Vecrion
Mrat
Nustton
Scort
MUST
Racumenca
Racumeno
6
4
5
5
7
9
7
9
8
"action": "click", "coordinate": (63, e80). "action, desc": "Click at the left side the screen"
"action": "click". "coordinate": [316, 1071), "action desc": "Now enter the snlary 9000 rupess,5 for tan thousand
"action": "click", "osordinata":
723) "action_ desc": "Select
Recurrence option"
keep my everyday steps
Google
weight
height
History
Stap2: Click the Continue Farida
v
About you
About you
About you
Tiik frmmal
"action": "click", "coordinate": (413, 1078j. baction_deec" "olick on Next at the bottom right of the screen"
"action": "direction": "down", "action, desc". "Bwipe
"action": "click", "coordinate" (132, 625), "astion, desc": "Click on Drop-down button next Welght
down for content"
Figure 7: Examples of action-level reward data used for UT-Genie-RE RM training. The visual prompts displayed here are for illustration purposes only and are not used during model training. For simplicity, omit the history image displays, though they are included in the actual training process.
29

---

Task
On the Expedia app, search one-way flight Amsttaddam Berlin December for one adult.
Successful Trajectory
X
Falled Trajectory
My old shoes damaged and I'm planning shoes find casual shoes for the Zara
ZARA
Successful Trajectory
ZRRA
X
ZERA
Failed Trajectory
Figure 8: Examples of trajectory-Ievel reward data showing successful and failed task completions. We omit the history of actions in this visualization, but the complete action sequences are utilized during training.
30

---

F22 Synthetic Trajectory example
We present illustrative examples of trajectories synthesized by UI-Genie, demonstrating both suc- cessful and unsuccessful attempts at completing UI tasks. Fig. 9 showcases a successful trajectory generated under the guidance of UI-GenieRMM. As shown in the sequence, the reward model ef- fectively steers the agent through series of actions, resulting in the successful completion of the assigned task. contrast, Fig. 10 presents a failure case that illustrates a limitation in trajectory exploration. Although the reward model provides guidance for individual steps within the trajectory, the final outcome reward is negative, indicating overall task failure.
Tark eenrraattt Thon m
foriano Buadbaani
Cridte DONACC
Creite COVENCE
Create CE HRUUT
Corgay
Conpee
Compan
Seppan
(US 2001 ledion desc The haadd
(713, 9011 antion dest "Tig the fine
selle deshh 1oyie the sine
T13, (10) "ecfon deshh Tag Lan
Tact on dest Toatr the but same dy
Conterortect
Craste cortect
Creite contact
Z
Beal
Srall
dop "acter" didl" "opendern ee"
Stp "usten" Typs" "ea".
Stat Fache 7"ias" Tooor hale's acder dash
Sea Tachben "heme "saaa"
Juac. Lod" Pett Cocouni qu Iok. Houund
the tus bece oo A all slle
comping nanc' E.24
tun glay (irearr
Figure 9: A successful trajectory example. Under the reward guidance of UI-Genie-RM. the agent successfully discovers and executes sequence of actions that complete the assigned task.
31

---

Tal:: mat hal.
Settings
Sound
Sound
Sound
Sound vibration
Cat
Kag bucico
Hogg Angkriiiiaatoooo
volame
Monn voleme
Murn vakure
Klam refume
Octooru
Do Dasuub
Eo Cie
Vibrdon hapiic
VOuSSe Trppic
Vitrrtin hapiit
(75 2900 "tatenntroe Ta Seend
1819, T0ML Sirceee" ee Yiae"
ELL 19ML Siotten 'ull "Mac"
R22, 2001, Troeuee TNB "ua
100, 1800, mwihee Tall "f""""
acei selliic Si
dader huidle the folliary
didertuidin Eauancs ballndy
3.707
Rlvarr
vibration
Merroniie returre
Dibrart
Vibitton hipiic
101, 1906 "akcciee lort. "ila".
lued allicy Rewerd L013)
Figure 10: A failure trajectory example. Despite receiving process reward guidance at each individual step, the sequence ultimately results in task failure as indicated by the negative outcome reward.
32

---

G Ul-Genie Trajectory Example on Online Evaluation
G.1 AndroidLab case
We provide examples demonstrating UI-Genie-Agent capabilities on AndroidLab tasks. Fig. 11 illustrates the agent executing a financial task in which it successfully adjusts an expenditure amount to 500 CNY for the specific date of May 15, 2024 in bluecoins app. This example showcases the model's s ability to navigate through the application's 's interface locate and modify transaction data according to precise user instructions. In Fig. 12, we present another example where UI- Genie-Agent-728 interacts with the calendar application. The agent demonstrates its capability to edit event details, specifically modifying the end time of an event titled "work" to 7:00 PM. This example highlights the model's proficiency in handling time-based modifications within scheduling applications. These examples from our Androidl ab evaluation illustrate UT-Gienie-Agen is ability to understand contextual instructions and execute precise actions within real-world mobile applications, showcasing its versatility across different types of mobile interfaces and task requirements.
Taske: Art et the expeend ture on May 15, 2024 to 500 CNY
singg
goiggo
Oungen
Bringg
magg
ISC218
SOND Piactton:: "KESS" "aar": 42
3: Paassarn diice
(laction" "ore": "ather, "Fag on
Step E'acdion "dipt "Top
43, Sc Casttaa "action bigt
the
Taction (nose "Tap May 15th
Trarnaactois day.
d00.000
200.00 00
May
hange
Mos Phgis O 1ire
Sai Wader
Welll
Call Nole:
file
can
Ciles
Crew traeer
Ciher
Ciles
EXTrES
WOVE
SE'Ve
Vactoo "0.30" "San""
(amaars Typee Nout"
Snap (action: "OlO" 'SoY" 4. Stop ('00000 "So TV FATC" "SoR0"" 500
"ma the Bald the experie amoor..) 3
1'501"" dhgeee amrourt))
sine the sdlood experie;)
scguted 500 CN "ecton desct Houudiieet caruutto
Figure I1: Example of Ul-Genie-Agent- 72B executing an AndroidLab task. The task instruction: Adjust the expenditure on May 15, 2024, to 500 CNY.
33

---

Task: Edit the event with tte "ookk" chanae the end time be 7:00 PM.
Evarts mide
19.
19.4
Doart
Fivett.
Acre
Togr
Tagi
1
Altee
2: Draciert saick "ooe
Siao Coccionn" "ORdC "S0n 2
Stop Eastoons the LMBG.))
Teat
Ciactenn "CR3O "001" col cront Mine
600-1
button create now evert
IOU cito the Oncit Mclc:)
01
00
07
00
07
00
accon deac. "Selec 7 The hoar for factoo desc SSelic PM tor
fecton dears CCenim m the ueleted
"ection SeRCt "Save event
"dtata "auceess Nadtion 1110 ever
edlling
Figure 12: Example of UI- UI-Genie-Are Peeeit-Ageet-ii 72B executing AndroidLab task. The task instruction: Edit the event with title "work", change the end time to be 7:00 PM.
34

---

G.2 Android Arena (A3) case
We further evaluate UI-Genie's capabilities using the Android Arena (A3) benchmark. Fig. 13 demonstrates EL-Genie-Agent- executing news search task within the CNN application. The agent successfully interprets the task instruction and navigates through the application 's interface locate and utilize the search functionality, ultimately retrieving panda-related news articles. This example highlights the model's ability to understand domain-specinn instructions and operate effectively within media consumption applications. Fig. 14 presents another example of Ul-Genie-Agent-2 versatility, showing the agent completing settings modification task in the Coursera application. The agent demonstrates its capacity navigate through application menus, locate configuration options, and toggle specific settings. These examples illustrates UI-Genie-Ar Agent's 's robust performance across diverse applications and task types, from content searches to application configuration changes.
Tasku news about 'oanda' CNN.
CNN
Q
Google
Tap Nows
Yo CNN
World
U.S. PNRROO
Recert Searches
Recert Searches
trump
Who Amenoaan chesen be popet
Heatery may clue 24d
about
prodeceecr's
winh Robert provectto no more ornical posss about Greese
come
sate Stop ("action": "cick" "coordinate": [228, 1471, "action desc" "Tap the search query.")
New
men
How
Nach
Soup ("action": "open", "sexp": "CNN" "action, disce "Open CNN app.")
Stap ("action": "click" "coordinata": [480, 711. "aciton, desc" "Tap the the icon top right corner
Stop ('action": "type", "text" "panda" "action desc "Enter "panda" search bar.")
a
a
Search
Search
pance
Connel
pancial
Connol
Recert tearches
Clesr
"pands
Porcess eut tankoe rennedd Explait why
hoise
caba honp Pooogilongg Kong
Strence
China oumere Thety
deijing
debx
DNB behind the scesse panddss
Wutch have loon voom service
glant get
& antios
& Milise
Bame
Step ("action:: "cick", natee: 510) accion
Step ("action"; "serminate", "stalus" "succias"" "acilon, desc":
Figure 13: Example of Ul-Genie-Agent executing task defined in Android Arena (A3), The task instruction: Search news about 'panda' in CNN.
35

---

Task: Open 'settiins'' in Coursera and switch to dark mode.
C
C
Explore
Culendeo
Devethck
Ceail
Topics
Racionni
Gurpede Si
Vataggan
Quira
arathe.
Blckngg
Cautters
Enable nottficaiions
Enable noti notifica fications
SeAll
Play
Mode
rec pi of what you post week. rech achioved.
fet ai what you post week much achioved.
Targel
nersideera yout and woaky orjectives
verindera meet goots weaktt oqjective
k
Earn Your Degros
See
Magioe
"coarcinata":
("action'ss "clikk"
"Counsera" laction pesc" "Open the Courseera app-")
(406, 684). "action desc": "Ciosa the
[106, 1047], "action desc": "Tap the
[464, 1882), "action desc": "Tap on the Proffie Krnn the brltom right comeer screers")
update rotifiation by tapping the close buttonn
maybe later option the bottom left
correr scraert'
Settings
Settings
Profile
Airaly Cautsaa Cauasona aecuast
Aawab Caurson
Unght u
APFRARNNCE
AFREAANCE
That feeling of completion just around the conner
Hede
*
Nede
0
Sta't
Change tedereetaggggg
2
Pederver
SUPPORT
SUMPORT
Center
Step ("action": "click", "coorcinate": 1492, 67). actionn cesc "Tp on the settings too right
Seanch
nar
Siee (266, 426].
cnd ("action": "cEck" "coorcinate", "
Step ("action's "tarminate"" "status"
"aciionn desc". TTa on Dark
"success" Paction desc' "Lask has completed")
Appear settings."
Figure 14: Example of Ul-Genie-Agent- 7B executing a task defined in Android Arena (A3). The task instruction: Open 'settings" in Coursera and switch to dark mode.
36


Reviews:
[{"review_id": "xzFc3QbvGV", "reviewer": "Reviewer_yoko", "summary": "This paper propose a self-improving framework, named  UI-Genie, designed to address two core challenges in GUI agent development: the difficulty of reliably verifying agent trajectories and the lack of scalable, high-quality training data. The proposed framework introduces UI-Genie-RM, a reward model with an image-text interleaved architecture for precise trajectory assessment, supported by targeted data generation strategies. The authors also establish an iterative self-improvement pipeline, enabling exponential growth in solvable GUI tasks and training data. The framework demonstrates state-of-the-art results on multiple benchmarks.", "questions": "1. The paper lacks experiments verifying the necessity of training UI-Genie-RM within the proposed framework. If the reward model were replaced with other MLLMs and the reward model fixed during training (i.e., not updated), could similar performance be achieved?\n2. How is the performance in the popular dynamic mobile benchmark AndroidWorld?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The workload of this paper is sufficient, and the motivation makes sense. \n2. The dataset collected using the proposed method is large in scale, and the experiments verify the effectiveness of training the agent with the dataset.  \n3. Both the proposed method and dataset will be open-sourced, which facilitates reproducibility for researchers and can also inspire future work. \n\nWeaknesses:\n1. Using the reward model to judge the correctness of the data and use it for subsequent training is a common idea, and the technical innovation of this paper should be further clarified.\n2. The newly proposed GUI trajectory reward task should be clearly defined before introducing the method, including the input/output information, task objectives, etc., and it would be best to correspond these directly to the content in Fig. 2. \n3. In the experiment of 4.4.2 Reward Model Evaluation, the test samples are drawn from the same source as the training set, meaning they are from the same distribution. The superior performance of UI-Genie-RM may result from overfitting. This setup may not be entirely fair; the test set should be drawn from a distribution different from that of the training set.\n4. The paper lacks ablation studies on the three complementary data generation techniques used during the Data Construction for Reward Model Training phase. Such experiments are important to demonstrate the effectiveness of the algorithmic design in improving the quality of the training data. It also lacks ablation on whether the reward model is updated or not.", "was_revised": true, "final_justification": "It has addressed most of my concerns, and considering the technical innovation of the method, I will maintain my borderline positive score.", "rebuttal": "Thank you for your thorough review and positive feedback on our paper. We appreciate your recognition of our motivation, workload, large-scale generated data and open-source commitment. We have carefully considered your valuable suggestions and will address your points below.\n\n**W1: The technical innovation of our paper.**\n\n**A1:** While using reward models is a known paradigm, its application to GUI agents is non-trivial and presents distinct challenges. Unlike tasks with verifiable final answers and well-defined reward datasets (e.g. math problems, code generation), GUI tasks involve dynamic screenshots and historical actions, making it difficult to define standard answers, and create large-scale, accurate training data for specialized GUI reward models. Consequently, there lacks a well-established reward model designed for GUI agents that can effectively assess GUI action quality.\n\nOur core innovations directly address these challenges:\n- **Specialized reward data generation techniques:** We develop automated techniques including rule-based verification, controlled trajectory corruption, and hard negative mining to transform existing GUI agent dataset into high-quality reward training data without manual annotation, enabling us to train an initial GUI reward model.\n- **Co-evolution self-improvement pipeline:** We establish a mutual enhancement loop where the agent and reward model continuously improve through dynamic interaction - the agent generates increasingly complex trajectories, the reward model evaluates them, and both model are continuously refined using the generated feedback data along with the initial data.\n\nThese contributions provide an efficient, scalable solution for getting the reward model's training data and training both reward and policy models in GUI environments.\n\n**W2: Better definition of the newly proposed GUI trajectory reward task.**\n\n**A2:** Thank you for this suggestion. We will add a new Section 3.1 to formally define the task and revise Figure 2 to align with these definitions. The new section will read:\n\n3.1 Task Formulation for GUI Trajectory Reward\n\n**Action-level reward:** Given a task goal $\\mathcal{G}$, current screenshot $\\mathcal{I}\\_t$, and historical observations $\\mathcal{O}\\_t={(\\mathcal{I}\\_0,\\mathcal{A}\\_0), (\\mathcal{I}\\_1,\\mathcal{A}\\_1), ..., (\\mathcal{I}\\_{t-1},\\mathcal{A}\\_{t-1})}$, the objective is to evaluate whether a candidate action $\\mathcal{A}\\_t$ constitutes a correct step toward achieving $\\mathcal{G}$. The reward model outputs a positive reward ($y=1$) for correct actions and negative reward ($y=0$) otherwise.\n\n**Task-level reward**: This task evaluates whether an entire trajectory successfully completes the goal $\\mathcal{G}$. The input consists of all actions and resulting screenshots in the trajectory, and the output is a single reward score indicating overall success or failure.\n\n**W3: Out-of-distribution test of UI-Genie-RM.**\n\n**A3:** We evaluate UI-Genie-RM on a out-of-distribution test set constructed from the AndroidWorld benchmark, using 11 apps that were unseen during UI-Genie-RM training. As shown in the following Table.1, UI-Genie-RM achieves performance competitive with the powerful proprietary model Gemini2.5-pro and significantly outperforms other open-source MLLMs, demonstrating strong generalization to unseen domain. We will add these results to Section 4.4.2 and Table.5 of our paper.\n\nTable. 1 OOD test of UI-Genie-RM\n| Model | **Step-level Acc (%)** $\\uparrow$| **Outcome Acc (%)** $\\uparrow$|\n| --- | --- | --- |\n| Gemini2.5-pro | 71.5 | 75.7  |\n| Qwen2.5-VL-7B | 46.8  | 64.4 |\n| Qwen2.5-VL-72B | 54.2 | 68.0 |\n| **UI-Genie-RM** | **74.5** | **76.2** |\n\n**W4: Ablation studies on three complementary data generation techniques and reward model updating.**\n\n**A4:** The results in the following Table.2 show the impact of three techniques used to construct reward model training data and whether the reward model should be updated in the self-improvement loops. \nResults of the first four experiments demonstrate each technique provides a significant boost to the reward model's accuracy. The comparision between the fourth and fifth experiments shows that updating the reward model in the self-improvement loops provide a large performance improvement. These results validate the effectiveness of data construction techniques and the design choice of self-improvement framework. We will add these results into the revised paper.\n\nTable.2 Ablation studies on three complementary data generation techniques and reward model updating.\n| Rule-based Verification | Trajectory Corruption | Hard Negative Mining | RM Updating | Step-level Acc (%)$\\uparrow$ | Outcome Acc (%)$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- |\n| × | × | × | × | 56.6 | 58.9 |\n| √ | × | × | × | 67.2 | 65.3 |\n| √ | √ | × | × | 67.4 | 75.9 |\n| √ | √ | √ | × | 69.2 | 76.7 |\n| √ | √ | √ | √ | **79.6** | **82.1** |\n\n**Q1**: **Necessity of training UI-Genie-RM.**\n\n**A:** To verify the necessity of training and updating a specialized RM, we adopt two variants: (1) using the base Qwen2.5-VL-7B as a fixed RM, and (2) using our initial UI-Genie-RM without updates during self-improvement. As shown in the following Table.3, a general MLLM (Qwen2.5-VL-7B) provides minimal benefit and fails to guide the agent effectively on complex tasks, leading to performance degradation. A specialized designed and fixed in the self-improvement phase reward model is helpful but its effectiveness is relatively limited on complex tasks. Our full framework, where the RM is continuously improved, achieves consistent gains, demonstrating the advantage of a **specialized and continuously improving RM.** We will include these analysis in the revised paper.\n\nTable.3 Ablation studies on reward models.\n| Reward Model | Reward Model Update | Reward Accuracy (%)$\\uparrow$ | Agent Success Rate (Round 0)$\\uparrow$ | Agent Success Rate (Round 1) $\\uparrow$| Agent Success Rate (Round 2)$\\uparrow$ | Agent Success Rate (Round 3)$\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- |\n| Qwen2.5VL-7B | × | 56.6 | 18.1 | 22.5 | 23.2 | 21.7 |\n| UI-Genie-RM | × | 69.2 | 18.1 | 29.8 | 31.2 | 35.5 |\n| UI-Genie-RM | √ | 79.6 | 18.1 | 29.8 | 33.3 | **38.7** |\n\n**Q2: Performance on AndroidWorld benchmark.**\n\n**A:** We evaluate our final agent models on the AndroidWorld benchmark. As shown in the Table.4 below, UI-Genie-Agent-7B achieves a success rate of 36.2%, surpassing strong proprietary models like GPT-4o. Our 72B model shows significant performance gains compared to its base model Qwen2.5-VL-72B, outperforming all other methods. We will incorprate these new evaluation results into our revised paper.\n\nTable.4 Comparisions on AndroidWorld benchmark\n| Models | Agent Success Rate (%) $\\uparrow$|\n| --- | --- |\n| GPT-4o | 34.5 |\n| Claude Computer Use | 27.9 |\n| Qwen2.5-VL-7B | 22.0 |\n| UI-TARS-7B | 33.0 |\n| **UI-Genie-Agent-7B** | **36.2** |\n| Qwen2.5-VL-72B | 35.0 |\n| Aguvis-72B | 26.1 |\n| **UI-Genie-Agent-72B** | **47.4** |"}, {"review_id": "77A5bqbHQP", "reviewer": "Reviewer_grio", "summary": "The paper presents UI-Genie, a self-improving framework for GUI agents that tackles two main challenges: (1) difficulty in verifying task outcomes, and (2) lack of scalable high-quality training data. To solve the first, it introduces UI-Genie-RM, a reward model with an image-text interleaved architecture that unifies action- and task-level feedback. It leverages strategies like rule-based verification, controlled trajectory corruption, and hard negative mining for synthesizing reward model training data. Then, UI-Genie uses a self-improvement pipeline that iteratively refines both the agent and reward model via reward model-guided exploration. The approach achieves state-of-the-art performance on multiple GUI benchmarks.", "questions": "**Equation 1 Typo**: There may be a typo in Equation 1. The two '-' symbols look questionable. Should the first one be a '+' and the second a '-'? A clarification here would be helpful.\n\n**Iterative Fine-tuning**: Does the iterative fine-tuning process incorporate any continual learning techniques? Without mechanisms to retain previously learned knowledge, the model may suffer from catastrophic forgetting over time. Clarification on this point would be useful.\n\n**Scope of GUI Evaluation**: The evaluation focuses primarily on mobile GUI tasks. It would be informative to see whether the proposed pipeline generalizes to other GUI domains such as desktop or web environments.", "limitations": "See Weaknesses part.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n**Performance**: The model demonstrates impressive performance across three Android-related benchmarks, outperforming some proprietary models despite its smaller size. This is a strong indicator of the method's efficiency and practicality.\n\n**Writing**: The writing is clear and accessible, and the method is presented in a simple yet effective manner. The motivation for self-improvement especially in the context of GUI tasks is compelling, given the high cost of human annotation in such domains.\n\n## Weaknesses\n\n**Claim About Novelty**: The authors claim this is the first method to train a reward model using open-source foundation models. However, the WebRL paper (published in Nov. 2024) also involves training outcome reward models, although in a different setting. While WebRL may not target multimodal GUI tasks specifically, it seems to share the basic method idea that could generalize to this domain. It would strengthen the paper to include a discussion of this overlap and clarify how the proposed approach differs.\n\n**Negative Sample Construction**: The process for constructing negative samples raises some concerns. The assumption that all non-ground-truth actions are incorrect might not hold in practice. For instance, when increasing the quantity of a product from 1 to 2, a user might either click '+' or directly input '2' and they are both valid actions. Rule-based methods may misclassify one as incorrect. It would be valuable to include a more rigorous human evaluation of the negative sample construction process and explore ways to make it more robust.\n\n**Reward Model Improvement**: The iterative training of the reward model raises questions about its long-term reliability. While the first iteration benefits from high-quality human-annotated data, subsequent rounds rely on predictions from a previous model, which may introduce noise. If the reward model starts to accept false positives as correct, this could significantly degrade the quality of future negative samples. More evidence is needed to support the claim that the reward model continues to improve over iterations.\n\n**Experiments**: More detailed results across iterations for both the policy and reward models would better illustrate the effectiveness of the self-improvement loop. It would also be interesting to study ablations, such as removing complex task generation or the other alternative processes, to understand the contribution of each component. Also, existing works on tree search (Koh et al., 2024) could offer competitive alternatives to beam search. A broader comparison to such baselines would strengthen the experimental section.", "was_revised": true, "final_justification": "With multiple rounds of discussion, the authors resolved most of my questions. I will raise my score.", "rebuttal": "Thank you for your thorough review and constructive feedback provided on our work. We appreciate your recognition of our method's impressive performance, clear writing, and compelling motivation for self-improvement in the costly domain of GUI annotation. Below are our clarifications and responses to the points raised:\n\n**W1: Novelty of our paper and the relation to other work like WebRL.**\n\n**A1:** We focus specifically on developing the first reward model tailored for GUI agents, addressing unique challenges in this domain that cannot be directly adapted from other areas. We will add a discussion distinguishing our work from WebRL in the revised paper.\n\n**Similarities with WebRL**: Both methods can provide outcome rewards for trajectories, enabling the model to learn from both successful and failed attempts.\n\n**Key distinctions from WebRL:**\n\n- **Scalable reward data generation:** The ORM training data of WebRL relies on existing WebArena-Lite training datasets with task-specific evaluation functions. In constrast, GUI agents lack such data, and justifying GUI task completion requires combining screenshot information, rather than easily applying task-specific evaluation functions. We propose a suite of **deliberately-designed techniques** (rule-based verification, controlled trajectory corruption, hard negative mining) to **create a large-scale reward dataset from scratch**, addressing a fundamental challenge for training GUI RMs.\n- **Fine-grained process rewards:** Unlike WebRL's outcome-only reward, our model provides both **task-level** and **action-level** rewards. This fine-grained feedback enables step-wise correctness evaluation and our novel reward-guided beam search, an efficient exploration strategy not present in other work.\n- **Reward model-guided self-improvement:** Different from WebRL, we utilize our reward model not just for trajectory validation, but for actively generating high-quality trajectories without human annotation. Then, our framework refines both reward and agent models iteratively with the increasingly complex training data. In contrast, WebRL's reward model remains fixed after pre-training, limiting its adaptability and preventing continuous generation of new high-quality data to enhance the reward model.\n\n**W2: Negative sample construction and human evaluation.**\n\n**A2:** In the negative sample construction process, we have established rigorous rules to evaluate the action alignment effectively, rather than a simple `non-GT = negative` assumption. Specifically, as discussed in Section B.1 of the supplementary, besides the basic action types, our action space includes a special parameter: `action_desc`, which describes the target of the action. With the sampled action and `action_desc`, we write the following code to use some pre-defined rule-based functions and potentially trigger an LLM to judge whether the sampled action is a true negative or equivalent to positive:\n\n```python\ndef check_action(pred_action, gt_action, task_instruction):\n    # return True for positive actions, False for negative actions\n    pred_type, pred_param, pred_desc = parse_action(pred_action)\n    gt_type, gt_param, gt_desc = parse_action(gt_action)\n    \n    # Rule-based verification\n    if gt_type == 'click':\n        # If types and coordinates match, the predicted action is a positive action.\n        if pred_type == gt_type and judge_coordinate(pred_param, gt_param): \n            return True           \n        # If coordinates differ or type is different,\n        # the predicted action might be a positive action (e.g. clicking `+` vs. typing `2`)\n        # it can also be negative (e.g. clicking `+` vs. clicking `-`).\n        # We thus use LLM to judge action equivalence based on `action_desc`.\n        elif pred_type in ['click', 'open', 'type', 'system_button']:       \n            return llm_judge_action(task_instruction, gt_desc, pred_desc)        \n        else:    \n            return False\n\n    # ... other rules for swipe, etc.\n```\n\n We provide the following two examples of sampled actions and corresponding judgement results. We will clarify the details and include the examples in the revised paper.\n\nTable.1 Examples of negative sampling result.\n| **Positive Action** | **Positive `action_desc`** | **Sampled Action** | **Sampled `action_desc`** | **Judgement Result** |\n| --- | --- | --- | --- | --- |\n| Click [720, 464] | Tap back arrow to exit. | System Button \"Back\" | Go back to manage accounts. | Positive |\n| Click [720, 464] | Tap back arrow to exit. | Type\"Martin Garrix\" |Enter Martin Garrix in the Name text field. | Negative |\n\n**Human evaluation:** We manually check 300 random samples each from AndroidControl, AMEX, and AndroidLab. Our above judgement method achieves **92.2% alignment** against manual labels, confirming the high quality of our generated reward data.\n\nTable.2 Sample type alignment with human evaluation.\n| **Data Source** | **Samples** | **Accuracy** $\\uparrow$ |\n| --- | --- | --- |\n| AndroidControl | 300 | 94.3% |\n| AMEX | 300 | 93.0% |\n| AndroidLab | 300 | 89.3% |\n| **Total** | **900** | **92.2%** |\n\n**W3: Validation of continuous improvement of the reward model.**\n\n**A3:** We have studied the performance evolution of the reward model in Section A.3 and Figure.1 of our supplementary. The key findings, summarized in the following Table.3, demonstrate that the RM's performance **continuously improves** with each iteration, which in turn boosts the agent's task success rate. This consistent improvement can be attributed to two main factors:\n\n- **High-quality foundation:** The reward model is initially trained on high-quality data from human-annotated trajectories, and subsequent rounds integrate base data with newly collected data, maintaining core capabilities while exploring diverse trajectories.\n- **Iterative enhancement:** The iterative process allows the agent model to handle more complex tasks and generate high-quality training samples, thus improving the reward model’s discriminative ability in challenging scenarios.\n\nTable.3 Performances of different round models.\n| Model | Metric | Round 0 | Round 1 | Round 2 | Round 3 |\n| --- | --- | --- | --- | --- | --- |\n| UI-Genie-RM | Reward Accuracy $\\uparrow$ | 69.2 | 73.6 | 75.4 | **79.6** |\n| UI-Genie-Agent | Task Success Rate  $\\uparrow$| 18.1 | 29.8 | 33.3 | **38.7** |\n\n**W4: More experimental results and ablation studies.**\n\n**A4:** We provide experimental results of iterative fine-tuning and more ablation studies as follows.\n\n**Iterative performance**: We provide a detailed iteration-by-iteration analysis in the above Table.3. The results demonstrate substantial and consistent improvement for both models, validating the effectiveness of the self-improvement loop.\n\n**More ablation studies:** To assess the contributions of each component, we conduct ablations on the following two aspects, as shown in Table. 4:\n\n(1) **Removing complex task generation**. Without introducing challenging tasks in the final round, the agent’s improvement is limited after Round 2 (31.2→32.6), highlighting the importance of a curriculum of increasing difficulty.\n(2) **Freezing reward model updates.** Updating the reward model during the self-improvement phase provides essential guidance for trajectory exploration, resulting in higher-quality synthetic data generation and enhanced performance of both models.\n\nTable.4 More ablation studies.\n| Complex Task Generation | Reward Model Update | Reward Accuracy-Round 0 $\\uparrow$| Reward Accuracy-Round 1 $\\uparrow$| Reward Accuracy-Round 2 $\\uparrow$| Reward Accuracy-Round 3 $\\uparrow$| Agent Success Rate-Round 0 $\\uparrow$| Agent Success Rate-Round 1 $\\uparrow$| Agent Success Rate-Round 2 $\\uparrow$| Agent Success Rate-Round 3 $\\uparrow$|\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| × | × | 69.2 | 69.2 | 69.2 | 69.2 | 18.1 | 29.8 | 31.2 | 32.6 |\n| × | √ | 69.2 | 73.6 | 75.4 | 77.1 | 18.1 | 29.8 | 33.3 | 34.1 |\n| √ | × | 69.2 | 69.2 | 69.2 | 69.2 | 18.1 | 29.8 | 31.2 | 35.5 |\n| √ | √ | 69.2 | 73.6 | 75.4 | 79.6 | 18.1 | 29.8 | 33.3 | **38.7** |\n\n\n**Discussion with tree search:** Our reward-guided beam search provides a diverse set of both successful and informative failed trajectories. This diversity is crucial for iteratively refining our reward model with various positive and negative examples. By contrast, the tree search method is designed to optimize toward a single best solution, pruning other low-reward trajectories. This would prevent us from collecting reward signals from failed trajectories. We will explore adapting tree search-based techniques in future work.\n\n**Q1: Equation 1 Typo**\n\n**A:** Equation 1 is not a typo. The trainining objective in Equation 1 is to maximize the log-likelihood of the correct label. For Positive samples $\\mathcal{A}_t^{+}$, we want to maximize $\\log P(y^{+}|\\cdot)$, equavelant to minizing $-\\log P(y^{+}|\\cdot)$. Similarly, for negative samples $\\mathcal{A}\\_t^{-}$, we want to maximize $\\log P(y^{-}|\\cdot)$, equavelant to minizing $-\\log P(y^{-}|\\cdot)$.\n\n**Q2: Avoiding catastrophic forgetting in iterative fine-tuning.**\n\n**A:** We do not implement specialized continual learning techniques. We employ a straightforward and effective data mixing strategy commonly used in VLM training. Specifically, in each fine-tuning round, we combine newly collected data with the data from the previous rounds to train both the reward model and agent model.\n\n**Q3: Extension to other GUI domains.**\n\n**A:** Other GUI domains also need scalable data generation and robust reward models. Our methods can create reward datasets and improve agent performance across web and desktop applications. However, this paper focuses specifically on mobile agents, and extending to other environments would require significant computation resources and time for data generation and model training. We consider this a promising direction for future work."}, {"review_id": "z4EuHNd0eg", "reviewer": "Reviewer_Q1Jb", "summary": "This paper introduces a framework for train GUI agents. The method allows to automatically generate data, and fine-tune an agent. The trained agent can be used to generate new data, allowing for a cycle of automatic improvements. The pipeline is composed by two models, a reward model for trajectory verification and the policy. They collected a dataset following their methodology, and train models of different sizes. They aim to open-source the framework.", "questions": "- Can you provide an analysis of the dataset as well? What are the type of actions there? How is the distribution of goals? \n- There are relevant works in web agents space: NNetNav (https://arxiv.org/abs/2410.02907) and inSTA (https://arxiv.org/abs/2502.06776).\n- Will you release the datasets and the models as well? Also, be sure to release the framework before the camera-ready, as it affects significance.", "limitations": "I will add a sentence about potential negative impact of fully automated bots for devices, for example in ease frauds.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper is well written, and shows promising results. They tested on different architecture sizes and against many models. The objective to automatizing data collection and allow self-improving agents is significant. They claim they will release the framework open-source, which will increase significance. I suggest to also release the dataset, and trained models with it. The work is original, hower, they may add some citations (see questions). The experiments shows interesting results, but I would add an analysis on the diversity of the dataset, which is often a problem with automatic generated datasets. \nThe paper claim that the methodology is fully-automated, however it seems to need humans for finding hard-negatives in the RM train.", "was_revised": true, "final_justification": "The paper studies an important aspect of training a foundation model for GUI agents. The paper is technically solid and well written; the results are promising. The rebuttal didn't change my assessment of the paper, which is to accept it due to its relevance.", "rebuttal": "Thank you for your insightful review and strong support for our work. We greatly appreciate your acknowledgement of the originality of our framework, the significance of automated data generation and the promising results across various models. Below are our detailed responses to the points your raised.\n\n**W1: Analysis on the diversity of the generated dataset.**\n\n**A1:** We provide the detailed statistics of the UI-Genie-Agent-16k dataset in the following three tables. We will include these results in our revised paper.\n\n**Action Distribution:** Our dataset contains 8 distinct action types with a realistic distribution of GUI interactions. While Click is naturally the most frequent action, the significant presence of Type (9.7%) and Swipe (6.7%) ensures the agent learns to handle complex data entry and navigation tasks.\n\nTable.1 Analysis on action distribution.\n| Action Type | Open | Click | Swipe | Long press | Type | System button | Wait | Terminate |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| Number | 2208 | 8428 | 1054 | 15 | 1526 | 218 | 25 | 2208 |\n| Percentage | 14.08% | 53.74% | 6.72% | 0.10% | 9.73% | 1.39% | 0.15% | 14.08% |\n\n**Goal Diversity:** We categorize task goals into 7 distinct categories, showing significant diversity across mobile GUI scenarios. The task goals span a wide range of realistic user intents, preventing model overfitting to a narrow set of tasks.\n\nTable.2 Analysis on goal category.\n| **Goal Category** | Number | **Percentage** |\n| --- | --- | --- |\n| Configuration & Settings | 518 | 23.46% |\n| Information Retrieval | 610 | 27.63% |\n| Task & Item Management | 291 | 13.18% |\n| Search Tasks | 194 | 8.79% |\n| Application Control | 178 | 8.06% |\n| Site Navigation | 183 | 8.29% |\n| Data Entry | 234 | 10.60% |\n\n**Task Complexity:** Crucially,  our dataset emphasizes challenging tasks over trivial simple tasks. We find that medium (5-10 steps) and hard tasks (>10 steps) comprise over 91% of the dataset, which is essential for developing highly capable agents.\n\nTable.3 Analysis on task difficulty.\n| Task Difficulty | Trajectory Length | Number | Percentage |\n| --- | --- | --- | --- |\n| Easy | <5 Steps | 182 | 8.24% |\n| Medium | 5-10 Steps | 1165 | 52.76% |\n| Hard | >10 Steps | 861 | 38.99% |\n\n**W2: Lack of some citations.**\n\n**A2:** We appreciate this feedback and will incorporate these relevant work in our revision. \n\nNNetNav proposes an unsupervised approach for training brower agenets by generating synthetic demonstrations through exploration and retroactive trajectory labeling. InSTA presents an internet-scale dataset construction pipeline that use LLMs to automatically generate agentic tasks, complete these tasks, and filter successful trajectories without human supervision. Both work exploit automatic data generation and trajectory filtering mechinisms, aligning with out UI-Genie's self-improving pipeline conceptually. However, our approach addresses mobile GUI tasks with visual information and introduce a specialized reward model UI-Genie-RM for both action-level and task-level trajectory assessments.\n\n**W3: Open-source release of framework, datasets, and trained models.**\n\n**A3:** We fully agree with your suggestion regarding open-source releases. To maximize the impact and significance of our contribution, we commit to releasing:\n\n- **Complete Framework**: The entire UI-Genie framework including data generation, training, and evaluation pipelines.\n- **Datasets**: UI-Genie-RM-517k (reward model training data) and UI-Genie-Agent-16k (agent model training data).\n- **Trained Models**: UI-Genie-RM (reward model) and UI-Genie-Agent in all tested sizes (3B/7B/72B).\n\n**W4: Human efforts needed for finding hard-negatives.**\n\n**A4:** Our hard-negative mining process is fully automated and requires **no** human effort. \n\nAfter obtaining the initial agent model (trained on existing trajectory datasets, Line153-154 main paper) and reward model (whose training dataset is constructed with *rule-based verfication* and *controlled trajectory corruption* techniques), the hard-negative mining process is adopted to find challenging negative examples in reward dataset. \n\nWe start with existing annotations from the AMEX dataset which provides task target and ground-truth actions. It also provides functional descriptions for other **target-unrelated** UI elements. We then use an open-source LLM (DeepSeek-v3) to convert these descriptions into executable actions. These actions are all regarded as step-level negative actions for the task target. After that, we use the initial reward model to score them. If the reward model mistakenly assign a positive score to one action, this action is identified as hard negative. This process is fully automatic without human annotations. We will revise our description in the paper to make this automated pipeline more clear.\n\n**Response to \"Limitations\"**\n\n**Potential negative impact of fully automated bots for devices**\n\n**A:** Thank you for highlighting this critical consideration. While the UI-Genie framework aims to advance automation in GUI interactions, we acknowledge the importance of recognizing potential risks associated with fully automated bots. If not properly monitored, these bots could lead to misuse, such as conducting financial fraud or stealing private data without authorization. We will expand the **Broader Impact** section of our paper to include a direct discussion of these risks."}, {"review_id": "aR1jYhQD3e", "reviewer": "Reviewer_afLe", "summary": "This paper introduces UI-Genie, a self-improving framework designed to enhance the capabilities of mobile GUI agents by addressing two primary challenges: the difficulty of verifying task outcomes and the scalability issues of creating high-quality training data. To tackle these problems, the authors developed UI-Genie-RM, a specialized reward model that assesses agent performance at both the individual action and full trajectory levels using an image-text interleaved architecture to process historical context. To train this reward model without costly human annotation, the framework employs deliberate data generation strategies, including rule-based verification, trajectory corruption, and hard negative mining. The core of the contribution is a self-improvement pipeline where the agent and reward models evolve iteratively; the reward model guides the agent's exploration in dynamic environments, and the resulting successful or informative failed trajectories are used to expand the training datasets for both models, enabling them to solve increasingly complex tasks. Through this process, the authors created two new datasets, UI-Genie-RM-517k and UI-Genie-Agent-16k, and demonstrated that their UI-Genie agent achieves state-of-the-art performance across multiple benchmarks.", "questions": "1. Confusions stem from reading Section 3\n- Why rule-based verification is needed for constructing data for the reward model? Will it filter out some data from the base GUI operation trajectory datasets? \n- In the iterative model fine-tuning, what is the definition of \"rounds\", \"steps\"?\n- UI-Genie-Agent-16k is not mentioned in Section 3 but is it generated during the iterative improvement? How?\n\n\n2. The iterative improvement seems only produces 59k data which is less than 15% of the total training data for the proposed reward model. How does this iterative process better than just using the trained RM as a guide to collect training data and fine-tune the GUI agent?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper introduces a novel self-improving framework that iteratively enhances both an agent and a reward model.\n\n2. The development of UI-Genie-RM is the first reward model specifically designed for evaluating GUI agent trajectories, providing a sophisticated solution to the complex problem of trajectory verification.\n\n3. UI-Genie demonstrates superior performance across multiple static and dynamic benchmarks, including AndroidControl, AndroidLab, and A3. \n\n4. The authors plan to open-source the entire framework, including the two novel datasets (UI-Genie-RM-517k and UI-Genie-Agent-16k) and model implementations. This will be a valuable resource for the research community and will undoubtedly facilitate future advancements in GUI agents.\n\n**Weakness**\n\n1. The model processes historical context by using only the five most recent screenshots and summarizing earlier steps as text. It may be insufficient for complex tasks requiring long-term memory while being unnecessarily inefficient for simpler tasks.\n\n2. Section 3 includes some vague descriptions without further explanation or examples, making it hard to follow. See some questions in the following Questions section.\n\n3. While the paper proposed a self-improvement pipeline with three iterative rounds, the experiments does not further analyze or quantify the performance gains from this pipeline.\n\n4. The experiments do not include studies on the out-of-domain performance of UI-Genie-RM.", "was_revised": true, "final_justification": "The paper introduces a novel and impactful self-improving framework for mobile GUI agents, along with a dedicated reward model and new datasets. After reading the rebuttal and discussion, several concerns were satisfactorily addressed:\n\n- Historical context handling is supported by thorough ablations showing the 5-screenshot window balances performance and efficiency.\n- Section 3 clarifications—on rule-based verification, dataset generation, and iterative fine-tuning—were clearly explained.\n- Iterative improvement effectiveness is validated through performance gains across rounds (Table 3).\n- Out-of-domain generalization is demonstrated with UI-Genie-RM performing competitively on unseen apps (Table 4).\n\nRemaining concern is only that the writing and presentation especially in Section 3 remain somewhat hard to follow and could be improved.", "rebuttal": "Thank you for your comprehensive review and supportive feedback on our paper. We appreciate your recognition of the novelty of our self-improving framework, the specific contribution of UI-Genie-RM as the first specialized reward model for the GUI agent domain, the superior performance across multiple benchmarks, and the value of our open-source commitment. Below we address specific points raised in your review.\n\n**W1: The sufficiency of using five most recent screenshots to process historical context.**\n\n**A1:** We conduct an ablation study analyzing our model's performance and efficiency across different task complexities.\n\nFor simple tasks(<5 steps), the actual input is naturally limited by the number of all available history(e.g., 3 screenshots at step 4). We evaluate preserving at most 1, 3, and 5 screenshots in the following Table.1. While using one screenshot is most efficient, it shows limited performance due to insufficient historical context, particularly in outcome reward evaluation. Using up to five screenshots achieves the best performance without introducing significant inference overhead, since the actual number of processed images remains low.\n\nTable.1 Model performance on simple task with different seen images.\n| Img Num | Step-level Acc( %) $\\uparrow$ | Outcome Acc( %) $\\uparrow$ | Inference Time(s/iter) $\\downarrow$ |\n| --- | --- | --- | --- |\n| 1 | 76.4 | 74.5 | 0.59 |\n| 3 | 78.2 | 79.8 | 1.05 |\n| 5 | 83.5 | 84.3 | 1.57 |\n\nFor complex tasks(≥8 steps), we evaluate the screenshot number at 5, 8, and 10. We observe that using 5 screenshots achieve comparable performance to 8 screenshots, while increasing to 10 leads to performance degradation, as presented in the following Table.2. This suggests that older screenshots beyond the 5 most recent ones may distract the model by less relevant information. Moreover, larger window sizes significantly increase the computational cost. These results demonstrate the effectiveness of our choice of using five most recent screenshots to achieve the best accuracy and efficiency trade-off. \n\nTable.2 Model performance on complex task with different seen images.\n| Img Num | Step-level Acc( %) $\\uparrow$ | Outcome Acc( %) $\\uparrow$ | Inference Time(s/iter) $\\downarrow$ |\n| --- | --- | --- | --- |\n| 5 | 72.9 | 73.5 | 2.46 |\n| 8 | 73.4 | 75.2 | 5.10 |\n| 10 | 69.7 | 71.3 | 7.81 |\n\n**W2&Q1: Clarifications on Section 3.**\n\n**A2:** We apologize for any lack of clarity in Section 3. We provide explanations as follows and will revise our paper accordingly.\n\n> Why rule-based verification is needed for constructing data for the reward model? Will it filter out some data from the base GUI operation trajectory datasets?\n\nWhen constructing the initial training dataset for the reward model, the rule-based verification method is used for getting negative step-level actions. Existing human-annotated GUI agent datasets (AndroidControl, AMEX, etc.) only provide successful trajectories consisting of action sequences, which are used as the *positive samples* in reward model training dataset. To get the *negative samples*, we provide the task instruction of GUI agent dataset to our initial agent model, let it sample the step-level actions, then rule-based methods are used to label these step-level actions as negative or positive (Line153-160, rule-based verfication). For example, given a sample click(\"empty_space\") and the corresponding positive action click(\"Send\"), our rules can label the sample click(\"empty_space\") as negative.\n\nThus, the rule-based methods will not filter out the original ground-truth trajectories, since we treat them as correct positive examples. The rule-based methods is only used to get the step-level negative actions.\n\n> In the iterative model fine-tuning, what is the definition of \"rounds\", \"steps\"?\n\n**Round**: The \"round\" refers to one full cycle of our iterative self-improvement framework, consisting of three phases: 1) Trajectory exploration: The agent generates new trajectories using reward-guided search. 2) Data expansion: Verifying the outcomes of these new trajectories, adding the successful ones to the agent training datasets, and expanding the reward model's dataset with all labeld steps. 3) Fine-tuning: Training both the agent and reward model on these expanded datasets. Based on the above definition, our fine-tuning process thus has three rounds (Line206).\n\n**Step:** A \"step\" refers to a single action taken by the agent within a task trajectory. At each step, the agent generates a candidate action, which is then evaluated by our UI-Genie-RM to produce a step-level reward.\n\n> UI-Genie-Agent-16k is not mentioned in Section 3 but is it generated during the iterative improvement? How?\n\nYes, the UI-Genie-Agent-16k dataset is directly generated during the iterative improvement loops (Line239). As detailed in Section 3.3 under \"Training Data Expansion\", we use UI-Genie-RM for outcome verification to identify successful task completions. These verified, successful trajectories are added to the agent's training set, finally resulting in 16k such trajectories.\n\n**W3: Quantifying performance gains of the self-improvement pipeline.**\n\n**A3:**  We have studied the performance evolution of both the reward model and agent model in Section A.3 and Figure 1 of our supplementary. The key findings is summarized in the following Table.3. The results demonstrate that the RM's performance **continuously improves** with each iteration, which in turn **boosts** the agent's task success rate. This consistent improvement can be attributed to two main factors:\n\n- **High-Quality foundation:** The reward model is initially trained on high-quality data derived from human-annotated trajectories. In subsequent rounds, we integrate this base data with newly collected data, improving the model through exploring diverse trajectories while maintaining its core capabilities.\n- **Iterative enhancement:** Through the iterative process, the agent model enhances its ability to tackle more complex tasks, generating high-quality reward training samples with increased difficulty. This iterative learning improves the reward model's discriminative ability in challenging scenarios.\n\nTable.3 Model performances after different rounds.\n| Model | Metric | Round 0 $\\uparrow$ | Round 1 $\\uparrow$ | Round 2 $\\uparrow$ | Round 3 $\\uparrow$ |\n| --- | --- | --- | --- | --- | --- |\n| UI-Genie-RM | Reward Accuracy | 69.2 | 73.6 | 75.4 | 79.6 |\n| UI-Genie-Agent | Task Success Rate | 18.1 | 29.8 | 33.3 | 38.7 |\n\n**W4: Out-of-domain performance of UI-Genie-RM.**\n\n**A4:** We evaluate UI-Genie-RM on a out-of-domain test set constructed from the AndroidWorld benchmark, using 11 apps that were unseen during UI-Genie-RM training. As shown in the following Table.4, UI-Genie-RM achieves performance competitive with the powerful proprietary model Gemini2.5-pro and significantly outperforms other open-source MLLMs, demonstrating strong generalization to unseen domain. We will add these results to Section 4.4.2 and Table.5 of our paper.\n\nTable.4 OOD performance of UI-Genie-RM.\n| Model | **Step-level Acc(%)** $\\uparrow$ | **Outcome Acc(%)** $\\uparrow$ |\n| --- | --- | --- |\n| Gemini2.5-pro | 71.5 | 75.7  |\n| Qwen2.5-VL-7B | 46.8  | 64.4 |\n| Qwen2.5-VL-72B | 54.2 | 68.0 |\n| **UI-Genie-RM** | **74.5** | **76.2** |\n\n**Q2: Advantage of iteratively improving the reward model.**\n\n**A:** To demonstrate the effectiveness of iterative fine-tuning, we conduct an ablation study where the reward model is not updated after its initial training. As shown in the following Table.5, fixing the RM results in less performance improvement compared with the iterative fine-tuning baseline.The performance gap is most significant in the final round when tackling more complex tasks. This demonstrates that even a relatively small amount of newly generated, high-quality reward data from challenging, dynamic interactions is critical for guiding the agent toward solving harder tasks. We will include these results into our revised paper.\n\nTable.5 Ablation study on reward model fine-tuning. \n| Reward Model Update | Reward Accuracy-Round 0 $\\uparrow$ | Reward Accuracy-Round 1 $\\uparrow$ | Reward Accuracy-Round 2 $\\uparrow$ | Reward Accuracy-Round 3 $\\uparrow$ | Agent Success Rate-Round 0 $\\uparrow$ | Agent Success Rate-Round 1 $\\uparrow$ | Agent Success Rate-Round 2 $\\uparrow$ | Agent Success Rate-Round 3 $\\uparrow$ |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| × | 69.2 | 69.2 | 69.2 | 69.2 | 18.1 | 29.8 | 31.2 | 35.5 |\n| √ | 69.2 | 73.6 | 75.4 | 79.6 | 18.1 | 29.8 | 33.3 | **38.7** |"}]

Author rebuttal:

Meta-review: The paper studies the training pipeline for GUI agents to operate mobile phone interfaces, and contributes two components that address the significant challenges in the domain.
The problem is well-motivated and timely as multi-modal LLM-based agents are being applied to different domains like web browsers, desktop OS and mobile interfaces.
The proposed solution components are described clearly and address the challenges of scalable, reliable trajectory collection & evaluation in a practical and effective manner.
The experiments in the paper left a lot of unanswered questions, and the reviewers initially all rated the paper as borderline as a result.
However, the authors substantially improved the paper with additional experiments, ablations, and analysis during the discussion period.
These include:
- Ablating the different data gathering and reward model updating schemes to isolate individual benefit and assess complementarity.
- Assess the learned reward model out-of-distribution.
- Assess the quality of reward labeling heuristics on the generated data.
- Analyze the diversity of generated data.
- Vary the amount of context used for reward modeling, to study accuracy-vs-latency trade-offs.
- etc.

Along with the several clarifications and discussions (e.g. clearer statement of problem formulation, clearer connections to related work on Web agents, etc.) make the paper substantially stronger and should all be included in the revision.
All the reviewers agree that the paper is above the bar for publication after these changes are incorporated.
Decision: Accept (poster)