GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
Authors: Yuqi Zhou
Sunhao Dai
Shuai Wang
Kaiwen Zhou
Qinglin Jia
Jun Xu
Venue: NeurIPS 2025 poster ()
Keywords: R1-Zero-Like Training
GUI Agents
Grounding
Primary area: other
TL;DR: This paper investigates what kind of R1-Zero-like training is suitable for grounding tasks in GUI agents.

Abstract: Recent Graphical User Interface (GUI) agents replicate the R1-Zero paradigm, coupling online Reinforcement Learning (RL) with explicit chain-of-thought reasoning prior to object grounding and thereby achieving substantial performance gains. In this paper, we first conduct extensive analysis experiments of three key components of that training pipeline: input design, output evaluation, and policy update—each revealing distinct challenges arising from blindly applying general-purpose RL without adapting to GUI grounding tasks. Input design: Current templates encourage the model to generate chain-of-thought reasoning, but longer chains unexpectedly lead to worse grounding performance. Output evaluation: Reward functions based on hit signals or box area allow models to exploit box size, leading to reward hacking and poor localization quality. Policy update: Online RL tends to overfit easy examples due to biases in length and sample difficulty, leading to under-optimization on harder cases. To address these issues, we propose three targeted solutions. First, we adopt a $\textbf{Fast Thinking Template}$ that encourages direct answer generation, reducing excessive reasoning during training. Second, we incorporate a box size constraint into the reward function to mitigate reward hacking. Third, we revise the RL objective by adjusting length normalization and adding a difficulty-aware scaling factor, enabling better optimization on hard samples. Our $\textbf{GUI-G1-3B}$, trained on 17K public samples with Qwen2.5-VL-3B-Instruct, achieves $\textbf{90.3\%}$ accuracy on ScreenSpot and $\textbf{37.1\%}$ on ScreenSpot-Pro. This surpasses all prior models of similar size and even outperforms the larger UI-TARS-7B, establishing a new state-of-the-art in GUI agent grounding.

GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
Yuqi Zhou', Sunhao Dai', Shuai Wang" Kaiwen Zhou', Qinglin Jia", Jun Xu
Gaoling School of Artificial Intelligence, Renmin University of China "Huawei Noah's Ark Lab sunbaadaa (uraaaiiruuueeeee edu.
Abstract
Recent Graphical User Interface (GUD) agents replicate the RI-Zero paradigm, coupling online Reinforcement Learning (RL) with explicit soning prior to object grounding and thereby achieving substantial performance gains. In this paper, first conduct extensive analysis experiments of three key components of that training pipeline: input design, output evaluation, and policy upda revealing distinct challenges arising from blindly applying general- purpose RL without adapting to GUI grounding tasks. Input design: Current templates encourage the model generate chain-of-thought reasoning, but longer chains unexpectedly lead to worse grounding performance. Output evaluation: Reward functions based hit signals or box area allow models exploit box size, leading to reward hacking and poor localization quality. Policy update: Online RL tends overfit easy examples due biases in length and sample difficulty, leading to under-optimizatio targeted solutions. First, we adopt Fast Thinking Template that encourages direct answer generation, reducing excessive reasoning during training. Second, we incorporate a box size constraint into the reward function to mitigate reward hacking. Third, revise the RL objective by adjusting length normalization and adding a difficulty-aware scaling factor, enabling better optimization on hard samples. Our GUI-G1-3B, trained on 17K public samples with Qwen2.5-VL-33 Instruct, achieves 90.3% accuracy on ScreenSpot and 37.1 % on ScreenSpot-PPe This surpasses all prior models of similar size and even outperforms the larger UI-TARS-77 establishing a new state-of-the-an art GUI agent grounding. The project repository is available at httttttp//ggiitei /gthhtttthb..... com com/Yuqi Chho/GUI--a
rea-
on harder cases. To address these issues, propose three
Multimodal Large Language Models
Reward Function
Ti
GRPO
Output:
02
T2
Length Au/inn)
Template:
<Think>
fnit
(You need to....
</Think>
fiou
Instruction:
<nnsers>
07
1>
Difficuity Reweight
(open the doc named Vclass)
[83. 12, 161)
fnox
CODB
</Answer>
0g
-earo
Figure 1: This framework employs the GRPO algorithm for optimization, emphasizing three critical input design, output evaluation, and policy update.
components:
*Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
SrepSeek-R1-Zer 12 revolutionizes the post-training pipeline of Large Language Models (LLMs) by introducing the RI-Zero paradigm, which applies RL directly to base LLMs without relying on supervised fine-tuning (SFT) as an intermediate step. Motivated by this approach, recent work in the domain of GUI agents [25] 28 40] has increasingly adopted RL, particularly the GRPO algorithm [34], in order to address two key limitations: (1) SFT requires large-scale, high-quality labeled datasets, resulting in significant computational costs; (2) existing open-source GUI agents trained with SFT often exhibit poor generalization to out-of-domain (OOD) scenarios [5]
While RL has emerged as popular choice for training GUI agents in recent work, attributing performance gains solely to the algorithm itself remains nontrivial. These RI-style models often differ in multiple dimensions- iissiions-incluudin inncluding backbone architectures, data sources, and training proto- cols-making it difficult to isolate the specific contribution of online RL. To better isolate the role of RL, we focus exclusively
the grounding task [1 23 which we consider the core capability (24). Building on these observations, this work rethinks the role of RL in R1 RI-style GUI agents training by (1) disentangling its algorithmic contributions from other system-level factors, and (2) focusing exclusively on grounding as the reinforcement objective.
for effective GUI interaction
For this, we decompose the RI-Zero-like training pipeline into three core components: input design, output evaluation, and policy update. Each reveals distinct challenges arising from blindly applying purpose RL. without adapting to grounding tasks. First, we observe that the grounding performance of the state-of-the-arr R1 -ssyle model, InfiGUI-RI [25, drops as reasoning increases in Sec. 3.1 suggesting that reasoning templates may not benefit grounding in GUI agents. Second, we find that commonly used reward functions based on hit signals or box area lead to opposite forms of reward hacking in Sec. 3.2 the former encourages smaller boxes with higher accuracy, while the latter favors larger boxes to increase Intersection over Union (loU). Finally, we identify two biases in the GRPO objective: length bias (26) and difficulty bias in Sec. 3.3 Length bias encourages longer but incorrect responses, which, as previously observed, further degrade grounding performance. Difficulty bias treats all samples equally, hindering the model's ability learn from more challenging examples. Together, these biases make it harder for the model to learn from difficult samples.
To address the above issues, we implement the following improvements. First, we introduce the Fast Thinking Template, which encourages the policy to generate answers directly during training. Second, to counteract the hacking in common reward functions that prefer boxes of different sizes during policy optimization, we propose a box-size-basee reward function as a constraint. Finally, remove the length normalization term from the original GRPO objective, following (26). We also introduce a difficulty coefficient for each sample's 's loss, allowing the model to receive greater gradients for more challenging samples. The difficulty coefficient is calculated from the relative box size, which serves as a proxy difficulty indicator in the grounding task [8].
Building on the above solutions, we train our model, GUI-G1-3B, using Qwen2.5 EL-V--3B-Instuue and a small (about 17K) set of grounding samples, showing strong performance with limited supervision from public datasets such as UI-BERT (3 and OS-Atlas (39]. Our model achieves new sfate-of-the-art performance on GUI grounding benchmarks, with 90.3% accuracy on ScreenSpot [8] and 37.1 %
on ScreenSpot-Pro [8.. It surpasses the previous best R R1-style GUI agent, InfiGUI-RI 25 while requiring significantly less data, fewer output tokens, and fewer training stages.
In summary, the contributions of this paper are as follows: (1) We identify three distinct challenges in the RI-Zero-like training pipeline of RI-sylle GUI agents: grounding is harmed by longer reasoning due to grounding's 's reliance on image tokens; common reward functions induce size-sensitive reward hacking: and GRPO biases agents toward simpler examples due to its objective.
(2) We further analyze and propose three solutions: a Fast Thinking Template for policy training, a box size basebaedd reward to regularize box size, and a modified GRPO with difficulty weighting and no length normalization. (3) Trained on only 17K fully open-source grounding samples, our GUI-G1-3B achieves state-of-the-art performance while using fewer tokens when testing.
2 R1-Zero-Like Training Paradigm for GUI Grounding
We begin by describing how to train Multimodal Large Language Models (MLLMs) for grounding tasks. Given screenshot and textual description d, the MLLM is trained to predict the target

---

location B, typically represented bounding box or point. Following prior work [ we formulate grounding as language generation task, where the MLLM produces response o that includes the predicted location as well as additional components such as the reasoning process objective descriptions Figure In our implementation, the predicted location expressed as bounding box Bprd (11, U1. 22 V2). where z and y denote the horizontal and vertical coordinates, respectively.
This prediction is evaluated against the ground-trun truth box B3tt = (01 y1, 22, y2).
When RL is applied via the algorithm like GRPO 34 template is first used to guide the response format, and the model generates candidate responses {or O2...., og....,oN): Each response is then evaluated using set of rule-baser reward functions, yielding a reward set fri, r2,..,,.., h. The relative advantage A, of each response is computed as:
72......
A std ,2...,,
(1)
where mean and std denote the mean and standard deviation the rewards, respectively. Finally, the policy model is optimized using the GRPO objective with Kl--divergence regularization.
3 How R1-Zero-Like Training Affects Grounding for GUI Agents?
We first aim to understand R1 R1-Zero-like training paradigm for grounding task GUI agents by examining three essential components: the input design (template) (.1c. the output evaluation (reward function) (Sec. 3.2). and policy update (RL objective) (Sec. 3.3). Finally, we present our model, GUI-G1, in Sec. 3.4 where also summarize and compare approach with existing RI RI-style agents to demonstrate its advantages in grounding tasks.
3.1 Analysis on Template
Recent R1 R1-style GUI agents have increasingly incorporated explicit reasoning by prompting the model to "think before action" (25] 28 40), as illustrated Figure example, InfiGUI-RI [25] uses Slow Thinking Template. While such reasoning-augnn auggnente agents achieve strong performance, remains unclear whether the gains truly arise from the reasoning process itself. In fact, we find that reasoning is often unnecessary for the grounding task in GUI agents. Before conducting the analysis, we formalize the model's input and output to ensure consistency across experiments. The input consists of image segment and instruction component tims which includes the user query and guiding template. The output o comprises the reasoning trace tuink and the final response tans where tans contains the predicted location Bped- We define the number of reasoning tokens Alink as output tokens, and the tokens derived from the image Timg as image tokens. The text ratio is Time + Titakk
given by
where Timm is the instruction tokens number.
+
Text
Text
High Text Ratio
Grounding Accuracy
1.0
Grounding Accuracy
1.0
Grounding Accuracy
1.0
Grounding Accuracy
1.3
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
75
85
45
105
115
Small
Midicle
Large
Small
Miriile
Large
Output Tokens
Image Tokens (x103)
Image
Size (Text)
Image
Size (Icon)
Figure 2: (Left) shows the grounding accuracy under varying numbers of output tokens and image tokens. "Text" refers cases where the target is textual element, while "Icon" refers to image targets. (Right) presents the grounding accuracy on the Text and Icon subsets across different image sizes. Within each group, samples evenly divided based their text ratio.
Longer thinking leads to worse grounding performance. While prior work, such as OpenAI- ol 30 and DeepSeek-R1 [12] demonstrates that longer reasoning chains can enhance performance on System-2 tasks like mathematics and programming, more recent studies [19 have found that introducing intermediate reasoning steps may impair performance in tasks such image classification and GUI grounding. Building these observations, further find that longer reasoning chains

---

AnPoint consistently degrade grounding accuracy on the ScreenSpot dataset [8], as shown in Figure Liftt.. This suggests that longer chains are not only unnecessary but can be increasingly detrimental GUI grounding, especially when the target item to be grounded is text.
Grounding henefits from appropriately scaled image tokens rather than from scaled text thinking. In Figure (Left (Left Middle), we observe that the grounding performance of InfiGUI-R1-3B improves as the number of image tokens increases. This observation raises a central question: Is grounding more reliant on image tokens or text tokens? To investigate this, we first partition the test samples based on the number of image tokens, ensuring each subset has a comparable level of visual input. Within each subset, we further divide samples into two categories according to their text ratio and evaluate grounding accuracy for each. As shown in Figure T (Right), a higher text ratio consistently correlates with lower grounding performance, indicating that enriching visual content is more effective than injecting additional textual reasoning.
As shown in Figure 2 explicit reasoning does not always improve grounding accuracy. Inspired by Thinking, Fast and Slow [15], we propose Fast Thinking Template that guides the model to generate grounding outputs directly without reasoning steps. As detailed in Appendix B models trained with this template outperform baselines on ScreenSpot and ScreenSpot-Pro.
Template (Fast Thinking Template) You are a helpful assistant. VUUser: Grounding instruction is: [question] Please help to locate and oulput the bbox coordinates in JSON formal. Ver ssistant:
Template 2 (Slow Thinking Template) You FIRST think about the reasoning process as an inter- nal monologue and then provide the final answer.aiThe reasoning process MUST BE enclosed within <think> </think> lags. VI User: The screen's 's resolution is to the UI element most relevant to "[question]", output its coordinates using JSON format:v "jsonnn[vvl pointt 2d" [x. yl. "label"; "object name/description" Iaciiioon"][/ [[n]" "/N1"""AAAA AAssiitant::
3.2 Analysis on Reward Function
The rule-based reward function introduced in DeepSeek-RI
12
exemplifies simple yet effective
approach based on exact match. In grounding tasks, current reward functions for RI-style GUI agents
are mainly categorized into Hit-based rewards [25] [28 [40] and IoU-based rewards (25 in Table I
Here, (Ip, 8p) the center of the predicted box, computed as Zp (.: + 2/2 yp (á) + 82) The Hit-based reward checks whether predicted box center hits within Bg. while the loUed
reward measures the IoU between Breed and Bg.- While prior work has employed Rua and RaO as reward signals for grounding- based RL, it remains unclear how these objectives jointly influence training dynamics. To answer this, we implement both types of reward functions for a comparative analysis. The detailed experimental settings and evaluation metrics can be found in Appendix C Unless otherwise specified, all subsequent analyses follow the same setup.
Table I: Comparison of rule-based reward functions and their effects on training dynamics. indicates failure to optimize (e.g., RBox alone). Driven By
Reward
Formula
Box Size(14)
Accuracy(tL)
ToU(
Ruit
1((2p) Up) Bg)
Point Accuracy
1
Ruu
lou(, Ba)
loU
1
t
RBa
Box Size
Epy tmpg tup tupg
Individually optimizing RHit and Rio leads to conflicting reward hacking behaviors. As shown in Figure 3 (Left), optimizing Rii improves accuracy but causes IoU to drop in later training. Conversely, optimizing Rio enhances overlap quality but reduces accuracy. This illustrates reward hacking in GUI grounding, where models overfit to one objective at the cost of others. These metrics capture complementary yet competing aspects: Rua focuses on correctly identifying the target box, while RIJ measures overlap with ground truth. Their conflict when optimized separately highlights the challenge of designing balanced rewards.
GRPO's sample selection bias toward different box sizes leads to reward hacking. To investigate
the cause of reward hacking, we visualize two cases with predicted bounding boxes Figure ( (Lett) Models trained with Ryn tend to produce boxes smaller than the ground truth, while Roo leads to

---

Hit
IoU
Hit+loU
Hit+Box
Hib+ieU+Baxx
Golden Relative Box Size
Accuracy
loU
Evaluation results
0.90
0.45
0.40
0.80
0.30
0.79
0.25
0.20
100
200
300
100
200
300
0
100
200
300
Policy iteration step
Pollcy Iteration step
Pollcy Iteration step
Figure 3: Changes in accuracy (Left), IoU (Middle), and relative box size (Right) across policy iterations during model training on the ScreenSpot dataset.
significantly larger boxes. This pattern is quantitatively confirmed in Figure (Right), where the relative size of predicted boxes increases over training under RiaU, but decreases under RHi- Further, illustrated in Figure 4 (Right), the cause of these opposite trends lies in how GRPO's sample selection interacts with the reward functions: optimizing Лчи encourages the model to pick smaller boxes that better capture the core target region, improving accuracy, whereas optimizing Rjo favors larger boxes that yield higher overlap with ground truth, thus boosting IOU.
RBox helps mitigate reward hacking by regularizing box size. To address reward hacking, straightforward solution is to jointly optimize both Ria and RioU- However, as shown in Figure training may still be dominated by one of the two, resulting in suboptimal balance. To alleviate this,
we introduce a new reward function Rgox in Table Here, Tp, = 1-181 -21 1/ /mmage width* with similar
definitions for the other terms. This reward encourages the predicted bounding box to match the ground truth in in both accuracy and IoU, with predicted box sizes becoming more aligned with the ground truth. We also experiment with using RBo alone, but the model fails to produce outputs in the correct format. We hypothesize this is because RBox assigns non-zero rewards even to poorly grounded predictions, encouraging optimization on uninformative samples. Therefore, RBox should be used in conjunction with Rum and Rio, which directly reflect the evaluation metrics and serve as auxiliary constraints.
of size. As shown in Figure incorporating RBo. leads to further improvements
Ground Truth
Rin optimized
Rio optimized
Raw Box
Optimized Box
Sertings
IoU: 0 0.05
Acc: 0
Excowoni
Payment Shipping
WeChhe
Bubscriptios
Weather
Weather
view password security
choose network
Larger
Smaller Size
Figure 4: (Left) Two cases with predicted bounding boxes and golden-truth boxes. (Right) Two examples illustrating why RoU favors larger boxes, while Rin prefers smaller ones.
3.3 Analysis on GRPO Objective
Recent approaches to improving GUI agents 25
28 40 have adopted RL techniques, such as the
GRPO algorithm proposed by DeepSeekMath 34 GRPO optimizes the policy me by sampling set of candidate responses {o; J from the old policy Th. for each input query q, where each response o; has length los|- The policy is updated based on a normalized advantage Á computed for each token, forming the objective (GRPO(TT):
(TR)O(TT) (-r
loil
To(ou, oi, New (oi.e Oi,ee)
min
Ai,, clip
e, -,,1+) Aie
N
To (Oi, q, Oi,<t)
(2)
where / is the index of the token in the response, is hyperparameter that controls the maximum allowed deviation from the old policy, and clip(; 1 e, + e) applies clipping to stabilize training.
In the setting of GUI grounding tasks, Eq. Pintroduces two biases (see also in Figure 5)
5

---

Correct
8.0.8
Incorrect
Wy vi-no
41
03
42
Au/loy
Easy Sample
Hard Sample
Box
Figure 5: Illustration of the response-lever length biases and query-level difficulty biases in GRPO.
Response-level length bias [6] has been observed 26 that GRPO introduces length bias: longer responses are preferred among incorrect ones, while shorter responses are favored among correct ones. This arises from dividing the objective (To) by (o,1, which amplifies the per-token gradient for shorter responses when the advantage is positive (A,,t > 0), pushing the policy toward simpler correct outputs. Conversely, encourages unnecessarily long incorrect answers. As shown in Figure 6 (Left), training gradually results in longer incorrect and shorter correct responses. This trend further harms performance, longer outputs shown to degrade accuracy in Section 3.1 Therefore, length bias in grounding tasks is especially problematic: it not only increases token count but also reduces overall quality.
Question-level difficulty bias: It has been noted in 226 that dividing the centered outcome rewards by (td. r2........ TN) can lead the model to focus disproportionately on either harder or easier samples. However, we argue that assigning higher weights harder samples during policy updates is desirable. In grounding tasks, the relative box size of the target can serve as proxy for task difficulty [8]. Based on this intuition, we modify the original objective to Wy [aRPo(TT)), where Wa reflects the difficulty of query q. The weight Wi is computed based on the relative box size, where a larger relative size indicates an easier grounding instance. Detailed computation is provided in Appendix C.3 Multiplying the objective by Wa assigns greater gradients to harder samples, thus encouraging the model to focus on more challenging instances. In fact, length bias can also be viewed as a form of difficulty bias, as it guides the model toward generating longer incorrect responses, which exacerbates the difficulty of learning from such examples and indirectly shifts the focus toward easier samples.
Length Blas
Extreme Sample (Correct)
Extreme Sample (Incorrect)
0.28
Output Tokans
0.30
48
Proportion
portion
46
0.20
44
Correct
GRPO
0.20
GRPO
42
Incorrect
Ours
Ours
0.16
75
150
225
300
0
75
150
225
300
75
150
225
300
Policy iteration step
Policy Iteration step
Policy Iteration step
Figure 6: Changes in output length and proportion of extreme samples during policy training.
Experimental Results. We implement both improvements, with the results reported in Table 2 and in Table 9 (Appendix D.2). Mitigating length and difficulty biases consistently enhances model performance. Figure 6 (middle and right) further tracks the ratio of extreme samples, where all sampled responses are either correct incorrect, throughout training. In the middle plot, our method initially lags on easy samples due to their lower weights, but gradually outperforms the original GRPO as these examples are eventually learned. the right plot, our method maintains lower proportion of extreme cases on hard samples, indicating that difficulty re-weighting facilitates better learning from challenging instances.
Table 2: Evaluation results on ScreenSpot after mitigating length and difficulty biases.
Mobile
Desktop
Web
Training Objective
Avg.
Text
Icon
Avg.
Text
Icon
Avg.
Text
Icon
Avg.
Standard GRPO 34
96.5
82.4
90.6
87.6
60.7
76.3
85.0
68.4
77.3
82.3
0.| >> Max_ Tokens 26
96.5
81.9
90.4
86.6
65.7
77.8
85.4
71.9
79.1
83.2
(T)) + up JoRea (T))
97.2
79.6
89.0
85.5
62.8
76.0
88.0
73.8
81.4
83.3

---

3,4 GUI-G1: A Tailored RL Visual Grounding Model
Based on the above analysis, we identify key limitations in existing training paradigms for grounding tasks. We now summarize our proposed improvements and present a comparison with prior methods.
Our method, GUI-GI, addresses the identified issues through:
Thinking leads to poorer grounding performance in Sec.3.11 We adopt a template without intermediate reasoning prevent the policy from generating long thinking during training.
Rum and Rio cause opposing types of reward hacking in Sec. 3.2) We combine Bui and Riol as the reward signal and introduce an additional Rgox term to regularize predicted box sizes, mitigating reward hacking caused by box size mismatch.
Original GRPO introduces length and difficulty biases in Sec.3.3 We remove these biases by replacing |o:| with a constant Max_Tokens [26] and by weighting the GRPO objective (To) with difficulty coefficient Wpp.
To make the distinctions clearer, Table 3 provides a structured comparison between GUI-GI and existing R1-style GUI agents in grounding tasks.
Table 3: Comparison of RI RI-style GUI agents for grounding tasks. focusing on RL, template, reward, and support for length control and difficulty awareness. and / are tunable hyperparameters.
Method
RL
Template Design
Reward Design
Length Control
Difficulty Aware
UI-RI
GRPO
Thinking Template
GUI-RI 40
GRPO
Thinking Template
REi
x
X
InfGUUIRR
25
RLOO
16
Thinking Template
Rim + Rual
x
X
GUI-GI (Ours)
GRPO
No-Thinking Tomplate
Rm + a
/
/
4
Experiments
In this section, we introduce the experimental setup for training and evaluating our proposed GUI- G1-3B agent. We outline the implementation details, describe the training dataset and evaluation benchmarks, and provide detailed comparison with state-of-the thettrrrrt
art methods.
Implementation Details. Our model is built upon the Qwen2.55 the VLM-R1 framework [B5). The reward function follows the form Ryn a R where a is set to 0.25 and / to 0.125. We conduct training on 4 NVIDIA H800 GPUs over 3 days, with a global batch size of 32 and learning rate of 10-6, No KL divergence regularization is applied. Only one training epoch is required.
and trained using
Training Dataset and Evaluation Benchmarks. We construct 17K-sample grounding dataset spanning three domains: Mobile (from UI-BERT (3), Web (from OS-Atlas (39]), and Desktop (from AS-Atlas, covering Windows, Linux, and MacOS), More details of the training dataset are provided in the Appendix D.I To ensure data quality, each sample is prompted eight times using Qwen2.5-VL. V---3B IIL-tutInttruct, and those with consistently correct or incorrect responses are discarded [6]. For evaluation, we adopt ScreenSpot [8] and ScreenSpot-Pro [18]. While ScreenSpot assesses grounding performance across diverse platforms, including Mobile, Web, and Desktop, ScreenSpot- Pro emphasizes more challenging desktop scenarios, featuring high-resolution screens.
Performance Comparison on ScreenSpot. We compare GUI-G1-3B with a range of ofate-of-the-art open-source and proprictary GUI agents, using results reported in their original papers. Table 4 summarizes performance the ScreenSpot benchmark. GUI-G1-3B achieves state-of-the-ard results, outperforming proprietary systems like Gemini 2.0 91 general-purpose purpose models such as the Qwen2.5 series 4. GUI-specific SFT models like OS-Atlas 39) and UGround [I], as well as RI-style models including UI-R1 28 GUI-RI 40) and InfiGUI-RI (25 It also surpasses larger models model is trained only 17K samples and requires no intermediate reasoning steps. Moreover, it achieves higher inference efficiency by generating significantly fewer tokens than other methods (see Appendix D.3 Table 10).
like OS-Atlas-7B 7B (9). Despite its strong performance,
Performance Comparison on ScreenSpot-Pro. As shown Table GUI-G1-3B achieves compet- itive performance on the challenging ScreenSpot-Pe benchmark, with an overall average score of
7

---

Table 4: Performance ScreenSpot across Mobile, Desktop, and Web. Bold highlights the best results, underlined the second-bes best. "_" indicates missing values due to unavailable results in the original paper, unreleased model checkpoints, and unreleased inference code.
Model
# #Training Samples
Accuracy (%)
Avg.
Mobile
Desktop
Web
Text
Icon
Text
Icon
Text
Icon
Proprietary Models
GPT-4o [31
30.5
23.2
20.6
19.4
11.1
7.8
18.8
Claude Computer Use
83.0
Gemini 2.0 (Project Mariner)
84.0
General Open-source Models
Qwen2-Ve VL-7B [37
61.3
39.3
52.0
45.0
33.0
21.8
42.9
Qwen2.5- VL-
55.5
Qwcn2.5-VLL-78
84.7
GUI-specific Models (SFT)
1ogAgent-18B [13
222M
67.0
24.0
74.2
20.0
70.4
28.6
47.4
SceClick-9.06 -.6.cck-.66B
LM
78.0
52.0
72.2
30.0
55.7
32.5
53,4
UGround-7B [
10M
82.8
60.3
82.5
63.6
80.4
70.4
73.3
UGround-vl -1-Bu
93.0
79.9
93.8
76.4
90.9
84.0
86.3
OS-Atlas-7B 39
13M
93.0
72.9
91.8
62.9
90.9
74.3
82.5
ShowU1-2B 23
256K
92.3
75.5
76.3
61.1
81.7
63.6
75.1
Aguvis-7B 41
IM
95.6
77.7
93.8
67.1
88.3
75.2
84.4
UI-TARS-2B 33
93.0
75.5
90.7
68.6
84.3
74.8
82.3
UI-TARS-7B 33
94.5
85.2
95.9
85.7
90.0
83.5
87.7
GUI-specific Models (RI.)
UI-R1-3B 2X
136
90.2
59.3
85.2
73.3
GUI-R1-3B 40
3K
93.8
64.8
89.6
72.1
GUI-RI-7B 40
3K
91.8
73.6
91.3
75.7
InfiGUI-RI-3B
32K
97.1
81.2
94.3
77.1
91.7
77.6
87.5
Ours GUI-G1-3B
17K
98.6
85.8
96.4
80.7
91.4
82.3
90.3
37.1%. It outperforms the larger UI-TARS-7B model (35.7%) and significantly surpasses the best- performing R1-based model, InfiGUI-R1-3 (35.7%). Although both GUI-G1-3B and OS-Atlas-7BB use the same training dataset, our model performs worse the OS subset (16.1% vs. OS-Atlas-7B's 16.8%), suggesting that its gains mainly result from post-training that activates pretrained knowledge rather than from task-specific data. This demonstrates the robustness and generalization ability of our approach in real-world scenarios. Additionally, we apply our improved training approach to the GUI-GI model across various model sizes and families, and reproduce the UI-R1 setting in the Appendix D.4, The results confirm that GUI-G1 maintains strong performance, highlighting the robustness and effectiveness of our method.
5 Related Work
Grounding for GUI Agents. Grounding is central to GUT agents research [38 47]. driving advances in data collection and model design. Farly works such as VUT 21 and Spotlight [17] aligned task structures and modalities (e.g., screenshots, instructions) using BERT-based [10] representations. while RUIG (45) employed reinforcement learning to map instructions to UI coordinates. With MLLMs, the focus shifted fine-tuning pretrained models for cross-platform interaction and GUI adaptation. ShowUI 23 improved efficiency by reducing redundant visual tokens, and Ferret- 2 22 enhanced understanding via high- resolutioo encoding and cross cross- ploss-platfomm generalization. introduced multi-turn grounding with sequential reasoning for dynamic multi-step
Aria- UI 43 interactions. More recently, OS-Atlas (39 and UGround [TT] advanced the field by releasing large open-source datasets and training models robust to out-of diittii-iitiittoonnn tasks. Unlike these data- intensive, supervised approaches, our work explores how minimal data and an R1-Zero-like method unlock MLLM grounding capability for GUI tasks.

---

Table 5: Comparison of agent models ScreenSpot- Pro across Text, Icon, and Average task metrics. Best results are shown in bold, with second best results underlined.
CAD
Development
Creative
Scientific
Office
OS
Avg.
Model
Text
Avg.
Avg.
Avg.
Aug.
Avg. Text
Avg. Text Avg.
Proprictary Models
GPT-4o HD
2.0
0.0
1.5
1.3
1.0
0.0
0.9
0.0
0.0
0.0
1.3
0.0
0.8
Claude Computer Use
2
14,5
3.7
11.9
22.0
3.9
12.6
25.9
3.4
16.8
33,9
15.8
25.8
30.1
16.3
26.9
11.0
4.5
8.1
23.4
7.1
17.1
General Open-source Models
Qwea2-VL-78 37
0.5
0.0 0.4 2.6
1.3
1.5
0.0
3.5
19
3.0
0.9
0.0
0.5
2.5 0.2
1.6
Qwen2.5. 3B
23.9
Qwen2.5 VL-7B
29.0
Kimi-VI. 36
34.5
GUN-spectfi Models (SFT)
SeoClick 8
2.5
0.0
1.9
0.6
0.0
0.3
1.0
0.0
2.0
1.1
0.0
0.9
2.8
0.0
1.5
1.8
0.0
1.1
CogAgent-199 18B 13
7.1
6.1
14.9
0.7
8.0
9.6
22.2
13.4
13.0
0.0
10.9
5.6
0.0
3.1
12.0
0.8
7.7
Aria-Ul [4]
7.6
1.6
16.2
8.4
23.7
14.7
18.1
20.3
16.1
4.7
0.0
2.6
17.1
2.0
11.3
0S-Atlas-4B 39
2.0
0.0
1.5
7.1
0.0
3.7
3.0
14
2.3
9.0
55
7.5
5.1
38
48
5.6
0.0
3.1
5.0
1.7
3.7
OS-Atlas-7B
10.3
27.4
28.1
18.9
ShowUI-2B (25
2.5
0.0
1.9
16.9
1.4
9.4
9.1
0.0
5.3
13.2
73
10.6
15.3
7.5
13.5
10.3
2.2
6.6
10.8
2.6
7.7
UGrond 11]
11.1
26.6
9.7
25.0
16.5
DGrund-VI--D
13.5
35.5
27.8
38.8
48.8
26.1
31.1
UI-TARS-2B
17.8
4.7
14.6
47.4
4.1
26.4
42.9
27.6
56.9
17.3
39.8
50.3
17.0
42.6
21.
5.6
14.3
39.6
8.4
27.7
UI-TARS-7B 83
20.8
9.4
18.0
58.4
12.4
36.1
50.0
32.8
63,9
318
50.0
63.3
20,8
53.5
30.8
16.9
24.5
47.8
16.2
35.7
GUl-sperific Models (RL.)
InfGUU-RI-3B
25
33.0
14,1
28.4
51.3
12.4
32.4
44.9
7.0
29.0
58.3
20.0
41.7
65.5
28.3
57.0
43.9
12.4
29.6
49.1
14.1 35.7
UI-RI-3B
2X
11.2
6.3
22.7
27.3
42.4
11.8
32.2
11.
13.1
4.5
17.8
GUI-RI-3B
40
26.4
7.8
33.8
4.8
40.9
61.8
17.3
53.6
17.0
28.1
5.6
GUI-RI-78
4D
23,9
6.3
49.4
4.8
38.9
8.4
55.6
11.8
58.7
26.4
42.1
16.9
Ourr
GUI-G1-3B
32.2 50.7 10.3
37.1
R1-Zero-like Training for MILMs. DeepSeek-R1-Z4 12 introduced GRPO -based post- training framework that enhances reasoning via structured outputs. This approach was extended to multimodal settings by R1 [14), MM-EUREKA (29]. and VisualThinker-RI
[46), all improving vision-language reasoning. LMM-RI 32 achieved strong results with a two-stage RL scheme and low cost. However, recent work [9 found that reasoning avverse models can outperform reasoning-base based ones in multimodal classification, suggesting reasoning is not always beneficial. In GUI agents, studies such UI-RI (28). GUI-RI [40], and InfiGUI-RI (25) verified the effectiveness of R1 R1-Zero--lic training in action prediction and grounding. showing consistent gains on ScreenS pot (8), ScreenSpot-Pro [18), and AndroidControl 20 This work focuses the grounding
task and revisits whether the standard RI R1-Zero-like settings are suitable for GUI scenarios.
6 Conclusion
In this work, we revisit the R1-Zero-style training setup for GUI grounding agents from three aspects: input design, reward evaluation, and policy update. We show that lengthy reasoning impairs grounding performance and propose Fast Thinking Template to address it. We then find and analyze opposing types of reward hacking issues in existing reward designs and introduce a box-size constraint to mitigate them, leading to improved performance. Finally, we examine GRPO's 's length and difficulty biases in grounding tasks and address them by removing length normalization and incorporating difficulty-bass weighting. With only 17K samples, our GUI-G1-3B surpasses larger RI-style models on ScreenSpot and ScreenSpot-Pro.
Acknowledgments
This work was funded by the National Key R&D Program of China (2023YAA the National Supported by fund for building world- class universities (disciplines) of Renmin University of China. Work partially done at Beijing Key I Laboratory of Research on Large Models and Intelligent Governance, and Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOF. We thank Yuchong Sun for helpful discussions and insights during the development of this work.
Natural Science Foundation of China

---

References
[1] A. Ahmadian, C. Cremer, M. Gallé, M. Fadace, J. Kreutzer, O. Pietquin, A. Üstün, and S. Hooker. Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12248-12261 2024.
[2] Anthropic. Developing a computer use model. ceglloing-coomptten use 2024. Accessed: 2025-04-12.
http://www.atrrroc......o.o
[3] C. Bai, X. Zang, Y. Xu, S. Sunkara, A. Rastogi, J. Chen, et al. Uibert: Learning generic multimodal representations for ui understanding. arXiv preprint arXiv:2107.137:1 2021.
[4] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang. S. Wang, J. Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
[5] Y. Chai, S. Huang, Y. Niu, H. Xiao, L. Liu, D. Zhang. P. Gao, S. Ren, and H. Li. Amex: Android multi-annotation expo dataset for mobile gui agents. arXiv preprint arXiv:2407. 17490, 2024.
[6] Z. Chen, Y. Min, B. Zhang. J. Chen, J. Jiang, D. Cheng, W. X. Zhao, Z. Liu, X. Miao, Y. Lu, et al. An empirical study on eliciting and improving rl-like reasoning models. arXiv preprint arXiv:2503.044 0444888 2025.
[7] Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic risual-linguisti tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14188-24418, 2024.
[8] K. Cheng, Q Sun, Y. Chu, F. Xu, L. YanTao, J. Zhang, and Z. Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9313-9332, 2024.
[9] G. DeepMind. Gemini-2.0 (project mariner). bttpss//deepnnt http:///wwmm....ggggl/////cc.ooo woogle/technoloogiieeeoooo project-mariner 2024. Accessed: 2025-04-12.
[10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolo- gies, volume (long and short papers), pages 4171-4186, 2019.
[11] B. Gou, R. Wang, B. Zheng, Y. Xie, C. Chang, Y. Shu, H. Sun, and Y. Su. Navigating the digital world humans do: Universal visual grounding for gui agents. In 13th International Conference on Learning Representations, ICLR 2025, 2025.
[12] D. Guo, D. Yang, H. Zhang, Song, Zhang, R. Xu, Q. Zhu, S. Ma, Wang, X. Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint WXXi:2501..1244 2025.
[13] W. Hong, W. Wang, Q. Lv, J. Xu, W. Yu, J. Ji, Y. Wang. Z. Wang. Y. Dong, M. Ding, al. Cogagent: visual language model for gui agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14281-14290 2024.
[14] W. Huang, B. Jia, Z. Zhai, S. Cao, Z. Ye, F. Zhao, 7. Xu, Y. Hu, and S. Lin. Vision-rl: Incentiviz- ing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025.
[15] D. Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, New York, October 2011 1.
[16] W. Kool, H. van Hoof, and M. Welling. Buy 4 reinforce samples, get a baseline for free! 2019.
[17] G. Li and Li. Spotlight: Mobile ui understanding using vasionnl models with a focus.
In 11th International Conference Learning Representations, ICLR 2023, 2023.
10

---

[18] K. Li, Z. Meng, H. Lin, Z. Luo, Y. Tian, J. Ma, Z. Huang, and T.-S. Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use. arXiv preprint arXiv:2504.07981, 2025.
[19] M. Li, J. Zhong, S. Zhao, Y. Lai, and K. Zhang. Think or think: A study of explicit thinking in rule-based visual reinforcement fine-tuning, arXiv e-prints, pages arXiv-2503 2025.
[20] W. Li, W. Bishop, A. Li, C. Rawles, F. Campbell- D. Tyamagundlu, and O. Riva. On the effects of data scale on computer control agents. arXiv e-prints, pages arXiv-2406, 2024.
[21] Y. Li, G. Li, X. Zhou, M. Dehghani, and A. Gritsenko. Vut: Versatile ui transformer for multi-modal multi-task user interface modeling. arXiv preprint arXiv:2112.05622 2021.
[22] Z. Li, K. You, H. Zhang, D. Feng, H. Agrawal, X. Li, M. P. S. Moorthy, J. Nichols, Y. Yang, and
Z. Gan. Ferret-ui 2: Mastering universal user interface understanding across platforms, arXiv preprint arXiv:2410. 18967, 2024.
[23] K. Q. L. Li, D. Gao, Z. Yang. S. Wu, Z. Bai, W. Lei, L. Wang. and M. Z. Shou. Showui: One vision- iitoo-language-actionn model for gui visual agent. arXiv preprint arXiv:2411.17465 2024.
[24] X. Liu, B. Qin, D. Liang, G. Dong, H. Lai, H. Zhang, H. Zhao, I. L. long, Sun, J. Wang, et al. Autoglm: Autonomous foundation agents for guis. arXiv preprint arXiv:2411.00820 2024.
[25] Y. Liu, P. Li, C. Xie, X. Hu, X. Han, S. Zhang, H. Yang, and F. Wu. Infigui-rl: Advancing multi- modal gui agents from reactive actors to deliberative reasoners. arXiv preprint arXiv:2504.14239 2025.
[26] Z. Liu, C. Chen, W. Li, P. Qi, T. Pang, C. Du, W. S. Lee, and M. Lin. Understanding rl-zero-like training: A critical perspective. arXiv preprint S.Ii::2503.27888 2025,
[27] Y. Lu, J. Yang, Shen, and A. Awadallah. Omniparser for pure vision based gui agent. arXiv preprint arXiv:2408.00200 00203, 2024.
[28] Z. Lu, Y. Chai, Y. Guo, X. Yin, L. Liu, H. Wang, G. Xiong, and H. Li. Ui-rl: Enhancing action prediction of gui agents by reinforcement learning, arXiv preprint arXiv:2503.21620, 2025.
[29] F. Meng, L.. Du, Z. Liu, Z. Zhou, Q. Lu, D. Fu, T. Han, B. Shi, W. Wang, J. He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.077 07365, 2025.
[30] OpenAl. Learning to reason with 1lms.
wttp:///lllannnnn-o-oeorrreeeeee www.rreannnihhhllmeeee Accessed: 15 March 2025.
[31] OpenAl. Gpt-4o, 2024. Accessed: 2025-01-03.
[32] Y. Peng. G. Zhang, M. Zhang, Z. You, J. Liu, Q. Zhu, K. Yang, X. Xu, X. Geng, and X. Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based arXiv preprint arXiv:2503.075366 2025.
[33] Y. Qin, Y. Ye, J. Fang, H. Wang, S. Liang, S. Tian, J. Zhang, J. Li, Y. Li, S. Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12222 2025.
[34] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprins axx1v:2402.033300 2024.
[35] H. Shen, P. Liu. J. Li, C. Fang, Y. Ma, J. Liao, Q. Shen, Z. Zhang, K. Zhao, Q. Zhang, R. Xu, and T. Zhao. Vlm-rl: A stable and generalizable rl-style large vision- lision-language model. arXiv preprint arXiv:2504.07615 2025.
[36] K. Team, A. Du, B. Yin, B. Xing, B. Qu, B. Wang, C. Chen, C. Zhang, C. Du, C. Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.074 07441, 2025.
11

---

[37] P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world any resolution. arXiv preprini arXiv:2409. 12191, 2024.
[38] S. Wang, W. Liu, J. Chen, Y. Zhou, W. Gan, X. Zeng. Y. Che, S. Yu, X. Hao, K. Shao, al. Gui agents with foundation models: A comprehensive survey, arXiv preprint arXiv:2411.000 1.04999 2024.
[39] Z. Wu, Z. Wu, F. Xu, Wang, Q. Sun, C. Jia, K. Cheng, Z. Ding, L. Chen, P. Liang, et al. Os-atlas: A foundation action model for generalist gui agents. In 13th International Conference on Learning Representations, ICLR 2025, 2025.
[40] X. Xia and R. Luo. Gui-rl: A generalist r ll-style vision-language action model for gui agents. arXiv preprint arXiv:2504. 10458, 2025.
[41] Y. Xu, Z. Wang, J. Wang, D. Lu, Xie, A. Saha, D. Sahoo, T. Yu, and C. Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint arXiv:2412.04454, 2024.
[42] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505. 003800888. 2025.
[43] Y. Yang. Y. Wang. D. Li, Z. Luo, B. Chen, C. Huang, and Li. Aria-ui: Visual grounding for gui instructions. arXiv preprint arXiv:2412.16256, 2024.
[44] L. Zhang, L. Gao, and M. Xu. Does chain-of-thouuht reasoning help mobile gui agent? an empirical study. arXiv preprint arXiv:2503.16788 2025.
[45] Z. Zhang, Xie, X. Zhang, and Y. Lu. Reinforeed instruction grounding: Towards generic ui task automation api. arXiv preprint arXiv:2310.0477 2023.
[46] H. Zhou, X. Li, R. Wang, M. Cheng, T. Zhou, and C.-J. Hsieh. RI-zero's" aha moment" in visual reasoning on a 2b non-sft model. arXiv preprint arXiv:2503.05132, 2025.
[47] Y. Zhou, Wang, S. Dai, Q Jia, Z. Du, Z. Dong, and J. Xu. Chop: Mobile operating assistant with constrained fregunryy optimized subtask planning. arXiv preprint arXiv:2503.03743, 2025.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state our key findings and contributions, including critical analysis of RI-Zero components and the proposal of a new method, which supported by experimental results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
12

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper includes a dedicated "Limitations" section in the Appendix A that clearly discusses the constraints of our approach. These include dataset coverage, key assumptions made during modeling, and the sensitivity of the method to specific configurations. The section also reflects on the generalizability of the results and the computational considerations, offering transparent view of the work's scope.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer[[NA]
Justification:
The paper does not include any theoretical results formal proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
13

---

Answer: [Yes]
Justification: The paper provides all necessary information to reproduce the main experimen- tal results. We include detailed descriptions of the training objectives in Sec The training settings, dataset preprocessing procedures, computing resources, evaluation protocols, and model architecture are specified in Sec
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We include code, data, and reproduction instructions (including environment and commands) via anonymous link in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
httii//ggiiiee//oeeuummiieeee public/ wublic/guides/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https //nips http://ccwii//eeee/e////eeaaaaaa
for more details.
14

---

The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
reproducible, they
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe the experimental settings and necessary details the appendix and supplemental material, with summary provided in Sec. of the main paper.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Due to the computational cost of large-scale experiments, we did not perform repeated trials statistical significance testing. All reported results are from single under fixed settings, ensuring consistency across comparisons.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
15

---

Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We report compute details in Sec. All training was conducted on NVIDIA H800 GPUs and took approximately days.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:
Answer: [Yes]
Justification: We have reviewed the NeurlPS Code of Ethics and confirm that our research complies with all relevant ethical guidelines. Our work does not involve human subjects, sensitive data, or foresecable negative societal impact.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper includes dedicated "Broader Impacts" section in the Appendix E. where we discuss potential positive outcomes such as improving accessibility and automation via GUI agents, as well as possible negative consequences, including reward hacking and model bias. We also highlight the importance of fairness and robustness in real-world deployment.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
16

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our paper does not release any new pretrained models datasets that pose risk of misuse. The study is based on public datasets and models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We used the publicly released Owwn2.5-VL-3B--1. Instruct model, which is licensed under the "Qwen RESEARCH LICENSE AGREEMENT," and cited the official release. We also used the ScreenSpot and ScreenSpot Pro datasets, which are publicly available and cited accordingly in the paper. All assets were used under their respective licenses and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcodd com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
17

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We do not release new model checkpoint, but we provide the dataset and code supplementary material. Comprehensive documentation, including usage instructions, dataset details, and limitations, included alongside the assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing or research with human subjects and thus no IRB approval was needed.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification: The research employs multimodal large language models (MLLMs) key component in the proposed method, specifically using Qwen2.5-VL-33 Instruct for training the GUI agent, which is central to the approach and results.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
19

---

A Limitations
While our method demonstrates strong performance GUI grounding. there remain several lim- itations that offer directions for future work: (1) Focus on grounding. The current work focuses on grounding, which essential for GUI agents, but does not cover tasks like action prediction or long-horizon planning. Future research can extend this approach to support full decision decision-making in GUI interaction. (2) Scope of RL analysis. The study mainly examines online reinforcement learning, especially GRPO. Other factors such as dataset composition, model design, and hyperparameter tuning are not fully explored and deserve further analysis. (3) Limited training data. Our model trained relatively small set of public datasets, which constrains its performance ceiling. In future work, we plan to scale up training using larger and more diverse datasets, such as those adopted in GUI-RI [40]. to further improve generalization and robustness.
B Fast Thinking Template versus Slow Thinking Template
To examine the role reasoning in grounding tasks, we formally define two instruction templates, the Fast Thinking Template and the Slow Thinking Template. We then train models under both settings to compare their effects on grounding performance. The detailed training configurations and implementation settings are provided in Section B.2
B.1 Formal Definition of Fast and Slow Thinking
Following Thinking, Fast and Slow [5]. Fast Thinking corresponds to System 1, characterized by intuitive, automatic, and effortless responses. Slow Thinking corresponds to System 2, characterized by deliberate, analytical, and effortful reasoning. In the context of large language models, Fast Thinking denotes direct answer generation without intermediate reasoning, exemplified by the non-thinking mode in Qwen3 [42]. Slow Thinking refers to explicit multi-step reasoning processes, such those elicited through chain-of-thought prompting. In work, Fast Thinking represents direct grounding prediction from visual inputs, whereas Slow Thinking involves reasoning over the observed scene before producing the final grounding output.
B.2 Experimental Setting
We train our Qwen2.5-VL-3B In-tructct model the CI-R1-3B-Train datase which consists of 101 samples with grounding annotations. During training, the vision encoder is kept frozen. We set the learning rate 1 10 and the sampling temperature 0.9, generating eight responses per prompt. Training is performed on four L20 (48G) GPUs with a batch size of four samples per GPU, B = and gradient accumulation step of one.
B.3
Experimental Results
The results in Table show that encouraging direct answers with Fast Thinking Template consistently leads to better performance, supporting our hypothesis that explicit reasoning can hinder grounding effectiveness.
Table 6: Results on ScreenSpot after optimization with Fast and Slow Thinking Templates.
Template
Mobile
Desktop
Web
Avg.
Slow Thinking Template
94.6
84.7
84.9
88.7
Fast Thinking Template
96.2
87.4
84.3
89.8
Train
20

---

Table 7: Results on ScreenSpot- after optimization with Fast and Slow Thinking Templates.
Template
CAD
Development
Creative
Scientific
Office
OS
Avg.
Slow Thinking Template
18.0
19.1
20.5
32.3
40.0
19.4
24.4
Fast Thinking Template
20.3
19.1
21.1
33.1
43.5
17.3
25.3
C Analysis Experiments Settings
C.1 Training Details
We fine-tune all parameters of the Owen2.5-VL-3B--nnst model using samples evenly drawn from three domains: mobile (UIBERT (3). web, and desktop (OS-Atlas (39). Due to computational constraints, we randomly sample 300 grounding examples from each domain. Despite the relatively small dataset, the model achieves strong performance after fine-tuning ensuring the reliability of our subsequent analysis. Training follows the default setup the VLM-RI repository [35], using
rollouts per example and no KI--divergence regularization by default.
C.2 Evaluation Metries
Specifically, we use the relative box size to measure the size of the predicted bounding box. is calculated as:
82 + 2t2 91 -
A
IMAGE_WINE + IMAGE_HEIGHT
where MMAGE_WIDTH and IMAGE_HEIGHE denote the pixel width and height of the input image, respectively.
C.3 Difficulty-Aware Weighting Strategy
To compute the difficulty weight Wi for each sample based on its relative box size Xg E (0, 1]. we first take the inverse of the size to reflect intuition that smaller boxes are harder: X₂ = x; We
X.-min, X3 then normalize the inverted values the range [0, 1] by computing λ₃ max: -min X7 Finally, linearly rescale the normalized scores to the interval (0.5, 1.5] to obtain the final difficulty weights: wy = 0.5 + Ay Putting everything together, the final formula is
min; N/+
A
weg = 0.5 +
max;;
min;
This ensures that harder samples (with smaller boxes) receive higher weights, while keeping the values a stable and bounded range.
D More Experiments Details
D.1 Training Data Composition
To provide a comprchensive grounding resource across diverse platforms, we construct dataset containing 17K samples distributed across three representative domains: Mobile, Web, and Desktop.
The Mobile domain is derived from the UI-BERT dataset [3] which consists of user interface data collected from Android applications.
The Web domain is sourced from OS-Atlas 39 including interactive web elements and browser- based environments.
The Desktop domain is also based on OS-Atlas, but focuses on native applications and interfaces from major desktop operating systems, including Windows, Linux, and MacOS.
21

---

Each domain contains a balanced set of grounding instances that pair natural language commands with corresponding UI elements. Table summarizes the number of samples collected in each domain.
Table 8: Statistics and sources of the grounding dataset across five platforms.
Mobile
Web
Windows
Linux
MacOS
Source
UI-BERT [3
OS-Atlas
[39
OS-Atlas [39]
OS-Atlas (39
OS-Atlas [39
Size
575
7,832
5,576
1,667
1,835
D.2 Debias Results on SereenSpot-Pro
The results in Table@indicate that both length bias correction and difficulty reweighting enhance performance on ScreenSpot-Pro. The limited effect of length bias removal may stem from the predominance of high-resolution images, where text tokens contribute little relative to image tokens. In contrast, difficulty reweighting produces notable improvements, validating its benefit for harder samples.
Table 9: Evaluation results on ScreenSpot-Pin after mitigating length and difficulty biases.
Model
CAD
Development
Creative
Scientific
Office
OS
Avg.
Standard GRPO 34
17.2
18.4
20.8
30.7
40.4
14.8
23.5
lo.| Nax, Tokens
26
16.1
19.7
21.4
32.3
40.4
14.8
23.9
Janpo(Te) mo) up (To)
20.1
D.3 Output Token Efficiency Analysis
To assess the efficiency of different models during inference, we compare the average number of output tokens generated per example across Mobile, Desktop, and Web domains on ScreenSpot [ As shown in Table TO GUI-G1-3B generates substantially fewer tokens than InffUU-RR-3B 25] in
all domains approximately one-third many on average- while maintaining or even improving task accuracy. This compact output not only reduces computational cost but also reflects the model's ability to produce precise and concise responses without relying on verbose intermediate reasoning.
Table 10: Average number of output tokens generated per example on ScreenSpot during inference.
Model
Mobile
Desktop
Web
InfiGUI-RI-3B 25
107
107
114
GUI-GI-3B
37
39
39
D.4 Experiments Across Various Model Sizes and Families
To evaluate the generality our method, we conduct experiments across different model families and sizes. We apply our approach to three models: Intern -.3-2BVL3--BBnsttee (a smaller model from a different family). Qwen2.5- 2oonn.55VL--- LL-3B-Instrrutt and Qwen2.5-VL-7B 1-n..-5--V-----sttrrrre (a larger variant). Fol- lowing the training protocol of UI-RI (28). we retrain the 7B and 2B models the same dataset, while Qwen2.5-VL-3B Instrruct is retrained using our proposed method for direct comparison. Due to resource constraints, LoRA fine-tuning is applied to the 7B model, whereas the others fully fine-tuned. All models use identical training data and hyperparameter settings for a fair comparison. Additional training details are provided in Appendix B.2
The results Table IT and Table 12 show that our method consistently improve performance across both Qwen and VL families, although Intern VL generally shows lower baselines. This trend
aligns with recent observations that most leading GUT agents 22 40 25 adopt Qwen as their backbone. In terms of model scale, increasing from 3B to 7B unexpectedly results in lower grounding
22

---

accuracy. We attribute this to two factors: (a) the 7B model relies on LoRA instead of full fine-tuning. and (b) larger models tend benefit less from task-specific grounding fine-tuning, as similarly observed in Table of GUI-RI 40) For example, UI-TARS-72B performs worse on ScreenSpot (88.4) than the smaller UI-TARS-7B (89.5). Our results on the Qwen series further confirm that grounding performance does not necessarily scale with model size.
Table 11: Performance ScreenSpot across model sizes and families under our method and the UI-RI 28 setting.
Model
Mobile
Desktop
Web
Avg.
Qwen2.5-VL-3B
Base Model
85.6
70.4
65.9
74.8
UI-RI [28]
85.4
Our Method
96.2
87.4
84.3
89.8
Qwen2.5-VL-7B
Base Model
80.5
80.1
83.4
UI-RI 28
80.6
78.7
84.2
81.4
Our Method
88.8
82.3
80.8
84.4
Y
Base Model
0.2
13.2
0.2
3.6
UI-R1 [8
0.5
Our Method
0.8
16.8
0.2
4.8
Table 12: Performance ScreenSpot-Prr across model sizes and families under our method and the
UI-R1
28
setting.
Model
CAD
Development
Creative
Scientifie
Office
OS
Avg.
Qwen2,5-VL-38
Base Model
14.4
UI-RI (28]
17.8
Our Method
20.3
19.1
21.1
33.1
43.5
17.3
Qwenn.55VVL.77
Base Model
13.4
14.7
25.2
35.7
14.3
18.3
UI-R1 (2
10.3
13.4
16.1
25.2
39.6
18.9
19.9
InternVL.3.22 Instrrct
Base Model
1.5
0.0
0.6
1.2
0.4
0.0
0.6
UI-RI (28
0.7
0.9
3.5
1.3
0.0
Method
0.0
0.9
3.9
1.3
0.5
E
Broader Impacts
Our work contributes to the development of more robust accurate GUT agents by addressing key training challenges in reinforcemen learning for visual grounding. This could improve the reliability of accessibility tools and human-computer interaction systems. However, care should be taken when deploying such agents in wolldwrrdd systems, reward design choices may cause unintended behavior such as reward hacking or bias toward easy cases. We encourage future research to further study fairness, robustness, and privacy considerations in GUI agent training and deployment.
23


Reviews:
[{"review_id": "S62wxX9cHp", "reviewer": "Reviewer_uv8E", "summary": "This paper analyzes the R1-Zero-like training approach applied to GUI grounding tasks, identifying three key issues in current methods: reward hacking resulting from the reward function, biases inherent in the GRPO training objective, and degraded performance due to excessive reliance on lengthy textual reasoning. To mitigate this issue, the authors propose corresponding improvements, including (1) a new prompt designed to reduce unnecessary textual reasoning, (2) a constraint on reward function based on bounding box size to deal with reward hacking, (3) normalization of the RL objective with response length and sample difficulty.  Leveraging these improvements, the authors train a GUI grounding model based on Qwen2.5-VL-3B-Instruct, achieving strong results on the ScreenSpot and ScreenSpot-Pro benchmarks.", "questions": "1. Lines 91-92 appear to use \"output evaluation\" and \"reward function\" together. It would be great if the authors could further elaborate the difference between these two terms. Otherwise, standardizing the terminology would enhance clarity.\n\n2. The concepts of \"Slow Thinking\" and \"Fast Thinking\" are under-specified and somewhat ambiguous. It would be great if the authors could provide a formal definition of these two concepts. \n\n3. In line 83, the term \"template\" is unclear. Could the authors explicitly clarify what a \"template\" entails in this context? Is it simply a specific part of the prompt describing the output format, or does it encompass additional instructions for the model's reasoning process?", "limitations": "No major limitation", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strength: \n1. The paper provides helpful empirical insights into the limitations of current GUI grounding training pipelines, such as reward hacking and biases in the RL objective. These insights contribute meaningfully to improving model training. \n\n2. The proposed solutions are effective and well-motivated, addressing the identified limitations. Experimental results validate that the proposed methods lead to good performance on two widely adopted GUI visual grounding benchmarks, ScreenSpot and ScreenSpot-Pro. \n\nWeakness:\n1. The performance of the baseline model (Qwen2.5-VL-Instruct-3B) is omitted (marked \"--\") in Tables 4 and 5 without explanation. This omission raises questions about the comparability and validity of the reported improvements. The authors should clarify why these baseline results are absent, such as indicating if the base model struggled significantly with instruction-following tasks.\n\n2. The writing and presentation could benefit from some improvements. I have listed some uncertainties in the question section in detail.", "was_revised": true, "final_justification": "The authors have addressed my concerns about the ambiguity of their results and terms. I will keep my scores unchanged.", "rebuttal": "We sincerely thank the reviewer for the constructive comments and suggestions, which are very helpful for improving our paper. We are also grateful that you recognized the strengths of our paper. Please kindly find point-to-point responses below.\n\n-------\n\n> **W1:Result Omission Explanation** The performance of the baseline model (Qwen2.5-VL-Instruct-3B) is omitted (marked \"--\") in Tables 4 and 5 without explanation. This omission raises questions about the comparability and validity of the reported improvements. The authors should clarify why these baseline results are absent, such as indicating if the base model struggled significantly with instruction-following tasks.\n\nThank you for your comment. The results for Qwen2.5-VL-Instruct-3B in Tables 4 and 5 are copied from the official technical report [1] (same for Kimi-VL). The report only provides overall scores and does not break down the results by category (mobile, web, desktop). Since the evaluation code was not released, we are unable to reproduce the detailed results and just copy the final result. We will clarify this omission in the captions of Tables 4 and 5. We appreciate your feedback in helping us improve the clarity of the paper.\n\n-------\n\n> **W2:** The writing and presentation could benefit from some improvements. I have listed some uncertainties in the question section in detail.\n\nThank you for your suggestions to help us improve the quality of our paper. We will provide point-by-point responses and corresponding revisions in the following.\n\n-------\n\n> **Q1. Relation between `output evaluation` and `reward function`:** Lines 91-92 appear to use \"output evaluation\" and \"reward function\" together. It would be great if the authors could further elaborate the difference between these two terms. Otherwise, standardizing the terminology would enhance clarity. \n\nThanks for your thoughtful comment. Sorry for the confusion about the relationship between output evaluation and reward function. In our framework, output evaluation is a general concept, and reward function is a specific instance of it in the context of RL. Other paradigms (e.g., SFT or unsupervised learning) may use different evaluation signals such as loss or entropy. Similarly, template and RL objective are concrete instances of input design and policy update, respectively. We will clarify this in the introduction to make these relationships clearer. Thank you again for helping us improve the writing quality.\n\n-------\n\n> **Q3. Formal Definition of \"Template\":** In line 83, the term \"template\" is unclear. Could the authors explicitly clarify what a \"template\" entails in this context? Is it simply a specific part of the prompt describing the output format, or does it encompass additional instructions for the model's reasoning process?\n\nThanks for your question. In our paper, **template** refers to the system-level instruction that appears before the user query. It includes both the expected output format and additional instructions that guide the model’s reasoning process. So it is not limited to one or the other.\n\nTo illustrate this, Table 1 in DeepSeek-R1 [3] provides an example prompt that begins with:\n\n> A conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> reasoning process here </think> <answer> answer here </answer>.\n> **User**: *prompt*. Assistant:\n\nThis system-level instruction before the user query is what we refer to as the *template*. We will revise the paper to make this definition clearer.\n\nWe will revise Section 3.1 to provide a clear and explicit definition of template. Thanks again for your suggestion, which helps us improve the clarity of the paper.\n\n-------\n\n> **Q2. Formal Definition of \"Slow Thinking\" and \"Fast Thinking\":** The concepts of \"Slow Thinking\" and \"Fast Thinking\" are under-specified and somewhat ambiguous. It would be great if the authors could provide a formal definition of these two concepts.\n\nThanks for your valuable suggestions. We believe it is necessary to provide clear definitions of key terms. We will include a clear comparison of the definitions of Fast Thinking and Slow Thinking from in the appendix to improve clarity:\n\n- Briefly, according to Kahneman’s Thinking, Fast and Slow, System 1 (Fast Thinking) is “fast, instinctive, and emotional,” while System 2 (Slow Thinking) is “slower, more deliberative, and more logical.”\n\n- In recent LLM research, such as the Qwen3 model [2], **Non-thinking mode** corresponds to **fast thinking**, where the model directly generates an answer without intermediate reasoning. **Thinking mode** involves multi-step reasoning before the final response, often using chain-of-thought (CoT) prompting. This corresponds to **slow thinking.**\n\n- **In our work**, **Slow Thinking** means the model observes the screen and performs reasoning before outputting a grounding coordinate, while **Fast Thinking** encourages direct answer generation without explicit reasoning, similar to the **Non-thinking mode** in Qwen3.\n\nWe thank the reviewer for the valuable suggestion, which helps improve the rigor and clarity of our presentation.\n\n-------\n\nWe hope the above response can fully address your concerns. We really appreciate your insightful and constructive comments to further help us improve the quality of our work. Please do not hesitate to let us know if you have any further questions. Thanks again!\n\n[1] Qwen2.5-vl technical report. arXiv.\n\n[2] Qwen3 Technical Report. arXiv.\n\n[3] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv."}, {"review_id": "J5BuGGXbt7", "reviewer": "Reviewer_jR9M", "summary": "This paper performs a comprehensive analysis of R1-style training for the GUI grounding objective. They identify three bottlenecks. 1. The GUI grounding performance actually decreases with increasing output tokens (thinking tokens). 2. Using only hit-based or IOU-based reward functions leads to reward hacking, with hit-based rewards favoring smaller boxes and IOU based reward functions favoring larger boxes. 3. The GRPO objective leads to overfitting easier to ground samples due to the generation of fewer output tokens. They see a trend where the examples that the model gets correct have increasingly smaller chains of thought and the examples the model gets wrong have increasingly longer chains of thought. \n\nThe authors propose GUI-G1 model to mitigate these three issues. This is done through the use of a Fast thinking template, which skips reasoning, a combination of IOU, hit-based, and additional box-based reward functions, and the use of a difficulty-aware objective, which favors the contribution of examples with smaller ground truth bounding boxes.", "questions": "Questions: \n1. In Figure 2, right, what are the exact specifications of Low and High Text ratios, and how do you get them? Is this an intervention experiment where low is with a fast thinking template and high is with a slow thinking template? Or is it just an observation where you threshold some value that defines how many is high and how many is low?\n2. From my understanding, higher image tokens are obtained with higher resolution tokens only. So, how do you get different text ratios for the same image size? Is this just by controlling the number of generated tokens? Is there some other way of enhancing the image and increasing the image tokens? \n\nSuggestions: \n1. Please include the UI-TARS-7B and UGround-v1-7B baselines in the GUI-specific models section of Table 4 for a better comparison, since you are comparing with other 7B models, and these models are compared in other tables.\n2. Line 198 should say Figure 6.\n3. The difficulty-aware objective should be validated on Screenspot-Pro.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths \n1. This paper is well-written and timely. The authors clearly analyse and describe the problems encountered by current GUI grounding models and propose well-motivated solutions to these problems. \n2. The performed analysis is thorough and offers useful insights that can be used in future work for improving on GUI grounding models. \n3. The GUI-G1 model shows good positive results on both ScreenSpot and ScreenSpot-pro. \n4. Interestingly, the method requires less training data, and the resulting model gets better results in fewer output tokens compared to previous works. \n\nWeaknesses\n1. The section on the thinking template is a bit confusing. I have added some follow-up questions in the question suggestions. More importantly, an ablation exclusively on the thinking templates is missing. While the analysis does point to better results with fewer tokens, this analysis is performed on a different model (InfiGUI-R1). Also, the analysis only gives a hypothesis that needs to be validated through an explicit ablation in the GUI-G1 model. \n2. The net increase from the difficulty-aware addition to the objective seems very small in Table 2. I think this objective should be evaluated on Screenspot-pro to reflect its benefit.", "was_revised": true, "final_justification": "The authors sufficiently addressed my key concerns regarding the following points: \n1. Explicit Ablation on thinking template \n2. Experiment showing the impact of DifficultyReweight on the Screenspot-Pro benchmark \n3. Inclusion of additional requested baselines\n\nI would like to maintain my positive score.", "rebuttal": "We sincerely thank the reviewer for the constructive comments and suggestions, which are very helpful for improving our paper. We are also grateful that you recognized the strengths of our paper. Please kindly find point-to-point responses below.\n\n-------\n\n> **W1. Thinking Template Ablation Results:** The section on the thinking template is a bit confusing. I have added some follow-up questions in the question suggestions. More importantly, an ablation exclusively on the thinking templates is missing. While the analysis does point to better results with fewer tokens, this analysis is performed on a different model (InfiGUI-R1). Also, the analysis only gives a hypothesis that needs to be validated through an explicit ablation in the GUI-G1 model. \n\nThanks for the thoughtful suggestion. We agree that an explicit ablation on the thinking templates is necessary to validate our hypothesis. To this end, we conduct a controlled experiment using the Qwen2.5-VL-3B-Instruct model. We chose to train on the dataset used in UI-R1 [1]. We train two versions—with and without the thinking templates—under identical settings copied from UI-R1, fully fine-tuning all model parameters. The results are as follows:\n\n**Performance on ScreenSpot**\n\n||Mobile|Desktop|Web|Avg.|\n|-|:-:|:-:|:-:|:-:|\n|Thinking|94.6|84.7|84.9|88.7|\n|No\\_Thinking|96.2|87.4|84.3|**89.8**|\n\n**Performance on ScreenSpot-Pro**\n\n||CAD|Development|Creative|Scientific|Office|OS|Avg.|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Thinking|39.5|39.5|37.8|48.8|51.3|40.3|42.4|\n|No\\_Thinking|44.3|39.4|39.1|49.9|56.1|35.5|**43.8**|\n\nThe results show that encouraging direct answers (Fast Thinking) consistently yields better performance, supporting our hypothesis. We appreciate your feedback in helping us strengthen the analysis and we will add this experiment to final version.\n\n-------\n\n> **W2 & S3. ScreenSpot-Pro Difficulty Reweighting Ablation:** The net increase from the difficulty-aware addition to the objective seems very small in Table 2. I think this objective should be evaluated on Screenspot-pro to reflect its benefit. **&** The difficulty-aware objective should be validated on Screenspot-Pro. \n\nThanks for the suggestion. We evaluate the same checkpoint used in Table 2 on the ScreenSpot-Pro dataset, and the results are shown below:\n\n||CAD|Development|Creative|Scientific|Office|OS|Avg.|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|GRPO|37.9|38.1|37.8|45.7|51.3|29.1|40.0|\n|-LengthBias|39.5|37.8|38.7|46.9|51.7|28.1|**40.5**|\n|+DifficultyReweight|41.4|44.2|42.5|49.6|59.6|38.3|**45.7**|\n\nThe results show that both the length bias correction and difficulty reweighting improve performance on ScreenSpot-Pro. The impact of removing the length bias is relatively small, which we suspect is because ScreenSpot-Pro consists primarily of high-resolution images—thus, the influence of text tokens is minimal compared to the large number of image tokens. In contrast, the proposed difficulty reweighting leads to significant gains, further validating its effectiveness in helping the model better handle harder samples.\n\nTips: We also observe that the results in this table are better than those reported in the main table, showing the same trend as results in **W1**. We suspect this is because both the UI-R1 and our analysis experiments used a `gradient_accumulation_steps` of 1, whereas the main table used a larger value 32. For GRPO training on the grounding task with Qwen2.5-VL-3B-Instruct, faster updates (i.e., smaller accumulation steps) appear to lead to better performance. Moving forward, we plan to train on the full dataset with smaller `gradient_accumulation_steps` and release the updated model.\n\nThanks again for your helpful feedback and we will add the results to the final version.\n\n-------\n\n> **Q1. Low/High Text Ratio Explanation:** In Figure 2, right, what are the exact specifications of Low and High Text ratios, and how do you get them? Is this an intervention experiment where low is with a fast thinking template and high is with a slow thinking template? Or is it just an observation where you threshold some value that defines how many is high and how many is low? \n\nThanks for your question. As defined in Line 105, the **Text Ratio** is computed as:\n**(n_ins + n_thinking) / (n_ins + n_thinking + n_img)**,\nwhich represents the proportion of text tokens in the input sequence before generating the bounding box. A higher text ratio indicates that more textual information is used to generate the grounding box.\n\nTo plot Figure 2 (Right) we first divide the data into three groups based on image size. Within each group, we calculate the Text Ratio for all examples and then split them in half: the 50% with shorter text ratios are categorized as **Low Text Ratio**, and the remaining 50% as **High Text Ratio**. The process is illustrated in the caption of Figure 2.\n\nWe appreciate your suggestion and will include detailed settings of this experiment in the appendix.\n\n-------\n\n> **Q2. Text Ratio Control Explanation:** From my understanding, higher image tokens are obtained with higher resolution tokens only. So, how do you get different text ratios for the same image size? Is this just by controlling the number of generated tokens? Is there some other way of enhancing the image and increasing the image tokens? \n\nThanks for the excellent question. As you pointed out, to control for confounding factors, it is important to analyze different text ratios under the same image size.\n\nWhile it is theoretically possible to manipulate text ratios for the same sample by controlling the length of generated text or altering the number of image tokens, such interventions may negatively impact model performance. Moreover, we currently do not have an effective method to precisely control these factors without introducing other variables.\n\nTherefore, as described in Q1, we group the test samples based on image size. Within each group, images are of similar resolution, allowing us to fairly compare samples with different text ratios.\n\n-------\n\n> **S1. Inclusion of UI-TARS-7B and UGround-v1-7B:** Please include the UI-TARS-7B and UGround-v1-7B baselines in the GUI-specific models section of Table 4 for a better comparison, since you are comparing with other 7B models, and these models are compared in other tables.\n\nThank you for the suggestion. Our model, with only 3B parameters, achieves better overall performance (89.8 on ScreenSpot and 43.8 on ScreenSpot-Pro) than UGround-v1-7B (86.3 on ScreenSpot), UI-TARS-7B-SFT (91.6 on ScreenSpot and 35.7 on ScreenSpot-Pro), and UI-TARS-7B-DPO (89.5 on ScreenSpot and 35.7 on ScreenSpot-Pro). We will include the results of UI-TARS-7B and UGround-v1-7B in both Table 4 and Table 5 for a more complete comparison.\n\n-------\n\n> **S2. Typos:** Line 198 should say Figure 6.\n\nThanks for your suggestion. We have revised the typo of reference.\n\n-------\n\nWe hope the above response can fully address your concerns. We really appreciate your insightful and constructive comments to further help us improve the quality of our work. Please do not hesitate to let us know if you have any further questions. Thanks again!\n\n[1] UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning. arXiv."}, {"review_id": "pu5c4gq6r7", "reviewer": "Reviewer_wn5i", "summary": "This paper performs a systematic study on the impact of using R1-zero-like training for GUI agents along 3 axes: input design, output evaluation, and policy update. The authors note that the current R1-zero paradigm causes performance degradation for GUI agents. They propose solutions along each axis tailored to GUI agents and achieve state-of-the-art performance in the same size model family and a larger UI-TARS-7B on the ScreenSpot and ScreenSpot-Pro datasets. This paper provides useful insight for the GRPO training design of GUI agents.", "questions": "1) Do the authors plan to release their checkpoint? This model could be very useful for the research community to use and build on top of. \n\n2) How does the proposed method perform when using different base model families? How does this method scale with model size?\n\n3) Would it be possible to perform multiple runs and report mean/std?\n\n4) Could there be any potential downstream effects on GUI control tasks when using the proposed training paradigm (e.g., degradation in downstream web navigation performance due to training to limit reasoning)?", "limitations": "Yes to negative societal impact. \nNo to limitations regarding statistical significance testing and base model ablations (please see the questions section).", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Thank you for sharing your work. Below are some strengths/weaknesses points. The main point that could be very useful to address is studying performance on different base models (please see the questions section). \n\nStrengths:\n-    Significance & Originality: This paper points to interesting insights regarding training GUI agents with GRPO. To the best of my knowledge, this is the first paper to do so. \n-    Clarity & quality: The paper is well organized and the writing is clear. Figures/tables are clear and helpful.\n\nWeaknesses:\n-    Clarity & quality:  \n      -    The insights highlighted in the paper are only evaluated on one base model. Thus, it is unclear whether the results found are model-specific or general. \n       -    It is generally important to evaluate RL methods over several runs due to high variance.", "was_revised": true, "final_justification": "Authors addressed most of my concerns. Hence I have reflected that in my updated scores.", "rebuttal": "We sincerely thank the reviewer for the constructive comments and suggestions, which are very helpful for improving our paper. We are also grateful that you recognized the strengths of our paper. Please kindly find point-to-point responses below.\n\n-------\n\n> **Q1. Model Checkpoint Availability:** Do the authors plan to release their checkpoint? This model could be very useful for the research community to use and build on top of.\n\nThanks for the valuable suggestion. We agree that open-sourcing the model is important for supporting the research community. Due to previous company policies, we are unable to release the original checkpoints and data used in this paper.\n\nHowever, during the rebuttal period, we re-trained the model (backbone: Qwen2.5-vl-3B-Instruct) using our proposed method on the dataset from UI-R1 on an external server (4×L20 GPUs), and achieve performance exceeding that reported in the original paper (will be presented in following response). This new checkpoint, training codes, and training datsets have all been released on HuggingFace. However, due to the rebuttal policy that prohibits including any form of URL links in the response, we are unable to provide the direct link here. We will ensure that all relevant materials remain publicly accessible for better reproducibility.\n\nIn addition, we find that using a smaller `gradient_accumulation_steps=1` (compared to 32 in the main table) can further improve performance. Based on this finding, we plan to continue training on larger datasets such as GUI-R1-3K and the public datasets used in the paper, and will release those checkpoints as well for supporting future research. Thanks again for the suggestion.\n\n-------\n\n> **Q2 & W1. Cross-Family and Scale Evaluation:** How does the proposed method perform when using different base model families? How does this method scale with model size? **&** The insights highlighted in the paper are only evaluated on one base model. Thus, it is unclear whether the results found are model-specific or general. \n\nThank you for the helpful suggestion. We agree that testing across different model families and sizes can better demonstrate the generality of our method. To this end, we conducted experiments on two additional models: **InternVL3-2B-Instruct** (a different mainstream family and smaller size) and **Qwen2.5-VL-7B-Instruct** (a larger size). We also retrain Qwen2.5-VL-3B-Instruct using our method for comparison.\n\nSince UI-R1 was originally trained on Qwen2.5-VL-3B, we follow the same training protocol from the UI-R1 paper and retrain Qwen2.5-VL-7B-Instruct and InternVL3-2B-Instruct on the UI-R1 dataset. Due to resource constraints, we used LoRA for training the 7B model, while other models were fully fine-tuned. For consistency and fair comparison with UI-R1-3B, we used the same training data and training settings.\n\n**Performance on ScreenSpot**\n\n|Model|Mobile|Desktop|Web|Avg.|\n|-|:-:|:-:|:-:|:-:|\n|Qwen2.5-VL-3B|||||\n|basemodel|85.6|70.4|65.9|74.8|\n|UI-R1||||85.4|\n|GUI-G1|96.2|87.4|84.3|**89.8**|\n|—|———|———|———|———|\n|Qwen2.5-VL-7B|||||\n|basemodel|88.2|80.5|80.1|83.4|\n|UI-R1(re-train)|80.6|78.7|84.2|81.4|\n|GUI-G1|88.8|82.3|80.8|**84.4**|\n|—|———|———|———|———|\n|InternVL3-2B-Instruct|||||\n|basemodel|0.2|13.2|0.2|3.6|\n|UI-R1(re-train)|0.4|14.4|0.5|4.1|\n|GUI-G1|0.8|16.8|0.2|**4.8**|\n\n\n**Performance on ScreenSpot-Pro**\n\n|Model|CAD|Development|Creative|Scientific|Office|OS|Avg.|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Qwen2.5-VL-3B||||||||\n|basemodel|28.4|22.1|26.7|36.6|33.0|22.4|28.1|\n|UI-R1|||||||17.8|\n|GUI-G1|44.30|39.40|39.10|49.90|56.10|35.50|**43.8**|\n|—|———|———|———|———|———|———|———|\n|Qwen2.5-VL-7B||||||||\n|basemodel|21.1|27.4|27.0|38.2|46.1|30.1|31.1|\n|UI-R1 (re-train)|22.60|27.80|29.60|37.80|50.90|39.30|**33.70**|\n|GUI-G1|20.70|28.80|28.70|40.60|45.20|36.20|32.60|\n|—|———|———|———|———|———|———|———|\n|InternVL3-2B-Instruct||||||||\n|basemodel|3.10|0.30|1.20|1.60|0.40|0.50|1.20|\n|UI-R1 (re-train)|7.30|0.00|1.50|5.90|1.70|1.00|2.80|\n|GUI-G1|6.90|1.30|1.50|5.50|1.70|0.50|**2.90**|\n\n**Key Findings:**\n\n- 1. **Model families:** Our method consistently improves performance across both Qwen and InternVL families, despite InternVL showing generally lower baseline performance. This aligns with recent trends, where leading GUI agents [1–3] predominantly adopt Qwen as the base model.\n- 2. **Model size**: Interestingly, scaling from 3B to 7B resulted in performance drops. We attribute this to two factors: (a) the 7B model was trained with LoRA instead of full fine-tuning, and (b) larger models may not benefit as much from task-specific grounding fine-tuning, as also observed in Table 1 of [2]. Moreover, UI-TARS-72B performs worse on ScreenSpot (88.4) than the smaller UI-TARS-7B (89.5). Our experiments on the Qwen series further support this observation, suggesting that grounding performance does not necessarily scale with model size.\n- 3. Across all models and settings (except Qwen2.5-vl-7B on ScreenSpot), our method consistently outperforms the UI-R1 approach, further validating its effectiveness.\n\nThank you again for the suggestion. It helps us better assess the robustness and scalability of our method, and we will include these results in the final version.\n\n-------\n\n> **Q3 & W2. Multiple Runs and Statistical Consistency:** Would it be possible to perform multiple runs and report mean/std?  **&** It is generally important to evaluate RL methods over several runs due to high variance.\n\nThank you for your suggestion. We have conducted three independent runs of our method (GUI-G1) on the UI-R1 dataset and settings, using random seeds 2023, 2024, and 2025. For evaluation, we used the ScreenSpot and ScreenSpot-Pro test sets respectively. The results are **highly consistent**, with **small variance** across runs.\n\n* On **ScreenSpot**, the average performance from the three runs are:\n  **0.43896**, **0.43833**, **0.43580**\n  → **Mean = 0.4377**, **Standard deviation = 0.0017**,\n  **95% confidence interval = \\[0.4344, 0.4410]**\n\n* On **ScreenSpot-Pro**, the average performance from the three runs are:\n  **0.89780**, **0.90094**, **0.89544**\n  → **Mean = 0.8987**, **Standard deviation = 0.0028**,\n  **95% confidence interval = \\[0.8938, 0.9035]**\n\nWe have updated the paper to include these statistics. Thank you again for your helpful feedback.\n\n-------\n\n> **Q4. Downstream GUI Task Impact:** Could there be any potential downstream effects on GUI control tasks when using the proposed training paradigm (e.g., degradation in downstream web navigation performance due to training to limit reasoning)?\n\nThank you for your helpful question. There are two types of GUI Agents in current systems:\n\n- 1. Agents that output both **action types** and **grounding coordinates.** These can complete tasks alone.\n- 2. Agents that split **planning** and **grounding.** The planner handles reasoning, and the grounding model only needs to find the correct UI element.\n\nOur model belongs to the second design. It does not perform reasoning and this does not hurt downstream performance. Instead, better grounding improves the agent’s success rate.\n\nWe follow the Table 4 setup from [4] and test on the Cross_Web split of Multimodal-Mind2Web. The planner was GPT-4o. The results are:\n\n|GroundingModel|element_acc|\n|:-:|:-:|\n|Qwen2.5-vl-3B-Instruct|45.82|\n|UI-R1-3B (w/ reasoning)|43.54|\n|GUI-G1-3B (w/o reasoning)|**46.04**|\n\nAs shown, GUI-G1 achieves the best performance, even without reasoning ability. This suggests that strong grounding is more important than reasoning in this setup.\n\n-------\n\nWe hope the above response can fully address your concerns. We really appreciate your insightful and constructive comments to further help us improve the quality of our work. Please do not hesitate to let us know if you have any further questions. Thanks again!\n\n[1] UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning. arXiv.\n\n[2] Gui-r1: A generalist r1-style vision-language action model for gui agents. arXiv.\n\n[3] Infigui-r1: Advancing multimodal gui agents from reactive actors to deliberative reasoners. arXiv.\n\n[4] Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. ICLR 2025 Oral."}, {"review_id": "GLRC1jaBsc", "reviewer": "Reviewer_c2Uh", "summary": "This paper studies methods for training\tagents to navigate Graphical User Interfaces (GUIs) using R1-style post-training using RL. They\tinvestigate 3 different\tdesign components: the effect of input templates, effect and limitations of reward functions used for GUI environments, and the policy objective. They propose several improvements to these\tcomponents based on GUI\tenvironments:\n- whereas templates which encourage long chains\tof though (CoT)\thave proven effective in reasoning-based tasks (like math and coding), here they find that they\tare detrimental\tfor more vision-based tasks like localizing objects in a screenshot, and they find that removing intructions\tencouraging CoTs improves performance for object grounding in GUIs.\n- they find that existing reward functions for object grounding in GUIs, such as hit rate or IOU, lead to reward hacking behaviors. Based on this observation, they propose a different\treward function\twhich is not prone to these types of reward hacks.\n- they find that the GRPO objective produces two biases for GUI-based tasks: 1) biased lengths due to a term dividing by the output length 2) a bias based on difficulty due to the division by standard deviation of rewards. They propose two modifications, where the division by\toutput length is replaced by a constant\tand the\tobjective is weighted by a proxy for question difficulty, which\there is\tbased on the bounding box size.\n\nWith these 3 modifications, the system is trained on GUI-based benchmarks, where it leads to improvements over previous\tmethods.\n\n\nFirst, I should mention that GUI-based agents are not my research area, and I am not very familiar with the main methods, benchmarks and prior work other, hence my review and recommendation should be weighted accordingly. \nOverall, this paper seems very specific to GUI-based environments: the template, reward functions and modifications to the GRPO\tobjective are motivated by this class of problems and failure modes of previous\tmethods on them.\nWhile the improvements are reasonable, it's unclear to me that these insights would transfer to other domains or have broader applicability (the paper does not claim or present any evidence of this). The paper seems quite application-focused and I think it might be better suited to a venue more focused on this area, rather than NeurIPS which (at least historically) tends to be more\tfocused on general ML methods. For this reason, I'm recommending reject.", "questions": "The inputs and output spaces described in lines 102-104 could be clearer. \n\n- For example, it wasn't clear to me what exactly the action space of the agent is: does it jointly produce text and a bounding box? are there separate heads for each? Is it all in tokens produced autoregressively? Are other keys part of the action space? \n\n- What exactly is the input - I understand it's an image, is it just RGB pixels? Or are there other post-processing steps applied.", "limitations": "Yes.", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n\n- The ablations and analyses appear carefully done, with the claims backed up by results and visualizations showing the described behavior and failure modes.\n- The empirical results\tfor the\tfinal system appear to improve noticeably over the baselines, and many different methods are included in the comparison.\n\n**Weaknesses**:\n\nAs mentioned above, the main weakness in my opinion is that the insights do not transfer much outside the area of GUI-based agents. They are quite specific to the modalities which the\tcurrent\tsystem uses (for example, bounding boxes over areas of the screen, reward functions based on\tbounding boxes and object grounding on the screen, etc). So, other than\tresearchers specifically working in this area, the paper may not be that relevant.", "was_revised": true, "final_justification": "One of my main concerns was that the scope of the paper is overly specific to GUI-based environments for NeurIPS (as noted in my review, this paper is outside my research area). However, after reading the other reviews and some of the references shared by the authors, I see that this work may be within scope for NeurIPS. Therefore, I am raising my score from 3 to 4. My confidence is still at 1 however, so I largely defer to other reviewers who have more expertise in this area.", "rebuttal": "We sincerely thank the reviewer for the constructive comments and suggestions, which are very helpful for improving our paper. We are also grateful that you recognized the strengths of our paper. Please kindly find point-to-point responses below.\n\n-------\n\n> **Weakness. Broader Impact Beyond GUI:** As mentioned above, the main weakness in my opinion is that the insights do not transfer much outside the area of GUI-based agents. They are quite specific to the modalities which the current system uses (for example, bounding boxes over areas of the screen, reward functions based on bounding boxes and object grounding on the screen, etc). So, other than researchers specifically working in this area, the paper may not be that relevant.\n\nThanks for your detailed comments. While our work is evaluated in the context of GUI agents, we believe its contributions are broadly relevant to the ML community. In fact, many recent works on GUI agents have been published at major ML venues [1–5], reflecting growing interest in this area within the community.\n\nIn particular, our methods offer insights that extend beyond GUI environments:\n- **Fast vs. Slow Thinking:** We introduce the **text ratio metric** to guide when a model should perform explicit reasoning. This principle can apply to a wide range of **multimodal models**, not just those for GUI tasks.\n- **Reward Design and Hacking:** Grounding tasks using bounding boxes are common in MLLM such as Qwen2.5-VL [6], VLM-R1 [7] and Visual-RFT [8], not just in GUI-based environments. Although our experiments focus on GUI agents, the reward design challenges we study and the methods we propose are applicable to these general grounding tasks as well. Our analysis provides insights for improving reward functions in common visual tasks and, as noted by **Reviewer jR9M**, also offers guidance for future work on GUI agents.\n- **Difficulty-Aware Learning:** Our use of difficulty coefficients in GRPO training is based on the **relative size of the target object.** This is a practical design that can be easily applied to other multimodal tasks where the difficulty of the target can be estimated.\n\nAlthough our experiments are focused on GUI environments, the core methods are applicable in broader multimodal and RL settings. We appreciate the reviewer’s perspective and would be happy to further elaborate on the borad impact of our work in the updated version.\n\n-------\n\n> **Q1. Model Output Action Space Details:** For example, it wasn't clear to me what exactly the action space of the agent is: does it jointly produce text and a bounding box? are there separate heads for each? Is it all in tokens produced autoregressively? Are other keys part of the action space?\n\nThanks for your question about the action space of our model. Our action space consists solely of **text tokens.** The bounding box information is represented as part of the text output, generated autoregressively by the model. There are no other elements in the action space.\n\nFor example, the model might output a JSON-like text sequence such as:\n\n```json\n[\n    {\"bbox_2d\": [1726, 226, 1787, 264], \"label\": \"locked rotation\"}\n]\n```\n\nIn this example, the bounding box coordinates `[1726, 226, 1787, 264]` are embedded within the text tokens generated by the model. This is consistent with the approach used in Qwen2.5-VL.\n\nWe will include a detailed example of the model’s input and output formats in the appendix. Thank you for your helpful question.\n\n-------\n\n> **Q2. Model Input Description:** What exactly is the input - I understand it's an image, is it just RGB pixels? Or are there other post-processing steps applied.\n\nThanks for your question about the model input. Our model takes two parts as input:\n\n- **Text input**, which includes the instruction template and the user question. For example:\n   *\"Grounding instruction is: {user question}. Help to locate and output its bbox coordinates using JSON format.\"*\n\n- **Image input**, which is provided as raw RGB pixels. The image is passed into an image encoder that processes the pixels into a sequence of tokens. These image tokens are then combined with the text tokens and fed jointly into the language model for forward computation and text generation.\n\nWe hope this clarifies the input processing of our model. Please let us know if you would like more details.\n\n-------\n\nWe hope the above response can fully address your concerns. We really appreciate your insightful and constructive comments to further help us improve the quality of our work. Please do not hesitate to let us know if you have any further questions. Thanks again!\n\n[1] Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration. NeurIPS 2024.\n\n[2] DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents. ICLR 2025.\n\n[3] Lightweight Neural App Control. ICLR 2025.\n\n[4] Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. ICLR 2025 Oral.\n\n[5] OS-ATLAS: A Foundation Action Model for Generalist GUI Agents. ICLR 2025.\n\n[6] Qwen2.5-VL Technical Report. Arxiv.\n\n[7] VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model. Arxiv.\n\n[8] Visual-RFT: Visual Reinforcement Fine-Tuning. Arxiv."}]

Author rebuttal:

Meta-review: This paper investigates R1-Zero-like training for GUI grounding tasks, identifying key limitations in current approaches and proposing targeted solutions. Through careful analysis of input templates, reward functions, and RL policy objectives, the authors show how existing practices can lead to degraded performance, reward hacking, or overfitting. They introduce a fast-thinking template to reduce unnecessary reasoning, a box-size-aware reward function, and a difficulty-aware objective, leading to a new model (GUI-G1) that achieves state-of-the-art results on ScreenSpot and ScreenSpot-Pro benchmarks, surpassing even larger models.

In general, the paper is well-written, timely, and provides valuable empirical insights into the failure modes of GUI grounding pipelines. Reviewers consistently praised the thorough analysis, strong empirical results, and clear presentation, and noted that the proposed solutions are both effective and well-motivated. However, as pointed out by the reviewers, there are some concerns about the scope of applicability beyond GUI agents, as well as requests for additional clarity and expanded evaluations across different model families and training runs. These concerns were largely addressed in the rebuttal through additional experiments, clarifications, and commitments to release code and checkpoints.

Overall, the paper makes a solid and impactful contribution to the understanding and improvement of GUI agent training. Given the novelty of the analysis, the effectiveness of the proposed methods, and the strong empirical performance, I recommend accepting the paper.
Decision: Accept (poster)