ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
Authors: Hanyu Lai
Xiao Liu
Yanxiao Zhao
Han Xu
Hanchen Zhang
Bohao Jing
Yanyu Ren
Shuntian Yao
Yuxiao Dong
Jie Tang
Venue: ICLR 2026 Poster (2026)
Keywords: large language model
computer use agent
reinforcement learning
Primary area: applications to robotics, autonomy, planning

Abstract: We introduce ComputerRL, a framework for autonomous desktop intelligence that enables agents to operate complex digital workspaces skillfully. ComputerRL features the API-GUI paradigm, which unifies programmatic API calls and direct GUI interaction to address the inherent mismatch between machine agents and human-centric desktop environments. Scaling end-to-end RL training is crucial for improvement and generalization across diverse desktop tasks; however, it remains challenging due to environmental inefficiency and instability during extended training. To support scalable and robust training, we develop a distributed RL infrastructure capable of orchestrating thousands of parallel virtual desktop environments to accelerate large-scale online RL. Furthermore, we propose Entropulse, a training strategy that alternates reinforcement learning with supervised fine-tuning, effectively mitigating entropy collapse during extended training runs. We employ ComputerRL on open models GLM-4-9B-0414 and GLM-4.1V-9B-Thinking, and evaluate them on the OSWorld benchmark. The GLM-ComputerRL-9B achieves a new state-of-the-art accuracy of 48.9%, demonstrating significant improvements for general agents in desktop automation. Our code is available at https://github.com/THUDM/ComputerRL.

Published as a conference paper at ICLR 2026
COMPUTERRL: SCALING END-TO-END ONLINE REIN- FORCEMENT LEARNING FOR COMPUTER USE AGENTS
Hanyu Lai
Xiao Liu Liu Yanxiao Zhao Han Xu², Hanchen Zhang' Bohao Jing"
Yanyu Ren Mt Shuntian Yao",, Yuxiao Dong's Jie Tang'
Tsinghua University
Z.AI
Uniiersity of Chinese Academy of Sciences
ABSTRACT
We introduce COMPUTERRI,, framework for autonomous desktop intelligence that enables agents to operate complex digital workspaces skillfully. COMPUT- ERRL features the API-GUI paradigm, which unifies programmatic API calls and direct GUI interaction to address the inherent mismatch between machine agents and human-centric desktop environments. Scaling end-to-end RL training is crucial for improvement and generalization across diverse desktop tasks; however, it re- mains challenging due to environmental inefficiency and instability during extended training. To support scalable and robust training, we develop a distributed RL infras- tructure capable of orchestrating thousands of parallel virtual desktop environments to accelerate large-scale online RL. Furthermore, we propose Entropulse, train- ing strategy that alternates reinforcement learning with supervised fine-tuning, effectively mitigating entropy collapse during extended training runs. We employ COMPUTERRL on open models GLM-4-9B-OKE 90-0414 and and evaluate them on the OSWorld benchmark. The W..M-COMPUTERRL... achieves new state-of-the-arr accuracy of 48.9% demonstrating significant im- provements for general agents in desktop automation. Our code is available at https://github. Sttpabbocoom/HLDMM//omuueeeere
55
48.9%
0.55
50-
Baseline
Success Rabe (%)
45
41.4%
42.5%
42.9%
Entropulse (Ours)
RL
Reward
0.45
30.7%
0.40
SFT
20
LLM-Computerfi
Claude 4.0
(Gemini-2.. S.Pro)
Agent 52
UATARS-1.5
Openti CUA
98
0.30
120 1802210 Step
270 300 330
(a) The success rates of agents on OSWorld.
(b) COMPUTERRL training reward curves (95% Cls).
Figure 1: COMPUTERRL enables efficient end-to-end online policy optimization for OS agents. (a) On OSWorld (Xie et al. 2024). CLM-COMPUTERRL, trained with COMPUTERRL. outperforms state-of-the-arr agents. (b) Our Entropulse approach yields higher average training rewards and
improves both learning efficiency and final performance over conventional methods.
INTRODUCTION
Large Language Models (LLMs) (Achiam et al. 2023 Touvron et al. 2023b Zeng et al. 2022 GLM
al. 2024
Team al. 2023 Guo al. 2025 Bai 2023a
Yang al.
2025) have dramatically
HI. XI. and YZ. contributed equally.
Work partially done while these authors interned 7.AL. Corresponding author: JT

---

Published as a conference paper at ICLR 2026
expanded the scope and depth of artificial intelligence capabilities, driving profound re-examination of our understanding of machine intelligence. Among all scenarios, the emergence of LLM- based
GUI (graphical user interface) agents, capable of independently perceiving, reasoning. and executing complex tasks on user devices, has aroused particular interest from researchers (Xi et al., 2023
Wang et al. 2023 Liu al.) (023). Given that desktops remain central to intelligence-intensivee
tasks, developing computer use agents is crucial for fundamentally transforming human-computer
interactions and elevating AI capabilities (Agashe al. 2025 Wu el al.
2024a).
Despite previous attempts to develop computer use agents (Agashe et al. 2025
Lei et al. 2024 Xie
al. 2025). enabling them to operate autonomously over extended periods in real-world scenarios remains a significant challenge. The first primary obstacle arises from the fact that GUIs are inherently
designed for human interaction, making the simulation of human actions by GUT agents (Liu et al..
2024b OpenAl, (025) Qin et al.) (0255 non-trivial and cumbersome endeavor. Second, current mainstream approaches of behavior cloning (BC) (Bain & Sammut 1995; Bratko et al.) 1995).
including manual annotation (He et al.] 2025) and model distillation (Sun et al. 2024 Xu et al. 2024),
are limited in scalability and effectiveness. Manual annotation, while precise, is prohibitively labor- intensive for complex tasks. Model distillation, on the other hand, is constrained by the performance of the teacher models, limiting overall capability. Both methods typically exhibit poor generalization and limited error recovery abilities. Finally, although reinforcement learning (RL) has shown potential for desktop automation tasks (Lu et al.] 2025 Feng et al., (202)) its practical application remains restricted due to computational complexity and methodological challenges. Complex environments. slow convergence, and known inefficiencies in RL training (Xie et al. [2024] Bonatti et al., [2024] Yu
al. 2025 Fu et al. 2025) severely limit its large-scale adoption in training computer use agents.
In this work, we propose COMPUTERRL, an end-to-end algorithmic framework designed to advance desktop-level planning, reasoning, and device operation. This framework includes a new API-GUI interaction paradigm, a scalable RL training infrastructure for computer environments, and an RL algorithm for extended effective training. First, we introduce API-GUI, a large-scale, automatically constructed API ecosystem that enables the agent to transcend the inherent biases of human-oriented operational paradigms. It instead leverages a more machine-orienter approach for device interaction, which combines API calls and GUI actions, thereby significantly enhancing both the versatility and overall performance of the agent. Second, we develop a distributed training infrastructure utilizing virtual machine clusters based Docker and gRPC protocols for scalability, which is fully compatible with AgentBBnnch (Liu et al., (023). This infrastructure supports thousands of parallel environments, ensuring high scalability and consistent interactions across all environments. Additionally, we integrate the training infrastructure with the AgentRL. framework (Zhang et al. 2025) to facilitate efficient asynchronous training. thereby accelerating the training process. Finally. to counteract stagnation and convergence issues in RL training-specifiicinn KL divergence- we propose Entropulse, which alternates between RL and SFT phases periodically. This approach maintains exploratory capacity and ensures continuous performance gains (Figure lb).
entropy collapse and rising
As a result, by harnessing end-to mnd-to-end RL and optimization in the desktop environment, COMPUTERRL has achieved remarkable improvement in understanding and operating GUTs. Evaluation on the significant improvements (see
OSWorld benchmark (Xie et al., 2024) shows COMPUTERRI. Figure aa) in computer use challenges, achieving a success rate of 48.9% (with 66% performance gain from RL), outperforming other state-of-the-ar models including OpenAl CUA o3 (42.9%), UI-TARS- (42.5%), and Anthropic Claude Sonnet (30.7%).
In summary, our contributions are as follows:
We propose a new interaction paradigm, a shift from human-centric to machine-oriented interaction by introducing large-scale, automatically constructed API ecosystem integrated with conven- tional GUI operations. This approach addresses the inherent mismatch between human-designed interfaces and artificial agent capabilities, while achieving superior operational efficiency and generalization performance on computer- based tasks.
We establish large-scale. distributed RL infrastructure for computer use agents by reconstructing virtual machine clusters, achieving unprecedented scalability with thousands of parallel environ- ments and seamless AgentBench compatibility, thereby overcoming the critical bottleneck that has limited RL-based computer use agent training to scale experiments and enabling breakthrough results in large-scee scale agent training.
2

---

Published as a conference paper at ICLR 2026
Rollout
Thought
Data
1000+ Q
Pirst, I should use Chrone to surf the web.
Queue
Environment
Traj
Actions
GUI
APIs
Traj
Click
Highlight
Type
Compare
Controller
Trai 4
Rollout Engine
Scroll
Play
Traj 5
Online
Tk+1
Actor Update
Sync Params.
Actor Update
Sync Params.
Update
Figure 2: Overview of COMPUTERRL framework. We introduce an API-GUI action paradigm that seamlessly integrates automatically constructed APIs with GUI actions improve agent efficiency and effectiveness. A large-scale parallel desktop environment with 1,000+ real-world instances, combined with asynchronous RL framework, enables efficient sampling and robust agent training.
We introduce Entropulse, novel training methodology that systematically addresses the challenges of entropy collapse and KL divergence accumulation in extended RL training through strategic alternation between RL and SFT phases, enabling sustained performance improvements and achieving ootte-of-the-ar performance computer automation.
2 THE COMPUTERRL FRAMEWORK
The human-oriented design of GUIs hinders agent efficiency, while limited environment scalability restricts large-scale training. This section presents the COMPUTERRL framework (see Figure 2 which features an API-GUI paradigm that integrates human-like GUI interactions with efficient API invocation. Additionally, develop scalable Ubuntu desktop environment for parallelism and utilize a fully asynchronous RL framework for efficient training.
2.1 GENERAL API-GUI PARADIGM
Existing GUI agents face challenges due their reliance on human-like interactions, while API- based control offers efficiency but introduces implementation complexity and security restrictions. To address these issues, we propose an API-GUI paradigm that unifies both action spaces, enabling agents to leverage API efficiency while retaining GUI versatility.
We develop an LL-M-based automated workflow for application API development (Yang al. 2024a Wang et al. 2024). significantly lowering the barrier for API creation. Users provide exemplar tasks, and our system autonomously generates API code and test cases through three stages:
Requirement Analysis: Users provide task examples for the target application. Our LLM analyzes these instances, extracts essential functionalities, and compares against existing API in- terfaces to identify gaps. New interfaces are automatically generated for uncovered functionalities, with a focus on general-purposs functions to minimize complexity and enhance usability.
API Implementation: The workflow iterates over each interface definition, implementing API functionalities using designated Python libraries. Error-handling mechanisms and logging implemented for debugging and maintenance purposes.
Test Case Generation: Similar to Li & Yuan (2024), verify API correctness by checking: (1) runtime error-free free invocation and (2) correct results across parameter inputs; failed APIs receive error feedback for autonomous correction.
This methodology enables the creation of application We have developed API sets for multiple Ubuntu applications and validated their effectiveness through experiments. Detailed API development workflow is provided in Appendix A The agent action space and prompt formulation detailed in Appendix and
APIs with minimal human intervention.

---

Published as a conference paper at ICLR 2026
2.2 STABLE UBUNTU ENVIRONMENT FOR LARGE- LARGE-SCALE PARALLELISM
A stable and scalable Ubuntu environment is essential for constructing behavior cloning data and large-scale RL training. Building on OSWorld (Xie al. 2024), we identify key limitations:
Resource Intensiveness and Stability: VMs are CPU-intensive and unstable under high concur- rency, causing performance degradation and system freezes.
Network Bottlenecks: Heavy workloads cause network overhead, connection failures, and IP address loss, hindering agent interaction and logging.
Lack of Native Distributed Support: OSWorld lacks multi-node clustering support, preventing efficient distributed deployment.
To address these limitations, we build robust and parallelizable OSWorld infrastructure (see Figure with the following innovations:
Standardized, Decoupled Interface: We refactor the environment via AgentBench API, provid- ing a unified interface that decouples environment execution from the computational back-end and enables flexible resource management.
Lightweight VM Deployment: Using gemu-in-dockere) we deploy containerized Ubuntu VMs with streamlined images that reduce network issues and optimize resource usage, significantly lowering per-instance CPU consumption.
We employ gRPC-based communication to link CPU nodes into a distributed cluster with centralized resource allocation and orchestration.
Distributed Multi-Node Clustering:
Web-based Visualization and Monitoring: A web interface provides real-time visualization of environment statuses, agent states, and resource allocation, improving usability and debugging capabilities.
Through these technical improvements, our system supports deployment of several thousands of concurrent environments on multi-node CPU cluster, validated by extensive empirical evaluation. Results confirm our platform's 's superior stability, resource efficiency, and scalability, making an enabling infrastructure for large-scale RI. and agent-based research.
2.3 FULL-ASYNCHRONOUS RL FRAMEWORK FOR EFFICIENT TRAINING
Existing RL frameworks rely on synchronous training paradigms, where rollout collection and parameter updates are alternated, resulting in training inefficiencies. To address the limitation, the AgentRL framework (Zhang et al. 2025) for fully asynchronous RL training with the following designs:
Resource Partitioning: Data collection runs on dedicated resources while the trainer streams
data from the replay engine, preventing mutual blocking.
Dynamic Batch Sizing: The trainer processes incoming data with flexible batch sizes, reducing idle time and improving efficiency.
Modular Component Isolation:
Actor, reference, and critic modules run independently with dedicated resources. We utilize PyTorh distributed groups and NCCL for efficient parameter sharing.
Off-policy Bias Mitigation: We limit the replay buffer size and sync trajectories after each update, ensuring trajectories remain close to the latest policy.
Through a stable, high-concurrency desktop environment and the decoupling of training from rollout, we markedly enhance the efficiency of sampling and RL training. Our system achieves a high average power consumption per GPU, reflecting optimal resource utilization. This design supports scalable, high-throughput RI training by enabling dynamic workload balancing, resulting in a significant improvement hardware efficiency and overall training throughput.
THE COMPUTERRL TRAINING
In Section 2
we establish a robust foundation for large-scale agent training. However, scaling end-to-end training still faces challenges in initializing capable base policy and entropy collapse
4

---

Published as a conference paper at ICLR 2026
Step 1:
Step 2: First RL Phase & Rollout data Collection
Step 3:
Behavior Cloning for Cold
Entropulse and Second RL Phase
Start
Tasks w/ eval func.
Rollout Data
Initial Samphing
RIVR:
from All Policy
Performance Entropy & KL ?
Entropulse:
100%
Performance
Init SET Model
Entropy
For Eachh Step
KL
Model Pool
86 O
Rollout Data from All Policy
Model w/ High Exploration
RLVR:
Filtering
Performance ?
Entropy &
Correct Steps
SFT
Data for Entropulsc
Final Agent Model
KL
Figure 3: Overview of COMPUTERRL, which includes three stages: (1) BC cold start with trajectories collected from general LLMs; (2) RL with step-level GRPO using verifiable, rule-based rewards; (3) Entropuise, which alternates RL with SFT correct rollouts to restore entropy and sustain learning.
during RL. This section details our scalable COMPUTERRL training approach and its algorithmic innovations for extended training in desktop environments.
3.1 BEHAVIOR CLONING SETUP
To perform cold start for our model, we employ BC as the initial stage of training. By imitating user interactions, BC enables agents to acquire foundational competencies, thereby facilitating rapid adaptation to computer operations and tasks.
Trajectory Collection with Multiple LLMs. We manually collect extensive tasks with corresponding evaluation functions (see Appendix and augment construct an 8k-task dataset. However, the large- scale collection of high-quality trajectories remains challenging. Manual annotation is prohibitively expensive, and relying on a single model for trajectory generation results in limited and homogeneous data distribution constrained by that model's capabilities. To address these limitations, we leverage the complementary strengths of multiple advanced models to collect diverse and high-quality set of interaction trajectories. Concretely, our data pipeline consists of three key stages:
Initial Sampling: For each task, we utilize closed-source LLMs to sample several trajectories per task independently. We record both the complete interaction trajectories and the outputs produced by the respective evaluation functions. This procedure yields a rich set of diverse trajectories that as the foundation for subsequent data augmentation and model adaptation.
Outcome Stratification: Following initial data collection, we perform a stratified analysis of task outcomes by categorizing all tasks into three groups based on achieved accuracy: Fully Solved (ace = 100%), Partially Solved (0 < acc < 100%), and Unsolved (ace 0%).
Maik-Orrentee Augmentation with Stratified Sampling: For partially solved tasks, we conduct SFT our backbone model using the initial trajectories input. The fine-tuned model is then used to sample additional trajectories for each task, thereby substantially expanding the coverage and quality of trajectories for tasks where model proficiency was previously limited.
For tasks classified as unsolved, we build a model pool of high-performing models and randomly select to determine each action. This approach leverages inter-model variance at the task level. as different models exhibit distinct areas of expertise despite comparable aggregate performance, enabling trajectory generation that is unattainable by any single model.
We systematically aggregate and filter the collected interaction data, retaining only successful trajectories (180k+ correct steps), and employ them for supervised fine-tuning of the model. This strategy equips the model with robust desktop manipulation capabilities and foundational reasoning abilities, significantly enhancing the performance of the base model.
5

---

Published as a conference paper at ICLR 2026
3.2 REINFORCEMENT LEARNING WITH VERIFIABLE REWARDS
Step-Level Group Relative Policy Optimization. We extend the GRPO algorithm (Shao et al. 2024) to the step-level, making it more suitable for agent RL training. For each task the policy me interacts with the desktop environment and samples G trajectories TI, .., TG. The i-th trajectory consists of L, step-level actions o,,,, O.L,, All steps from the same task are grouped, and the advantage A is computed for each step. The overall loss aggregates all step advantages as follows:
In
To( 96.))
JStepG (0) = -E
T-P))) ((.s))))
Ei=1 La
min
Thord (or,)
A
1=1
clip To(on,1l1,))
c, (1)+)()))))))
BDx - (
Tomid (ot5)9b.5)
A., Ti. (R) std
R= (ra,e u 1, 1,....,.., G, v 1, Lnd
Reward Design. We select a subset of the constructed human-annotated data (in Section 3.1 for RL and employ rule-based verification function to provide verifiable training signals for each trajectory. Successfully solved trajectories receive a reward of for every correctly formatted action that contributes the solution; failed trajectories or improperly formatted actions receive a reward of O. Unlike conventional approaches that propagate step-wise returns via the Bellman equation, our methodology treats each prompt-response pair as an independent training instance with rewards based on the final trajectory outcome. This direct reward assignment provides explicit feedback by coupling agent behaviors with task success, facilitating effective policy optimization.
3.3 ENTROPULSE FOR SCALING RI. TRAINING
observe that model performance plateaus after hundreds of training steps, with stagnating task completion rates and decreasing entropy. This premature convergence motivates policy exploration. Inspired by DAPO (Yu et al., (025). we experiment with increasing the clipping threshold, which attenuates the decline in entropy but significantly slows down policy improvement.
In the RL training in Section 3.2,
investigate strategies for extending effective training and enhancing
To address the issue, we propose Entropulse, motivated by the observation that SFT and RL objectives differ markedly during training. As entropy decreases during RL optimization, integrating SFT at critical junctures enhances exploration and trajectory diversity, facilitating further policy optimiza- tion. During initial RL training, we aggregate and retain all successful rollout trajectories. While conventionally discarded after single use, these trajectories from various policies at different training steps represent valuable and diverse behavioral data.
We process this dataset by randomly selecting successful trajectories per unique task to construct new SFT training set, which exhibits the following attributes:
1. High quality: All data comprises completed, high-fidelity trajectories.
2. Diversity: Rollouts originate from heterogeneous policies in different training steps, offering variety of problem-solving strategies.
3. Computational efficiency: The dataset leverages existing interaction data, climinating the need for additional environment rollouts.
SFT on this dataset produces notable shifts in policy behavior. While evaluation task performance remains stable, the resulting policy shows increased entropy relative to the original one, indicating enhanced exploration. Building upon this enhanced exploration capability, we conduct second round of RL training, which yields significant performance improvements and enables us achieve sate--oftthe-ane results in computer automation. The training and hardware details are in Appendix
4 EXPERIMENTS
We employ COMPUTERRL on GLM44-9B-0414 GLM et al. 2024 and GL--4,1V-9B- Thinking (Hong al. 2025). to produce CLMM-COMPUTERRI.9 We conduct extensive

---

Published as a conference paper at ICLR 2026
Table I: GLM-COMPUTERRL performance on OSWorld and OSWorld -SWorid-Verifieed (updated in 2025.08)
We compare GLM-COMPUTERRI with tate-of-the-ar agents, including both proprietary and open models.
Agent Model
Paaaas
OSWorld
OSWorld-Vere
Proprietary Models
Aria-UI w/ GPT-4o Yang et al.
2024b
15.2
Aguvis- B2B w/ GPT-40 [uu et al. 2024
17.0
Claude 3.7 Sonnet (Anthropic
2023
28.0
35.8
Claude 4.0 Sonnet (Anthropic 2023
30.7
43.9
Agent S2 w/ Claude-3.-Sonncl Aggahh et al.
2025
34.5
InfantAgent (Lei ct al. 2024 2024
35.3
OpenAT CUA 40 (OpenAT | 2025
38.1
31.3
Agent $2 w/ Gemini-2.5-Pri (Agashe el al. 2025
41.4
45.8
UT-TARS-1.5 (Qin et al. al. 2025
42.5
OpenAl CUA 03 (OpenA) 2025
42.9
Open Models
Owen2.5-v1-720. (Bai
20236
72B
8.8
5.0
PC Agent-E (He 2025
72B
14.9
S-TTAR--7B-SSA
Jm ala 2025
72B
18.8
UI-TARS-72B-DPO [Qm ara
72B
24.6
27.1
UI-TARS- 1.5-BB (Qin el 2025
2025
26.9
27.4
Jedi-7B w/ GPT-40 (Xle al. 2025
7B+
27.0
29.3
UI-TARS-7B-1.5 ARPO (LU 2025
29.9
COMPUTERRL (ours)
w/ GLM-4-99-006 9--444444 1V-LM-4.I-9B--hhnnke.
9B
48 18.10100
47.3
9B
08.9
48.0
periments across various scenarios to evaluate GLMM-COPPUTERRL'S performance within the computer environment.
4.1 MAIN RESULTS
To closely reflect the real user experience, evaluate GLM-COMPUTERRY the OSWorld (Xie et al. 2024) and (SWorld benchmark, comparing its performance against state-of-the-ar models, including CUA (Op2nAA]2025), Claude-4 (Anthropic) 2023). and UI-TARS QQin et 2025 among others. The comparative results are in Table The results indicate that GLM-COMPUTERR achieves superior performance across a range of domains, with its advantages most pronounced the challenging multi-app setting. Moreover, by employing the API-GUI strategy, GLM- COMPUTERRL can accomplish tasks using at most 1/3 of the steps required by the strongest baseline approaches, demonstrating remarkable gains in execution efficiency. These results underscore the potential of COMPUTERRL to advance the state of the art in computer automation across various applications.
4.2 OFFICE APPLICATION PERFORMANCE
As a critical interface for delivering and presenting, office application constitutes an important testbed for evaluating computer use agents. To assess agent performance in this domain, we curate set of 180 challenging tasks from three sources: SpreadsheetBench (Ma et al. 2024). PPTC (Guo et al. 2023 and in ho-h developed Writer domain tasks. These tasks adapted necessary to integrate them into the OSWorld framework. The resulting benchmark, termed World. enables systematic measurement of agent capabilities in office-oriented scenarios. The results are in Table
4.3 ABLATION STUDY
To evaluate the influence of various algorithms and training datasets agent performance, we present an ablation study on the OSWorld benchmark in Table
Framework Ablation. We compare the performance of the GUI-only approach with proposed API-GUI paradigm using GPT-4o. The results demonstrate that the API-GUI paradigm substantially outperforms the GUI-only baseline across all domains. Specifically, the API-GUI strategy achieves
7

---

Published as a conference paper at ICLR 2026
Table GLM-COMPUTERRA performance on OfficeWorld compared to common baselines, We employ the same framework (with tools) and test settings to ensure a fair comparison.
Agent Model
Word
Excel
PPT
Average
DeepSock-V3. (Liu al. 2024a
6.7
35.0
21.7
21.1
DeepScek-RI Guo 1202>
13.3
36.7
18.3
22.8
Claude 3.7 Sohner (Anthropic
2023
15.0
25.0
25.0
21.7
Claude 4.0 Sonnet (Anthropic
202.3
18.3
35.0
20.0
24.4
Gemini-2.5-Pru (Team cl al. 2023
5.0
11.7
20.0
12.2
GPT-4o (Hurst ef al. 2024
18.3
21.7
8.3
16.1
GPT-4.1 (Achiam ct 2023
21.7
25.0
28.3
25.0
OpenAl 03 Jaech et at. 2024
23.3
36.7
41.7
33.9
COMPUTERRL (ours) w/ GLM-4-9B 9B-0414
21.7
58.3
43.3
41.1
w/ GLM-4. V-99-Thinking
30.0
58.3
41.7
43.3
Table 3: Ablation study framework designs and training methods. We categorize OSWorld into
five distinct domains facilitate a granular comparison of different strategies across various domains.
Method
OS
Office
Daily
Professional
Workflow
Avg.
Framework Ablation (w/ GPT-4o)
GUI Only
41.7
6.2
12.3
14.3
7.5
11.2
API-GUI
52.6
27.9
25.7
41.6
10.8
26.2
Training Ablation (w/ Qwen2.5-1453
Untrained
20.8
17.2
19.7
22.9
3.3
15.2
+) Behavior Cloning
54.2
35.0
37.2
45.8
10.8
31.9
+) RL Phase
83.3
46.1
45.1
56.3
16.1
42.0
+) Entropulse
75.0
42.3
50.6
52.1
18.9
41.5
+) RL Phase 2
83.3
46.2
46.7
60.4
27.2
45.8
an average success rate of 26.2%, representing a 134% improvement over the GUI-only approach (11.2%). The most significant gains are observed in the Office (27.9% 6.2%) and Professional (41.6% vs. 14.3%) domains, where API-GUI provides 350% and 191% improvements, respectively. These results validate our core hypothesis that combining API calls with GUI interactions enables more efficient and reliable task execution, particularly for complex professional workflows that benefit from programmatic control.
Training Ablation. We study the progressive impact of different training stages using Qwen2.5-14B Starting from the backbone, Behavior Cloning (BC) establishes a solid foundation with 31.9%. The first RL phase (RL1) yields substantial gains, increasing the performance to 42.0% (+10.1%% Interestingly, Entropulse phase maintains similar performance (41.5%) while significantly increasing action entropy, which enhances exploration diversity and enables the final R1.2 phase to achieve further improvements. The RI.2 phase achieves the best performance at 45.8 45.8% (+3.8% from RI.1), benefiting from the increased exploration capacity introduced by Entropulse. Notably, the Workflow domain shows the most dramatic improvement throughout training (10.8 + 27.2%), while the other domains maintain consistently high performance, highlighting the importance of multi-stage training.
RL Scalability. We present the RL training reward and entropy curves in Figure 4 to study the impact of Entropulse on the extended RL training dynamics. After the first RL phase converges, we compare the second RL phase with and without Entropulse. To ensure a fair comparison, we reset the reference model in both scenarios.
The results demonstrate that incorporating Entropulse increases the model's entropy. thereby restoring its exploratory capacity. This enhanced exploration substantially scales the effective training steps, ultimately leading to improved overall performance.
4.4 CASE STUDY AND ERROR ANALYSIS
8

---

Published as a conference paper at ICLR 2026
RL Phase
RL Phase
Entropulse
RL Phase
Entropulse
0.55
0.30
0.50
0.25
Reward
0.45
Entropy
0.40
0.35
0.10
0.30
150
300
150 180 330
Step
Step
Figure 4: COMPUTERRL training curves of reward (left) and entropy (right) with 95% confidence intervals. The red line denotes the training with entropy recovery via Entropulse after the first RL stage, while the grey line denotes continued training with only reference resetting.
We conduct a case study in the desktop environment to identify po- tential avenues for system optimization. Although model exhibits robust performance across most scenarios, several limitations have been identified. In particular, can be categorized into four primary types: visual perception errors, multi-application coordination failures, operational illusions, and other errors. The distribution of these error types is presented in Figure 5
34.4%
encountered during task execution
29.0%
25.8%
Ervor Type
Illusine
Vision
Others
Multiple App
4.5 EXPERIMENTS DIFFERENT BASE MODELS
Figure 5: Error distribution.
To further verify the effectiveness of our method, we conduct repetitive experiments with different base models (both text and multimodal). demonstrating the stability and superiority of our approach. The results are reported in Figure 6
0.55
ComputerRL Qwen2.5-14B
ComputerRL w/ GLM-4-98-0414
0.50
ComputerRL w/ GLM--..V-9B-Thhinking
Average
Entropulse
Average Reward
0.45
0.35
0.30
40
60
100
120
140
160
180 Step
200
220
240
260
280
300
320
340
360
Figure 6: Repetitive experiments with different base models (95% Cls).
Appendix E presents more experimental results. Additional examples (including both good and bad) provided in Appendix to further illustrate the model's capabilities and limitations.

---

Published as a conference paper at ICLR 2026
5
RELATED WORK
Large Language Models. LLMs, such as GPT (Achiam et al. 2023), , Gemini Team et al. Claude (Anthropic 2023). Llama (Touvron et al.
2023
2023a), GL.M (Zeng et al. 2022 Du et al. 2022).
Qwen [Team] 2024). and Deepseek (Liu et al. 2024a). have demonstrated remarkable capabilities in knowledge representation and language understanding, leading to diverse downstream applications.
Vision-Language Models (VLMs) (Hong et al. 2023 2025; Bai et al. 2023b Hurst et al. 2022)
further extend LLMs multimodal inputs, enabling joint reasoning over text and images.
Computer Use Agents. CogAgent (Hong et al. 2023) introduces multimodal GUI understanding. AutoGLM (Liu al. 2024b) decouples planning and grounding with online RL improvement. OS-
Atlas (Wu et al. 2024b) proposes a foundational GUI action model. Aguvis (Xu et al. 2024) enables
cross-platform interaction through visual training. PC-Agent-E. (He al. 2025) utilizes trajectory
boosting for enhanced proficiency. UI-TARS (Qin et al. (025) performs human- Mikenn-ikeeea GUI interactions from screenshots. Agent S2 (Agashe et al. 2025) integrates grounding with hierarchical reasoning.
CUA (OpenAI 2025)
offers programmable desktop automation.
Computer Use Benchmarks. WebArena (Zhou et al. 2023) provides simulated websites for online interactions, but has limitations: discrepancies from weal-world environments and web-only focus. Similar issues exist other web-focused benchmarks (Yao al. 2022 Koh et al., 2024 Chezelles
ct al., 2024, Miyai al.] 2025). Software engincering benchmarks (Jimenez et al. 2023 Yang
ctt al. 2024a Li et al. 2024; Zan et al. 2025 Padigela et al.] (025) lack comprehensive desktop
evaluation. OSWorld (Xie al. 2024) addresses these gaps with 369 tasks with 134 evaluation
functions. Windows Agent Arena (Bonatti et al. 2024) expands this with 150+ Windows-basse tasks.
RL and Entropy Management for LLMs. PPO (Schulman et al. 2017) addresses instability in policy gradients for RL training. GRPO (Guo et al. 2025) extends PPO with group sampling and re- moves value updates. Maximum entropy RL (Haarnoja al. 2018) and ensemble methods (Lee et al., 2021] De Paola et al.] (025) maintain diversity through regu regularization or multiple models. Recent work identifies entropy collapse
critical challenge in LLM RL (Cui al. (025), with proposed solutions including DAPO Yuu et al. (205)) with adaptive clipping and token-level interventions (Hao) al. 2025). Entropulse takes a different approach by actively restoring collapsed entropy through targeted SFT training on diverse rollout data, achieving extended training.
6 CONCLUSION
In this work, we present COMPUTERRL, novel computer use agent that integrates API-based and GUI-based actions with scalable RL training. Our experiments on OSWorld and OfficeWorld demonstrate superior performance compared to prior approaches, laying the groundwork for more capable autonomous computer use agents.
ACKNOWLEDGMENT
This work was supported by Fundamental and Interdisciplinary Disciplines Breakthrough Plan the Ministry of Education of China (No. JY2005XXXX1001) Natural Science Foundation of China (NSFC) 62425601 and 62495063, the New Cornerstone Science Foundation through the XPLORER PRIZE.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint arXiv:2303.08777 2023.
Saaket Agashe, Kyle Wong. Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist-specialist framework for computer use agents, arXiv preprint vrXiv:2504.00900 00906, 2025.
Anthropic. Model card and evaluations for claude models. 2023.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.1660 2023a.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308. 100.8/229666 /2966, 2023b.
Michael Bain and Claude Sammut. A framework for behavioural cloning. In Machine intelligence 15, pp. 103-129, 1995.
Rogerio Bonatti, Dan Zhao, Francesco Bonacci, Dillon Dupont, Sara Abdali, Yinheng Li, Yadong Lu, Justin Wagle, Kazuhito Koishida, Arthur Bucker, et al. Windows agent arena: Evaluating multi-modal os agents scale. arXiv preprint arXiv:2409.08266, 2024.
Ivan Bratko, Tanja Urbančič, and Claude Sammut. Behavioural cloning: phenomena, results and problems. IFAC Proceedings Volumes, 28(21) 1995.
De Chezelles, Thibault Le Sellier, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han Lù, Ori Yoran, Dehan Kong, Frank F Xu, Siva Reddy, Quentin Cappart, et al. The browsergym ecosystem for web agent research. arXiv preprint 2rXiv:24/2.05460) 2024.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, al. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22611 2025.
Vincenzo De Paola, Riccardo Zamboni, Mirco Mutti, and Marcello Restelli. Enhancing diversity in parallel agents: A maximum state entropy exploration story. arXiv preprint M.Niv:2505.013:: 2025.
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp- 320 335, 2022.
Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-growd policy optimization for Ilm agent training. arXiv preprint 1007925505.0077, 2025.
Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, et al. Areal: A large-scale asynchronous reinforcement learning system for language reasoning. arXiv preprint arXiv:250S.24229. 2025.
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang. Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, et al. Chatglm: family of large language models from glm- 130b to glm-4 all tools. arXiv preprint arXiv:2406.1223 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint arXiv:2501.12945 2025.
Yiduo Guo, Zekai Zhang, Yaobo Liang, Dongyan Zhao, and Nan Duan. Ppte benchmark: Evaluating large language models for powerpoint task completion. arXiv preprint arXiv:2311.0/ 10.1767, 2023.
11

---

Published as a conference paper at ICLR 2026
Tuomas Haarooja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Internationall conference machine learning. pp. 1861-1870. Pmlr, 2018.
Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. Rethinking entropy interventions in rlvr: An entropy change perspective. arXiv preprint arXiv:2500. 10150, 2025.
Yanheng He, Jiahe Jin, and Pengfei Liu. Efficient agent training for computer use. arXiv preprint arXiv:2505. 13909, 2025.
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang. Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. arXiv preprint 2rXiv:2312.089/1 2023.
Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Qi, Junhui Ji, Lihang Pan, al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv e-prints, pp. arXiv-2507, 2025.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint 162:: 2024.
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770. 2023.
Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po- Pu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. arXiv preprint arXiv:2401.13649 2024.
Kimin Lee, Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Sunrise: A simple unified framework for ensemble learning in deep reinforcement learning. In International conference machine learning, pp. 6131-6141, PMLR, 2021.
Bin Lei, Yuchen Li, Yiming Zeng, Tao Ren, Yi Luo, Tianyu Shi, Zitian Gao, Zeyu Hu, Weitai Kang, and Qiuwu Chen. Infant agent: A cool-integrated, logic-driven agent with cost-effective api usage. arXiv preprint 2xXxx:2411.0//11 2024.
Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, al. Devbench: A comprehensive benchmark for software development. CORR, 2024.
Kefan Li and Yuan Yuan. Large language models as test case generators: Performance evaluation and enhancement, 2024. URL https://arxiv.orrr htwpaaaaiii.oo////////////// 133444444331.000
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024a.
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating 1lms as agents. arXiv preprint arXiv:2308.00 03688, 2023.
Xiao Liu, Bo Qin, Dongzhu Liang, Guang Dong, Hanyu Lai, Hanchen Zhang, Hanlin Zhao, lat Long long, Jiadai Sun, Jiaqi Wang, et al. Autoglm: Autonomous foundation agents for guis. arXiv preprint (tt::24//.00820 2024b.
Fanbin Lu, Zhisheng Zhong, Shu Liu, Chi- Wing Fu, and Jiay Jia. Arpo: End-to-end policy optimization for gui agents with experience replay. arXiv preprint arXiv:2505. 16282, 2025.
12

---

Published as a conference paper at ICLR 2026
Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. Spreadsheetbench: Towards challenging real world spreadsheet manipulation. Advances in Neural Information Processing Systems, 07:94871-94908 2024.
Atsuyuki Miyai, Zaiying Zhao, Kazuki Egashira, Atsuki Sato, Tatsumi Sunada, Shota Onohara, Hiromasa Yamanishi, Mashiro Toyooka, Kunato Nishina, Ryoma Maeda, et al. Webchorearena: Evaluating web browsing agents on realistic tedious web tasks. arXiv preprint arXiv:2506.01955: 2025.
OpenAL. Computer-using agent: Introducing universal interface for ai to interact with the digital world. 2025. URL /Rthhttss://ppeaai wwwpeaaa..cooiiiiaaa httom/inexx/cooe http://pwaa...oom///m///oo/oooot -ggntt
Harshith Padigela, Chintan Shah, and Dinkar Juyal. MI-dev-hench: Comparative analysis of agents on ml development workflows. arXiv preprint arXiv:2502.0096/ 2025.
Yujia Qin, Yining Ye, Junjie Fang. Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents, arXiv preprint urXiv:2501. 123:6, 2025.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy 2017.
optimization algorithms. arXiv preprint
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemat- ical reasoning open language models. arXiv preprint arXiv:2402.03300, 2024.
Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, et al. Os-genesis: Automating gui agent trajectory construction via reverse task synthesis. arXiv preprint arXiv:2412.19723, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
Qwen Team. Qwen2 technical report. arXiv preprint 2w..iv2407..0000 10671, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.0922 09288, 2023b.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang. Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji- Ji-Rong Wen. A survey large language model based autonomous agents. arXiv preprint arXiv:2308. 11432, 2023.
Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, al. Openhands: An open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407. 16741, 2024.
Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. Os-copilot: Towards generalist computer agents with self-improvement. arXiv preprint arXiv:2402.07456, 2024a.
Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang. Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, et al. Os-atlas: foundation action model for generalist gui agents. arXiv preprint
2024b.
13

---

Published as a conference paper at ICLR 2026
Zhiheng Xi. Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang. Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongx- iang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang, and Tao Gui. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.0755 2023.
Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li. Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng. Dongehan Shin, Fangyu Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37:52040-52000 52094, 2024.
Tianba Xie. Jiaqi Deng, Xiaochuan Li, Junlin Yang, Haoyuan Wu, Jixuan Chen, Wenjing Hu, Xinyuan Wang, Yuhui Xu, Zekun Wang, al. Scaling computer-use grounding via user interface decomposition and synthesis. arXiv preprint arXiv:2505. 00.5500../322,, /3227, 2025.
Yiheng Xu, Zekun Wang. Junli Wang. Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint 2rXiv:24/2.04455 2024.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang. Jianxin Yang. Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL https:
httgg/ab//2500/ 09388
John Yang, Carlos Jimenez, Alex L. Zhang, Kilian Lieret, Joyce Yang. Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R Narasimhan, et al. Swe-bench multimodal: Do ai systems generalize to visual software domains? arXiv preprint arXiv:2410.03859, 2024a.
Yuhao Yang, Yue Wang, Dongxu Li, Ziyang Luo, Bei Chen, Chao Huang, and Junnan Li. Aria-ui: Visual grounding for gui instructions. arXiv preprint arXiv:2412.162555 2024b.
Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 01220744-207577 2022.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjum Liu, et al. Dapo: An open-souree Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503.14470 2025.
Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, et al. Multi-swe-bench: A multilingual benchmark for issue resolving. arXiv preprint arXiv:2504.02605 2025.
Aohan Zeng, Xiao Liu, Zhengxiao Du Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. Glm-130b: An open bilingual pre-trained model. In The Eleventh International Conference Learning Representations, 2022.
Hanchen Zhang. Xiao Liu, Bowen Lv, Xueqiao Sun, Bohao Jing, lat Long long, Zehan Qi, Hanyu Lai, Yifan Xu, Rui Lu, Zhenyu Hou, Hongning Wang. Jie Tang, and Yuxiao Dong. Agentrl: Reinforce all-round agents from zero. arXiv preprint, 2025.
Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo. Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. In Second Agent Learning in Open-Oneeee Endedness Workshop, 2023.
14

---

Published as a conference paper at ICLR 2026
A API DEVELOPMENT WORKFLOW
In this section, we detail the methodology for leveraging LLMs to automate API construction. We propose semi-automated workflow wherein users need only supply exemplar tasks performed within the target application; the LLM then autonomously generates both the necessary API code and corresponding test cases. The workflow comprises three primary stages: requirement analysis, API implementation, and test case generation.
Requirement Analysis During the requirements analysis phase, users provide of task examples related to the target application as input. The workflow leverages the LLM to analyze these task instances, extracting the essential functionalities required for task completion. It then compares these requirements against the existing API interface definitions to identify potential gaps. If uncovered functionalities are detected, the system automatically generates new API interfaces along with their corresponding parameter specifications.
Notably, we limit the generated interfaces to encapsulate only general-purpose functionalities, thereby avoiding excessive complexity and the proliferation of APIs. This design choice mitigates implemen- tation difficulty and reduces the adaptation burden on the agent.
API Implementation Upon obtaining the interface definitions, the workflow systematically iterates over each interface and associated parameters. For each specification, leverages the designated Python libraries of the target application implement the corresponding API functionalities. Ad- ditionally, the workflow incorporates error- grror-haddling mechanisms and logging to facilitate human debugging and maintenance. This automated approach not only streamlines API development but also enhances consistency and reusability across different application contexts.
Test Case Generation Following the implementation of API functionalities, the workflow conducts fundamental unit testing to ensure the correctness and robustness of each API. Specifically, the testing process verifies: (I) whether the API can be invoked without runtime errors, and (2) whether the API returns correct results across range parameter inputs. For API implementations that fail these tests, the workflow provides detailed error feedback to the API implementation module, which then autonomously attempts corrections until the APIs pass all tests.
This automated methodology substantially lowers the manual effort required, enabling the creation of application-specific API sets with minimal human intervention. As result, the barrier for users to develop APIs for diverse applications is significantly reduced. We have developed API sets for multiple commonly used default applications in Ubuntu and integrated them into our Ubuntu virtual machine environment.
15

---

Published as a conference paper at ICLR 2026
B ACTION SPACE
Our action space for GLM-COMPUTERRL is shown in Table and our API number for each application in Table
Table 4: Action space of GLM-COMPUTERRL.
Function
Description
open_app(app_name click(coordinates, num_clicks, wutticckssuttto._o.. www.bbuutttotytoooo type)
Open specified application (e.g.. Chrome, Terminal).
Click coordinates [x, yJ with the specified mouse button and number of clicks.
type (ypordddiissttes, text, overwrite, enter) dog_and_drropddra__rrrr drop_on)
Type text at coordinates; optionally overwrite existing content and/or press Enter.
Drag from [x1, yı] and drop onto [x2. y2].
(croldtccod,,nattss,, direction)
Scroll at coordinates in direction (up down).
Switch focus window with given ID.
hotkey (keys)
Press a key combination (e.g., [ctrl, cl).
quote(content)
Record content for memory.
wait()
Pause execution temporarily.
exit (success)
Terminate task with success (True) or failure (False).
Table 5: Statistics of the number of available APIs per application
Application
Number APIs
Code
12
Chrome
11
LibreOffice Calc
27
LibreOfffice Impress
22
LibreOfffice Writer
19
VLCC
Total
103
16

---

Published as a conference paper at ICLR 2026
C PROMPT FORMULATION FOR LLM-COMPUTERRY
The design of the observation space pivotal, as it directly constrains the upper bound of the agent's performance. In this section, we detail the integration of the API set with GUI operations, alongside the incorporation of contextual information from the desktop environment, to systematically construct both the agent's observation space and action space. This unified framework ensures that the designed observation and action spaces capture the complexity of real-world tasks, providing solid foundation for robust agent learning and generalization.
Action space formulation The integration of a large number of APIs with GUI operations. while ensuring effective agent interaction, remains significant challenge. In practice, mitigate this complexity by dynamically detecting the currently active application to infer potentially relevant APIs, thereby reducing the number of available APls and lowering the agent's adaptation overhead. Furthermore, we use Python classes and descriptive docstrings to delineate each operation type, ensuring they are clearly interpretable by most LLMs. This object-oriented strategy enhances the model's understanding and precision in performing operations. These classes are provided to the agent via the system prompt, enabling interaction through Python function calls. This design facilitates rapid agent adaptation and efficient generalization of operations across diverse applications. Additionally, the agent's output format standardized the system prompt, which encourages the agent to interleave reasoning and action execution. This approach promotes enhanced planning and reflective capabilities within the agent, thus improving its overall performance in complex task exccution scenarios.
Observation formulation To facilitate the effective perception and manipulation of GUIs by the model, we leverage the Python Accessibility Toolkit Service Provider Interface (pyatspi) to extract comprehensive attributes of desktop elements systematically. Each GUI element encompasses the element's 's semantic type, visible text content, precise screen coordinates, and spatial dimensions. This structured representation enables the LLM agent to parse, ground, and reason over the GUI in a manner analogous to human users.
We present the element format of the environment l tree in our observation space as follows: tag
text
position (center x y) size
(w h)
The tag is the XML tag of the element, such as div button. The text is the text content of the element, which can be empty for elements that do not have text. The posi tion is represented by the center coordinates (x, y) of the element, and the size is represented by its width (w) and height (h).
For the multimodal model, the al ly tree is removed from the input. Instead, we capture the GUT screenshot at a resolution of 1920 1080 pixels (1080p) and subsequently resize it 1280 720 pixels (720p), which serves as the input representation of the desktop environment.
Beyond the extraction of individual GUI components, augment the input space with rich contex- tual metadata to provide a holistic depiction of the agent's 's operational environment. Specifically, we provide a comprehensive enumeration of open desktop windows, including their hierarchical relationships, well as the name and additional information of the currently focused application. To promote consistent and adaptive hehavior, we also deliver feedback from the most recent GUI action or tool call, which may include environmental status updates, confirmations, or error signals.
The app format of the observation space is as follows:
Window ID
App
Name
Title
The Window ID is the unique identifier of the application window, App Name is the name of the application, and Title is the title the application window.
History formulation Given the extensive length of GUI observations and the inherent constraints imposed by the model's context window, it is necessary to efficiently manage the input history across multiple interaction rounds. For each interaction, we omit redundant and detailed interface information while preserving the complete sequence of the agent's 's reasoning process, actions taken, and the corresponding operation feedback. This approach ensures the retention of the essential opera- tional trajectory, thereby maximizing the informativeness of the historical context while maintaining compatibility with the model's capacity limitations.
17

---

Published as a conference paper at ICLR 2026
Collectively, the components above constitute the observation space and action space of agent. This representation not only enhances the agent's environmental cognition but also enables better strategies for long-horizon planning and reasoning. As result, the agent is better equipped to execute complex, multi-step tasks across diverse applications in the desktop environment.
Below is detailed prompt organization for GLM--OMMUTERRL..
You are an agent which follow instruction and perform desktop computer
tasks instructed.
You have good knowledge of computer and good internet connection and assume your code will run on a computer for controlling the mouse and keyboard.
For each step, you will get an observation of the desktop by 1) screenshot; 2) current application name; 3) accessibility tree, which based on AT-SPI library; 4) application info; 5) last action result.
You should first generate plan for completing the task, confirm the previous results, reflect the current status, then generate operations to complete the task in python-stye pseudo code using the predefined functions.
Your output should STRICTLY follow the format:
<think>
YOURRU--PA -AI-LLAN-AA-TTIIIIINN----.
</think>
'python
[** [**ONE-LINE -FFCNEE+)
You will be provided access to the following methods to interact with the
UI:
1. class Agent, grounding agent which provides basic action space interact with desktop.
2. class [tool_class_namer which provides tools to interact with the current application [app_name).
Here are the defination of the classes:
.. ppython
"class_content
Note:
Your code should be wrapped in "'python""" and your plan and thinking should be wrapped in <think></think> Only -OOEE-LINE-OF-COEE
at time.
Each code block is context independent, and variables from the previous round cannot be used in the next round.
Do not put anything other than python code in "'python
You **can only use the above methods to interact with the UI**, do not invent new methods.
Return with 'Agent.exit (successssss)))))))))) immediately after the task is completed.
If you think cannot complete the task, *xDO NOT keep repeating actions, just return with 'Agent. exit (succes===alsse)) '3** The computer's environment is Linux, e.g., Desktop path is //home / Desktop
My computer's 's password is 'password', feel free to use it when you need sudo rights
IMPORTANT** You asked to complete the following task: (instruction)
Below is our history and input prompt for GLM-COMPUTERRL:
<Juser EnEviirnment
State (Omi tetdtted+=
18

---

Published as a conference paper at ICLR 2026
<|assistant
<think>
(rounde_thinking)
</think>
"'pythn
[round@_operation)
<|user
EnEiironnent State (Omitted)**
Previous Action Result: (rounde feedback)
<lassistant|> |>
<think>
(round1_thinking)
</think>
''python
[round1_operation)
<|user
WAEnvironment State (0mitted)**
Previous Action Result: Fronntloperatiion_fedbee


<think>
[hound__thinking
</think>
''python
[round2_operation)
</user|>
Apps: [all_apps)
Current App: (cur_window_id)
* A11y Tree: (elly_trre_for_texx)
App Info: (app_info)
Previous Action Result: ceeeratioonfeebacck
19

---

Published as a conference paper at ICLR 2026
D TRAINING & HARDWARE DETAILS
D.1 TRAINING PROCESS & HYPERPARAMETER SETTINGS
During the behavior cloning stage, we construct approximately 8,000 tasks through manual annotation and data augmentation. We employ multiple advanced models to generate diverse samples for each task, and subsequently apply the eval function to filter out successful trajectories. This process yields a high-quality BC dataset comprising roughly 180,000 steps, which is then used for SFT training. We employ 16-node computing cluster for fine-tuning, with maximum learning rate set x 10-s, sequence length of 32, 768 tokens, and global batch size of 256, over total of three training epochs.
In the RL stage, the key training hyperparameters are summarized in Table 6 We initially train the BC policy (using the 1 I-epoch checkpoint for diversity) for 180 steps, after which performance improvements began to plateau. At this point, we collect rollouts during RL, perform task-level random selection, and curate approximately 130,000 additional steps of data for Entropulse training. The hyperparameters in this phase are identical to those used in the BC stage, except for a reduced learning rate of 10-6, RL training is then resumed until total of 360 steps have been reached.
D.2 TRAINING CLUSTER CONFIGURATION
Our training infrastructure consists of a high-penn gigh-performance GPU cluster. The complete specifications, including GPU, CPU, cache, memory, and network configuration, are detailed in Table Our training pipeline requires at least 4 GPU nodes run distributed RL training.
D.3 ENVIRONMENT CLUSTER CONFIGURATION
For running distributed RL environments, we employ a dedicated compute cluster with 7 nodes. The complete specifications shown in Table g In empirically validated deployment:
Each GPU achieves optimal utilization when paired with approximately 80 rollouts.
Each environment server can reliably host 200 concurrently running virtual environments.
This ratio maintains equilibrium between GPU computation and environment sampling, minimizing idle computational resources.
D.4 VIRTUAL ENVIRONMENT INSTANCE CONFIGURATION
Each RL task is executed within a dedicated virtual machine instance. The specifications are detailed Table
Each virtual environment instance runs an independent Ubuntu 20.04 desktop environment for executing GUI-based tasks. The lightweight resource configuration (2 cores/4GB) ensures high concurrency under limited hardware resources, supporting the environment parallelism required for large-scale distributed RL training.
Note:
D.5
TRAINING DURATION AND FLOPS STATISTICS
Table T0 presents the complete training time and FLOPs statistics for the multimodal training.
20

---

Published as a conference paper at ICLR 2026
Table 6: Training configuration for RL training of GLM-COMPUTERRL.
Category
Parameter (Value)
Data
Task type
Multi-turn chat
Max prompt length
63,488 tokens
Max response length
2,048 tokens
Train batch size
32
Responses per prompt
Concurrency
1024
Shuffle
True
Seed
42
Actor
Exchange size
x 1020
Gradient checkpointing
Enabled
Strategy
FSDP
FSDP offloading
Param Optimizer
Sequence parallel size
Max tokens / GPU
32,768
Precision dtype
b
Algorithm
Advantage estimator
GRPO
Discount factor y
1.0
GAE parameter À
1.0
Use remove padding
True
Use dynamic bsz
True
Mini-batch size
32,768
Micro-batch size GPU
Logprob micro-batch size GPU
KL loss
Enabled (low_var_k), coef = 0.0003
Entropy coefficient
0.0
Clip ratio
0.2
Optimizer
Actor learning rate
x 10
LR warmup steps ratio
0.0
Warmup style
constant
Gradient clipping
1.0
Save frequency
25
Rollout
Enable chunked prefill
True
Max new tokens (generation)
2,048
Do sample
True
Sampling temperature
0.8
Max turns
30
GPU memory utilization
0.7
Pools rollout
2
Pools other
21

---

Published as a conference paper at ICLR 2026
Table 7: Training Cluster Specifications
Configuration
Specification
Cluster Overview
Cluster Size
16 nodes
GPU Configuration
GPU Type
NVIDIA H800
GPUs per Node
8
Total GPUs
128 (16 nodes x 8 GPUs)
CPU Configuration
CPU Model
Intel Xeon Gold 6430 x86_64
Architecture
Physical Sockets
Cores Socket
32
Total Cores
64
Threads
64 (1 thread/core)
Base Frequency
GHz
Minimum Frequency
800 MHz
Instruction Set Extensions
AVX-512, AVX512 FPI6, AMX (INT8/BFL6/Tilc)
Cache Configuration
LI Data Cache
MiB (64 instances)
LI Instruction Cache
MiB (64 instances)
L2 Cache
128 MiB (64 instances)
L3 Cache
120 MiB (2 instances)
Memory Configuration
Total Memory Capacity
2.0 TiB
Available Memory
1.9 TiB
NUMA Nodes
NUMA Node 0 CPUs
0-31
NUMA Node CPUs
32-63
Swap
Disabled B)
Network Configuration
Interconnect
InfiniBand/High-speed Ethernet
Address Width
Physical 46-bit, Virtual 57-bit
22

---

Published as a conference paper at ICLR 2026
Table 8: Environment Cluster Specifications
Configuration
Specification
Cluster Overview
Cluster Size
7 nodes
Total Cluster Memory
~.. TiB
CPU Configuration
CPU Model
Intel Xeon 6986P-C (Granite Rapids)
Architecture
x86_64
Physical Sockets
1
Cores per Socket
120
Total Cores
120
Threads
240 (2 threads/core, hyper-threading enabled)
Base Frequency
3.3 GHz
Max Turbo Frequency
3.9 GHz
Minimum Frequency
800 MHz
Instruction Set Extensions
AVX-512, AVXS12_ AMX, SHA-NI
Cache Configuration
L1 Data Cache
5.6 MiB
LI Instruction Cache
7.5 MiB
L2 Cache
240 MiB
L3 Cache
504 MiB
Memory Configuration
Memory per Node
1.1 TiB
Available Memory
949 GiB
NUMA Nodes
3
NUMA Node 0 CPUs
0-39, 120-159
NUMA Node CPUs
40-79, 160-199
NUMA Node CPUs
80-119, 200-239
Swap
Disabled (0 B)
Virtualization and Features
Virtualization Technology
Intel VT-x, EPT, VPID Enhanced IBRS, IBPB, Spectre/Meltdown mitigations AES-NI, SHA-NI, AVX512_VAIE VAES AMX, AVX512 BF16, AVX512 VNNI
Security Features
Cryptographic Acceleration AI Acceleration
Address Width
Physical 52-bit, Virtual 57-bit
Table Virtual Environment Instance Specifications
Configuration
Specification
Operating System
Ubuntu 20.04 LTS
vCPU Cores
2
Memory Allocation
4 GB
Runtime Average Bandwidth
0.4 Mbps
Virtualization Platform
KVM/QEMU
Table 10: Training Time and FLOPs Statistics (Multimodal)
Training Stage
Duration (hours)
Total FLOPs
SFT (Behavior Cloning)
16
1.67 10
SFT (Entropulse)
11
1.22 10
RL (Two-stage)
58
3.21 x 17 (estimated)
23

---

Published as a conference paper at ICLR 2026
D.6 KEY PERFORMANCE TRADE-OFFS AND BOTTLENECKS
Our observations identify two principal hyperparameters that significantly influence the balance between training efficiency and convergence stability.
1. RESPONSES PER PROMPT (RPP) EXPLORATION UPPER BOUND Vs. SAMPLING EFFICIENCY
Role. RpP defines the breadth of the search space in Best-of-N (BoN) sampling.
Trade-off:
A higher RpP broadens exploration, increasing the likelihood discovering high-quality trajecto- ries, but sampling latency grows roughly linearly.
A lower RpP yields faster sampling but may omit promising solutions, constraining exploration scope.
Bottleneck. Excessively large RpP values incur substantial sampling overhead with diminishing marginal gains.
2. BATCH SIZE (B) TRAINING STABILITY VS. ITERATION THROUGHPUT
Role. B specifies the number of samples processed in each gradient update.
Trade-off:
A larger B improves gradient estimation accuracy and stabilizes training, but extends iteration time.
A smaller B accelerates iterations but introduces higher gradient variance, potentially destabilizing convergence.
Bottleneck. Too small values cause pronounced oscillations in the training curve, while too large values extend iteration time.
Optimal Configuration: RpP 16, = 32. Systematic experimentation confirms that RpP 16
and B = 32 represent an optimal balance across competing objectives:
Exploration Adequacy RpP = 16 affords sufficient BoN sampling scope to cover the majority of feasible solution trajectories.
Training Stability B = 32 maintains variance in gradient estimates within acceptable bounds, promoting smooth convergence.
Resource Efficiency This configuration ensures balanced utilization both GPU and environ- ment clusters, avoiding throughput bottlenecks.
Performance Outcome Using this configuration, achieved the reported final performance, outperforming other settings in the accicie trade-off.
D.7 ADDITIONAL OBSERVED BOTTLENECKS
1. ENVIRONMENT HETEROGENEITY
Issue. Significant variance in task execution time results in some GPUs waiting for slower environ- ments to complete.
Mitigation. An asynchronous rollout collection mechanism allows fast environments to submit results without delay.
INTER-CLUSTER NETWORK BANDWIDTH
Issue. High concurrency in environment simulation can saturate network bandwidth due to frequent transmission of screenshots and state data, occasionally causing Docker network stalls.
Mitigation. Employing image compression reduces network load; optimizing Docker networking decreases virtual NIC overhead.
24

---

Published as a conference paper at ICLR 2026
3. INTERNET BANDWIDTH CONSTRAINTS
Issue. Large-scale simultaneous environment instances can generate excessive external network traffic.
Mitigation. Packet-level traffic analysis enables elimination of unnecessary transmissions; construct- ing an IP proxy pool mitigates service blocking risks.
E ADDITIONAL EXPERIMENTAL INDICATORS
To more comprchensively validate the effectiveness of our method, we report detailed experimental indicators to support our conclusions, shown Figure 7 These indicators include Average Reward, Entropy Loss, KI. Loss, PPO KI, Average Margin, BoN Reward, Average Turns, and Response Length. Based on these metries, we make the following observations:
Entropulse effectively increases the stochasticity of the policy, leading to a substantial improve- ment BoN after activation. This, turn, drives the growth of the margin and enables the policy continue learning and improving.
After applying Entropulse, the response patterns of the policy (including response length and number of dialogue turns) become closer to those before the first-stage RL training (i.e., shorter), while maintaining comparable scores. This indicates that Entropulse helps the policy discover better solutions along shorter trajectories, thereby suppressing excessive reasoning and redundant steps.
After resetting the reference model, the KL Loss is also reset, allowing the policy to explore larger space relative to the new reference. This prevents the policy from being overly constrained by its previous strategy.
F HUMAN ANNOTATION PROTOCOL
Our annotation process involves ten trained annotators with master's degrees, who are recruited and compensated in compliance with local labor laws and regulations. Annotators provided with clear written guidelines to ensure consistency and accuracy, as outlined in our annotation protocol (see Figures X and 9 All tasks are designed avoid sensitive personal data, and all annotated content in English with no identifying information. The process includes task expansion-trrassfommiee transforming generalized instructions into explicit, executable tasks-and strict result verification minimize errors. Quality control measures include verification passes and clear formatting rules to improve annotation reliability. No annotator exposed harmful, discriminatory, or unsafe content during the process, and all work adheres to the Code of Ethics regarding faimesss, privacy, and legal compliance.
25

---

Published as a conference paper at ICLR 2026
RL Phase
Entropulse + RL Phase
Entropuise
0.30
0.50
Average Revard
Entropy Lass
0.45
0.15
0.35
0.10
60
120
180
180
240
300
360
60
120
180
160
240
300
360
0.0175
0.05
0.0150
0.0125
LOGS
0.03
PPO
0.0075
0.0050
0.01
0.0000
-0.0025
120
180
180
240
300
360
120
160
180
240
300
360
0.550
0.12
0.525
Average Margin
0.10
BoN Reward
0.500
0.475
0.08
0.450
0.06
0.425
0.400
0.04
0.375
104
Average Turns
9.5
Response Length
100
100
8.5
Step
Step
Figure 7: Detailed experimental indicators for COMPUTERRL (95% Cls).
26

---

Published as a conference paper at ICLR 2026
Annotation Guidelines
 Project Background
Currently, when agents perform different application tasks, various execution issues may arise. address this problem, core annotation task is:
1. First, expand the user's seed instruction into executable instructions. Then, conduct strict result verification after each execution ensure the accuracy the execution chain.
II. Data Description
Phase 1: Task Expansion
Fields be completed need execute virtual machine, only need in):
1. Expanded Task
Modify the generalized task description into independent, executable task.
No pronominal references allowed.
All operations must explicitly specify that they are performed
located under
hhome//ser/work/ (the prefix /home/user/wort is fixed).
Instructions must be as clear and executable as possible.
Prohibited example: "Help me find some good movies" (too vague).
You may appropriately reduce the difficulty of the task, for example:
If the original task is "Collect data from 2013-2020." you may adjust it to "Collect data from 2013-2015" (simplify content preparation for Phase 2).
2. Notes Provide guidance for Phase (described in Chinese for convenience).
Example:
"Prepare test. file containing multiple sentences, where the first letter of each sentence lowercase. Verify using
compare_ hecpaare_wtt__oe Iden (esuutt_ptth, golden_patte _aatt)_pahhhh_ The result passes if the first English letter of each sentence capitalized, other characters remain unchanged, and the content is exactly the same the golden file."
You must specify:
a. What type of file to prepare
b. The file content format
Figure 8: Document for Human Annotation (1 22)
27

---

Published as a conference paper at ICLR 2026
c. Which function to use for verifcation
d. Which fields to check during verification
Phase 2: Virtual Machine Function Verification & Field Completion
Fields be completed:
Target File
All files in the task that require model operations (unprocessed files).
compare_wite stree_wwthhhh_dee_eeeeep (only supports xlsx docx, csv ):
task involves other file types functions, these can be omitted.
Detection Target
The name(s) of file(s) that need to be compared.
Golden File (only filled when the verification function is compare_with_geee
Name of the standard answer file used for comparison.
4. check list (required if the Notes task description explicityy specifies formatting):
Optional content comparison is default.
Single/multiple choice:
font Compare font name, color, size, bold, italic, etc.
fill Compare cell fill color.
aignment Compare alignment (e.g., center).
para_format Compare paragraph formatting (alignment, line spacing).
table Compare text in inserted tables (format comparison not currently supported).
5. Files
All files required for the task.
You must create your file names (arbitrary). but the file names must be consistent across fields (important).
III. Annotation Notes
Focus on evaluation ensure model's required operations properly executed; avoid
cases where detection passes without actual modifications.
2. All data must be English, ensure logical correctness after translation.
For manually typed scenarios, verify multiple times to minimize error rate:
Especially check IDs and file paths.
OCR (from screenshots) is recommended for accuracy.
4. Report issues promptly in the annotation group to avoid unnecessary rework.
5. For color-related tasks, use red, yellow, blue, green; avoid visually similar colors.
6. For downloading plugins or assume they are already installed.
Figure 9: Document for Human Annotation (2 / 2)
28

---

Published as a conference paper at ICLR 2026
G FUTURE DIRECTION
While our advancements with COMPUTERRL mark significant leap forward intelligent desktop automation, we see this work as foundation for radical transformation in human-computer interaction. Unleashing the full potential of autonomous agents on the desktop frontier demands reimagining standingn paradigms across several axes.
G.1 TOWARDS ROBUST PERFORMANCE
GLM-COMPUTERRL has demonstrated remarkable proficiency across spectrum of desktop tasks. However, genuine universality requires transcending current boundaries in coverage and general- ization. Real-world digital environments characterized by continual flux and heterogeneity, encompassing unfamiliar applications, emergent workflows, and rare edge cases that lie beyond the scope of existing datasets. A next-generation agent must dynamically adapt shifting GUIs, unpre- dictable pop-ups, and entirely novel interfaces. To this end, we are re-architecting data pipelines to facilitate exponential expansion in training diversity and pioneering infrastructure distill knowledge from real-world user interactions at scale continuously.
G.2 BREAKTHROUGHS IN LONG-HORIZON AUTONOMY
Envisioning the autonomous desktop assistant as an always-available cognitive collaborator necessi- tates mastering sustained, long-duration workflows. While current solutions excel bounded, atomic tasks, they fall short of orchestrating complex, multi-step objectives over extended horizons. Our ambition is to endow the agent with hierarchical planning capabilities, allowing it to reason, learn, and revise strategies dynamically across arbitrarily long task sequences. Realizing this vision will catalyze paradigm shift: automating not just discrete operations, but entire workstreams and creative processes end-to-end, fundamentally reshaping the productivity landscape for the cloud-native era.
G.3 FOUNDATIONS FOR SAFE AND ALIGNED AUTONOMY
Autonomous control over desktop platforms raises profound questions about safety, trustworthiness, and user agency. The margin for error narrows dramatically when agents are empowered to modify files, access sensitive data, or execute unbounded actions. Mitigating these risks requires a rigorous and principled approach to safe behavior and aligmment. Our roadmap includes architecting granular pernissioning frameworks, embedding robust pre-action validation, and multi-stage approval proto- cols. Ultimately, we aim establish safety standards and best practices that serve foundational infrastructure, not just for our agent but for the future ecosystem intelligent digital collaborators.
29

---

Published as a conference paper at ICLR 2026
H DEMONSTRATION
This section presents examples drawn from a variety of application scenarios, including the initial four positive cases and the final two negative ones. Each example illustrates our API-GUI operational paradigm, which addresses the diverse challenges and requirements that arise within different applications.
[A]: Copy an image on Dasktop paste into LibreOffice Writer donument, then save as marge.dox on the Desktop.
Use command 'sysstat' toolitt monitor Ubuntu sy: resources every second for seconds, and save the results to
System Resources Report.tx! on the Desktop.
23
2:
(C): Create table with 'Mothh and Total' headers in new sheet to show the total sales for months.
[D]: Convert uppercase text to lowercase in document consistency.
Figure 10: Examples of GI.M-COMPUTERRL'S execution four tasks, including image processing between GIMP and LibreOffice Writer, monitoring system resource usage in Terminal, table calculation LibreOffice Calc, and document formatting LibreOffice Writer.
H.1 CREATE MONTHLY SALES SUMMARY TABLE
The targeted task be executed is "Create table with two headers ('Month' and 'Total') in new sheet to show the total sales for all months." The actual execution steps can be summarized as follows:
Stepl: Switch to 'Sheet2'
Step2: Set cell 'Al' "Month"
Step3: Set 'B1' "Total"
Step4: Get monthly sales data from 'Sheet1
Step5: Switch to 'Sheet1"
Step6: Switch to 'Sheet2'
Step7: Set month names in column A
Step8: Set total formula for Jan in 'B2'
Step9: Set total formula for Feb in 'B3'
Step 10: Set total formula for Mar in 'B4"
Stepl1: I: Set total formula for Apr in 'B5'
1tep12: Set total formula for May in 'B6'
Step13: Set total formula for Jun in 'B7'
30

---

Published as a conference paper at ICLR 2026
Step14:
Exit after completion
As Figure 12 shows, summary table is created successfully.
H.2
CONVERT UPPERCASE TEXT TO LOWERCASE IN DOCUMENT
The targeted task to be executed "Convert all uppercase text to lowercase the document for consistent text presentation." The actual execution steps be summarized as follows:
Step1: Use the text case conversion tool to change all text to lowercase.
Step2: Click the formatting menu.
Step3: Select the specific formatting option.
Step4: Open the relevant submenu.
Step5: Select all text in the document.
Step6: Click the formatting menu again.
Step7: Select the specific formatting option again.
Step8: Open the relevant submenu again to apply the changes.
Step9: Exit after finishing the operation.
As Figure 13 shows, all uppercase has been successfully converted lowercase, ensuring uniform and polished document appearance.
H.3 GENERATE SYSTEM RESOURCE MONITORING REPORT
The targeted task to be executed is "Use the 'sar' command in the "sysstat' toolkit monitor system activity and output the results to 'System Resources "eppottxx_ under Desktop." The actual exccution steps can summarized as follows:
Stepl: Open the Terminal application.
Step2: Type the command sar 30 / txt.
Resources_ Step3: Execute sudo apt install sysstat to install the toolkit.
Step4: Enter the password if prompted.
Step5: Re-type the sar command to start monitoring.
Step6: Wait for seconds while system activity is being recorded.
Step7: Open the Files application.
Step8: Navigate to and click on the report file under Desktop.
Step9: Exit after confirming the report has been generated.
As Figure 14 shows, system resource report file was generated successfully on the Desktop after monitoring, fulfilling the target task.
H.4 COPY IMAGE FROM XCF FILE TO LIBREOFFICE WRITER AND SAVE
The targeted task to be executed is "Copy the image from an xcf file located on the Desktop and paste into LibreOffice Writer document, then save the document as image, docx on the Desktop." The actual execution steps be summarized as follows:
Stepl: Open the Files application.
Step2: Navigate to the Desktop folder.
Step3: Double-click to open the xcf image file.
Step4: Copy the image in the image editor.
Step5: Open LibreOffice Writer.
31

---

Published as a conference paper at ICLR 2026
Step6: Paste the copied image into the Writer document.
Step7: Save the document.
Step8: Enter the filename as image. docx and select the Desktop as the save location.
Step9: Navigate to the Desktop folder.
Step10: Confirm and click save.
As Figure 15 shows, the image from the xccf file successfully pasted into LibreOffice Writer document and saved as image. docx on the Desktop, completing the task.
H.5
QUESTION MISUNDERSTANDING ERROR
The targeted task to executed is "Append <br to the end of each line in '1'n2\n3' and save in The actual execution steps can be summarized as follows:
output.txt."
Step1: Type the command echo -e '1\n2\n3' in the terminal.
Step2: Exit after execution.
As Figure [6 shows, the agent misunderstood the requirement and only echoed the content without appending <br/> to each line or saving it into output.txt. This results in task failure due to misunderstanding of the question.
H.6 CLICK OPERATION ERROR
The targeted task to be executed is "Please help change GIMP's theme from dark to light." The actual execution steps can be summarized as follows:
Stepl: Click on the menu in GIMP.
Step2: Attempt to click the Preferences option.
Step3: Repeat the click Preferences.
Step4: Try to use the shortcut Shi ft+fCr+++++ to open Preferences.
Step5: Exit without successfully changing the theme.
As Figure shows, the theme remains dark, indicating that the agent failed change GIMP's theme to light due to incorrect click operations.
32

---

Published as a conference paper at ICLR 2026
Step 1: Calctools.switchh active sheet
Step 2: CalcTools.set cell_ valuel'AA, 'Month")
Step 3: CalcTools.set cell value('B1, TTtaal)
Step 4: CalcTools.get cectet__ooo__eeeeeeeee data
Step 5: CalcTools.switch aativee_tiveeeeeeeeeeeeeee sheet((Shett1))
Step CalcTools.switch active sShet((Sheet22)
Step CalcTools.set, column_ columo_valuue(( A',', l'Jan', 'Feb'...
Step CalcTools.set cell valuel SSUM(Sheet1.BB2
Step 9: saleTools.set_ cell valuee""e) '-SUM[$hee11.C2 C2...
Step 10: CalcTools.set, seeecollsset__ellll call_value[(B B4', '=SUM(Sheet1..) .2...
Figure 11: Task (Step 1-10): Create a table with two headers ("Month" and "Total") in a new sheet to show the total sales for all months.
33

---

Published as a conference paper at ICLR 2026
Step 11: CaleTools.set cell_ val Shhe
Step 12: CalcTools. vall_vale""'"" '=huM,Sheet..FF..
Step 13: Calcfools.set cell_ value SSUU[Sheee11.GG
Step 14: Appent.wi//success-rroo
Figure 12: Task (Step 11-14): Create table with two headers ("Month" and "Total") in sheet to show the total sales for all months.
34

---

Published as a conference paper at ICLR 2026
Step wolterrfolss.caanee text cosersooer, /....
Step Agent.click((20, 76)1
Step Agent.click((392, 101))
Step Agent.click[[67, 430))
Step Agent.hotkey([ ctrl', 'x')
Step Agent.click((90,, 76))
Step 7: (genttclick(339, 1011)
Step Apent.click[[670, 4301)
Step 9: Agent.ett(suceesssTre)
Figure 13: Task: am currently engaged in text processing and require assistance in converting all uppercase text to lowercase within my document. This precision is critical for maintaining uniform and polished presentation. Could you help me on this?
35

---

Published as a conference paper at ICLR 2026
Step Agent.open_ (ppppppnnpppppteiaal
Step Agenttyyee(textt'sar /t0e
Step ggent.typeetext-'uue apt install sysstat', ent...
Step Ayent.ty "teeetoyyettext--passood password', enter=True)
Step 5: Agent.lype(text=ssr1 30 "/Desktop/Systemm Re...
Step Agent.wait()
Step 7: Agent.open (
Step Agent.click[116, 172))
Step 9: Agent.exi(ssuceess-rru
Please use the "sar" command in the 'sysstat" toolkit to monitor system activity, eval- uate the status once every second for 30 seconds, output the results to "System Resources Report.txt" under Desktop.
Figure 14: Task:
36

---

Published as a conference paper at ICLR 2026
Aggentopee_appp(iiee
Step Agent.click(117, 172))
Step Agent.click((A08, 153], num_clicks-2)
Step Agent..otkeylli. 'c])
Step Agent.open_appe wwwp:/opeenaappiiipieeee (ibrrioffoooffcc writer")
Step hotken 'v]]
Step hotkeyllt 's]]
Step Agent 206], text- image dock', over...
Step 9: Agent.click[|610, 290])
Step 10: Agent.click(1427, 206
Figure 15: Task: I've stored my xcf file on the Desktop. Can you assist me in copying the image and pasting into LibreOffice Writer document? Save the document as 'image.docx' on the Desktop, please.
37

---

Published as a conference paper at ICLR 2026
Step 1: Agent.typee(eett "echiteexxx""ehhoooo -e '1)2(3' | sed '////bbr...
Step Agenteedtssucces-Trrug
Figure 16: Fail Task (Question Misunderstandiing Error): Append "<br/>" to the end of each line in "IAn2In3" and save in output.txt
Step 1: A.e.nt.click(1122. 76))
Step 2: Agent.click((633, 446))
Step 3: Agent.clic((113, 446))
Step 4: hoteeyhhokky((sshitt 'ctrl', 'p'1)
Step S: Aggeeteeni(successFalls
Figure 17: Fail Task (Click Operation Error): Please help change GIMP's theme from dark to light.
38

---

Published as a conference paper at ICLR 2026
1 THE USE OF LARGE LANGUAGE MODELS
During the preparation of this work, we employed L.LMs to assist with language refinement and gram- mar correction. All research ideas, methodologies, experiments, and analyses were independently conceived, designed, and validated by the authors.
39


Reviews:
[{"review_id": "KF1L15njow", "reviewer": "Reviewer_6WJM", "summary": "This work presents ComputerRL, a scalable framework for training autonomous computer-use agents via end-to-end reinforcement learning. ComputerRL introduces an API-GUI paradigm that unifies APIs and GUI actions, enabling higher efficiency and generalization on computer-based tasks. To support large-scale training, the authors develop a distributed RL infrastructure orchestrating thousands of virtual desktops. They also introduce Entropulse, a hybrid training strategy that alternates between RL and supervised fine-tuning, in order to prevent entropy collapse. Applied to GLM-based models, ComputerRL achieves 48.9% success on OSWorld, surpassing prior state-of-the-art agents such as OpenAI CUA o3, Claude 4.0, and gemini 2.5 pro, while demonstrating superior efficiency and stability. This work establishes a robust foundation for scaling RL-based desktop agents.", "questions": "1. Could Entropulse be generalized to other settings, such as text-based reasoning or code synthesis tasks?\n2. Why do you think Entropulse's performance plateaued below 50%? What would it take to scale it up?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is overall well written.\n- Novel API-GUI paradigm for more generality in computer-based tasks.\n- New Entropulse training strategy that mitigates entropy collapse by alternating between supervised learning and RL.\n- Scalable and asynchronous RL training pipeline\n- Strong empirical results", "weaknesses": "Overall, the contribution lies more in engineering execution than in theoretical advancement.\n- Limited algorithmic novelty: primarily builds upon GPRO, and alternating between SFT and RL is similar to exploration-refresh or replay strategies.\n- The paper does not fully disentangle how much gain comes from ComputerRL’s methods compared to having strong pre-trained models.\n- Limited experimental diversity: they mostly evaluate on OSWorld and OfficeWorld, with no long-horizon or multi-user adaptation or interactions.\n- Limited accessibility to reproduction: no analyses on FLOPs or cost of the experiments, and a complex engineering setup.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6axCux8Tto", "reviewer": "Reviewer_oLgc", "summary": "This paper presents **ComputerRL**, a framework for training computer use agents capable of performing desktop operations through a unified API-GUI interaction paradigm. The framework integrates:\n1. a distributed RL infrastructure that supports training across thousands of virtual desktop instances;\n2. aan LLM-driven module that automatically derives application programming interfaces (APIs) to extend and complement traditional GUI-based controls;\n3. Entropulse, a hybrid training regime that periodically alternates between RL and SFT phases to counteract entropy collapse and maintain exploration over extended training trajectories.\n\nEvaluations on OSWorld and OfficeWorld benchmarks show state-of-the-art results (48.9% success rate) using open-weight models (GLM-4-9B-0414, GLM-4.1V-9B-Thinking), outperforming both proprietary and open baselines such as OpenAI CUA, Claude 4.0, and UI-TARS.", "questions": "- Please specify the number of seeds, compute hardware, and total training cost (GPU hours, cluster size). How reproducible are the reported results on smaller scales?\n- What are the hardware requirements and cost implications for scaling your distributed RL infrastructure to the reported scale? What performance trade-offs or bottlenecks did you observe in practice?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Strong systems and engineering contribution:** The distributed RL infrastructure is technically impressive, enabling large-scale online RL across thousands of virtualized desktop environments. Such scale is rare in open research and represents a substantial engineering achievement.\n\n- **Practical API-GUI paradigm:**  The unified action space combining GUI operations with automatically constructed APIs addresses a key bottleneck in desktop automation. The LLM-driven API construction pipeline is pragmatic and lowers the barrier for generalization.\n\n- **Entropulse training strategy:** Alternating RL and SFT phases effectively combats entropy collapse and stabilizes long-horizon training. While conceptually simple, it appears empirically effective and easy to adopt in practice.\n\n- **Empirical performance:** The results on both OSWorld and OfficeWorld are strong and consistent. The proposed approach achieves superior performance and sample efficiency (fewer steps per task) compared to all evaluated baselines. Ablation studies indicate the importance of multi-stage training and the API-GUI design.\n\n- **Writing and organization:** The paper is very well written, clearly structured, and visually well presented.", "weaknesses": "- **Unsubstantiated claims about diversity and exploration:** The paper claims that alternating SFT with RL increases exploration and diversity, yet no quantitative evidence is provided. Metrics such as action entropy, trajectory variance, or coverage are not analyzed. The only evidence is a qualitative entropy curve, which is insufficient.\n\n- **Incomplete empirical rigor and reproducibility:** All training curves appear to represent single runs without confidence intervals or variance estimates. The number of seeds, randomization strategy, or statistical robustness is not discussed. Given the scale (9B-parameter models, thousands of desktops), compute and reproducibility details are critically missing — no information on hardware setup, training duration, or cluster configuration. For a paper emphasizing scalability, this omission is significant.\n\n- **Limited methodological novelty:** The paper’s main innovations are engineering-oriented. The API-GUI paradigm is conceptually straightforward since it. The novelty lies in the automation pipeline, not the interaction paradigm itself. Similarly, Entropulse is a *training schedule* rather than a new RL algorithm; no theoretical or comparative justification is provided beyond empirical observations.\n\n- **Missing broader impact and ethical discussion:** Given that this system trains autonomous agents capable of full computer control, a discussion of potential misuse, safety mechanisms, or privacy implications is notably absent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uisD79DmCw", "reviewer": "Reviewer_fU96", "summary": "This paper introduces ComputerRL, a large-scale framework for training computer use agents that can operate desktop environments via a unified API-GUI interaction paradigm. The framework integrates (1) a distributed RL infrastructure supporting thousands of parallel virtual desktops, (2) an automated pipeline for generating APIs from LLMs to augment GUI control, and (3) a training strategy named Entropulse, which alternates between reinforcement learning (RL) and supervised fine-tuning (SFT) to mitigate entropy collapse during long training runs.\nThe proposed system is evaluated on the OSWorld and OfficeWorld benchmarks, achieving state-of-the-art results (48.9% success rate) on computer automation tasks with open-weight models.", "questions": "Could you clarify what you consider the core algorithmic innovation of ComputerRL beyond the engineering scale-up?\n\nYou mention that Entropulse increases exploration and behavioral diversity — could you provide quantitative evidence (e.g., entropy statistics, trajectory variance, or action coverage) supporting this claim?\n\nCould you provide details on the hardware setup used — such as GPU type/count, CPU cluster size, memory, and network bandwidth?\n\nHow does your approach compare to prior work addressing entropy collapse or exploration in RL (e.g., maximum entropy RL, population-based methods)?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The API-GUI unification is an interesting engineering contribution that bridges the gap between human-designed interfaces and agent-level programmatic control.\n\nThe distributed RL infrastructure is impressive in scale and demonstrates strong engineering capability, enabling parallelized desktop environments at large scale.\n\nThe Entropulse idea addresses an important issue in long-horizon RL (entropy collapse), and the empirical results suggest measurable benefits in maintaining exploration and training stability.\n\nThe evaluation across multiple benchmarks (OSWorld, OfficeWorld) provides strong empirical evidence of performance improvements.", "weaknesses": "The paper’s novelty lies primarily in implementation and scaling, not in new algorithmic contributions. The API-GUI paradigm is conceptually straightforward—it effectively automates API construction via LLMs rather than introducing a new interaction or reasoning mechanism. Similarly, the Entropulse training alternation between RL and SFT is more of a practical training schedule than a novel learning algorithm.\n\nThe training curves in the figures appear to correspond to single runs, with no error bars or indication of variance across seeds. It is therefore unclear how stable the reported improvements are. The lack of information about the number of runs or random seeds undermines the reliability of the reported trends.\n\nThe paper does not include prior literature on diversity or exploration in RL, which would be relevant given the focus on entropy restoration and SFT alternation. Existing works in exploration-based RL, policy diversity, or ensemble-based approaches could provide valuable context, but they are not mentioned.\n\nGiven the large-scale infrastructure described (thousands of virtual desktops and 9B-parameter models), the paper lacks concrete information on hardware setup, training cost, and total compute used. For a work that emphasizes scalability, this omission is significant. Details such as training duration, GPU/CPU utilization, and cluster size should be clearly reported to assess feasibility and reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)