ColorGUI-32B

ColorGUI-32B is a vision-language action model for mobile GUI automation. It is post-trained from Qwen3-VL-32B-Instruct to read screen images, understand user instructions, reason about the next GUI operation, and output executable actions for an OS Agent runtime.

The model is designed for GUI Agent / Mobile Agent scenarios and is intended to be used with the ColorMobileAgent runtime in the ColorAgent technical stack for mobile screen understanding, action planning, and multi-step interactive execution.

The training direction follows the technical report ColorAgent: Building A Robust, Personalized, and Interactive OS Agent. Starting from Qwen3-VL-32B-Instruct, the main training focus includes long-horizon interaction, step-wise reinforcement learning, and self-evolving training.

Model Highlights

GUI-oriented action model: optimized for mobile screen understanding, UI grounding, action selection, and multi-step task completion.
ColorMobileAgent-oriented runtime integration: can be used as the vision-language action model in a mobile GUI Agent execution pipeline.
Reinforcement learning for long-horizon tasks: trained with GUI-specific multi-dimensional rewards, curriculum learning, multi-path exploration, and hard-sample mining.
Stronger robustness on dynamic tasks: achieves clear improvement on a 500-case dynamic benchmark covering social, travel, shopping, entertainment, tool, system-app, and cross-app scenarios.

Evaluation Results

Dynamic GUI Benchmark: 500 Tasks

This test suite is designed to evaluate the task-completion capability of GUI agents. It comprises 500 real-world operational tasks spanning over 80 mainstream mobile applications, including frequently used apps such as Dianping, Amap, and JD. With diverse task types and broad scenario coverage, the suite aims to systematically assess an agent’s perception, decision‑making, and execution performance in complex interface environments, providing an objective and reproducible quantitative benchmark for model evaluation.

Below are two example test tasks (one single-app task and one cross-app task).

Example1(single-app task):

{
        "app_name": "途牛",
        "user_query": "想去国外旅行，帮我在途牛上查看泰国的出境游，路线要包含曼谷和芭提雅，把销量最高的产品添加到收藏",
        "checkpoints": [
            "进入到出境游页面",
            "将目的地设置为泰国",
            "添加筛选条件游玩路线为曼谷和芭提雅，销量优先",
            "进入第一个套餐详情页并完成收藏操作"
        ]
}

Example2(cross-app task):

{
        "app_name": "跨app",
        "user_query": "要出门旅行了帮我在去哪儿上看看从昆明到北京的机票，要最便宜的直飞机票，顺便去飞猪上看看北京的4A以上级景点，顺便查看第一个景点的营业时间",
        "checkpoints": [
            "在去哪儿进入到机票页面",
            "完成起点位置切换为昆明，目的地位置为北京",
            "筛选条件价格低到高、直飞",
            "在飞猪中进入到北京景点搜索页面",
            "添加筛选条件4A及以上",
            "进入到第一个景点详情页成功查看营业时间"
        ]
}

The main benchmark contains 500 dynamic GUI tasks. Each task includes one or more human-defined checkpoints. A task is counted as successful when the final execution result satisfies the user request.

Model	Success	Success Rate	Delta
Qwen3-VL-32B-Instruct	219/500	43.80%
ColorGUI-32B	304/500	60.80%	+17.00 pp

Difficulty Breakdown

Difficulty is defined by the number of human-defined checkpoints in the 500-task benchmark:

Easy: 1-2 checkpoints
Medium: 3-4 checkpoints
Hard: 5 or more checkpoints

Full 500-task benchmark:

Difficulty	Tasks	Avg. Checkpoints	Qwen3-VL-32B-Instruct	ColorGUI-32B
Easy	156	1.76	71.79%	83.33%
Medium	274	3.44	34.67%	52.55%
Hard	70	5.47	17.14%	42.86%

The model improvement is more visible on long-horizon tasks. On the full 500-task benchmark, ColorGUI-32B improves over its Qwen3-VL-32B-Instruct baseline by +25.72 pp on hard tasks.

Task Examples and Execution Trajectories

To make the model capability easier to inspect, we include three representative GUI Agent tasks below. These examples are only used to illustrate applicable scenarios and do not disclose internal training data details.

Task Type	Example Task	Main Capabilities Evaluated
Local-life search	Open the dessert and drinks category in Ele.me and search for "Bawangchaji"	Category-entry recognition, search input, result-page grounding
Vertical-domain filtering	I just got my driver's license and want to buy a used car. Help me find used cars on Autohome with a price of RMB 50k-100k, vehicle age within 1 year, and 0 ownership transfers	Multi-condition filtering, list understanding, constraint preservation
Cross-app long-horizon task	Find 2024 action movies in Migu Video, check the introduction of the first movie, then search for an explanation video about this movie on Bilibili and coin the first video	Cross-app information transfer, long-horizon goal tracking, content retrieval and interaction

Execution Trajectories

The following trajectory figures show the model's step-by-step operations on real mobile interfaces. The images can be replaced with final high-resolution versions before release. We recommend keeping the action type, action parameters, concise reasoning, and semi-transparent coordinate markers so readers can understand each model decision.

Trajectory 1: Searching for Bawangchaji in Ele.me

User instruction: 打开饿了么甜品饮品，搜索霸王茶姬

Task characteristics: This is a short local-life search task. It evaluates whether the model can identify a category entry, enter the search box, type the specified keyword, and keep the target consistent on the result page.

Trajectory figure:

Result summary: The model enters the dessert and drinks category, searches for "Bawangchaji", and reaches the related merchant and product result page. This task demonstrates basic GUI operation ability over category cards, search input, and result pages.

Trajectory 2: Used-car Filtering on Autohome

User instruction: 驾照到手了想买个二手车，帮我在汽车之家找找价格在 5-10 万，车龄 1 年以内，0 过户的二手车

Task characteristics: This is a medium-length vertical-domain filtering task with multiple hard constraints. The model needs to enter the used-car section in Autohome and preserve constraints such as price, vehicle age, and ownership-transfer count.

Trajectory figure:

Result summary: The model enters the used-car channel, selects the price range, applies the vehicle-age filter, applies the ownership-transfer condition, and returns a vehicle list that satisfies the constraints. This task demonstrates parameter preservation in multi-condition filtering scenarios.

Trajectory 3: Cross-app Task from Migu Video to Bilibili

User instruction: 帮我在咪咕视频找一下 2024 年的动作电影，查看第一部影片的简介，然后去哔哩哔哩搜一下这部电影的解说，给第一个视频投币

Task characteristics: This is a long-horizon cross-app task involving content retrieval, movie information reading, app switching, keyword transfer, video search, and interaction. The model needs to preserve the task goal across multiple pages and apps.

Trajectory figure:

Result summary: The model first filters 2024 action movies in Migu Video and reads the introduction of the first movie. It then transfers the movie information to Bilibili, searches for explanation content, and coins the first video. This task demonstrates context preservation and goal progression in cross-app long-horizon workflows.

Future Roadmap

We will continue improving GUI Agent capability along three directions: stronger base models, more stable offline training, and more realistic online interaction validation.

Stronger Model Capabilities

Improve complex-page understanding: further enhance the model's understanding of icons, text, UI hierarchy, pop-ups, lists, tabs, input boxes, and cross-page state, reducing incorrect actions caused by UI recognition errors.
Improve long-horizon stability: focus on state tracking, error recovery, goal backtracking, and intermediate-result usage in multi-step tasks, reducing loops, stalls, premature stopping, and goal drift.
Improve action-parameter precision: continue improving parameter prediction for clicks, swipes, typing, back actions, waits, and other basic operations so the model behaves more consistently across resolutions, app versions, and dynamic pages.
Enhance cross-app and cross-device generalization: improve transfer ability across search, travel, shopping, content consumption, tool apps, and system settings, and integrate more tightly with TopoClaw's cross-device execution framework.

Intended Use

This model is intended for GUI Agent research and development, including:

mobile GUI task automation,
screen-grounded action planning,
long-horizon OS Agent research,
TopoClaw GUI-model integration,
benchmark and Agent-framework development.

Limitations

The model may produce incorrect actions in unfamiliar apps, changed layouts, login/payment flows, or high-risk tasks.
The reported results are from a self-evaluation benchmark and should not be treated as a universal conclusion for all mobile GUI Agent scenarios.
GUI task results can be affected by app version, device resolution, language/region settings, network status, account state, and permission state.
Human confirmation is recommended for irreversible, financial, privacy-sensitive, or account-modifying operations.

Safety Recommendations

When deploying this model in a GUI Agent framework, we recommend adding execution safeguards:

require explicit user confirmation for payment, purchase, deletion, posting, messaging, and account-modification actions;
isolate workspaces and file-access permissions for desktop tasks;
configure app allowlists/blocklists for mobile tasks;
record action audit logs;
pause and request user confirmation when the model is uncertain.

Citation

@misc{li2025coloragent,
  title={ColorAgent: Building A Robust, Personalized, and Interactive OS Agent},
  author={Ning Li and Qiqiang Lin and Zheng Wu and Xiaoyun Mo and Weiming Zhang and Yin Zhao and Xiangmou Qu and Jiamu Zhou and Jun Wang and Congmin Zheng and Yuanyi Song and Hongjiang Chen and Heyuan Huang and Jihong Wang and Jiaxin Yin and Jingwei Yu and Junwei Liao and Qiuying Peng and Xingyu Lou and Jun Wang and Weiwen Liu and Zhuosheng Zhang and Weinan Zhang},
  year={2025},
  eprint={2510.19386},
  archivePrefix={arXiv},
  primaryClass={cs.AI}
}
@software{madeagents_topoclaw_2026,
  title={TopoClaw: Your All-Scenario AI Digital Assistant},
  author={MadeAgents},
  year={2026},
  url={https://github.com/MadeAgents/TopoClaw}
}