O2-27B-Preview

A 27B agentic model trained from interactions generated inside runtimes deployed across real office and software-development workflows.

Preliminary Evaluation

O2-27B-Preview was evaluated across eleven reasoning, tool-use, and coding benchmarks. We report its same-size Qwen3.5-27B base alongside the smaller O2-9B-Preview checkpoint and cross-scale reference models. Scores are shown on their original scales, and higher is better. The average is the unweighted mean of the eleven displayed benchmarks.

		Reasoning			Tool Use						Coding
Model	Avg	GPQA	HMMT 2025	HMMT 2026	bfcl v3	bfcl v4	Tau2 Retail	Tau2 Airline	Tau2 Telecom	Tau3 Banking	Agent Bench OS	SWE- Verified
DeepSeek V4 Flash (284b–a13b)	71.84	90.40	93.33	87.88	48.50	57.60	85.09	90.00	95.61	22.68	50.00	69.20
MiniMax- M2.5	62.33	85.9	75.60	73.70	37.00	49.27	79.24	70.67	90.94	8.93	41.00	73.40
Qwen 3.5–397B– A17B	70.56	86.9	90.00	81.80	58.90	63.66	87.13	85.33	93.57	12.37	51.30	65.20
Qwen 3.5– 27B	69.97	86.04	90.00	81.80	58.10	62.15	85.96	82.67	90.94	15.46	56.90	59.60
Qwen 3.5– 9B	60.27	80.81	77.78	67.17	48.70	62.36	83.33	78.67	95.32	5.50	40.97	22.40
O2-9B-Preview (Ours)	64.07	87.37	84.44	69.70	61.90	51.83	86.26	82.67	96.20	9.28	45.14	30.00
O2-27B-Preview (Ours)	73.40	90.40	93.33	90.01	63.38	67.10	89.47	86.00	95.61	16.48	63.19	52.40

O2-27B-Preview improves ten of the eleven reported point estimates over its Qwen3.5-27B base model and raises the aggregate score from 69.97 to 73.40.

These results are preliminary. They come from an author-provided evaluation report without repeated seeds, uncertainty intervals, or matched training and runtime-compute budgets. They characterize the released checkpoint but do not isolate the causal contribution of any individual training component.

Overview

O2-27B-Preview is a 27B-parameter preview model developed by Ant International and initialized from Qwen3.5-27B. It is designed for agentic reasoning, tool use, and coding.

The defining feature of O2-27B-Preview is where its post-training experience comes from. Instead of relying only on static prompt-response datasets, we deployed an executable agent runtime in real operational environments—including international merchant customer-service workflows and the daily development workflows of dozens of engineers—and trained the model from the resulting interactions.

Within these runtimes, the model acts on a task, observes the consequence, receives an executable verification result, and, when necessary, repairs its action. These student-visited states are preserved as structured training records and distilled back into the model. O2-27B-Preview is therefore trained not only on what a successful answer looks like, but also on what happened when an action was executed, why it failed, and how it was repaired.

What Makes O2 Different

Trained where agents actually work

The training loop is connected to deployed office and development environments rather than an isolated text-only data pipeline. It captures interactions with tools, repositories, tests, workflow state, and domain rules under the same kinds of conditions in which agents are expected to operate.

Learning from executable outcomes

For each task, the runtime constructs a task-conditioned verifier suite that checks observable evidence such as tool results, environment state, code execution, repository tests, schema constraints, and domain rules. A verifier pass is treated as a certificate under a specific, versioned suite—not as unrestricted ground truth.

Preserving failures and repairs

The runtime records the task context, model action, executed consequence, verifier outcome, localized failure feedback, and repaired action when one exists. Accepted, rejected, repaired, and unresolved attempts remain available for learning instead of discarding everything except the final answer.

Runtime on-policy distillation

O2-27B-Preview is post-trained with Runtime OPSD, an on-policy sequence-distillation recipe over states visited by the student in the deployed runtime. Oracle outcomes anchor complete trajectories, while teacher and student score tokens under the same task context, runtime state, and generated prefix. This keeps the learning signal aligned with the information available to the model at inference time.

Training Loop

1. A task and its runtime profile are compiled into a versioned verifier suite.
2. The model acts in the executable task environment.
3. The runtime checks the resulting state and returns an auditable outcome.
4. Failed obligations produce localized, failure-only feedback for bounded repair.
5. The interaction is stored as a versioned training record, including failures and repair transitions.
6. Runtime OPSD distills these student-visited records into the next model checkpoint.

This process turns deployed task environments into a renewable source of grounded training experience.

Model Details

Field	Description
Model	O2-27B-Preview
Developer	Ant International
Parameters	27B
Base model	Qwen3.5-27B
Training paradigm	Verifier-guided runtime data synthesis and Runtime OPSD
Primary capability groups	Reasoning, tool use, and coding
Release stage	Preview

Intended Use

O2-27B-Preview is intended for research and evaluation involving:

• agentic reasoning over multi-step tasks;
• tool-using assistants operating in controlled environments;
• software-development and repository workflows;
• office and business-process automation with explicit validation; and
• research on runtime learning, executable verification, and on-policy distillation.

For state-changing or business-critical actions, the model should be deployed with sandboxed tools, least-privilege access, independent checks, and human review.

Limitations

• Preview evidence. The current results are single-report point estimates without multi-seed uncertainty or matched-compute controls.
• Capability-specific regression. Despite a higher aggregate score, O2-27B-Preview scores lower than its base model on SWE-bench Verified in the reported evaluation.
• Partial verification. Executable verifiers encode partial specifications. They can miss valid behavior, accept incorrect behavior, or reward shortcuts when checks are incomplete.
• Runtime dependence. Performance can depend on tool availability, verifier coverage, environment state, and the compatibility of the deployment runtime with the training setup.
• Operational claims. Deployment of the training runtime in merchant-service and engineering workflows establishes real-world operational scope, not a controlled improvement in business outcomes or developer productivity.
• Reproducibility. Exact data mixtures, teacher checkpoint, optimizer schedule, final Runtime OPSD hyperparameters, generated-token budget, runtime-call budget, decoding settings, and immutable evaluator identifiers are not included in this preview.

Safety and Responsible Deployment

O2-27B-Preview can generate incorrect content, malformed tool calls, insecure code, or actions that satisfy an incomplete verifier without satisfying the user's broader intent. Do not treat a verifier pass as proof of unrestricted correctness. Deployers should use held-out checks, audit false acceptances and rejections, isolate executable environments, minimize access to sensitive data, and require human confirmation for consequential actions.

The model has not been established as suitable for autonomous use in high-stakes domains such as medical, legal, financial, or safety-critical decision-making.

Acknowledgements

O2-27B-Preview is initialized from Qwen3.5-27B. We thank the teams who built and operated the office, customer-service, software-development, verification, and evaluation runtimes that made this model possible.