arex-2-p300

A tt-model v6 thin bundle that serves BAAI/AREX-2 (revision d4e3502f92d9e889031c2d04e387ab2eb3520268) on 2 Tenstorrent blackhole chips (mesh P150x2). The tt-orchard harness staged it from a run of that model.

License

The weights this bundle points to are licensed Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0). The bundle ships no weights. tt-model downloads them from BAAI/AREX-2 with your own Hugging Face account.

What it runs

• Weights: BAAI/AREX-2 at revision d4e3502f92d9e889031c2d04e387ab2eb3520268.
• Model code: the Tenstorrent implementation of Qwen/Qwen3.8-27B, taken from the bundle qwen3.8-27b-dflash2-p300. BAAI/AREX-2 has the same text architecture, so only the weights differ.
• At each start the launcher builds model-dir/ from the configuration files of Qwen/Qwen3.8-27B (config.json, generation_config.json, preprocessor_config.json, video_preprocessor_config.json, shipped in base_config/) and the tokenizer and weights of BAAI/AREX-2. It sets MODEL_WEIGHTS_DIR and HF_MODEL to that directory, so the chips load the weights of BAAI/AREX-2.
• Context length 262144 tokens; up to 4 sequences at a time.
• BAAI/AREX-2 ships no generation_config.json, so the stop tokens and the sampling defaults the server applies when a request sets none are those of Qwen/Qwen3.8-27B.
• Speculative decoding uses the drafter incoai/Qwen3.8-27B-DFlash2, which was trained on Qwen/Qwen3.8-27B. Its acceptance rate on this model is listed under Not measured.
• BAAI/AREX-2 ships no mtp.* tensors, so the drafter has no MTP head of its own. The launcher reads the 15 mtp.* tensors of Qwen/Qwen3.8-27B from that model's pinned snapshot (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, file model-00018-of-00018.safetensors, downloaded alone on the first start). They are added at each start to model-dir/ as one more weight file, model-mtp-from-base.safetensors (graft.json names them). The head was trained on Qwen/Qwen3.8-27B, so how well it suits BAAI/AREX-2 is shown only by the rows below.

Intended use

BAAI/AREX-2 is a reasoning model for long, multi-step work. Its authors describe it as built for iterative problem solving: it proposes a solution, measures it, reflects on the result and revises it over several rounds. They name long-horizon agents, machine-learning engineering, algorithmic coding and tool-augmented deep research as the intended uses. This package serves it on 2 Blackhole chips for those uses, through the OpenAI-compatible chat endpoint.

It is not a quick-answer chat model. What that means in practice:

• It thinks before it answers, and the thinking is long. The chat template turns thinking on by default at its highest reasoning effort. The server separates the thinking into a reasoning_content field and puts the answer in content. Set max_tokens high. In the benchmark runs below, many replies that hit the lower limit came back empty, because the whole budget went on thinking.
• It is meant to call tools. The server parses tool calls into the standard tool_calls field. In the speed runs, the two tool-use prompts each returned a well-formed call (run_tests, read_file) in place of a text answer, which is what an agent loop expects.
• It works best where feedback exists. The authors trained it on tasks with checkable results (test scores, logs, timings) and its strengths show when a loop feeds those results back to it.
• Text only. The model is multimodal upstream, but this package skips the vision tower. It does not accept images or video.

Reasoning effort and thinking can be set per request through chat_template_kwargs (the model's template accepts reasoning_effort of xhigh, medium or low, and enable_thinking: false). Only the default, xhigh with thinking on, was measured here. The other settings are taken from the chat template and were not tested on this server.

Example request with a tool:

curl http://localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "BAAI/AREX-2",
  "messages": [{"role": "user", "content": "Run the tests under tests/utils and tell me which fail."}],
  "tools": [{"type": "function", "function": {"name": "run_tests", "description": "Run pytest on a path",
             "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}}],
  "max_tokens": 32768
}'

The port is vLLM's default (8000) unless you pass --port after the bundle id. Long-running agent use, long contexts and several requests at once were not measured here.

Expected performance

Every number is labelled. A measured number names its evidence files; the first file holds the value. The files are in this repository under evidence/, at the paths shown, with the run's directory, the home directory and the host name replaced by <RUN_DIR>, <HOME> and <HOST>. TODO means not measured.

Number	Value	Label	Evidence
top1 agreement with the CPU reference, stage 2 (2 chips, bundle qwen3.8-27b-dflash2-p300)	0.96875 fraction	measured	stages/2/result.json, stages/2/evidence/swap-check.json, stages/2/evidence/server.log
server ready after start, stage 2 (empty tensor cache)	160.1 s	measured	stages/2/result.json, stages/2/evidence/swap-check.json, stages/2/evidence/server.log
top1 agreement with the CPU reference, this package (stage 7)	0.96875 fraction	measured	stages/7/verify/evidence/verify.json, stages/7/verify/evidence/server.log
server ready after start, this package (stage 7, fresh install, empty tensor cache)	714.2 s	measured	stages/7/verify/evidence/verify.json, stages/7/verify/evidence/server.log
GSM8K score, this model	0.9433 fraction	measured	stages/7/benchmark/benchmark.json
GSM8K replies cut off at the token limit, this model	0 items	measured	stages/7/benchmark/benchmark.json
GSM8K score, nearest model, same conditions	0.9533 fraction	measured	stages/7/benchmark/benchmark.json
GSM8K paired sign test p, this model against the nearest model	0.5811 p-value	measured	stages/7/benchmark/benchmark.json
IFEval score, this model	0.8928 fraction	measured	stages/7/benchmark/benchmark.json
IFEval replies cut off at the token limit, this model	18 items	measured	stages/7/benchmark/benchmark.json
IFEval score, nearest model, same conditions	0.8595 fraction	measured	stages/7/benchmark/benchmark.json
IFEval paired sign test p, this model against the nearest model	0.06297 p-value	measured	stages/7/benchmark/benchmark.json
decode speed, this package (drafter on, grafted head), one request at a time, median of four prompts	65.79 tokens/s	measured	stages/7/benchmark/speed.json
decode speed, same model with the drafter off and sampling on the host, same prompts	19.88 tokens/s	measured	stages/7/benchmark/speed.json
decode speed, nearest model served the plain way, same prompts	16.73 tokens/s	measured	stages/7/benchmark/speed.json
time to first token, this package, median of four prompts	0.19 s	measured	stages/7/benchmark/speed.json
time to first token, same model with the drafter off, same prompts	0.32 s	measured	stages/7/benchmark/speed.json
GSM8K score (first 300 items), same model with the drafter off	0.9467 fraction	measured	stages/7/benchmark/graft-comparison.json
GSM8K paired sign test p, this package against the drafter-off run (first 300 items)	1.0 p-value	measured	stages/7/benchmark/graft-comparison.json
IFEval score, same model with the drafter off (8192-token first pass, empty replies asked again at 32768)	0.8484 fraction	measured	stages/7/benchmark/graft-comparison.json
IFEval paired sign test p, this package against the drafter-off run	0.009683 p-value	measured	stages/7/benchmark/graft-comparison.json

Speed and benchmark conditions

The speed rows come from one request at a time, greedy decoding, three repetitions of each of four short prompts, on the same 2-chip bundle. The same prompts ran against BAAI/AREX-2 with the drafter off and sampling on the host (an earlier configuration of this package) and against Qwen/Qwen3.8-27B served that way. The per-prompt values are in evidence/stages/7/benchmark/speed.json. The prompts are short. Longer contexts and several requests at once were not measured.

The accuracy rows have different conditions, and the differences change what they show:

• Both models ran with thinking on, as the chat template sets it, at temperature 0.
• GSM8K covers the first 300 items only, 5-shot, with the flexible-extract filter and a 32,768-token limit. The harness's default stop string Question: cuts a reasoning model off when it repeats that word while thinking, so these runs stop only on </s>. The nearest model and the drafter-off run were scored on the same 300 items. A full-set run of this package is not in the table yet.
• IFEval is the prompt-level strict score over all 541 prompts. This package ran every prompt at a 32,768-token limit. The nearest model and the drafter-off run used an 8,192-token limit, and only their empty replies were asked again at 32,768. About 20 more of their replies were cut off at the lower limit and stayed scored as they were. The IFEval difference between this package and the drafter-off run may come from that limit and not from the drafter.
• On GSM8K this package and the nearest model cannot be told apart on these 300 items. On IFEval the paired difference against the nearest model is not clear either.
• None of the authors' reported benchmarks (research, agent and coding tasks) was run here. Tool calling and long-running agent use were not tested.

Reported by the authors

These scores come from the BAAI/AREX-2 model card on Hugging Face (revision d4e3502f92d9e889031c2d04e387ab2eb3520268), which says the results follow the protocols in the AREX-2 paper. They were not measured here: the authors' protocol, tools and judge differ from the checks in the table above, and nothing in this repository reproduces them.

Benchmark	Reported	Note
Frontier-CS (188-task Agent Track)	70.7
MLE-Lite	81.8	Any Medal, averaged over three seeds
BrowseComp	84.0
HLE	52.6	text-only subset
GAIA	92.2
DeepSearchQA	93.8

Limitations

• Tested on 2 chips (mesh P150x2) only; other configurations are not claimed by this bundle.
• Accuracy was compared with the CPU reference on one short fixed prompt only.
• The drafter's MTP head is the nearest model's (Qwen/Qwen3.8-27B), added to BAAI/AREX-2 at start. How often the drafter's guesses are accepted on this model was not measured.
• The stage 2 and stage 4 numbers were measured without the graft (the drafter off and sampling on the host). The stage 7 rows were measured with this package's settings.
• Default sampling settings are those of Qwen/Qwen3.8-27B, because BAAI/AREX-2 ships no generation_config.json. Send your own settings or use those on the model's card.

Risks and safety considerations

The check above does not cover long outputs, tool calling or safety behaviour. Output can differ from BAAI/AREX-2 run on a CPU or GPU in ways it does not show.

Not measured

• the drafter's acceptance rate on this model
• a download of the weights through tt-model pull, and a boot of the package from the Hub

Boot check

Stage 7 of the run installed this bundle, served it on a leased board and compared its tokens with the CPU reference. The Expected performance table has the result.

How to serve

tt-model serve episod/arex-2-p300