quest-35b-rl on Tenstorrent p300x2 and p150x4 and p300 and p150x2 and p150 and p100

Derived from osunlp/QUEST-35B-RL by OSU NLP (a fine-tune of Qwen3.5-35B-A3B). The checkpoint is used as published; expert weights are converted to lower precision when the server loads them (see Limitations). See the original model card for training and evaluation details.

QUEST-35B-RL, a deep-research agent model: Qwen3.5-35B-A3B (35B-parameter mixture of experts, 3B active per token: 256 experts, top-8 plus a shared expert; 40 layers mixing Gated DeltaNet linear attention with gated full attention; vision encoder) after mid-training, SFT and RL on synthetic research tasks. Served with vLLM: a reasoning model that thinks before it answers by default, with text and image input (text only on one die), tool calling and a 262,144-token context, for up to 32 concurrent users.

Runs on Tenstorrent p300x2 or p150x4 or p300 or p150x2 or p150 or p100 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p100, p150, p150x2, p150x4, p300, p300x2
Context	262,144 tokens
License	apache-2.0

Intended use

Search-and-browse research agents and tool use, plus chat and reasoning over text and images, through an OpenAI-compatible API.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p300x2 or p150x4 or p300 or p150x2 or p150 or p100

Quickstart

tt serve raahemnabeel/quest-35b-rl

tt serve downloads the Docker image and the osunlp/QUEST-35B-RL at cfe1bd34132816e76bbbd338cddfcef8ba289c1c weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve raahemnabeel/quest-35b-rl.

Serving profiles

Select a different serving profile using --profile.

profile	hardware	mesh	max_num_seqs	max_model_len	description
p300x2	p300x2	P300x2	32	262144	Fastest profile: four dies, text and images.
p150x4	p150x4	P150x4	32	262144	p300x2 on four p150 boards.
p300 (default)	p300	P300	32	262144	Default: two dies, text and images, at about 60 % of the four-die single-user speed.
p150x2	p150x2	P150x2	32	262144	p300 on two p150 boards.
p150	p150	P150	32	262144	One die: text only, routed experts at bfp4.
p100	p100	P150	32	262144	p150 on one p100.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "osunlp/QUEST-35B-RL" — the weights id, not this package's name.

Tool calling is enabled (--tool-call-parser qwen3_xml): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.

Reasoning output is separated (--reasoning_parser qwen3): the thinking text arrives in reasoning, apart from content.

Thinking is on by default. Give thinking requests a large max_tokens (thousands): when the budget runs out inside the reasoning, content is empty. Turn thinking off per request with "chat_template_kwargs": {"enable_thinking": false}.

The checkpoint ships no generation config; a request without sampling parameters uses the base model's thinking defaults (temperature 1.0, top_p 0.95, top_k 20). Sampling runs on the device; top_k is limited to 32.

Tools: pass them as OpenAI tools; the model emits Qwen XML tool calls, parsed into tool_calls by the server. QUEST was trained to drive search and page-fetch tools; the package serves the model only, bring your own tools.

Images: up to 4 per request as image_url parts, resized to at most 4 MP (two- and four-die profiles). Video input is not accepted.

Expected performance

Accuracy

Pending. OSU NLP's own scores (BrowseComp, GAIA, HLE with tools and the other deep-research benchmarks) are on the original model card; measurements of this package on those benchmarks will be added here.

Performance

Served as shipped (tt-model serve --profile <profile>), vLLM's random-token benchmark with ignore_eos, medians; one request per user at a time.

p300x2 (four dies)

Single user on real text (12 chat prompts x 384 output tokens, temperature 0): 216 tok/s.

users	input / output tokens	TTFT (ms)	TPOT (ms)	tok/s per user	output tok/s, all users
1	128 / 128	121	4.6	217	181
1	1,024 / 256	233	4.7	214	179
1	4,096 / 256	656	4.8	208	141
1	32,768 / 256	2,719	5.2	191	63
8	128 / 128	1,165	9.7	103	425
8	1,024 / 256	1,998	10.0	100	451
32	1,024 / 256	7,402	26.6	38	577

p300 (two dies)

Single user on real text (12 chat prompts x 384 output tokens, temperature 0): 126 tok/s.

users	input / output tokens	TTFT (ms)	TPOT (ms)	tok/s per user	output tok/s, all users
1	128 / 128	134	7.9	126	112
1	1,024 / 256	275	8.1	124	110
1	4,096 / 256	821	8.2	122	91
1	32,768 / 256	3,716	8.6	116	43
8	128 / 128	1,242	18.3	55	288
8	1,024 / 256	2,337	18.5	54	291
32	1,024 / 256	8,768	45.3	22	406

p150 (one die)

Single user on real text (12 chat prompts x 384 output tokens, temperature 0): 45 tok/s.

users	input / output tokens	TTFT (ms)	TPOT (ms)	tok/s per user	output tok/s, all users
1	128 / 128	162	22.2	45	43
1	1,024 / 256	371	22.3	45	42
1	4,096 / 256	1,224	22.5	44	38
1	32,768 / 256	6,253	23.2	43	21
8	128 / 128	1,428	42.1	24	152
8	1,024 / 256	3,079	42.6	23	147
32	1,024 / 256	11,768	104.5	10	214

Limitations

• The checkpoint has no MTP head: decoding produces one token per step on every profile.
• Expert weights run at reduced precision (bfp8 routed experts on two and four dies, bfp4 on one die), so output can differ from the bf16 original where two candidate tokens are close.
• One die (p150, p100): text only, no prefix caching.
• The p150x4, p150x2 and p100 profiles run the same meshes as p300x2 / p300 / p150 but were not run on those boards.
• Prefix caching resumes only at 2,048-token boundaries; requests with images are not cached.
• The device sampler supports top_k up to 32; a larger top_k is clamped to 32.
• Not for unattended high-stakes decisions (medical, legal, financial, safety-critical) or as a source of facts without checking: the model can state wrong things confidently. Review generated code and tool calls, or run them in a sandbox, before they act on real systems. No safety evaluation was run on this package beyond the original model's.

Related packages

• raahemnabeel/ornith-1.5-35b-a3b: Ornith-1.5-35B-A3B, another Qwen3.5-35B-A3B fine-tune served by the same code.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/raahemnabeel/quest-35b-rl/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	ecb84e2dc47435b7f7b33d47bf0ae00fcaed411f (dirty tree — the image includes uncommitted changes)
vLLM	v0.26.0
vllm-tt-plugin	a local checkout — commit not published
code/ digest	2cea76f9d057d3cb (sha256, first 16 hex digits)
built	2026-11-03T18:47:04+00:00 by tt-model 0.1.0