OsaurusAI/Raptor-0.6-4B-JANG_6M

Raptor 0.6 — a lightly tuned Spark-X2.5-4B, quantized to 3.41 GiB for Apple Silicon and framed for the Osaurus harness.

The release write-up — what happened to Raptor 0.5, why the base changed, and how 0.6 was built and measured — is at osaurus.ai/models/raptor-06; the announcement is at osaurus.ai/blog/raptor-06.

Raptor 0.6 is a supervised fine-tune of XHToken/Spark-X2.5-4B, a 4.112B dense reasoning and tool-use model. The tune is deliberately light: it touches attention only — the fused q_k_v_proj and out_proj of all 36 layers — and leaves the MLP, the tied embedding, every norm and the per-head attention gate byte-identical to the base. Tokenizer and chat template are unchanged from the base as well.

This is a JANG_6M bundle: attention and the tied embedding at 8-bit, the MLP carrying the 6-bit budget, every group scale in bfloat16, and the per-head gate and all norms kept in full precision.

This is a different backbone from Raptor v0.5 and from the withdrawn 0.6 preview. Raptor 0.6 runs on the spark2_5 architecture. Raptor v0.5 was a Ling-3.0-tiny hybrid MoE and the withdrawn 0.6 preview was a Nanbeige looped transformer; none of their runtime, parser or cache assumptions carry over to this bundle.

Quality — measured, not estimated

15,839 teacher-forced positions on held-out prompts, scored against the model's own bf16 weights. Calibration and evaluation prompts come from disjoint corpora, asserted at eval time.

	Raptor-0.6-4B-JANG_6M	base Spark-X2.5-4B-JANG_6M
Size on disk	3.41 GiB	3.41 GiB
bits/weight	7.126	7.126
median KL ↓	0.0054	0.0048
mean KL ↓	0.0132	0.0143
top-1 agreement ↑	97.67%	97.69%
decode	104.8 tok/s	104.7 tok/s
prefill	5042 tok/s	5046 tok/s

Decode is the median of 4 probes at a fixed condition (512-token prompt, 128 generated), first probe discarded, on an M5 Max; spread across the retained probes was 0.6%.

The margin-conditioned flip curve decreases monotonically across the first five bins (13.68% → 0.00%), then reads 0.04% in the final [8, ∞) bin. That final tick is one token out of 2,611 and is reported here rather than smoothed away; at that count it is sampling noise, not a structural defect.

There is no stock-MLX comparison row, and that is not an omission. mlx-lm has no spark2_5 architecture — not in any release, not on main — so no stock MLX quantization of this model exists to score against. The MLX runtime ships with the Osaurus harness.

Calibration

1,759,685 calibration tokens across code, agentic and tool-call transcripts, academic multiple-choice, general chat, Chinese, long-context, science and security prompts; 7.3% of tokens self-generated. One capture drives activation-aware scaling, per-channel importance and the error-corrected weight fit together, covering all 36 layers.

The base model's calibration was not reused. The tune moved attention, so the activation statistics moved with it; this bundle was captured against its own weights.

Applied: activation-aware scaling on 72/72 norm fold sites and 36/36 attention gate projections, error-corrected fitting on 180/181 tensors. The single holdout is the tied embedding — a lookup with no input activation to condition on, so it is fit directly.

Running it in Osaurus

Requires Osaurus 0.25.0 or newer — that is the first release whose engine carries the spark2_5 runtime. osaurus.json in this repo declares the requirement so the harness can tell you before it tries to load.

Point Osaurus at the repo and it serves with the sampling, reasoning and tool-call contract below already declared in the bundle: jang_config.json carries the parser names, dialect and delimiters, so the harness wires itself with no per-model configuration.

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("OsaurusAI/Raptor-0.6-4B-JANG_6M")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is 84 * 3 / 2?"}],
    add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=1.0, top_p=0.95)))

A bare mlx_lm.load will not resolve spark2_5 on its own — the runtime ships with the harness.

On Windows, Osaurus (now in beta at osaurus.ai/windows) loads GGUF rather than MLX, and its runtime carries spark2_5 — use the GGUF sibling below there. For llama.cpp-family runtimes, and for Osaurus for Windows, there is a GGUF sibling built from the same merged bf16 checkpoint: OsaurusAI/Raptor-0.6-4B-GGUF (Q6_K and Q4_K_M, importance-matrix calibrated). Osaurus runs this MLX bundle, not the GGUF.

Serving contract

• Sampling: temperature=1.0, top_p=0.95, top_k=-1 (no truncation, full-vocab top-p), no repetition penalty. Vendor defaults, carried verbatim in both generation_config.json and jang_config.json.
• Reasoning is ON by default. The template force-opens the rail: the generation prompt ends with <|Bot|><think>, and with enable_thinking=false it ends <|Bot|></think>. A reasoning parser that waits for a literal <think> in the output stream will never see one, because the opener is in the prompt. Reasoning off is a supported, tested lane.
• Give reasoning room. Reasoning traces on hard prompts can exceed 8,192 output tokens. Budget for real input plus output inside the context window rather than silently truncating history.
• Tools render into the first system block (## Tools plus a <tools> JSON list). Adding or removing a tool rewrites the system prefix, so a tool-set change is a full re-prefill, not a suffix append — any prefix-cache reuse accounting must treat it that way.
• Tool call shape, no newlines: <tool_call>NAME<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>. Tool results go in a <|Tool|> block; consecutive tool messages are merged.
• generation_config.json carries the base model's nonstandard max_tokens key and no max_new_tokens. HF generate() ignores it. Left verbatim on purpose — pass your own cap.

Known limits — please read before deploying

Raptor 0.6 is a practical checkpoint. It was selected under an explicit tolerance for ordinary 4B-model mistakes. It is not a zero-failure model, and the following are measured or documented, not hypothetical.

Outside a harness, keep a system prompt. Osaurus always sends one that describes the available tools, and reasoning is on by default. If you serve the model elsewhere with reasoning off and no system prompt at all, a tool offered mid-conversation can occasionally get a written answer instead of a call; a one-line instruction to call available tools keeps it reliable. This was measured on both the tuned weights and the untuned base, pre-exists in the base, and is not quantization damage — this bundle tracks its own bf16 source within measurement noise.

Also carried over from the checkpoint that produced these weights:

• Long-context recall is not guaranteed. A 64K-token retrieval prompt was answered correctly on a clarified retry, both with reasoning on and off, but an earlier long prompt failed to answer at all and instead copied and re-examined the source text. Allow clarification and retry rather than promising reliable completion of every long prompt.
• Ordinary agentic mistakes remain: stale or wrong tool names, schema and JSON errors, unsupported success claims, invented identifiers, dependent calls issued before their prerequisite, and arithmetic or recall slips. Do not execute generated mutations without your harness's normal approval step.
• Identity responses are not enforced. "Raptor" names the tuned artifact; the model will often still describe itself as Spark. Quality was preferred over persona enforcement.

Provenance

Quantized by Jinho Jang (eric@osaurus.ai). Base model XHToken/Spark-X2.5-4B, apache-2.0. Iteration 1 — see osaurus.json (model_version) for the bundle iteration and required_osaurus_version for the minimum runtime. First published on 2026-09-10 as Raptor-0.6.1-preview-JANG_6M and renamed to Raptor-0.6-4B-JANG_6M for the release; the weights are unchanged (model_version 1) and the old name redirects here.

License

Raptor 0.6 is a derivative of XHToken/Spark-X2.5-4B by the SparkLLM Team, which is released under the Apache License, Version 2.0. The full license text is in the LICENSE file in this repository, carried over unchanged from the base model as the license requires.

Raptor 0.6 is distributed under the same Apache License, Version 2.0. In accordance with Section 4 of that license, the changes made to the original work are: (1) a supervised fine-tune of the attention projections (q_k_v_proj and out_proj, rank-4 LoRA merged into the weights; all other tensors unchanged), and (2) quantization of the merged weights to the JANG_6M format (affine 6-bit MLP, 8-bit attention and embedding, group size 64) with AWQ, GPTQ and importance-matrix calibration. The tokenizer, chat template and generation defaults are unchanged from the base model. Osaurus, Raptor and the Osaurus banner are trademarks of Osaurus and are not covered by the Apache license.

If you use this model, please also credit the base model:

@misc{sparkx2.5,
    title  = {Spark-X2.5 4B&1.7B: Pushing the Limits of Agentic Capabilities in On-Device Models},
    author = {SparkLLM Team},
    year   = {2026}
}