vasilee/span-02-gguf

span-02 GGUF - a frontier decision model you can run anywhere

respanai/span-02 is a frontier decision model: send it a state (text or JSON) plus typed questions and it answers with a calibrated probability per option - no free text, no parsing. It is a Gemma 4 26B-A4B fine-tune (~4B active parameters), #4 on the Decision Index with 62.6 full score at 33 ms median.

The upstream release only runs on NVIDIA Blackwell (RTX PRO 6000 / RTX 5090): the weights ship pre-quantized and are executed by a bundled Rust/CUDA engine with hand-written sm_120a kernels, so transformers, vLLM, Mac and CPU setups cannot load them at all. These GGUFs are the portable form of the same model: plain llama.cpp, one file for chat and for decisions, on Mac (Metal), CPU, CUDA, Vulkan and friends.

Files

file	size	layout	pick it when
span02-mxfp4.gguf	14.7 GB	MoE experts MXFP4 (4-bit e2m1, power-of-two block scales), rest Q8_0	smallest file, keeps the native 4-bit-experts / 8-bit-dense shape
span02-q4_k_m.gguf	16.8 GB	mixed Q4_K / Q6_K / Q8_0, BF16 embeddings	widest backend support, runs on every llama.cpp build

Both carry the decision-model metadata, so one file serves text generation and the TypeSafe /v1/systemone decision API (output_modalities: ["text", "decisions"]).

The upstream NVFP4/FP8 weights were dequantized exactly per the engine's math (FP8: w * scale[row]; NVFP4: e2m1 * fp8(scale) * gscale), converted with llama.cpp's Gemma 4 support, and re-quantized.

Measured results

Decision-suite benchmarks through the /v1/systemone endpoint: systemone mode, prefixed prompt style, 0-shot, whole scored split, 0 request errors.

benchmark	native engine (RTX PRO 6000)	span02-q4_k_m.gguf	span02-mxfp4.gguf
GPQA Diamond, accuracy (196 scored)	46.94 %	46.94 %	47.45 %
GPQA Diamond, chance-corrected skill	0.2925	0.2925	0.2993
GPQA Diamond, ECE (lower is better)	-	0.131	0.121
Home appliances, case exact (88 cases)	100 %	93.18 %	94.32 %
Home appliances, field accuracy (1,590 fields)	-	99.56 %	99.69 %
Home appliances, ECE (lower is better)	-	0.013	0.007

How to read it:

• Q4_K_M reproduces the native engine's Decision Index numbers on GPQA exactly (raw accuracy 0.4694, skill 0.2925).
• MXFP4 is one question ahead on GPQA and one case ahead on the home-appliance simulator. Neither gap is significant (two-proportion z-test, p = 0.92 and p = 0.76): the two quantizations are indistinguishable in accuracy.
• MXFP4 is the better calibrated of the two: lower ECE and Brier on both benchmarks, and confidence closer to accuracy.
• Both GGUFs sit about 6 pp below the native engine on the home-appliance suite, and they fail the same ambiguous cases (a request that is blocked or already satisfied, a device that is not in the household, a single-device pronoun). The gap follows the conversion path (no-think readout, llama.cpp prompt path), not the quantization.

Native-engine numbers are the Decision Index lab run on one RTX PRO 6000 with the author's own serving.

The original, for reference

one RTX PRO 6000, native Rust/CUDA engine
Decision Index full score	62.6 (rank 4)
Median latency	33 ms (p95 822 ms)
Throughput, 64 requests in flight	167 req/s without thinking, 150 req/s with auto
Requests answered	110,201 / 110,201

Original model card: respanai/span-02 - announcement: Introducing span-2.

Use

llama-server -m span02-mxfp4.gguf -c 8192 --repeat-penalty 1.1 --temp 0.8

Chat needs repeat_penalty ~1.1 and temperature ~0.8; this fine-tune loops without a repetition penalty.

Chat - /v1/chat/completions

curl localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "messages": [{"role": "user", "content": "tell me a joke in Romanian"}],
  "max_tokens": 300
}'

Decisions - /v1/systemone

Send a state (any JSON) plus typed questions (choice, noul, score); get back calibrated probabilities:

curl localhost:8080/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"customer": "My package arrived damaged and nobody answers my emails. I want a refund now."},
  "questions": {
    "dept":   {"type": "choice", "instructions": "Which team should handle this?",
               "criteria": {"returns": "Exchanges, wrong or damaged items",
                            "shipping": "Delivery status, delays, lost packages",
                            "billing": "Charges, invoices, payment problems"}},
    "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"},
    "anger":  {"type": "score", "instructions": "How frustrated is the customer?",
               "criteria": ["Not frustrated: calm or positive",
                            "Somewhat frustrated: mild annoyance",
                            "Very frustrated: angry, hostile or threatening to leave"]}
  }
}'

{"answers": {
   "dept":   {"type": "choice", "choice": "returns",
              "probabilities": {"returns": 0.957, "shipping": 0.029, "billing": 0.014}},
   "refund": {"type": "noul", "noul": 0.99997},
   "anger":  {"type": "score", "score": 1.98, "legend": {"0": ..., "1": ..., "2": ...}}}}

Checked against the native engine's golden tests: same choices, probabilities within ~0.01.

Limitations vs the native engine

• No-think readout only (no think-gating or thought blending), so close calls can differ.
• No number questions (the endpoint rejects them), no images (text-only weights), max 255 options.
• Chat quality is that of a decision fine-tune: fine for simple prompts, not base-model grade.
• Latency here is not comparable to the native engine's 33 ms median; that number is Blackwell + CUDA specific.

License

Weights derive from Google's Gemma 4 under the Gemma 4 license. See the original model card (Respan, 2026) for details and citation.