D3-Lite-NPU2

vllm-sr/d3-lite v3.1.3 (Decision 3.0, the decision model of vLLM Semantic Router) for AMD Ryzen AI NPUs, served by FastFlowLM 1.0.10+. It answers System One requests -- choice, yes / no and score questions about a state, with a probability for every option -- over the same POST /v1/systemone API as the upstream package.

flm serve --decision d3:lite          # optionally next to a chat model: flm serve qwen3.5:4b --decision d3:lite
curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" \
  -d '{"state": "The order arrived damaged.", "questions": {"route": {"type": "choice",
       "criteria": {"returns": "Refunds and replacements", "billing": "Payments", "general": null}}}}'

What was changed

Weights: the Qwen3.5 text tower quantized to FastFlowLM's Q4NX format: every projection Q4_K with GPTQ-calibrated rounding (calibration: 300 Decision Index prompts, none of them in the evaluation sets below), token embedding Q8_0, norms and the Gated DeltaNet parameters unquantized. vision_weight.q4nx holds the vision tower for a later FastFlowLM release; FastFlowLM 1.0.10 answers text requests.
Unchanged: readout.safetensors, decision_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja and LICENSE are the upstream files byte for byte (MANIFEST.sha256).
Same contract: prompts, token ids and answer objects are byte-identical to the upstream runtime (checked on every FastFlowLM build against 1,308 upstream-rendered requests).

Accuracy (against the upstream BF16 model)

	this package on the NPU	upstream BF16
Decision Index public-suite subset, gold accuracy (6,824 questions, 35 benchmarks)	60.6%	61.2%
Top-1 agreement with BF16 on the same questions	90.5%	--

Paired gold accuracy, upstream BF16 minus this package: +0.59 pt (95% CI [-0.00, +1.18] pt).

Speed (AMD Ryzen AI MAX+ PRO 395 (Strix Halo, XDNA 2 NPU), 128 GB, Ubuntu 24.04, XRT)

request	median latency
1 question, 128 tokens	112 ms
1 question, 330 tokens	159 ms
1 question, 1,024 tokens	518 ms
1 question, 4,096 tokens	3652 ms
4 questions, 330 tokens each	574 ms
16 questions, 330 tokens each	2198 ms