fafstmobel

A locally built 27B derivative of the Swift abliterated Qwen3.8-27B checkpoint: Huihui's observed abliteration delta transplanted onto Swift, quantized to an NVFP4/FP8 text allocation, and exported to a single NInfer v3 artifact with the published DFlash2 draft component bundled alongside vision and MTP.

Built end to end on one workstation (RTX 5090, CUDA 13.1, Python 3.11) by a reproducible pipeline. Every number below was measured on that machine.

Artifact

File	fafstmobel.ninfer — 23,719,715,844 bytes
SHA-256	70752ce85422f9d716438f85e80e6b68c496197c41c2aed32b126f1c23ce7364
Container	NInfer v3, 1246 objects / 1240 tensors / 6 resources / 1513 bindings / 844 uses, 1 file
Components	text, vision, mtp, dflash2, plus the indexed proposal head
Text model	64 layers, hidden 5120, vocab 248320, 16 full-attention + 48 linear-attention
Draft	DFlash2, 5 layers, target taps [5,19,33,47,61], block 8, selector top-k 16
Metadata name	fafstmobel

What was changed relative to upstream

1. Delta transplant onto Swift (scripts/abliterate.py). Huihui's abliteration was not re-derived; the base→Huihui weight delta was applied to Swift: result = (swift.float() + (huihui.float() - base.float())).to(bfloat16). Exactly 70 tensors changed, in language-model layers 17–51: mlp.down_proj plus self_attn.o_proj (full-attention layers) or linear_attn.out_proj (linear-attention layers). Max |Δ| = 2.83e-2, mean |Δ| = 2.02e-4, 4,220,518,400 elements rewritten. The other 1129 tensors — all vision, MTP, embeddings, norms and the output head — are byte-identical to the Swift checkpoint.
2. Text quantization to the published NVFP4/FP8 allocation: 168 NVFP4 matrices (MLP gate/up/down, group size 16, 4-bit weights, local-dynamic 4-bit activations with calibrated input scales) and 233 FP8 per-channel matrices (attention q/k/v/o, GDN in/out projections, output head, MLP layers 56–63). 32 UltraChat-200k calibration samples × 2048 tokens. Vision, MTP and GDN gate projections stay unquantized.
3. Export to NInfer v3 with the official tools.convert (NInfer revision 98dada0e03cb073fd07f905400b5904bc6e82759) and the unmodified qwen3_8_27b_nvfp4 recipe, including the maintained Qwen3.8 chat template.
4. Naming normalization of the text intermediate (model.language_model.* → model.*) so the checkpoint is consistent with its flattened text-only config; only safetensors headers were rewritten, verified against per-shard payload digests.

Measured results (RTX 5090, 32 GB; context 16,384; fp8 KV; CUDA graphs on)

Check	Result
Arithmetic (17 × 19)	323; same-seed replay identical
Code prompt	valid slicing def, 30 tokens
NInfer DFlash2 integration test	exit 0
Serving	/health ok, /v1/models id fafstmobel, tools.smoke.serve_contract v2 pass
Multimodal	256×256 red/blue image → "left is red, right is blue"
Speculation (seed 42, greedy, 256 tokens, batch 1)	autoregressive 68.5–77.1 tok/s · MTP 210.0–215.3 tok/s (2.79–3.14×, acceptance 75.9%) · DFlash2 304.1–312.9 tok/s (4.06–4.57×, acceptance 54.2%)

Autoregressive throughput varies with machine state across runs; both speculation acceptance rates reproduced exactly in every run.

Usage

ninfer-serve fafstmobel.ninfer --host 127.0.0.1 --port 8088 --model-id fafstmobel \
  --spec dflash2 --draft-tokens 7 --lm-head-draft --vision \
  --kv-dtype fp8 --max-context 16384 --kv-capacity 16384 --max-concurrency 1

ninfer fafstmobel.ninfer --prompt "What is 17 multiplied by 19?" --greedy --max-new 32 \
  --no-thinking --vision --kv-dtype fp8 --max-context 16384 --spec dflash2 --draft-tokens 7

Licensing and notices

This artifact contains three upstream contributions, each retained under its own terms:

• Swift contribution (ukisai/Swift-Qwen3.8-27b) — Swift Open License v1.0, bundled as LICENSE.swift. That licence grants reproduction and distribution of derivative works, including converted weights, subject to its conditions: recipients receive a copy of the licence, modified files carry prominent change notices, attribution notices are retained, and Commercial Use is not licensed for a Legal Entity exceeding the one-million-USD Threshold (Section 5). A separate Swift Enterprise License is required above that threshold. This derivative is distributed under the same terms for the Swift contribution.
• Base model (Qwen/Qwen3.8-27B) — Apache-2.0, bundled as LICENSE-APACHE-2.0. Qwen3.8-27B portions remain under the Apache License 2.0.
• Huihui abliteration delta (huihui-ai/Huihui-Qwen3.8-27B-abliterated) — Apache-2.0, bundled as LICENSE.huihui. Only the observed weight difference was transplanted.
• DFlash2 draft (z-lab/Qwen3.8-27B-DFlash2) — Apache-2.0; bundled by conversion, unmodified and unquantized in its file.
• NInfer (https://github.com/Neroued/ninfer, Apache-2.0) — the converter and runtime used.

NOTICE carries the upstream notice plus this derivative's transformation notice; PROVENANCE.md records the exact repository revisions.

Claim boundaries

• This is a delta transplant, not a newly fitted refusal direction, and it is not proof that all refusals are removed. Behavioural claims about refusal removal are not made here.
• The DFlash2 draft is pretrained upstream and bundled by conversion; no drafter was trained, and Swift was not retrained.
• The quantization is a reproduction of the published allocation, not a bit-exact replication of any other derivative; no other derivative's weights were downloaded or compared.
• Not suitable for safety-critical use. Outputs are unmoderated model text.