kingjones777/Ling-3.0-flash-ROCmFPX-Q6-AGENT-GGUF

Ling-3.0-flash — Research Q6 AGENT (ftype 114)

This is the high-fidelity end of my research family — 117 GB, 8-bit heads, and the slowest of the three. It is not my serving build. If you just want to run the model, take Ling-3.0-flash-ROCmFP4-STRIX_LEAN-GGUF instead.

⚠️ "Research" here means the QUANT RECIPE, not a modified model

This is the same aligned inclusionAI/Ling-3.0-flash checkpoint as my serving build — same weights, same behavior, same alignment. Not uncensored, not ablated. What makes it "research" is how I quantized it: I follow my own research quant recipe rather than my serving recipe, so I can measure what each precision choice buys.

What kind of build this is

ftype 114 Q6_0_ROCMFPX_AGENT — a 6-bit ROCmFPX body with Q8_0_ROCMFPX (tensor type 103) heads and attention output. The AGENT routing keeps attention-output tensors at higher precision, which is the part that matters most for tool-calling and structured output.

⚠️ Don't confuse tensor type 103 with ftype 103. Tensor type Q8_0_ROCMFPX is 8-bit. Ftype 103 (FAST) is 4-bit. Same number, completely different things.

Where this sits in the family — all three are the same checkpoint, differing only in quantization:

build	bytes	output.weight	decode	use it for
serving 106 STRIX_LEAN	68,136,565,408	Q6_K	30.7 t/s	default — serve from this
research 106	68,020,249,248	Q4_0_ROCMFP4_FAST (4-bit)	32.0 t/s	measuring the cost of an unprotected head
this — research 114	117,608,292,512	Q8_0_ROCMFPX (8-bit)	25.8 t/s	fidelity ceiling / reference point

The honest trade: this costs 73 GB more and 4.9 t/s versus my serving build. I keep it as the upper-bound reference — when I want to know whether a 4-bit build lost something real, I compare against this, not against a guess. Unless you specifically need the fidelity ceiling or you're doing heavy tool-calling with memory to spare, the serving build is the better pick.

⚠️ Requires a ROCmFPX-capable llama.cpp build — these tensor types aren't in mainline, so stock llama.cpp / Ollama / LM Studio won't load it. ⚠️ At 117 GB this is tight even on a 128 GB Strix Halo box. Check your headroom before pulling it.

Receipts

bytes	117,608,292,512
sha256	7e8679475a7697dfeda2cd21d7e245f18737622099754f8aa2caab6652fb4a4b
general.file_type	114
arch / tensors / ctx	bailingmoe3 / 938 / 262144
blocks	43, nextn_predict_layers = 1 (MTP head at blk.42)
dry-run → built	112153.77 MiB (7.38 BPW) → +6.238 MiB
decode	25.8 t/s (prompt 78.7 t/s)
histogram	F32×390, type102×338, type103×210

Heads read back by exact tensor name — never a substring, since output.weight matches inside attn_output.weight:

tensor (exact name)	type
output.weight	Q8_0_ROCMFPX (103)
token_embd.weight	Q8_0_ROCMFPX (103)

I also dry-ran --output-tensor-type q6_K on this tier and rejected it: it would have demoted the head from 8-bit to 6-bit. The native Q8 heads are already better than the override, so I left them alone.

Source: inclusionAI/Ling-3.0-flash @ 42766a814ab117e75e2e61465d5e131b72d931a3 → 255,091,083,232-byte BF16 GGUF (938 tensors) → quantized.

llama-quantize \
  Ling-3.0-flash-I-BF16.gguf \
  Ling-3.0-flash-I-Research-Q6_0_ROCMFPX_AGENT.gguf \
  Q6_0_ROCMFPX_AGENT 16

Measured on my own hardware (gfx1151, Strix Halo). Every number is read back from the built file.