maxgam/Qwopus3.8-27B-Flash-V2-ROCmFPX-GGUF

Qwopus3.8-27B-Flash-V2 (ROCmFPX & ROCmI4 GGUF Quants)

Hardware-optimized, sub-byte GGUF quantizations of Jackrong/Qwopus3.8-27B-Flash-V2 targeting AMD Strix Halo (Radeon 8060S / gfx1151) and RDNA 3.5 unified memory architectures.

These builds leverage custom ROCmFPX sub-byte kernel layouts to bypass standard dequantization overhead, saturating Strix Halo's 256-bit memory bus for autonomous coding agent loops (Kilo Code, Claude Code, Cline, Roo Code).

---

Architecture & Post-Training Lineage

• Foundation Architecture: Qwen 3.8 27B Coder.
• Base Checkpoint: Jackrong/Qwopus3.8-27B-Flash-V2
• *Core Philosophy — "Preserve Capability, Collapse Cost":*
  • In iterative multi-turn agent workloads, wall-clock latency compounds rapidly. A 5-second delay per turn adds over 4 minutes of idle waiting across a 50-turn debugging loop.
  • Flash-V2 Post-Training: Built on a revised reward and reinforcement learning pipeline engineered to prune unproductive chain-of-thought bloat, eliminate pathological reasoning loops, and prevent indentation/AST drift during file edits.
  • Bundled NextN/MTP Draft Head: Includes native Multi-Token Prediction layers trained directly into the weights, enabling zero-memory-overhead speculative decoding with ~75%–88% draft acceptance on structured code.
• Hardware Quantization: Quantized natively with AVX-512 vector pipelines targeting Strix Halo compute units via Heretek-AI and charlie12345/ROCmFPX toolchains.

---

Quantization Variants

Filename	Type ID	BPW	Size	Silicon Target & Trade-offs
Qwopus3.8-27B-Flash-V2-ROCmI4.gguf	118	4.36	~13.86 GB	Peak Throughput (Integer WMMA): Native signed-nibble 4-bit integer weights. Dispatches directly to v_wmma_i32_16x16x16_iu4 instructions with zero codebook lookups, delivering peak tokens/sec on Strix Halo's 256-bit LPDDR5X bus.
Qwopus3.8-27B-Flash-V2-ROCmFP4-STRIX_LEAN.gguf	106	4.34	~13.81 GB	Balanced Agent Adherence (Micro-scaled FP4): Combines a fast FP4 feedforward body with preserved Q5_K token embeddings and attention K/V routing to prevent schema drift during deep monorepo tool-calling.

⚠️ Runtime Engine Requirement:
These models utilize specialized tensor formats (Q4_0_ROCMI4 and Q4_0_ROCMFP4_STRIX_LEAN). They will not load inside vanilla llama.cpp, Ollama, or LM Studio releases. You must use a ROCmFPX-enabled server binary compiled for gfx1151, such as Heretek-AI/ROCmFPX-BUILDER or charlie12345/ROCmFPX.

---

Performance & Hardware Benchmarks (Strix Halo)

Measurements gathered on AMD Strix Halo (Radeon 8060S, 40 CUs, 128GB LPDDR5X-8533 unified memory @ ~273 GB/s):

• Decode Speed (Base Non-Speculative): ~28–34 tok/s.
• Decode Speed with MTP Speculation (--spec-type draft-mtp): 48–60+ tok/s effective throughput.
• MTP Draft Acceptance Rate: ~80–88% on structured programming tasks (Go, TypeScript, Python, and JSON schema emission).
• VRAM Footprint: ~14.2 GB model weights + ~8.1 GB KV Cache (64k context @ Q8_0). Fits comfortably with massive headroom for deep context windows.

---

Quickstart: Running on AMD Strix Halo (Windows PowerShell)

Ensure your server environment targets the gfx1151 driver dispatch before initializing:

# 1. Force ROCm driver dispatch to gfx1151
$env:HSA_OVERRIDE_GFX_VERSION = "11.5.1"

# 2. Launch the hardware-accelerated server
.\llama-server.exe `
  -m "Qwopus3.8-27B-Flash-V2-ROCmI4.gguf" `
  --alias "qwopus-v2-rocmi4" `
  -dev ROCm0 `
  --n-gpu-layers 999 `
  --threads 16 `
  --batch-size 1024 `
  --ubatch-size 512 `
  --ctx-size 65536 `
  --flash-attn on `
  --cache-type-k q8_0 `
  --cache-type-v q8_0 `
  --spec-type draft-mtp `
  --spec-mtp-strict-qwen `
  --spec-draft-ngl 999 `
  --spec-draft-n-max 3 `
  --temp 0.20 `
  --top-p 0.95 `
  --min-p 0.05 `
  --jinja `
  --port 8080

To run the FP4 Strix Lean variant, simply replace the -m flag with "Qwopus3.8-27B-Flash-V2-ROCmFP4-STRIX_LEAN.gguf".

---

Agent Integration & Best Practices (Kilo Code / Claude Code / Cline)

1. Sampling Parameters

Flash-V2 features condensed reasoning heads and is post-trained for deterministic code generation:

• Temperature: 0.15 – 0.25 (High temperatures cause JSON schema drift in sub-byte quants).
• Min-P: 0.05
• Top-P: 0.95
• Presence / Frequency Penalty: 0.0

2. KV Cache Quality Over Memory Savings

• Always use --cache-type-k q8_0 and --cache-type-v q8_0.
• Do not down-quantize KV cache to q4_0 on Strix Halo. On a 128GB unified memory pool, saving 4 GB of VRAM at the expense of needle-in-a-haystack attention precision will degrade cross-file symbol resolution in large monorepos.

3. MTP Speculative Decoding Tuning

• --spec-type draft-mtp utilizes the embedded 1-layer NextN head directly inside the weights.
• If your coding agent enters an edit loop or repeatedly fails diff application on complex multi-chunk files, drop --spec-draft-n-max from 3 to 2.

4. Diff & Tool Calling Schema

• Qwopus3.8-Flash-V2 natively adheres to standard Search/Replace diff blocks (<<<<<<< SEARCH, =======, >>>>>>> REPLACE) and <tool_call> JSON blocks.
• Ensure your agent system prompt explicitly instructs the model to output strict unified diffs or exact search/replace blocks with a minimum of 3 lines of surrounding context.

---

Credits & Acknowledgements

• Base Model Author: Jackrong for the Qwopus architecture and Flash-V2 post-training pipeline.
• ROCmFPX Kernel & Tooling: charlie12345 and Heretek-AI for the Strix Halo WMMA and FP4/I4 compiler implementations.
• Quantization, Benchmarking & Packaging: Built and maintained by Maxim Gamurar (@maxgam).