Laguna-S-2.1-Chadrock-ROCmFP4-StrixKVSpine-V4-GGUF

High-performance tuned Laguna S 2.1 Chadrock ROCmFP4 quant with custom ROCmFP4 tensor types + Laguna architecture support, optimized for AMD Strix Halo / Ryzen AI Max+ 395 systems.

⚠️ Important Engine Requirement

This quant uses custom ROCmFP4 tensor formats + Laguna architecture support. It does NOT load in stock upstream llama.cpp. You MUST use the specialized Ciru ROCmFPX Runtime V3.

Repository: github.com/ciru-ai/ROCmFPX
Branch: agent/laguna-s21-runtime-v3
Pinned Commit: 54f5fe06c74350fb8b6aec21d8749071bc195bdb

🖥️ System Requirements & Specs

Target Hardware: AMD Ryzen AI Max+ 395 / Radeon 8060S (Strix Halo gfx1151) with 128 GB Unified Memory.
Backend: Vulkan (Vulkan0 via Mesa RADV driver).
Max Stable Context: 131,072 tokens (131K).
Model File: laguna-s-2.1-ROCmFP4-StrixKVSpine-v4.gguf (65.4 GB).
KV Cache Precision: F16 / F16.

⚡ How to Run (Validated Strix Halo Profile)

1. Clone & Pin Runtime

git clone --branch agent/laguna-s21-runtime-v3 --depth 1 https://github.com/ciru-ai/ROCmFPX.git
cd ROCmFPX
git checkout --detach 54f5fe06c74350fb8b6aec21d8749071bc195bdb

2. Install Vulkan Deps & Build

scripts/install-laguna-vulkan-deps.sh --install
JOBS=8 BUILD_TYPE=Release scripts/build-laguna-strix-vulkan.sh

3. Launch Supervised Endpoint (Recommended)

HOST=0.0.0.0 PORT=8089 scripts/run-laguna-vulkan-supervised.sh /path/to/laguna-s-2.1-ROCmFP4-StrixKVSpine-v4.gguf

This applies driver preflight, DeviceLost safeguards, safe 131K context profile, Flash Attention, and F16 KV cache.

🌐 Open WebUI / PrivChat Integration

Connect Open WebUI (Settings → Connections → OpenAI API):

OpenAI API Base URL:
- Docker: http://host.docker.internal:8089/v1 or http://172.18.0.1:8089/v1
- Host / Browser: http://127.0.0.1:8089/v1
API Key: sk-no-key-required (or any string)
Model Name: laguna-s21-rocmfp4-strixkvspine-v4

📊 Benchmark Summary

Tested on AMD Ryzen AI Max+ 395 / Radeon 8060S with 128 GB LPDDR5X-8533:

Metric	Result
Generation Speed (Decode)	33.1 – 38.3 t/s
Prompt Eval Speed (Prefill)	79.3 – 92.7 t/s
Time to First Token (TTFT)	~560 – 618 ms
Validated Max Context	131,072 tokens (131K)
Default Thinking	Off

Downloads last month: -; Downloads are not tracked for this model. How to track