Instructions to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
llama cli -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
llama cli -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
./llama-cli -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
./build/bin/llama-cli -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Use Docker

docker model run hf.co/kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

LM Studio
Jan

vLLM

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Ollama
How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with Ollama:
```
ollama run hf.co/kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
```

Unsloth Studio

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF to start chatting

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with Docker Model Runner:
```
docker model run hf.co/kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
```

Lemonade

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Run and chat with the model

lemonade run user.Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF-Q4_0_ROCMFP

List all available models

lemonade list

Hermes Agent

How to use kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Run Hermes

hermes

Atomic Chat

Laguna-S-2.1 ROCmFP4 STRIX_LEAN (GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

First public-ready Laguna-S-2.1 ROCmFP4 "Strix Lean" quant for AMD Ryzen AI Max+ 395 (gfx1151 / Radeon 8060S).

⚠️ Not compatible with upstream llama.cpp. Requires the charlie12345/ROCmFPX fork built with HIP + ROCmFP4 kernels.

Files

File	Size	Notes
`Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf`	~58.34 GiB	4.26 BPW (quantize report)
`SHA256SUMS`

Base model: poolside/Laguna-S-2.1
F16 source: poolside/Laguna-S-2.1-GGUF laguna-s-2.1-F16.gguf (235202258240 bytes)

Hardware / stack (validated)

Box: Ryzen AI Max+ 395, gfx1151, 128 GB unified
ROCm 7.2.4 (/opt/rocm-7.2.4, clang 22)
Fork: charlie12345/ROCmFPX @ 3edc3d31ee5ebcea47fd7e0f42c89767bb4245db

Build recipe (HIP-only)

export PATH=/opt/rocm-7.2.4/bin:$PATH
export HIP_PATH=$(hipconfig -R)
export HIPCXX=$(hipconfig -l)/clang
cmake -B build \
  -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_ROCWMMA_FATTN=ON -DGGML_HIP_NO_VMM=ON -DGGML_HIP_MMQ_MFMA=ON \
  -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build -j 4   # prefer -j 4..8 on 128GB Strix; avoid -j32

Quantize

build/bin/llama-quantize \
  laguna-s-2.1-F16.gguf \
  Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN 8
# quant size = 59739.82 MiB (4.26 BPW)

Runtime (required)

export LD_LIBRARY_PATH=<build/bin>:/opt/rocm/lib
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1

llama-server --host 127.0.0.1 --port 8099 \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --model Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 \
  --temp 0.2 --top-p 0.95 --top-k 20 --min-p 0.05 --repeat-penalty 1.1 --repeat-last-n 256 \
  --chat-template-kwargs '{"enable_thinking":true}' --reasoning-budget 512

-dio is required for reliable cold load of large Laguna GGUFs on this stack (mmap path can hang).

A/B vs Q4_K_M (same binary, same flags, cold load)

Quant	prompt_n	prompt tok/s	decode tok/s (256-cap gen)	GGUF size
Q4_K_M	8116 (~8K)	445.6	17.36	71 GiB
ROCmFP4 STRIX_LEAN	8116 (~8K)	381.8	28.23	58.3 GiB
Q4_K_M	32223 (~32K)	372.4	14.15	71 GiB
ROCmFP4 STRIX_LEAN	32223 (~32K)	327.3	20.32	58.3 GiB

Decode speedup: +62.6% @ ~8K, +43.6% @ ~32K
Size: −18% (58.3 vs 71 GiB)
Prompt processing slightly slower on ROCmFP4 (−14% / −12%)

Quality / tools / reasoning

Shared prompts (reasoning, code, tool-call plan, short math with thinking):

Check	Q4_K_M	ROCmFP4	Verdict
Bat/ball $0.05	correct	correct	parity
is_palindrome plan	coherent thinking	coherent thinking	parity
get_weather Paris/Tokyo tool JSON	clean calls	clean calls	parity
enable_thinking / reasoning-budget	works	works	parity

Did we lose anything? No quality regression observed on the four shared prompts. We gain decode speed and lose model file size; prefill is slightly slower.

⚡ DFlash speculative decoding — +23% decode (added 2026-08-08)

DFlash works with this quant and is worth turning on: 32.6 → 40.1 tok/s (1.234×) on my box, same binary, same flags, only the speculation changed.

config	decode tok/s	vs off
off	32.6	1.000×
dflash `n-max 2`	39.6	1.215×
dflash `n-max 4`	40.1	1.234×
dflash `n-max 8`	25.2	0.773× — too deep, don't

Draft head: wimmmm/poolside-Laguna-S-2.1-DFlash-GGUF, Q8_0, 1.19 GB (the GGUF twin of poolside/Laguna-S-2.1-DFlash-NVFP4). Keep the head at Q8_0 — draft quality drives acceptance.

Two things will stop you cold. Both are fixable.

1. The fork commit pinned above is too old for DFlash

Building at 3edc3d3 (the commit in the recipe above) gives:

error loading model: done_getting_tensors: wrong number of tensors; expected 76, got 69

The 7 unmapped tensors are 6× blk.N.attn_gate.weight plus enc.aux_norm.weight. Current ROCmFPX main knows them; that commit does not. Quick check on your own build:

grep -ac aux_norm    build/bin/libllama.so   # newer: non-zero, old: 0
grep -ac decoder_arch build/bin/libllama.so  # newer: non-zero, old: 0

Build ROCmFPX main (add -DLLAMA_BUILD_WEBUI=OFF — the WebUI asset step fails without Node/npm and takes llama-server down with it at 100%). You can keep your existing build for everything else and point only Laguna at the new one.

2. The DFlash head declares a target layer that doesn't exist

The head ships dflash.target_layers = [2, 11, 20, 30, 39, 48], but Laguna-S-2.1 is block_count = 48, so valid indices are 0–47. llama.cpp asserts and aborts:

llama-context.cpp: GGML_ASSERT(lid < model.hparams.n_layer) failed

vLLM tolerates 48 as "the final hidden state"; llama.cpp does not. Patch the last element to 47 — it is a single 4-byte value of the same width, so it rewrites in place with no re-encode:

import struct, shutil
src="laguna-s-2.1-dflash-Q8_0.gguf"; dst="laguna-s-2.1-dflash-Q8_0-fix47.gguf"
shutil.copy(src, dst)
f=open(dst,"r+b"); assert f.read(4)==b"GGUF"
struct.unpack("<I",f.read(4)); struct.unpack("<Q",f.read(8))
n_kv=struct.unpack("<Q",f.read(8))[0]
def rs():
    n=struct.unpack("<Q",f.read(8))[0]; return f.read(n).decode()
SZ={0:1,1:1,7:1,2:2,3:2,4:4,5:4,6:4,10:8,11:8,12:8}
for _ in range(n_kv):
    k=rs(); t=struct.unpack("<I",f.read(4))[0]
    if t==9:
        et=struct.unpack("<I",f.read(4))[0]; ln=struct.unpack("<Q",f.read(8))[0]
        start=f.tell()
        if k=="dflash.target_layers":
            fmt={4:"<I",5:"<i",10:"<Q",11:"<q"}[et]
            f.seek(start+(ln-1)*SZ[et]); f.write(struct.pack(fmt,47))
            print("patched last target layer -> 47"); break
        if et==8:
            for _ in range(ln): rs()
        else: f.seek(SZ[et]*ln,1)
    elif t==8: rs()
    else: f.seek(SZ[t],1)
f.close()

Serving with DFlash

llama-server --host 127.0.0.1 --port 8099 \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --model Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --spec-type draft-dflash \
  -md laguna-s-2.1-dflash-Q8_0-fix47.gguf -ngld 999 \
  --spec-draft-n-max 4 --spec-draft-p-min 0.5 \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1

⚠️ You must set --spec-draft-n-max. The default of 16 exceeds the head's trained block size of 15 and the server aborts in the DFlash constructor before it ever listens.

Credit for the GGUF draft head to wimmmm; the target-layer patch and the build-version finding are mine.

License

Follow the base model (poolside/Laguna-S-2.1) license terms.

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Repository	Largest model file	Variant	Ships	Downloads	Likes
`kkuspa/Laguna-S-2.1-NVFP4-0804`	4.26 GiB	NVFP4	safetensors	1220	5
`kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF` (this repo)	58.34 GiB	STRIX_LEAN	single model file	461	2
`1337Hero/Laguna-S-2.1-Q4_0-ROCMFP4-GGUF`	58.34 GiB	ROCmFP4	single model file	219	0
`raulvidis/Laguna-S-2.1-ROCmFP4-STRIX-GGUF`	58.38 GiB	STRIX	single model file	455	4
`jcbtc/Laguna-S-2.1-Chadrock-ROCmFP4-StrixKVSpine-V4-GGUF`	60.94 GiB	STRIX	single model file	5094	33
`raulvidis/Laguna-S-2.1-ROCmFP4-COHERENT-GGUF`	61.78 GiB	COHERENT	single model file	455	2
`cnuland/legunas21-custom`	—	—	drafter	80	1

Base model: poolside/Laguna-S-2.1. Generated from Hub metadata; download counts move over time.

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

Downloads last month: 607

GGUF

Model size

118B params

Architecture

laguna

Hardware compatibility

4-bit

Model tree for kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF

Base model

poolside/Laguna-S-2.1

Quantized

(91)

this model

kingjones777
/

Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF

Laguna-S-2.1 ROCmFP4 STRIX_LEAN (GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

Files

Hardware / stack (validated)

Build recipe (HIP-only)

Quantize

Runtime (required)

A/B vs Q4_K_M (same binary, same flags, cold load)

Quality / tools / reasoning

⚡ DFlash speculative decoding — +23% decode (added 2026-08-08)

1. The fork commit pinned above is too old for DFlash

2. The DFlash head declares a target layer that doesn't exist

Serving with DFlash

License

Other public builds of this model

Acknowledgements

Model tree for kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF

Space using kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF 1