j2ipper/Laguna-S-2.1-Q4-lean

Laguna-S-2.1 Q4-lean (GGUF)

A 4-5-bit GGUF of poolside's Laguna-S-2.1, sized for a 128 GB AMD Strix Halo machine (Ryzen AI Max+ 395, Radeon 8060S) with room for the model's full 256K context. It is the Laguna-S-2.1 weight file that Talus ships with.

It is a requantization of Unsloth's Laguna-S-2.1 UD-Q4_K_XL: the routed experts are Unsloth's, byte for byte; the attention and shared-expert tensors go from Q8_0 to Q5_K / Q6_K.

Files

File	Size	SHA-256
Laguna-S-2.1-Q4-lean-00001-of-00003.gguf	3.68 MB (metadata only)	777fe1c031ceb7e424420e86c72c9e2b964054c3bd3ad1285ca96d4fc1dac0e7
Laguna-S-2.1-Q4-lean-00002-of-00003.gguf	49.13 GB	e2f86845e65cd9c184a4069bcd282513aa4cc5f6dac2a1236f78ceeae15bbde2
Laguna-S-2.1-Q4-lean-00003-of-00003.gguf	23.08 GB	1856e320077070f3fda08035872aa8d88bbd18c92a98bbc718b078d5d5c243b4

Every file, including the recipe, is listed in SHA256SUMS. Point Talus or llama.cpp at the first shard; the others are found next to it.

Talus drafts with prompt lookup, so no drafter is needed. The 64K draft vocabulary laguna-dv-65536.bin (for DFlash drafters in other engines) is in j2ipper/draft-vocab.

Tensor types

814 tensors, 67.24 GiB.

Tensors	Type	Notes
Routed experts ffn_gate_exps, ffn_up_exps	Q4_K (layer 47: Q5_K)	Unsloth's, unchanged
Routed experts ffn_down_exps	Q5_K (layers 46-47: Q6_K)	Unsloth's, unchanged
Attention attn_q, attn_output	Q5_K	from Q8_0 (this quant)
Attention attn_k, attn_v	Q6_K	from Q8_0 (this quant)
Shared experts ffn_*_shexp	Q6_K	from Q8_0 (this quant)
Dense FFN (layer 0), attn_gate	Q8_0	unchanged
output	Q6_K	this quant
token_embd	Q8_0
Router, norms, biases	F32	unchanged

By type: Q4_K 38.81 GiB, Q5_K 25.83 GiB, Q6_K 2.04 GiB, Q8_0 0.43 GiB, F32 0.14 GiB.

Recipe

One step with llama.cpp's llama-quantize --allow-requantize --keep-split, from Unsloth's UD-Q4_K_XL, with Unsloth's importance matrix (imatrix_unsloth.gguf from the same repository; not re-hosted here):

• a per-tensor type file: attn_q / attn_output -> q5_k; attn_k / attn_v and the shared experts -> q6_k; every other tensor pinned to its current type (copied byte for byte);
• --output-tensor-type q6_k --token-embedding-type q8_0.

recipe/ holds the type file (laguna-tt-q4-lean.txt) and the script that generates it from the UD-Q4_K_XL files (make_laguna_q4_lean_types.py).

Use it with Talus

Build Talus (Quick start), download this repository, extract the tokenizer as the Quick start shows, and serve with a run config like this one (the arguments are Talus's shipped defaults for this model, from docs/MODELS.md):

{
  "exe": "rust/target/release/talus-cli",
  "args": ["--model", "/models/Laguna-S-2.1-Q4-lean-00001-of-00003.gguf",
           "--max-context", "262144",
           "--ubatch", "2048",
           "--eos-ids", "2,24",
           "--spec", "7",
           "--sparse", "1024,16384,128,1024,16",
           "--dflash-gguf", "/models/laguna-s-2.1-dflash-Q8_0.gguf"],
  "tokenizer": "/models/laguna-s-2.1/tokenizer",
  "model_name": "laguna-s-2.1",
  "stop_tokens": ["〈|EOS|〉", "</assistant>"],
  "env": {},
  "lib_dirs": ["/path/to/rocm/lib", "/path/to/rocm/lib/llvm/lib"]
}

STRATA_API_KEY=<choose-a-key> rust/target/release/talus-server --config laguna.json --host 0.0.0.0 --port 8000

Talus serves it with 256K context, q8_0 KV and block-sparse decode attention past 32K. It also loads in llama.cpp builds that support the laguna architecture.

Quality

Measured by the Talus freeze run rel-final-e6cf (Talus commit e6cf959e1cad, 2026-10-28); methods in Talus's docs/METHODOLOGY.md:

Gate	Result
Teacher-forced KL, this release build vs the previously gated Talus build (served-speed numerics at 32K)	KL 0.00185 (@32K; @2K 0.000), top-1 98.5% (gate <= 0.003)
Needle in a haystack, 60K / 81K	15/15 / 15/15
Agentic coding suite	5/5

The KL line checks Talus's engine from build to build; it is not a measurement of this quant against the original weights.

Speed on Strix Halo: Talus's docs/RESULTS.md.

License and credits

The Model Materials are licensed under the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1), the license of the base model. A copy of the agreement is in LICENSE.md, and the notices of origin are kept below, as the agreement requires for redistribution. Note the agreement's patent clause: rights terminate for anyone who sues claiming the Model Materials infringe a patent or copyright.

• Base model: poolside/Laguna-S-2.1, by poolside, OpenMDW-1.1. poolside asks that the model be used consistently with its Acceptable Use Policy.
• Source quantization: unsloth/Laguna-S-2.1-GGUF (OpenMDW-1.1); the routed experts and the importance matrix are Unsloth's.
• Quantized with llama.cpp (MIT).