Quacken-Ornith-35B-FP8

The Rock8 - Got any weights? 💪🦆

Native fp8 E4M3 GGUF of Ornith-1.0-35B - a Mixture-of-Experts GatedDeltaNet-hybrid (multimodal-capable) model - for AMD RDNA4 (gfx1201 - Radeon AI PRO R9700 / RX 9070 / 9070 XT / W-series), quantized with AMD Quark from the full-precision BF16 weights by The Rock8.

The Rock8's llama.cpp fork runs this fp8 on RDNA4's native WMMA fp8 tensor cores - no dequant-to-f16 fallback.

What it is

• Format: fp8 E4M3 (F8E4M3), block-scaled, produced by AMD Quark from BF16.
• Architecture: MoE + GatedDeltaNet SSM-hybrid. No MTP head (the checkpoint ships zero MTP tensors) - so self-speculative --spec-type draft-mtp is not available for this model; run it as a plain target.
• Target: AMD RDNA4 / gfx1201; ~38 GB -> runs 2-GPU (tensor-split across two 32 GB R9700s) or one large card.
• Runtime: The Rock8 (llama.cpp fork with native RDNA4 fp8 kernels) on TheRock ROCm 7.13.
• File: ornith-1.0-35b-quark-fp8-authentic.gguf (37.8 GB).

Important run note

This is an SSM-hybrid architecture. Run it with llama-server or llama-perplexity - not llama-bench (llama-bench cannot allocate the SSM-hybrid context and will fail on this arch). That is a tooling limitation, not a model defect; the model is validated and generates correctly under llama-server.

Source model + license

• Source: Ornith-1.0-35B.
• License: Apache-2.0 (redistribution of this quantized derivative is permitted with attribution). This is a derivative work.

Validation (real gfx1201 hardware)

Metric	Value
Perplexity (wikitext, 20 chunks, n_ctx=512), 2-GPU	6.7010

Validated via llama-perplexity on dual R9700 (gfx1201).

Run it (2-GPU)

# use llama-server (NOT llama-bench); -ngl 999 tensor-splits across both R9700s
llama-server -m ornith-1.0-35b-quark-fp8-authentic.gguf -ngl 999 --host 0.0.0.0 --port 13305

curl -s http://localhost:13305/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"What do you call a dried grape? Answer in one word."}],"max_tokens":16}'
# expect: raisin

Lemonade appliance (container)

podman run -d --rm --runtime crun --name lemonade \
  --device /dev/kfd --device /dev/dri \
  --group-add keep-groups --security-opt seccomp=unconfined \
  -v /path/to/quacken-ornith-35b:/models:ro \
  -e MODEL=/models/ornith-1.0-35b-quark-fp8-authentic.gguf -e MODEL_NAME=Quacken-Ornith-35B-FP8 \
  -p 13305:13305 \
  ghcr.io/the-monk/the-rock8:rdna4-tr713 serve
# ~38 GB - needs both GPUs; do NOT pin HIP_VISIBLE_DEVICES to a single card

Container (same image on each registry; --runtime crun is required for GPU): ghcr.io/the-monk/the-rock8:rdna4-tr713 - docker.io/gorilla4x/the-rock8:rdna4-tr713 - quay.io/the-monk/the-rock8:rdna4-tr713 (images may not be pushed to every registry yet).

The Rock8 - RDNA4 fp8 (links)

• GitHub: The-Rock8 - kernels, patch series, appliance recipe, full feature doc.
• Collection: The Rock8 - RDNA4 fp8.
• Sibling models: Quacken-8B-FP8 - Quacken-R1-14B-FP8 - Quacken-27B-FP8 - Quacken-35B-A3B-FP8.

Every artifact links to the others - land on any one, reach them all.