keptn/ornith-1.5-122b-fp8

Ornith 1.5 122B · FP8 · for radiance

ornith-ai/Ornith-1.5-122B in one 118.0 GiB file for the radiance inference engine on AMD RDNA4. A mixture-of-experts reasoning model with 10B active parameters, for four 32 GB cards.

File	ornith-1.5-122b-fp8.rad, 118.00 GiB
Needs	radiance 1.9.0 or newer; 4 cards with 32 GB plus a pinned host pool of 3 GiB per card (tested on Radeon AI PRO R9700)
Format	weights FP8 (E4M3), one bf16 scale per 128×128 block; activations FP8; lm_head FP8, 128×128 blocks
Speculation	the model's own MTP head, 3 draft tokens
Inputs	text · image
Context	262,144 tokens
Version	1.0 (2026-10-22)

Speed

Radeon AI PRO R9700 at 215 W, radiance 1.9.0, fp8 KV, 3 draft tokens, thinking off. BetterBench 0.6.1, full run, temperature 0.7. Tokens per second (median):

	1 card	2 cards	4 cards
decode, one stream	does not fit	does not fit	165.1
prefill at 24k tokens	does not fit	does not fit	7,020
4 streams, total	does not fit	does not fit	241

Quality

Distance to the bf16 original on the 80 MT-Bench questions, both turns. The text is the GPT-4 answers from lmsys/mt_bench_human_judgments, with the model's chat template and thinking off. The reference is the bf16 original in HF transformers, with fp32 arithmetic and the full vocabulary. radiance scores this file with --kld-ref, on the answer tokens only (42,305 tokens).

	mean KL	99th percentile KL	top token the same	perplexity ratio
this file	0.0118	0.139	97.2%	×1.003
keptn/ornith-1.5-122b-fp8-dflash2	0.0118	0.139	97.2%	×1.003
keptn/ornith-1.5-122b-w4a8	0.0174	0.222	96.4%	×1.002
keptn/ornith-1.5-122b-w4a8-dflash2	0.0174	0.222	96.4%	×1.002

Quick start

hf download keptn/ornith-1.5-122b-fp8 ornith-1.5-122b-fp8.rad --local-dir ~/models
docker run -d --init --name ornith --network host --group-add video --group-add render \
    --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
    -v ~/models:/models:ro -e ROCR_VISIBLE_DEVICES=0,1,2,3 \
    stilldeadcode/radiance:1.9.0 --model /models/ornith-1.5-122b-fp8.rad --host 0.0.0.0 --port 8000 \
    --tp 4 --p2p on --placement expert_tiered --host-pool-mib 3072 --vision-tower-placement ram \
    --kv-cache-dtype fp8 --max-model-len 262144 --max-num-seqs 8 \
    --max-num-batched-tokens 2048 --num-speculative-tokens 3 \
    --temp 1.0 --top-p 0.95 --top-k 20 --presence-penalty 1.5

	change
host pool	--host-pool-mib is per card. The weights fill the cards; 3 GiB per card keeps the rest in host memory
coding	--temp 0.6 --presence-penalty 0 (Ornith's setting for precise coding)
thinking	on by default; per request "chat_template_kwargs": {"enable_thinking": false} turns it off

Memory and context

KV pool at --max-model-len 262144, fp8 KV, radiance 1.9.0, from radiance's startup report. Tokens of context in total, shared by all sequences:

	2 sequences	4 sequences	8 sequences
1 card	does not fit	does not fit	does not fit
2 cards	does not fit	does not fit	does not fit
4 cards	524,352	1,048,720	614,208

Images, audio and video

The same inputs go to radiance and to the bf16 original in HF transformers. Greedy, thinking off, 4 cards, radiance 1.15.1 (release image). There is one measurement for each vision or audio encoder, made with ornith-1.5-122b-fp8.rad.

input	cases	same answer	mean KL	top token the same
images (DocVQA and AI2D, fixed selection)	20	16/20	0.0328	91/95

How it was made

1. Started from Ornith's FP8 release ornith-ai/Ornith-1.5-122B-FP8 (revision 31b76f63).
2. Requantised every linear to FP8 with one scale per 128×128 block, rounded to nearest. radiance's expert kernels read only this form.
3. Added a 2-bit copy of the lm_head for the draft rounds, as radiance's recipe for Qwen3.6-35B-A3B does.

rad-convert Ornith-1.5-122B-FP8 --tokenizer Ornith-1.5-122B-FP8/tokenizer.json \
    --recipe ornith-1.5-122b-fp8.recipe -o ornith-1.5-122b-fp8.rad

Calibration: no calibration data.

Model card with: radiance-model-cards-framework 0.3 (codeberg.org/keptn42/radiance-model-cards-framework).

Limits

• radiance runs on AMD RDNA4 cards.
• The file needs four cards. On two cards radiance 1.9.0 has no attention kernel for this model: it has 16 query heads per KV head there.
• The FP8 weights of the release are rounded a second time, into blocks.
• With thinking on, hard tasks can need more than 32k output tokens.

Credits

• Model: the Ornith team (github.com/ornith-ai/Ornith-1), built on Qwen3.5 (Apache 2.0).
• radiance: StillDeadcode (codeberg.org/StillDeadcode/radiance).
• Test sets: MT-Bench, DocVQA, AI2D.

License

MIT, the license of the base model. See LICENSE (from github.com/ornith-ai/Ornith-1).