Shrike-3B: d1-3B healed from 1.5 GB down to 0.68 GB

Shrike is a set of low-bit GGUF files of LiquidAI/d1-3B, a decision model (yes/no, choice and score answers in one forward pass, zero output tokens). They run on stock llama.cpp with /v1/systemone. No code changes, no extra flags.

Plain 1- and 2-bit quants of d1-3B lose a lot of decisions. We "heal" them: a LoRA is trained so the low-bit model copies the full model's answers, then it is merged and the file is re-quantized. Same size, same speed, far closer to F16.

Modified files. These files are derived from LiquidAI/d1-3B and were changed by us (LoRA distillation, merged, re-quantized). They are not made or endorsed by Liquid AI. Licensed under the LFM Open License v1.0 (see LICENSE).

Files

file	size	same answers as F16 (mean of 4 evals)	plain quant of the same size
Shrike-3B-IQ4_XS.gguf	1.52 GB	97.7%	97.8%
Shrike-3B-IQ2_M.gguf	0.98 GB	93.5%	91.8%
Shrike-3B-IQ2_XXS.gguf	0.82 GB	90.5%	85.0%
Shrike-3B-IQ1_S.gguf	0.68 GB	85.6%	61.6%

Pick IQ4_XS as a safe default (smaller than Liquid's Q4_K_M, same quality), the smaller files when memory matters. For images, use Liquid's mmproj-d1-3B-Q8_0.gguf (583 MB) or -F16. CPU only, Shrike-3B-IQ1_S + the Q8_0 mmproj peaked at 1.44 GB RAM (text ~450 ms, image ~2 s on 6 threads).

For the most faithful file, use Liquid's official Q4_K_M (97.5%). Shrike is for when you want it smaller.

Quick start

Needs a llama.cpp build with d1 support (lfm2-d1, PR #30110 or newer).

hf download DogukanUrker/Shrike-3B Shrike-3B-IQ1_S.gguf --local-dir .
hf download LiquidAI/d1-3B-GGUF mmproj-d1-3B-Q8_0.gguf --local-dir .   # only for images
llama-server -m Shrike-3B-IQ1_S.gguf --mmproj mmproj-d1-3B-Q8_0.gguf

curl localhost:8080/v1/systemone -d '{"state": "The food was cold and nobody came to our table.",
  "questions": {"happy": {"type": "noul", "instructions": "Is the customer happy?"}}}'

If you send several questions about one state, start the server with -np 8 -kvu (or more): the questions then share the state's prefill (8 questions: 803 ms -> 186 ms on an RTX 3060).

Results

Agreement = the top answer is the same as d1-3B F16. Accuracy = against the dataset labels. All runs: stock llama-server (llama.cpp 3d65c90; master 781dbc5ac gives identical answers), RTX 3060. scored with the release code (eval/).

file	size	english (13 tasks) agree / acc	images (doom) agree / acc	multilingual agree / acc	MASSIVE (unseen) agree / acc	gpu ms
F16	5.4 GB	1.000 / 0.845	1.000 / 0.954	1.000 / 0.786	1.000 / 0.844	30.3
official Q4_K_M	1.67 GB	0.979 / 0.841	0.983 / 0.958	0.966 / 0.783	0.972 / 0.839	22.8
IQ4_XS plain	1.52 GB	0.980 / 0.839	0.992 / 0.954	0.970 / 0.789	0.969 / 0.853	20.7
Shrike-3B-IQ4_XS	1.52 GB	0.986 / 0.845	0.983 / 0.954	0.970 / 0.786	0.969 / 0.861	20.8
IQ2_M plain	0.98 GB	0.923 / 0.831	0.942 / 0.938	0.881 / 0.793	0.925 / 0.825	25.1
Shrike-3B-IQ2_M	0.98 GB	0.945 / 0.847	0.967 / 0.963	0.913 / 0.791	0.917 / 0.822	25.0
IQ2_XXS plain	0.82 GB	0.862 / 0.805	0.929 / 0.946	0.814 / 0.728	0.794 / 0.747	21.8
Shrike-3B-IQ2_XXS	0.82 GB	0.922 / 0.841	0.971 / 0.958	0.873 / 0.781	0.853 / 0.789	21.8
IQ1_S plain	0.68 GB	0.670 / 0.668	0.762 / 0.767	0.596 / 0.559	0.436 / 0.433	23.2
Shrike-3B-IQ1_S	0.68 GB	0.884 / 0.820	0.963 / 0.954	0.814 / 0.757	0.764 / 0.714	23.1
d1-omni-600M Q8_0	0.41 GB	0.792 / 0.781	0.762 / 0.671	0.593 / 0.562	0.547 / 0.533	9.0

Evals:

• english: 13 tasks x 100 (boolq, sst2, rte, mrpc, qnli, cola, qqp, imdb as yes/no; ag news, trec, emotion, dbpedia, tweet sentiment as choice).
• images: 240 doom frames, "is there a living monster in this picture?", labels from the game engine. accuracy = best single threshold.
• multilingual: xnli (9 languages), paws-x (6), sentiment (7), ~1300 decisions.
• MASSIVE: 360 voice-assistant requests in 9 languages, 18-way choice. no MASSIVE data was used in healing.

Independent check: BenchKit

Two more benchmarks, run with BenchKit (a different harness, different prompts, none of this data was used in healing). 32 parallel requests on the same stock llama-server, RTX 3060.

file	code-verifier acc	agree with d1-3B	GPQA acc	agree with d1-3B	dec/s (32 parallel)
d1-3B Q8_0 (Liquid)	51.8%	100%	33.8%	100%	12.6 / 16.8
Shrike-3B-IQ4_XS	51.8%	98.4%	29.8%	86.9%	13.6 / 18.4
Shrike-3B-IQ2_M	52.4%	94.6%	32.8%	68.2%	11.2 / 15.3
Shrike-3B-IQ2_XXS	49.6%	92.2%	28.3%	44.9%	12.8 / 17.3
Shrike-3B-IQ1_S	49.4%	84.4%	23.2%	32.3%	12.1 / 16.6

• Agreement on code-verifier matches our evals (98 / 95 / 92 / 84% vs 98 / 94 / 91 / 86%) on a domain we never touched.
• On hard reasoning, d1-3B itself is near chance: code-verifier is balanced yes/no (50%) and d1 answers "yes" 93% of the time; GPQA has 4 options (25%). When the original is guessing, its answers flip easily, so agreement drops (GPQA) and the accuracy differences between files are noise (IQ2_M vs IQ4_XS: paired p = 0.39 on GPQA, 0.73 on code-verifier). Use d1 / Shrike for classification, routing and images, not for graduate-level reasoning.
• On a GPU, lower bits are smaller, not faster. Decisions are prefill only, which is compute-bound when batched, and the 1-2 bit kernels cost more compute to unpack. IQ4_XS is the fastest file; the win of the small files is memory.

How it was made

1. llama-quantize with an imatrix from decision prompts (a plain wikitext imatrix scored the same).
2. Healing: the d1-3B text model with the dequantized low-bit weights (exactly what llama.cpp computes) plus a rank-16/32 LoRA on every linear layer, trained 3000-6000 steps to match full d1-3B's top-32 answer distribution (KL). Data: ~13k text prompts (yes/no, choice, score; 10+ public datasets; english + 15 languages) and ~4.1k image prompts (COCO, doom, mario) fed as the exact llama.cpp input embeddings. Teacher answers only, no labels.
3. soup: IQ2_M, IQ2_XXS and IQ1_S are each healed twice with different data mixes, and the two adapters are averaged (llama-export-lora --lora-scaled a.gguf:0.5,b.gguf:0.5). the soup beat both parents on mean agreement.
4. IQ4_XS gets one gentle heal instead (learning rate 2e-5, 1000 steps): the 1-2 bit recipe (2e-4) made it worse.
5. llama-export-lora merges into F16, then llama-quantize re-quantizes to the same type.

Honest caveats

• At 4 bits there is little left to heal: Shrike-3B-IQ4_XS ties a plain IQ4_XS (97.7% vs 97.8% mean agreement; better on english, 2 doom frames worse). Both beat Liquid's Q4_K_M (97.5%) at 9% smaller. The healing recipe used for 1-2 bits made IQ4_XS worse (96.5-97.3%), so it got a 10x gentler heal.

• Healing helps most where the plain quant is worst. At IQ2_M the gain is small, and on the unseen task (MASSIVE) the soup is ~1 point below plain IQ2_M (a single heal was ~2 below; the soup fixed most of it).

• The multilingual training prompts come from the train splits of the same datasets as the multilingual eval (different sentences). MASSIVE is the fully unseen check.

• Image healing used doom frames from the same game as the image eval (different frames).

• Evals are small (100 per task, 360 MASSIVE items): treat differences under ~2 points as noise.

• IQ1_M heals too, but its llama.cpp kernels were 2x slower here, so it is not included.

Credits

Base model: LiquidAI/d1-3B (LFM Open License v1.0). Commercial use is not licensed for legal entities with annual revenue of $10M or more (license section 5). Code: https://github.com/DogukanUrker/Shrike.