AtomicChat/Muse-Glimmer-30B-GGUF

Muse Glimmer 30B — GGUF

GGUF builds of Meta's Muse Glimmer 30B, made from the original BF16 weights with a calibrated importance matrix and a bit layout chosen by measurement.

Support was merged into llama.cpp master in #26841 (commit 62bf73d). You need a build from master — the last tagged release predates it.

Files

File	Size	bpw	Fits	Notes
Muse-Glimmer-30B-AD-Q8_0.gguf	32.1 GB	9.23	40 GB	highest fidelity
Muse-Glimmer-30B-Q8_0.gguf	29.6 GB	8.50	32 GB, no vision	plain preset
Muse-Glimmer-30B-AD-Q6_K.gguf	28.3 GB	8.02	32 GB	see note
Muse-Glimmer-30B-AD-Q5_K_M.gguf	22.7 GB	6.44	24 GB, no vision
Muse-Glimmer-30B-AD-Q4_K_M.gguf	19.1 GB	5.47	24 GB
Muse-Glimmer-30B-AD-IQ4_XS.gguf	16.0 GB	4.58	24 GB + vision + drafter	recommended
Muse-Glimmer-30B-AD-IQ3_M.gguf	14.9 GB	4.28	16 GB, no vision
Muse-Glimmer-30B-AD-IQ3_XXS.gguf	12.2 GB	3.51	16 GB + vision
Muse-Glimmer-30B-AD-IQ2_M.gguf	11.2 GB	3.21	16 GB + vision
Muse-Glimmer-30B-AD-IQ2_XXS.gguf	9.3 GB	2.67	12 GB + vision	smallest
BF16/	55.7 GB	16.0	64 GB	reference for all metrics
mmproj-…-BF16.gguf	3.9 GB	—	+3.7 GB	required for images
dflash-…-BF16.gguf	5.1 GB	—	+5.5 GB	drafter, 1.8x faster generation

AD-Q6_K at 28.3 GB is dominated by plain Q8_0 at 29.6 GB, which is 1.3 GB larger and 31 % more accurate. Take Q8_0 instead unless you need exactly that size.

imatrix/ holds the importance matrix, layouts/ the per-tensor type map of every file, experiments/ the control builds. All logs: AtomicChat/Muse-Glimmer-30B-GGUF-metrics.

Everything on one RTX 5090

AD-IQ4_XS plus the vision projector plus the speculative drafter fits in 24.6 GB. Full multimodal, tool calling, and 1.8x faster generation on a single consumer card.

	VRAM	Generation	Prompt	Acceptance
model + mmproj	19.3 GB	39.6 t/s	242.6 t/s	—
model + mmproj + dflash	24.6 GB	71.8 t/s	198.5 t/s	0.177

Measured on one RTX 5090 at -c 16384 -fa on, nine prompts across code, prose and reasoning, greedy sampling.

Speculative decoding costs nothing in quality. The drafter only proposes tokens; the full model verifies every one of them, and under greedy sampling the output is identical to running without it. What it costs is 5.5 GB of VRAM and 18 % of prompt processing speed, since prefill runs through both models and gains nothing from speculation.

Running it

Build from master:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-mtmd-cli

--jinja is required. Without it neither the chat template nor tool-call parsing works.

llama-server -m Muse-Glimmer-30B-AD-IQ4_XS.gguf \
    --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
    --jinja -ngl 99 -c 16384 -fa on \
    --temp 1.0 --top-p 0.95 --top-k 64

With speculative decoding:

llama-server -m Muse-Glimmer-30B-AD-IQ4_XS.gguf \
    -md dflash-Muse-Glimmer-30B-BF16.gguf \
    --spec-type draft-dflash --spec-draft-n-max 15 \
    --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
    --jinja -ngl 99 -c 16384 -fa on

Reasoning depth is set by a line in the system prompt, not a flag: Reasoning strength: low|medium|high|xhigh. Default is high — at that setting short answers can spend their entire token budget inside the reasoning block, so raise max_tokens or drop to low for quick replies.

Quality

Reference is our own BF16 GGUF, verified bit-identical to the original weights by UUID and all 731 tensor hashes. Two corpora: neutral text in 30 languages, and agentic dialogue in the model's native chat markup with tool calls. Lower KLD is better.

Build	Size	KLD neutral	top-1	KLD agentic	top-1
AD-Q8_0	32.13	0.000819	98.99 %	0.018382	98.58 %
Q8_0	29.61	0.000861	98.95 %	0.019968	98.58 %
AD-Q6_K	28.26	0.001247	98.71 %	0.023174	98.36 %
AD-Q5_K_M	22.67	0.003136	97.82 %	0.039615	97.57 %
AD-Q4_K_M	19.06	0.008866	96.27 %	0.065472	96.39 %
AD-IQ4_XS	15.95	0.019418	94.55 %	0.091624	95.41 %
AD-IQ3_M	14.92	0.026290	93.60 %	0.121391	94.68 %
AD-IQ3_XXS	12.22	0.064353	89.97 %	0.168065	92.90 %
AD-IQ2_M	11.16	0.104788	87.31 %	0.236144	92.00 %
AD-IQ2_XXS	9.31	0.266591	79.66 %	0.333133	87.90 %

The same file scores an order of magnitude higher KLD on agentic dialogue than on neutral text. Agentic text is far more predictable (base PPL 2.45 vs 5.46), so the model is very confident and small probability shifts produce large divergence. A KLD number without its corpus is meaningless.

Against another publisher

Twelve builds from another publisher were downloaded and measured against the same BF16 base, with the same binary, on the same hardware.

Below 16 GB our curve sits below theirs everywhere, by 12 % to 41 % depending on the size and corpus. Two of our files win outright — smaller and more accurate:

Ours	Theirs	Result
AD-IQ3_XXS, 12.22 GB, 0.0644	UD-IQ2_M, 12.26 GB, 0.0826	smaller, 22 % lower KLD
AD-IQ3_XXS, 12.22 GB, 0.0644	UD-Q2_K_XL, 12.44 GB, 0.1020	smaller, 37 % lower KLD
AD-IQ3_M, 14.92 GB, 0.0263	UD-IQ3_M, 14.12 GB, 0.0426	+5.7 % size, 38 % lower KLD
AD-IQ4_XS, 15.95 GB, 0.0194	UD-Q4_K_XL, 15.88 GB, 0.0202	+0.4 % size, 4 % lower KLD

Above 19 GB theirs is better, by 13 % to 18 %. Their high rungs raise token_embd and leave the FFN at the base type; ours does more or less the opposite. The layout study below explains why that is the right call at 2–4 bits and the wrong one at 6–8.

How the layout was chosen

The first attempt, and why it failed

The first release raised attn_k/v to q8_0, attn_q/output/gate one step, and both token_embd and output, leaving the FFN at the base type. It beat the matching llama.cpp preset at every rung — 22 % lower KLD at Q4_K_M — but below 15 GB it sat above the preset curve once size was accounted for.

The reason is arithmetic. FFN is 20.7 of 27.9 billion parameters, 74 % of the model. The whole extra budget had gone to the other 26 %.

Measuring what a gigabyte buys

Seven builds from one IQ3_XXS base, each differing by exactly one override. Recovered KL divergence divided by added size gives a price per gigabyte.

Raised one step	Added	ΔKLD per GB, neutral	ΔKLD per GB, agentic
attn_k + attn_v → q8_0	0.11 GB	0.0466	0.0027
ffn_down	0.32 GB	0.0451	0.0296
ffn_gate + ffn_up	0.65 GB	0.0345	0.0590
attn_q + attn_output + attn_gate	0.22 GB	0.0106	0.0092
token_embd	0.35 GB	0.0080	0.0340
output	0.18 GB	0.0018	0.0003

output is the worst investment in the model and the first layout raised it at every rung. An earlier 8-bit study had suggested it mattered for top-1 agreement; at 8 bits everything is saturated and that signal was noise.

attn_k and attn_v are the bargain. GQA 32/2 makes them 6656 x 256 — together 0.64 % of the weights. Holding both at q8_0 costs 110 MB.

The optimum moves with bit width

That table is measured at three bits, and it does not transfer upward. At 8 bits the same experiment gives the opposite answer for token_embd: raising it to BF16 takes Q8_0 from 0.000861 to 0.000838 on neutral text and from 0.019968 to 0.018397 on agentic — a 7.9 % improvement, the largest single lever at that size.

The mechanism is straightforward. At low bit widths the FFN is coarse and every bit spent there pays. At high bit widths it is saturated, and the residual error collects in token_embd — the one tensor that is quantized blind, because it is a row lookup rather than a matrix multiply and the importance collector never sees it.

So the recipe is not one rule across the ladder:

• IQ2 through IQ4_XS — raise all three FFN matrices one step, hold attn_k/v at q8_0, leave everything else alone.
• Q4_K_M and above — the same, plus token_embd one step.

llama-quantize --imatrix muse-glimmer.imatrix.gguf \
    --tensor-type attn_k=q8_0 --tensor-type attn_v=q8_0 \
    --tensor-type ffn_gate=q4_K --tensor-type ffn_up=q4_K --tensor-type ffn_down=q4_K \
    Muse-Glimmer-30B-BF16-00001-of-00002.gguf out.gguf IQ4_XS 96

Base type and FFN type move together: IQ2_XXS/iq2_s, IQ2_M/iq3_xxs, IQ3_XXS/iq3_s, IQ3_M/iq4_xs, IQ4_XS/q4_K, Q4_K_M/q5_K + embd q6_K, Q5_K_M/q6_K + embd q8_0, Q6_K/q8_0 + embd q8_0.

What did not work

Per-layer bit allocation. Importance concentration in ffn_down varies enormously with depth — ten channels out of 19 968 hold 1.6 % of the total at layer 6 and 60 % at layer 42. Raising only the concentrated layers 36–47 versus raising all 52 uniformly gives builds within 1.6 % of each other, in both directions depending on corpus. Indistinguishable. Control published as experiments/…-IQ4_XS_FLAT.gguf.

Beating the presets on layout alone. The control builds carry the same importance matrix as ours and differ only in layout. Against them our advantage is inconsistent — better at some sizes, worse at others. Our lead over the other publisher comes largely from the calibration, not from the bit layout. The controls are published so anyone can check that.

Calibration

imatrix/muse-glimmer.imatrix.gguf is published so anyone can rebuild these files or build their own on the same calibration.

An importance matrix records, for every weight matrix, the mean squared activation arriving on each of its input channels. The quantizer uses those numbers to weight reconstruction error when it searches for block scales: error on a channel that usually carries large activations is penalized more than error on a channel usually near zero.

Source	our BF16 GGUF, bit-identical to the original weights
Corpus	5.64 M tokens over 689 windows
Composition	25 % agentic dialogue in native chat markup, 18 % code, 15 % long chain-of-thought, 12 % long-context documents, 12 % multilingual, 10 % vocabulary sweep, 5 % structured, 3 % graphics
Context	8192 for short documents, 32768 for the long-context slice, merged
Flags	--parse-special, --process-output
Coverage	all 417 collected tensors woke up more than 99.9 % of their input channels

Both flags are off by default and both matter. Without --parse-special the chat control tokens are tokenized as literal punctuation, so 40 % of the corpus would be calibrated on sequences the model never produces. Without --process-output, output.weight — 1.34 B parameters sitting directly under the logit softcap — gets no importance data at all.

What the calibration shows

Muse Glimmer has 52 layers with 8 collected matrices each, plus the output matrix, for 417 importance vectors. They reduce to five distinct profiles per layer: attn_q, attn_k, attn_v and attn_gate all read the output of attn_norm, so their vectors are identical to the last digit; ffn_gate and ffn_up likewise share ffn_pre_norm. These groups cannot be told apart by importance, only by cost — which is why attn_k/v are raised and attn_q/gate are not.

Y lists the five profiles; X is the share of a tensor's importance held by its 10 strongest input channels and by its strongest 1 %, averaged over 52 layers. Ten channels out of 6656 would hold 0.15 % under a uniform distribution; on the attention input they hold 38.8 %, a factor of 259. For ffn_down the ten strongest of 19 968 hold 19.8 % against a uniform 0.05 %, a factor of 396.

X is the layer index, Y the share of ffn_down importance in its 10 strongest channels. Layers 5–11 sit between 1.6 % and 3.6 %; layers 41 and 42 reach 59 % and 60 %.

K-quants assign one scale to a block of 32 adjacent weights along the input dimension. A single dominant channel forces that scale upward and costs precision on the other 31, so concentration makes a block harder to quantize, not easier. It marks where calibration has the most leverage — but as the per-layer control above shows, that does not translate into a usable rule for moving bits between layers.

One note on attn_gate. Gated attention is new to this architecture and adds a 6656 x 4096 matrix per layer, roughly a third of the attention parameters. No published quantization work has had importance data for it. Its profile is identical to attn_q, attn_k and attn_v, so it cannot be distinguished from them by importance alone.

Reproducing it

llama-imatrix -m Muse-Glimmer-30B-BF16-00001-of-00002.gguf \
    -f calib_train.txt -o part.gguf \
    -c 8192 -b 2048 -ngl 99 \
    --parse-special --process-output --no-ppl

--no-ppl is required at long context: the perplexity path allocates n_ctx x n_vocab x 4 bytes for logits, which is 26.5 GB at 32768 context with this model's 202 048 token vocabulary. Perplexity is not used by importance collection, so disabling it costs nothing.

Merging is exact addition of sums and counts, so the corpus can be split across machines and recombined:

llama-imatrix -m any-model.gguf -c 8192 \
    --in-file part-00.gguf,part-01.gguf,part-long.gguf \
    -o muse-glimmer.imatrix.gguf

Corpus: AtomicChat/calib-corpora. Per-tensor statistics: imatrix/imatrix-stats.txt in the metrics dataset.

Credits

Model by Meta Superintelligence Lab. llama.cpp support by @pcuenca and the ggml-org reviewers.