original-el8/Muse-Glimmer-30B-NVFP4

Muse Glimmer 30B NVFP4

Model ID: original-el8/Muse-Glimmer-30B-NVFP4
Base model: meta-models/Muse-Glimmer-30B
Created: August 11, 2026
Runtime status: qualified on one NVIDIA RTX PRO 6000 Blackwell GPU (SM120) with native vLLM Muse Glimmer support, the FlashInfer b12x NVFP4 kernel, piecewise CUDA graphs, and the Muse Glimmer DFlash assistant at speculative depth 16.
Quality result: on matched 140-question MMLU-Pro smoke tests, BF16/NVFP4 scored 109/109 with DFlash and 109/106 without DFlash; both passed all 16 paired core-workload canaries. Held-out WikiText-2 token perplexity was 5.6884 versus 5.4567 for BF16, a 4.25% increase.
Distribution fidelity: mean full-vocabulary next-token KL divergence of 0.091815 nats against BF16 across 24 matched contexts.
Container: ghcr.io/liquidgravityai/vllm-muse-glimmer:cu132-b12x-pr51655-a52f4d9c

Muse Glimmer 30B NVFP4 is an NVIDIA ModelOpt NVFP4 W4A4 conversion of the 29.8-billion-parameter Muse Glimmer vision-language model. The published safetensors files occupy 23,380,728,424 bytes (23.38 GB, 21.78 GiB); tokenizer and configuration files are additional.

Quantization specification

Property	Value
Format	NVIDIA NVFP4 W4A4
Producer	NVIDIA ModelOpt 0.47.0.dev48+g41b18a856
Quantized modules	416 decoder linear projections across 52 layers
Quantized projections per layer	attention Q, K, V, output, and gate; MLP gate, up, and down
Weight values	4-bit floating point, 16-value blocks
Activation values	4-bit floating point, 16-value blocks, calibrated static global scales
Weight scale calibration	MSE search across all 128 FP8 E4M3 global-scale candidates
Activation scale calibration	max(16384 × p1(block_amax), p99.99(block_amax))
Calibration workload	16,000 deterministic records: 9,987 agentic/coding, 4,994 diverse text, 819 deep-context text, and 200 multimodal
Calibration token coverage	13,406,156 post-template non-padding tokens; 23,677,561 padded token slots across 4,765 forward batches
Sequence limits	2,048 tokens for agentic/diverse text, 8,192 for deep-context text, and 4,096 for multimodal
Preserved BF16 modules	vision tower, vision adapters and projections, token embeddings, normalization layers, and LM head
KV cache	Not quantized by the checkpoint

The safetensors index contains 2,684 tensors and 23,380,728,424 indexed bytes. Validation checked 1,572,733,760 scale values; all were finite and positive. All 104 packed Q/K/V and gate/up groups passed shared-global-scale equality. BF16-excluded tensors matched the source checkpoint byte-for-byte.

The checkpoint uses standard ModelOpt/Compressed-Tensors metadata in hf_quant_config.json. model-inputscales.safetensors stores one calibrated input scale for each quantized projection.

The complete deterministic first-$N$ selection rule, dataset hashes, template mode counts, per-source token counts, exact TOML configuration, and template revision are published under evaluation/. Calibration used the original Meta release template; the distributed checkpoint now carries Meta's corrected template, with both revisions and SHA-256 hashes recorded in evaluation/provenance.json.

Quantization and calibration attribution

The quantization and export pipeline is based on Luke Alonso's local-inference-lab/quant-toolkit, using revision 8bdb1016e52dd15f91e42d35c09096c0f31325f3. The toolkit provides the calibration runner, NVIDIA ModelOpt integration, resumable activation-statistics collection, fused-projection scale handling, and Hugging Face export path. The Muse Glimmer conversion adds its model adapter, calibration profile, quantization boundary, and validation gates. NVIDIA ModelOpt performs the calibration and NVFP4 conversion.

The toolkit's text-data builder declares the following public calibration source pool:

• agentic, coding, reasoning, math, and multilingual data: Orca AgentInstruct, CodeAlpaca, Self-OSS-Instruct, StarCoderData, Hermes Function Calling, Glaive Function Calling, OpenOrca, GSM8K, MathInstruct, Alpaca Chinese, COIG-CQIA, and Zhihu-KOL
• diverse conversations: WildChat-nontoxic and LMSYS-Chat-1M
• multimodal images: COCO val2017, selected and paired with calibration prompts by the toolkit's tools/build_mm_calib.py

The normalized text JSONL files retain message content but not a per-record upstream source identifier. The manifests under evaluation/ identify and verify the exact local files and first-$N$ selections with SHA-256 hashes, but they do not recover record-level provenance. Each upstream dataset remains subject to its own license and terms.

Quality validation

MMLU-Pro

BF16 and NVFP4 were evaluated twice on the same 140-question, five-shot, high-reasoning MMLU-Pro slate. Both comparisons use the same final-answer extraction rule for BF16 and NVFP4.

Run	BF16	NVFP4	NVFP4 minus BF16
Historical DFlash	109/140 (77.86%)	109/140 (77.86%)	0.00 points
No DFlash, corrected template	109/140 (77.86%)	106/140 (75.71%)	-2.14 points

The DFlash run had 100 both-correct, 9 BF16-only, 9 NVFP4-only, and 22 both-incorrect outcomes (exact two-sided McNemar $p=1.0$). The no-DFlash run had 93 both correct, 16 BF16-only, 13 NVFP4-only, and 18 both incorrect ($p=0.711$).

The historical runners had stored 106/140 for BF16 and 108/140 for NVFP4 because they used inconsistent extraction behavior; applying the shared rule to both unchanged output sets produces the DFlash tie above. The no-DFlash comparison removes the assistant-model confound and uses Meta's corrected chat template, but its -2.14-point observed difference means this small slate does not support a tight non-inferiority claim. Neither result establishes statistical equivalence or superiority.

The exact question IDs, rendered prompts, per-request seeds and sampling parameters, all four output sets, shared scorer, replay tool, immutable no-DFlash runtime digest, and paired summaries are published in evaluation/mmlu-pro/.

Core-workload canaries

BF16 and NVFP4 both passed all 16 deterministic no-DFlash canaries:

Workload	BF16	NVFP4
Coding reasoning	4/4	4/4
Tool selection and exact arguments	4/4	4/4
Synthetic image understanding	4/4	4/4
Needle retrieval at 32,768 and 128,000 rendered tokens	4/4	4/4

The paired comparator verified that every request was identical except for the served model name. These are targeted behavior canaries, not broad benchmark estimates. Protocols, generated images, complete long-context prompts, requests, responses, hashes, runner, and comparator are published in evaluation/core-workloads/.

Held-out token perplexity

BF16 and NVFP4 were measured without speculative decoding on the WikiText-2 raw test split. The evaluator concatenated all 4,358 rows with double newlines, tokenized once with the Muse Glimmer tokenizer, and scored each of 289,691 non-BOS tokens exactly once using 4,096-token windows and a 2,048-token stride. Overlapping prefixes supplied context only; raw token IDs bypassed the chat template.

Checkpoint	Mean NLL	Token perplexity	Bits/token
BF16	1.696842 nats	5.456689	2.448026
NVFP4	1.738425 nats	5.688377	2.508017

NVFP4 increased perplexity by 4.2459% relative to BF16. A 100,000-sample sequential 2,048-token block bootstrap placed the PPL ratio at 1.0399–1.0452 for its central 95% interval. This misses the provisional no-more-than-2% relative-regression gate; the checkpoint is therefore not qualified against that strict PPL threshold.

Token perplexity depends on the tokenizer, corpus construction, and windowing protocol. Compare these values only under this matched setup. The pinned dataset revision and hashes, per-checkpoint summaries, all 289,691 paired raw token log-probabilities, runner, and paired comparator are published in evaluation/perplexity/.

Full-vocabulary next-token fidelity

The fidelity measurement compares exact next-token distributions from BF16 and this NVFP4 checkpoint.

Conditions

• Hardware: one NVIDIA RTX PRO 6000 Blackwell GPU (SM120).
• Runtime: the same native vLLM Muse Glimmer implementation for both checkpoints.
• Protocol: 12 fixed bare-completion prompts with an Answer: cue, each measured before generation and after eight tokens along the same fixed NVFP4 greedy trajectory.
• Vocabulary: all 202,048 output tokens; no top-$k$ approximation.
• Direction: $D_{KL}(P_{BF16}\parallel P_{NVFP4})$ using natural logarithms.
• Computation: raw log probabilities renormalized in float64.

Metric	Result
Mean KL divergence	0.091815 nats / 0.132461 bits
Median KL divergence	0.066785 nats
Sample standard deviation	0.100880 nats
95th-percentile KL divergence	0.222844 nats
Maximum KL divergence	0.455607 nats
Prompt-cluster bootstrap 95% interval for mean	0.055015–0.136542 nats
Mean Jensen–Shannon divergence	0.021053 nats
Mean total-variation distance	0.123759
BF16/NVFP4 top-token agreement	22/24, 91.67%
Mean KL divergence at trajectory depth 0	0.104314 nats
Mean KL divergence at trajectory depth 8	0.079316 nats

The maximum-divergence context was the coding prompt at trajectory depth 8, where the checkpoints selected different top tokens. These values measure token-distribution fidelity, not downstream task accuracy.

The exact contexts, raw [24, 202048] BF16 and NVFP4 arrays, per-context results, and independent NumPy verifier are published in evaluation/kl-fidelity/.

Qualified runtime

A compatible runtime must provide all of the following:

1. Native Muse Glimmer model, processor, reasoning-parser, and tool-parser support.
2. NVIDIA ModelOpt NVFP4 checkpoint loading.
3. An NVFP4 linear kernel for the target GPU. The qualified SM120 runtime used FlashInfer b12x.
4. Transformers 5.15.0 or another release with MuseGlimmerForConditionalGeneration.
5. The Muse Glimmer assistant checkpoint for DFlash speculative decoding, when DFlash is enabled.

The qualified target-plus-drafter process loaded 27.1 GiB of weights. At --gpu-memory-utilization 0.94, it allocated a 1,963,206-token GPU KV cache and reported 14.98× maximum concurrency at 131,072 tokens. DFlash and a 131K context budget are not claimed to fit on 24 GB or 32 GB GPUs. Remove DFlash and reduce context/concurrency for lower-memory systems, then qualify that configuration independently.

The production command is:

vllm serve original-el8/Muse-Glimmer-30B-NVFP4 \
  --host 0.0.0.0 \
  --port 8000 \
  --linear-backend flashinfer_b12x \
  --attention-backend FLASHINFER \
  --max-model-len 131072 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.94 \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --enable-auto-tool-choice \
  --tool-call-parser muse_glimmer \
  --reasoning-parser muse_glimmer \
  --speculative-config '{"model":"meta-models/Muse-Glimmer-30B-assistant","method":"dflash","num_speculative_tokens":16,"draft_sample_method":"probabilistic"}'

The b12x backend is selected explicitly. Do not assume that auto selects the same kernel.

Text request

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Muse-Glimmer-30B-NVFP4",
    "messages": [
      {"role": "user", "content": "Explain why idempotency matters for payment APIs."}
    ],
    "temperature": 0,
    "max_tokens": 256
  }'

Muse Glimmer accepts OpenAI-style multimodal message content. Supply an image_url content part before the text content part in the same user message.

Published GHCR image

The Linux/amd64 image contains the qualified CUDA 13.2.1 runtime, the pinned b12x/FlashInfer stack, Transformers 5.15.0, the integrated Muse Glimmer model and parsers, and the DFlash fixes used for the validation above.

Property	Value
Package	liquidgravityai/vllm-muse-glimmer
Tag	ghcr.io/liquidgravityai/vllm-muse-glimmer:cu132-b12x-pr51655-a52f4d9c
Immutable digest	sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871
Local image ID used for validation	sha256:1fa569b8c44c3d1575c9f8ee8390337ced3be23ded5b889c54cf4a09e2baddb9
vLLM base revision	371085e9e4ee3471125d69cfbfcfc66864634ee4
Muse Glimmer upstream revision	99a10304dce8945119bd0b1a072297803c52a749
Muse follow-up revision	a52f4d9c8640332842bfc3428382f5ad1fc5bdda
Integrated source-diff SHA-256	d835a5b0618ee306a5b7a09f4386d439077c0cc710d783bccb89b863ad965c3d

Docker

The assistant repository is gated. Export a Hugging Face token with access before starting the DFlash configuration.

export HF_TOKEN=hf_...

docker pull \
  ghcr.io/liquidgravityai/vllm-muse-glimmer@sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871

docker run --rm --gpus all --ipc=host -p 8000:8000 \
  -e HF_TOKEN \
  -v "${HF_HOME:-$HOME/.cache/huggingface}:/root/.cache/huggingface" \
  ghcr.io/liquidgravityai/vllm-muse-glimmer@sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871 \
  vllm serve original-el8/Muse-Glimmer-30B-NVFP4 \
    --host 0.0.0.0 \
    --port 8000 \
    --linear-backend flashinfer_b12x \
    --attention-backend FLASHINFER \
    --max-model-len 131072 \
    --max-num-seqs 8 \
    --max-num-batched-tokens 16384 \
    --gpu-memory-utilization 0.94 \
    --enable-prefix-caching \
    --enable-chunked-prefill \
    --enable-auto-tool-choice \
    --tool-call-parser muse_glimmer \
    --reasoning-parser muse_glimmer \
    --speculative-config '{"model":"meta-models/Muse-Glimmer-30B-assistant","method":"dflash","num_speculative_tokens":16,"draft_sample_method":"probabilistic"}'

Docker Compose

Save the following as compose.yaml:

services:
  muse-glimmer:
    image: ghcr.io/liquidgravityai/vllm-muse-glimmer@sha256:34173f7bc4f6a5a1e590a899ef4753b22e7d87e18b156f74958ec3b2dbf74871
    gpus: all
    ipc: host
    shm_size: 16gb
    ports:
      - "8000:8000"
    environment:
      HF_TOKEN: ${HF_TOKEN:?Set HF_TOKEN}
    volumes:
      - ${HF_HOME:-./hf-cache}:/root/.cache/huggingface
    command:
      - vllm
      - serve
      - original-el8/Muse-Glimmer-30B-NVFP4
      - --host
      - 0.0.0.0
      - --port
      - "8000"
      - --linear-backend
      - flashinfer_b12x
      - --attention-backend
      - FLASHINFER
      - --max-model-len
      - "131072"
      - --max-num-seqs
      - "8"
      - --max-num-batched-tokens
      - "16384"
      - --gpu-memory-utilization
      - "0.94"
      - --enable-prefix-caching
      - --enable-chunked-prefill
      - --enable-auto-tool-choice
      - --tool-call-parser
      - muse_glimmer
      - --reasoning-parser
      - muse_glimmer
      - --speculative-config
      - '{"model":"meta-models/Muse-Glimmer-30B-assistant","method":"dflash","num_speculative_tokens":16,"draft_sample_method":"probabilistic"}'

Then run:

docker compose up -d
docker compose ps
curl --fail http://127.0.0.1:8000/health

vLLM implementation provenance

This image is not a stock vLLM wheel. It overlays the validated Muse Glimmer sources onto a pinned CUDA 13.2 b12x base. The relevant pull requests are listed explicitly:

Pull request	Role in this image
vllm-project/vllm#51655	Native Muse Glimmer model, processor, reasoning parser, tool parser, and DFlash integration
xianbaoqian/vllm#1	DFlash architecture registration, assistant config, weight-name mapping, and multimodal target unwrapping fixes
xianbaoqian/vllm#2	Muse Glimmer model-support corrections
xianbaoqian/vllm#3	Non-causal block-diffusion DFlash attention fix
xianbaoqian/vllm#4	Muse source pre-commit, typing, and parser-registration corrections
xianbaoqian/vllm#5	Consolidated Muse parser/config regression tests
xianbaoqian/vllm#6	Registry, vision-depth override, parser-state, and CI follow-up fixes; image pins head a52f4d9c8640332842bfc3428382f5ad1fc5bdda
vllm-project/vllm#48735	Correct explicit --linear-backend filtering and FlashInfer b12x NVFP4 discovery
vllm-project/vllm#40082	FlashInfer b12x FP4 GEMM backend for SM120/SM121

Local compatibility changes preserve the Muse assistant's encoder.fc and encoder.output_norm_enc checkpoint names with the pinned base loader, retain the validated SM120 vision-parallel API, and keep b12x explicit rather than automatic.

Runtime verification of the published image

The immutable image above was loaded with this checkpoint and meta-models/Muse-Glimmer-30B-assistant on one RTX PRO 6000 Blackwell GPU. Verification observed:

• healthy service startup with zero container restarts;
• FlashInferB12xNvFp4LinearKernel selected for NVFP4 GEMM;
• piecewise CUDA graph capture and DFlash speculative depth 16;
• deterministic text generation with final content MUSE GLIMMER READY;
• a valid get_weather({"city":"Paris"}) tool call through the Muse parser;
• successful image preprocessing and a correct one-sentence description of the test scene;
• the same Tetris generation protocol used for the earlier baseline completed 4,501 output tokens in 32.876 seconds (136.91 end-to-end output tokens/s); because sampling is stochastic, this is a runtime check rather than a controlled performance claim;
• eight concurrent 512-token requests all completed, producing 4,096 tokens in 7.362 seconds, after which the service remained healthy with zero restarts.

Limitations

• The published container is qualified for Linux/amd64 and NVIDIA SM120. A one-DGX-Spark, TP=1 SM121 canary loaded the checkpoint and selected both b12x and CUTLASS NVFP4 paths, but the available ARM64 runtime failed its profile run with CUDA error: no kernel image is available for execution on the device. SM121 is therefore not qualified; exact attempts are recorded in evaluation/single-spark-canary.json.
• ModelOpt NVFP4 checkpoint support in vLLM is experimental and its serialization contract can change.
• The DFlash assistant increases memory use and requires access to the gated assistant repository.
• Quantization can change rare-token probabilities and downstream behavior even when aggregate benchmarks pass.
• Validate task-specific accuracy, long-context behavior, safety, and tool permissions before deployment.

License and use

The base model and this quantized checkpoint are distributed under the Apache License 2.0. Review the base model card for architecture, intended use, limitations, and safety information.