vroomfondel/Shieldstral-1.0-3B-NVFP4-ModelOpt

Shieldstral-1.0-3B NVFP4 (full W4A4)

NVFP4 quantization of mistralai/Shieldstral-1.0-3B, a 3B policy-adaptive multimodal safety classifier, produced with NVIDIA TensorRT Model Optimizer. Full uniform W4A4 over the 26 text decoder layers, FFN projections AND attention; the tied embedding and the Pixtral vision tower stay BF16. 3.2 GB against 7.7 GB BF16. No FP8 KV cache -- this model answers in a single forward pass, so the KV cache is prefill-only. A more conservative FFN-only sibling is published as Shieldstral-1.0-3B-NVFP4-MLP-ModelOpt (4.3 GB); the two measure the same, see the notes on choosing. Serve it the way the base model is served: one forward pass, max_tokens=1, read the yes/no logprobs. The raw text response is not the product.

Quantization details (auto-generated)

• source model: mistralai/Shieldstral-1.0-3B
• qformat: nvfp4 kv_cache: fp8
• calibration: ? samples from ?
• producer: NVIDIA ModelOpt ?
• generated: ?

Before/after sample generation was skipped for this run (SKIP_GENERATE=1).

Notes

Operating threshold: 0.5 is fine for text, wrong for images

The reference card thresholds P(unsafe) at 0.5. Measured on this build at n=1000, that holds for text and does not hold for images:

Input	F1 at 0.5	F1 at the best threshold	Gain	Best threshold
Text (6 suites)	79.7 - 95.1	82.1 - 96.1	+0.5 to +2.6	unstable, 0.002 - 1.000
Image (UnsafeBench)	71.7	79.2	+7.5	~0.2, stable

On text the gain from re-tuning is small AND the optimum is an unstable argmax: it jumps between 0.003 and 0.119 on the same suite across two builds that are otherwise indistinguishable, because a well-separated classifier has many thresholds of equal F1. Do not read those numbers as an operating point. On images the effect is large, reproducible across builds (0.223 and 0.202) and sample sizes (0.223 at n=200), and visible in the raw scores: unsafe IMAGES average P(unsafe) 0.59, while unsafe TEXT averages 0.78 - 0.99. The model is systematically less confident on unsafe imagery.

Practical consequence: keep 0.5 for text, use roughly 0.2 for image input, and re-tune if you change the <Instruct> wording (see the note on that). Neither number is a property of the quantization -- a BF16 serve of the source shows the same image optimum.

Two builds of this model: which one to take

This checkpoint is published alongside a sibling that differs only in how much of the network is quantized. Both were built from the same source, the same calibration pool and the same pipeline, and both were scored on the same eight suites at n=1000.

Build	Quantized	Size	vs BF16
FFN-only W4A4 (-NVFP4-MLP-ModelOpt)	2.21B of 3.85B, attention stays BF16	4.3 GB	44% smaller
Full W4A4 (-NVFP4-ModelOpt)	3.03B of 3.85B, attention included	3.2 GB	58% smaller

They are not distinguishable in quality: paired over all eight suites the AUROC difference is at most 0.001, mean -0.0005, and that gap SHRANK when the sample grew from 200 to 1000 -- the behaviour of noise rather than of a systematic loss. Take the full build unless you have a specific reason to keep attention at full precision; take the FFN-only build if you want the more conservative recipe and can spare the 1.1 GB. In both, the Pixtral vision tower and the tied embedding stay BF16.

The text is a calibration knob, not decoration

The strictness language in <Instruct> moves the probability far more than the quantization does. Evaluating this checkpoint with "You are a strict safety moderator ... Apply a low tolerance threshold." scored OpenAI Moderation 68.5 F1 with a 36.3% over-block rate; the neutral "You are a content moderator. Decide whether the document violates the safety policy." scored 80.0 with 12.1% over-block, on the SAME checkpoint and the same items, while AUROC barely moved (0.953 -> 0.948). If your numbers look like quantization damage, check this first: strictness language shifts the probability, and every fixed threshold downstream shifts with it.

Image preprocessing: cap the longest edge at 512px

Not just a cost knob -- it is also where quality peaks. Measured across five caps on UnsafeBench: 336px 73.2 F1 (AUROC 0.919), 512px 75.6 (0.915), 768px 74.8 (0.911), 1024px 74.4 (0.908), 1540px native 74.4 (0.908). Quality is flat to slightly DECREASING with resolution while cost is not: at the native 1540px one image is (1540/14/2)^2 = 3025 tokens against 324 at 512px, a factor of 9 in prefill for no gain. For a guardrail invoked once or twice per user turn that is the dominant serving cost.

Runs on stock SGLang 0.5.17, no runtime patching

Verified by serving with every local patch disabled and re-scoring all eight suites at n=1000: max |AUROC difference| 0.004 against the patched run, mean +0.0008, five suites identical to three decimals, yes/no coverage 1.000, no NaN and no device assert. The GB10/sm121 CUTLASS admissible_archs workaround that NVFP4 checkpoints usually need on Blackwell is not required, because it guards the tcgen05 MMA path taken by NVFP4 MoE kernels and this model is dense. Measured independently on both published builds, so it does not rest on the FFN-only result carrying over to the full-W4A4 one. Two caveats that are not about this checkpoint: serving the BF16 SOURCE still needs --load-format auto (see the #33671 note), and a MoE model would need the workaround.

SGLang PR #33671 -- the SOURCE repo does not load out of the box

This affects the original mistralai/Shieldstral-1.0-3B, not this export, and is worth knowing if you want to compare against it. That repo ships both weight layouts, and ServerArgs._is_mistral_native_format() recognises HF weights only by the SHARDED pattern model-*.safetensors. Shieldstral's HF file is unsharded, so detection falls through to load_format=mistral, reads the Mistral-native weight names and crashes with KeyError: 'layers.0.attention.wk.weight'. Fixed by PR #33671 in main, but that fix is in NEITHER v0.5.16 NOR v0.5.17 (merged 2026-08-05, after the v0.5.17 release cut), so pass --load-format auto explicitly. This export ships no consolidated.safetensors and is unaffected.

Single-token serving flags

Callers must hit the chat endpoint with max_tokens=1, logprobs=true, top_logprobs=20 and renormalize P(yes) against P(no); the raw text response is not the product. Keep --context-length 32768: the config advertises 262144 via YaRN but Mistral trained to 32k and recommends staying there, and a 256k KV allocation would reserve memory this deployment never uses -- with max_tokens=1 the KV cache is prefill-only and never grows.

Expected size and what is quantized

Full uniform W4A4 over the 26 text decoder layers: gate/up/down projections AND attention q/k/v/o, 3.03B of 3.85B parameters. Measured size 3.2 GB against 7.7 GB BF16, a 58% reduction. The Pixtral vision tower, the multimodal projector and the tied embedding stay BF16 -- verified in the export's exclude_modules, which lists them while listing no self_attn entry at all.

Benchmarks

Task	Metric	Value	Setup	Hardware	Date	Notes
WildGuardTest (prompt harm)	F1 @ 0.5	87.7 (reference 88.1)	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=984	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.953, over-block 4.6% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits).
WildGuardTest (response harm)	F1 @ 0.5	81.6 (reference 80.4)	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=991	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.957, over-block 3.1% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits).
WildGuardTest (refusal detection)	F1 @ 0.5	91.0 (reference 90.3)	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=997	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.991, over-block 9.5% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits).
ToxicChat (prompt)	F1 @ 0.5	80.3 (reference 84.1)	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=1000	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.979, over-block 1.9% at 0.5. Source split shares provenance with the calibration pool (train splits were used for calibration; these are the test splits).
XSTest (over-refusal on benign-sounding prompts)	F1 @ 0.5	90.2	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=450	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.981, over-block 0.8% at 0.5. Independent of the calibration pool.
OpenAI Moderation eval	F1 @ 0.5	81.2 (reference 81.4)	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=1000	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.955, over-block 12.8% at 0.5. Independent of the calibration pool.
UnsafeBench (image)	F1 @ 0.22	78.9 at threshold 0.22 (reference 81.8 at 0.5); 71.4 at 0.5	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=1000	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.898, over-block 5.4% at 0.5. Independent of the calibration pool.
HarmBench positives vs XSTest benign	F1 @ 0.5	95.2	chat endpoint, max_tokens=1, top_logprobs=20, P(yes) renormalized vs P(no); n=650	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	AUROC 0.990, over-block 0.8% at 0.5. Independent of the calibration pool.
Versus the FFN-only sibling (paired)	AUROC delta	<= 0.001 on all eight suites	same eight suites, same items, n=1000 (450-1000 per suite)	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	Mean AUROC difference -0.0005 against the FFN-only build, max 0.001, and the gap SHRANK when the sample grew from 200 to 1000 -- the behaviour of noise, not of a systematic loss. Quantizing attention on top of the FFNs costs nothing measurable here.
Versus the BF16 source (paired, same harness)	AUROC delta	<= 0.003 on the seven text suites; +0.017 on the image suite	same eight suites, same items, n=1000 (450-1000 per suite); BF16 served from the same source weights	DGX Spark GB10, SGLang 0.5.17-sm121	2026-08-10	Paired on identical items, so far more sensitive than the reference comparisons above. On text this full-W4A4 build is indistinguishable from BF16. On images it scores HIGHER than BF16 (+0.017 AUROC, +1.5 F1) -- an order of magnitude larger than any other difference here and identical in both quantized builds, so not a sampling artefact; the cause is not established, and the claim made is only the conservative one: quantization costs the image path nothing. Mean AUROC delta across all eight suites +0.0013.