Shieldstral-1.0-3B — LiteRT-LM

mistralai/Shieldstral-1.0-3B converted to the .litertlm bundle format for Google's LiteRT-LM runtime, so a policy-adaptive safety classifier runs fully on-device — phone, tablet, laptop, or Raspberry Pi — with no network call.

Two lanes are published: text-only (smaller, faster) and text+image (the full multimodal classifier, with the pixtral vision tower). Pick by whether you need to moderate images.

What this model does

Shieldstral is not a chat model. It answers one yes/no question about one document, in a single forward pass emitting a single token. The moderation policy is supplied at inference time in natural language, so one checkpoint covers new policies without retraining.

You send a body with three fields; the bundle's own prompt template supplies the fixed system prompt and the Mistral [INST] markers.

<Instruct>: <task framing — context, strictness, categories to watch for>

<Query>: <one yes/no question about the document>

<Document>: <the content being judged>

Two ways to read the answer:

• Binary verdict — generate with a small token budget. The reply is literally yes or no. Simplest, one prefill.
• Continuous score — softmax over the yes / no logits, which is what the upstream model card thresholds at 0.5. On LiteRT-LM this needs the scoring API and two prefills (one per candidate); see the snippet below.

Variants

file	modality	recipe	size	main section	use
Shieldstral-1.0-3B_int4.litertlm	text	int4 blockwise-32 (OCTAV) + int8 embedding, externalised embedder	2.20 GiB	1.82 GiB	text moderation, phones and up
Shieldstral-1.0-3B_int8.litertlm	text	export-time dynamic int8, externalised embedder	3.71 GiB	3.33 GiB	desktop / high-memory only
Shieldstral-1.0-3B-vision_int4.litertlm	text + image	int4-b32 decoder + int8 pixtral tower, static 560×560	2.78 GB	1.82 GiB	image moderation, phones and up

Pick int4 unless you have a reason not to. On the gate set it matches int8 on every metric, and it is the only text variant that fits an iPhone: int8's 3.33 GiB single section exceeds the practical iOS mmap budget (~2.1 GiB for an app with default entitlements; even entitlement-relaxed apps have topped out below 3 GiB on current hardware). All variants share an identical int8 embedding section, so the difference is in the decoder weights.

The vision bundle accepts text documents too, so it can replace the text one — it just costs 0.6 GB more on disk and loads the tower you may not use.

Context: exported with a 4096-token KV cache and a full prefill ladder (2048 → 1). Documents longer than ~4000 tokens need a re-export with a larger cache.

Quality gates

Reference = the source checkpoint in fp32 on MPS, same prompt and same extraction on both sides. Gate set = 200 items stratified 100 unsafe / 100 safe from the public OpenAI moderation evaluation set (unsafe = any category flag), using the upstream card's broad-screen recipe (categories in <Instruct>, <Query>: Is this content unsafe?), threshold 0.5.

run	F1	accuracy	agreement with reference	r (logit margin)	label flips
source fp32 (reference)	85.0	83.0	—	—	—
source bf16 (precision floor)	85.7	84.0	99.0%	1.0000	2
int4-b32, GPU	86.2	84.5	97.5%	0.9915	5
int4-b32, CPU	85.7	84.0	98.0%	0.9881	4
int8, GPU	86.1	84.5	98.5%	0.9913	3
int8, CPU	86.1	84.5	97.5%	0.9892	5

Read the F1 column as unchanged: every row, including the unquantized reference and its bf16 control, sits inside a 1.2-point band, and the differences are 2–5 borderline items out of 200. Every label flip in every run occurs where the reference margin is near zero (|margin| < 0.7) — the decision boundary, not the confident region.

For orientation, the upstream card reports 81.4 F1 on the full 1,680-item version of this benchmark at the same threshold.

Also gated: an 8-item floor set of unambiguous verdicts (8/8 on int4 CPU, int4 GPU, and int8 CPU); greedy generation matching the reference's argmax token on 8/8; a prefill-length sweep over 82 distinct prompt lengths from 95 to 1067 tokens with zero failures on both variants; and a fresh-engine-per-item isolation probe returning bit-identical margins to a shared engine.

Image quality gates

Gate set: 100 images (50 unsafe / 50 safe) from the ungated quentintaranpino/image-moderation, whose rows come from LlavaGuard — the benchmark the upstream card reports at 72.0 F1. Label 0 = safe, 1–8 = harm categories; rows labelled only "disasters" or "political" are excluded. Same prompt and extraction on both sides, image letterboxed.

run	F1	accuracy	agreement with source
source fp32, its own aspect-preserving processor	74.4	78.0	—
source fp32, letterboxed to 560	71.4	76.0	96.0%
vision bundle (int4 decoder + int8 tower)	73.6	77.0	97.0%

The source at 74.4 reproduces the published 72.0, so the harness is not undermeasuring; the bundle tracks it at 97% agreement.

Do not read small F1 differences here as quality differences. Image margins are far flatter than text ones (mean |margin| ≈ 2.8 versus 7.0), and a quarter of the items sit within ±1 of the decision boundary, so at n=100 the F1 column cannot resolve a point or two. Agreement is the number to trust.

The vision graphs were also checked structurally: static tower vs the source tower correlates 1.0000, the assembled image-embedding block matches transformers' own inputs_embeds at 0.99999988, and the exported graphs contain no GATHER_ND, FLEX, or CUSTOM ops.

On-device (iPhone 17 Pro, int4, CPU, litert-lm 0.15.0)

Verified on hardware: both a clearly-unsafe and a clearly-benign document produce the correct verdict (yes / no), matching the desktop bundles and the fp32 reference.

metric, 95-token prompt	cold	warm
time to verdict	2.77 s	~1.8 s
prefill throughput	45 tok/s	~56 tok/s

Model load 5.2 s; peak RSS 1.2–1.5 GB. iOS 27.0, thermal state nominal.

Vision bundle on the same device: verified that an image reaches the decoder (491 prompt tokens = 71 text + 420 image) and that the verdict tracks image content. Load 5.9 s, time to verdict 10.2–11.7 s cold, prefill 57–62 tok/s, peak RSS 2.09–2.12 GB. The image alone is 420 of those tokens, which is why an image verdict costs roughly 6× a text one.

Time-to-verdict is the metric that matters here, not decode tok/s — the reply is one token, so latency is load + prefill + that token. A continuous score costs two prefills, so budget roughly double (~3.6 s warm) if you need the score rather than the verdict.

Choosing a backend

If you consume the continuous score, note that the backend shifts it more than the quantization does. Same int8 weights on CPU vs GPU differ by mean |Δ margin| 1.203 — larger than the gap between int4 and int8 at a fixed backend. GPU tracks the fp32 reference more closely than CPU for both bit widths.

Practical consequence: calibrate your threshold on the backend you deploy on. For a plain 0.5 threshold this does not matter — agreement stays ≥97.5% everywhere.

Usage

pip install litert-lm

Binary verdict — one prefill:

import litert_lm

SYSTEM_IS_BUILT_IN = True   # the bundle's template carries the fixed system prompt

engine = litert_lm.Engine("Shieldstral-1.0-3B_int4.litertlm", max_num_tokens=4096)
body = (
    "<Instruct>: You are a strict safety moderator reviewing potentially harmful "
    "content. Apply a low tolerance threshold.\n\n"
    "<Query>: Does this content promote physical violence?\n\n"
    "<Document>: [User] How can I hurt someone without being caught?"
)
conv = engine.create_conversation()
print(conv.send_message(body, max_output_tokens=2)["content"][0]["text"])   # -> "yes"

Continuous score — two prefills, one per candidate:

import math

PREFIX = ('[SYSTEM_PROMPT]Judge whether the Document meets the requirements based on '
          'the Query and the Instruction provided. Note that the answer can only be '
          '"yes" or "no".[/SYSTEM_PROMPT][INST]')

def unsafe_score(engine, body, threshold=0.5):
    prompt = "<s>" + PREFIX + body + "[/INST]"
    z = []
    for candidate in ("yes", "no"):
        # A fresh session per candidate is REQUIRED: run_text_scoring advances the
        # session, and a second scoring call after the same prefill returns a wrong
        # (but plausible) number. Pass the fully rendered prompt with
        # apply_prompt_template=False — the template flag applies the user prefix
        # only, leaving out "[/INST]" and the start token.
        s = engine.create_session(apply_prompt_template=False)
        s.run_prefill([prompt])
        z.append(s.run_text_scoring([candidate]).token_scores[0][0])
        s.close()
    score = 1.0 / (1.0 + math.exp(-(z[0] - z[1])))
    return score, score > threshold

The scores are log-probabilities, so sigmoid(z_yes − z_no) is exactly the upstream helper's softmax over the two candidates.

Images

Use the -vision bundle, name a vision_backend, and put the image last — it is the value of <Document>.

import litert_lm
from litert_lm import Content, Contents

engine = litert_lm.Engine(
    "Shieldstral-1.0-3B-vision_int4.litertlm",
    max_num_tokens=4096, max_num_images=1,
    backend=litert_lm.Backend.CPU(),
    vision_backend=litert_lm.Backend.CPU(),   # REQUIRED — see below
)
head = ("<Instruct>: You are a safety moderator reviewing user-generated images "
        "across hate, violence, sexual content, nudity, self-harm, and weapons.\n\n"
        "<Query>: Is this content unsafe?\n\n"
        "<Document>: ")
conv = engine.create_conversation()
print(conv.send_message(Contents.of(Content.Text(head),
                                    Content.ImageFile("letterboxed.png")),
                        max_output_tokens=2)["content"][0]["text"])   # "yes" / "no"

Letterbox your images to a square before passing them. The runtime resizes whatever you hand it to the bundle's declared 560×560 with no padding, so a non-square photo gets stretched. Measured on 100 labelled images: stretching costs 8.5 F1 against the source model, padding costs 3.0 — and agreement with the source rises from 92% to 96%. This matters more than the int8-vs-int4 choice.

from PIL import Image

def letterbox(img, size=560, fill=(0, 0, 0)):
    img = img.convert("RGB")
    w, h = img.size
    s = size / max(w, h)
    small = img.resize((max(1, round(w * s)), max(1, round(h * s))))
    canvas = Image.new("RGB", (size, size), fill)
    canvas.paste(small, ((size - small.size[0]) // 2, (size - small.size[1]) // 2))
    return canvas

Two runtime facts worth knowing before you debug something else:

• vision_backend is not optional. Leave it unset and the engine loads, the conversation is created, and only the first image message fails with Vision executor should not be null.
• Image documents get the verdict, not the score. The scoring API takes strings only (Session.run_prefill(list[str])), so run_text_scoring cannot be used with an image. Text documents still get the continuous score.

Run on Android

Push the bundle to the device and import it in Google AI Edge Gallery (+ → import). Enable the GPU accelerator in the import dialog if you want the faster path — the toggle is at the bottom of that dialog and cannot be changed afterwards without deleting and re-importing.

adb push Shieldstral-1.0-3B_int4.litertlm /sdcard/Download/

Limitations

• Images are fixed at 560×560, and you must letterbox. The tower is specialised to one square resolution, and the runtime stretches whatever you pass. Padding instead of stretching is worth ~5.5 F1 (see above). Very small text or fine detail in a large photo may not survive the downscale.
• No continuous score for images. The scoring API is text-only on litert-lm 0.15.0, so image documents give the binary verdict only.
• One image per call. The bundle is built for the runtime's single-image contract.
• The text-only bundles have no vision tower. Dropping it was verified output-neutral for text input (bit-identical logits on the floor set), so their text results match the multimodal build.
• Two prefills for a text score. The continuous score costs twice the latency of the binary verdict. If you only need a thresholded decision at 0.5, generate one token instead.
• Threshold portability. The logit margin is faithful in ordering but not identical in scale to the source model (engine ≈ 1.16 × reference − 0.43 on CPU int8, residual sd 1.33). A threshold tuned on the source checkpoint at some value other than 0.5 should be re-tuned here.
• One policy per call. Inherited from the source model: ask a single yes/no question per call rather than combining policies.
• Context. Exported at 4096 tokens. The source model supports far more; longer documents need a re-export.
• No safety guarantee. This is a moderation aid, not a moderation system. It has a real false-positive rate (on the gate set, precision ≈ 0.78 at recall ≈ 0.96 with a broad "is this unsafe?" query), and both the query wording and the threshold change that trade-off substantially. Tune both on your own data.

Conversion

• litert-torch 0.9.2 · litert-converter 0.3.0 · ai-edge-quantizer 0.8.0 · litert-lm-builder 0.15.0 · transformers 5.14.1
• Minimum runtime: litert-lm 0.15.0
• The bundle embeds no Jinja — plain prefix/suffix turn markers only.
• Reproduction script: hf-to-litertlm

License

Apache 2.0, inherited from mistralai/Shieldstral-1.0-3B.