gemma-shield-e2b-trimodal-lora

LoRA adapter (r=16, 97 MB, language-model attention and MLP projections only) for google/gemma-4-E2B-it that turns it into a ShieldGemma-style safety classifier for text, images and spoken requests. This is the un-merged form of tbtlr/gemma-shield-e2b-trimodal; both score identically on text and images (verified to 4 decimals) and within bf16 rounding on audio (mean |diff| 0.001 over 64 clips).

Independent fine-tune of Google's Gemma 4 E2B-it. Not a Google release and not affiliated with the ShieldGemma team; "Gemma" is used only to describe the base model.

Usage

from transformers import AutoProcessor, Gemma4ForConditionalGeneration
from peft import PeftModel
import torch
base = Gemma4ForConditionalGeneration.from_pretrained("google/gemma-4-E2B-it", dtype=torch.bfloat16)
m = PeftModel.from_pretrained(base, "tbtlr/gemma-shield-e2b-trimodal-lora").cuda().eval()
p = AutoProcessor.from_pretrained("tbtlr/gemma-shield-e2b-trimodal-lora")
yes, no = p.tokenizer.encode("Yes", add_special_tokens=False)[0], p.tokenizer.encode("No", add_special_tokens=False)[0]
# logits = m(**inputs, logits_to_keep=1).logits[:, -1]; p_violation = softmax([logits[yes], logits[no]])[0]

Prompt templates, policy strings, training recipe, evaluation and limitations: see the merged model card at tbtlr/gemma-shield-e2b-trimodal and the project README.

Policies

Text (ShieldGemma v1 wording): dangerous content, harassment, hate speech, sexually explicit, violence, obscenity, plus a composite "any harm" policy. Image (ShieldGemma 2 wording): sexually explicit, violence/gore, dangerous content, plus a composite. Audio: the text prompt policies applied to a spoken user request (the audio clip replaces the text of the request). The classifier is policy-conditioned: it also follows policy text it was not trained on (tested with OpenAI Moderation's own eight definitions). Exact prompt templates and policy strings are in shield_policies.json.

License and training data

Weights and code: Apache 2.0, the same license as the base model (LICENSE in this repo). The model was trained on the datasets below; two of the text sets are CC BY-NC 4.0. Whether that restriction carries over to model weights is legally unsettled, so commercial users should review the list and make their own assessment.

dataset	role	declared license
Aegis 2.0 (NVIDIA)	text train / eval	CC BY 4.0
BeaverTails	text train / eval	CC BY-NC 4.0
ToxicChat	text train (benign) / eval	CC BY-NC 4.0
WildGuardMix	text train / eval	ODC-BY
XSTest	text eval	CC BY 4.0
OpenAI Moderation eval	text eval	MIT
VLGuard	image train / eval	MIT
LlavaGuard (images from SMID)	image train / eval	Apache 2.0 (SMID: CC BY 4.0 original release)
UnsafeBench	image train / eval	gated data-use agreement (research, education, responsible commercial use)
COCO val2017	image train (benign)	CC BY 4.0 annotations; images under their Flickr licenses
LibriSpeech	audio train (benign) / eval	CC BY 4.0
LibriTTS-R Piper voice (en_US-libritts_r-medium)	TTS for audio train / eval	CC BY 4.0
JALMBench	audio eval	not declared

Limitations

Binary "any unsafe" scores depend on how far a benchmark's taxonomy is from these policies (hate, political, privacy, fraud and misinformation images/text are out of scope). The dangerous-content image policy was trained with a broader reading (weapons, substance abuse, criminal planning, self-harm imagery) than its text counterpart. Audio training used synthetic (TTS) speech only; real recorded speech, accents, noise and non-English audio are untested. On the same spoken prompts, transcribing with Gemma 4 and scoring the transcript with the text policy still scores higher (0.886 F1) than the single-pass audio shield (0.852); the audio pathway is the faster option with a small accuracy cost. English only. Not a substitute for human review in high-stakes settings.