Galvanize
Collection
1 item • Updated
Galvanize-60M: Agent-Native Prompt Injection Classifier
Galvanize-60M is a 60-million parameter distilled transformer classifier engineered specifically to detect prompt injection, instruction overrides, and delimiter smuggling attacks in autonomous agent workflows and Model Context Protocol (MCP) tool pipelines.
Derived from answerdotai/ModernBERT-base through structural distillation into 4 transformer layers, it incorporates native Rotary Position Embeddings (RoPE) for up to 8,192 tokens of context and a specialized MultiHeadSecurityPooling architecture designed to maintain high precision on structured tool schemas.
Base Model, Lineage & License Transparency
In accordance with open science practices and upstream licensing requirements:
- Base Architecture: Sliced and distilled from
answerdotai/ModernBERT-basecreated by Benjamin Clavié, Dylan Slack et al. (Answer.ai & LightOn). - Upstream License: Apache 2.0.
- Model License: Apache 2.0. This model is distributed permissively for open research, commercial deployment, and private use.
- Training Objective: Distilled using cross-entropy and margin ranking loss against an ensemble of certified security adversarial suites and benign function-calling schemas, focusing on wrapper invariance.
Empirical Benchmark Evaluation
The model was evaluated against 9 industrial security validation gates on blind test splits. All numbers below reflect measured empirical results:
| Benchmark / Evaluation Metric | Galvanize-60M (zn) | ProtectAI-DeBERTa-v3 | Meta-Prompt-Guard-2-86M | Meta-Prompt-Guard-2-22M |
|---|---|---|---|---|
| Tool False Positive Rate (FPR) | 1.00% (0.67% @ $\tau=0.80$) | 90.33% (Fails on tool calls) | Not published | Not published |
| Tool Benign False Alarm Rate | 0.53% | 88.90% | Not published | Not published |
| Deep Injection Recall (OOD Deepset) | 91.60% | 20.42% | 9.58% | 8.33% |
| Long Needle Recall (2,000–8,000 tokens) | 77.00% – 97.00% | 1.00% (Truncates at 512) | 7.00% (Truncates at 512) | 5.00% (Truncates at 512) |
| Inference Latency (p50 single CPU core) | 11.52 ms (FP32) / 18.18 ms (INT8) | 55.79 ms | 45.36 ms | 24.12 ms |
| Supported Context Window | 8,192 tokens (Native RoPE) | 512 tokens | 512 tokens | 512 tokens |
| Deployment Footprint | 176 MB (INT8 ONNX) | ~440 MB | ~340 MB | ~90 MB |
Visual Comparisons
1. Tool-Calling False Positive Rate (Lower is Better)
Galvanize-60M (1.00%) [■] 1.00%
Meta-PG-2-86M [?] Not published
ProtectAI-DeBERTa-v3 [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] 90.33%
Takeaway: Standard sentence classifiers produce false alarms on normal tool schemas (JSON arguments, SQL queries, code snippets). Galvanize-60M was trained with structured negative pairs to prevent breaking legitimate agent function calls.
2. Out-of-Distribution Deep Injection Recall (Higher is Better)
Galvanize-60M (91.60%) [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] 91.60%
ProtectAI-DeBERTa-v3 [■■■■■■■■] 20.42%
Meta-PG-2-86M [■■■■] 9.58%
Meta-PG-2-22M [■■■] 8.33%
3. Native Context Window (Tokens)
Galvanize-60M (8,192 tok) [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] 8,192 tokens
ProtectAI-DeBERTa-v3 [■■] 512 tokens
Meta-PG-2-86M [■■] 512 tokens
Architecture: MultiHeadSecurityPooling
Standard CLS or average mean pooling causes representation collapse when evaluating mixed syntax (e.g. valid JSON wrapping a malicious instruction).
Galvanize-60M replaces generic pooling with MultiHeadSecurityPooling: 4 learned attention queries probe the sequence across all 8,192 positions and are concatenated into a 3,072-dimensional representation:
This ensures that the decision boundary separates benign syntax wrapping from adversarial intent, preventing both false rejections on complex JSON structures and false negatives on obscured injection payloads.
Limitations and Failure Modes
We encourage responsible deployment and transparency regarding operational boundaries:
- Task Specificity: Galvanize-60M is strictly trained for binary prompt injection classification. It is not an LLM, does not generate text, and is not calibrated for general toxic speech, hate speech, or topic categorization.
- Layer Pruning Trade-off: Slicing from 12 layers down to 4 layers achieves sub-15ms CPU speeds, but reduces broader semantic nuance compared to full-depth foundational encoders.
- Threshold Calibration: The default calibrated decision threshold is $\tau = 0.80$. Increasing $\tau$ reduces false alarms further; decreasing $\tau$ maximizes recall at the cost of higher rejection rates on borderline prompts.
Quickstart: CPU Inference with ONNX Runtime (<15 ms)
The repository includes a ready-to-run INT8 ONNX graph in onnx/model_quantized.onnx:
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
# 1. Load tokenizer and ONNX session
tokenizer = AutoTokenizer.from_pretrained("usezn/Galvanize-60M")
session = ort.InferenceSession("onnx/model_quantized.onnx", providers=["CPUExecutionProvider"])
# 2. Tokenize input
payload = '{"tool": "database_query", "parameters": {"sql": "SELECT * FROM users"}}'
inputs = tokenizer(payload, max_length=8192, truncation=True, return_tensors="np")
ort_inputs = {k: v.astype(np.int64) for k, v in inputs.items()}
# 3. Predict probability
logits = session.run(None, ort_inputs)[0]
prob = 1.0 / (1.0 + np.exp(-logits[0][0]))
# 4. Apply threshold tau = 0.80
verdict = "BLOCK" if prob >= 0.80 else "ALLOW"
print(f"Risk Score: {prob:.4f} | Decision: {verdict}")
Model Context Protocol (MCP) Drop-in
To use Galvanize-60M as an automatic guardrail layer for Claude Desktop, Cursor, or OpenCode:
npx -y zn-guard-mcp
Or query the managed cloud gateway:
curl -X POST https://api.usezn.com/analyze \
-H "Authorization: Bearer $ZN_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": "text from untrusted agent context"}'
Citation & Acknowledgments
@misc{usezn2026galvanize60m,
author = {zn Security Team},
title = {Galvanize-60M: Agent-Native Prompt Injection Classifier},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/usezn/Galvanize-60M}
}
@article{clavi2024modernbert,
author = {Benjamin Clavi{\'e} and Dylan Slack and others},
title = {ModernBERT: Bringing Modern Architectures to Bidirectional Encoders},
year = {2024},
journal = {arXiv preprint arXiv:2412.13663}
}
- Downloads last month
- 19