saroku-guard

saroku-guard is a classifier that judges whether a proposed AI agent tool call is safe to execute, before it runs. It is the runtime Policy Decision Point in saroku, sitting in the execution path of agent tool calls to catch unsafe actions in single-digit milliseconds, with no API calls and no data leaving the local environment.

Model Details

• Type: Text classifier (safe / unsafe), with a secondary violation-type output
• Task: Pre-execution safety judgment for agentic tool calls
• License: Apache 2.0
• Output: Binary label (safe / unsafe) plus, on an unsafe verdict, an associated violation category

Intended Use

saroku-guard is designed to run as the first-line check in an agent execution pipeline, immediately before a tool call is invoked. It is built for high-throughput, low-latency screening: the large majority of routine actions are cleared locally, and only flagged actions are escalated to a more expensive analysis step (an LLM judge).

In scope: judging a single, discrete tool call (name, arguments, and surrounding context) for safety prior to execution.

Out of scope: conversational content moderation, jailbreak or prompt-injection detection in free-form chat, and post-hoc audit of actions that have already executed. The violation category returned alongside an unsafe verdict is intended as supporting context for logging and review, not as the sole input to automated policy branching.

How to Use

The recommended way to use saroku-guard is through the saroku SDK, which handles input formatting automatically:

pip install saroku

from saroku import SafetyGuard

guard = SafetyGuard()  # saroku-guard runs by default, no configuration required

result = guard.check(
    action="DELETE FROM users WHERE last_login < '2023-01-01'",
    context="Production database agent.",
    operator_constraints=["Never DELETE on prod without confirmation"],
)

if not result.is_safe:
    print(result.summary())

Direct use

The model can also be loaded directly with transformers. Weights are merged and self-contained — no adapter or separate base model is required.

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained(
    "karanxa/saroku-guard",
    num_labels=2,
    id2label={0: "unsafe", 1: "safe"},
    label2id={"unsafe": 0, "safe": 1},
)
tokenizer = AutoTokenizer.from_pretrained("karanxa/saroku-guard")

Direct use requires reproducing the model's expected input structure (action, context, constraints, and related fields, assembled in a fixed format). The saroku SDK implements this natively; see its serializer for the reference implementation.

Where This Fits

agent action
     │
     ▼
saroku-guard  ── safe ──▶  allow
     │
   flagged unsafe
     │
     ▼
LLM judge (if configured)  ──▶  allow / block, with full analysis
     │
   no LLM judge configured
     │
     ▼
block, with the violation category attached

Evaluation

Evaluated on a held-out set of 1,920 agent tool-call decisions across 16 domains, disjoint from training and validation data.

Metric	Score
Accuracy	97.7%
Unsafe recall	97.9%
Unsafe precision	98.6%
Latency, p50 (single request)	~6.5ms
Latency, p50 (batched, n=32)	~0.7ms

The model is tuned to favor recall on the unsafe class, consistent with its role as a pre-execution safety gate.

Limitations

• English-language tool-call text.
• Training data was generated synthetically across 16 domains; performance on tool-call patterns well outside that distribution has not been independently verified.
• saroku-guard is a fast first-pass filter, not a complete safety system. For irreversible or high-consequence actions, pair it with saroku's LLM-based judge for deeper analysis.

Citation

If you use saroku-guard in your work, please reference the saroku project.

---

Built on microsoft/deberta-v3-base.