Interlock

Interlock judges a proposed agent tool call against what the user actually asked for. Content shields read the request and ask whether it is harmful. Interlock reads the request/action pair and asks a different question: is this call more than what was asked?

It is deliberately blind to where a deviation came from. A widened path argument looks the same whether it originated in a prompt injection, a jailbreak, or an ordinary model mistake, and the defence is the same either way.

Output

Three-valued, never binary:

decision	meaning
allow	the call stays within what the request asked for
confirm	the call should be put to the user before it runs
block	the call goes materially beyond the request

The three decisions are allow, confirm, block. Alongside the decision the model predicts a deviation type, one of none, scope_expansion, unrequested_action, irreversible_unconfirmed, target_mismatch, out_of_scope, privilege_escalation.

confirm is a first-class outcome. Collapsing it into block would trade a question for a refusal on exactly the cases where the request is ambiguous.

Two tiers

Tier 1 -- linear (1080 parameters). A two-headed multinomial logistic regression over 107 structured request/action alignment features. Pure NumPy at inference: no tokeniser, no torch. This is the tier that runs on every agent turn. Shipped as interlock_linear.npz, with the fitted operating point in policy.json.

Tier 2 -- encoder (optional). ModernBERT-base with a decision head and a deviation head, reading a role-annotated text rendering of the same pair. More accurate where the wording rather than the structure carries the signal, and correspondingly slower. Shipped as interlock.onnx when it has been trained and exported; the export is verified against the torch model to within 1e-3 on the raw logits before it is published.

A third file, interlock-linear.gguf, is a weight container for tier 1 with its feature names and operating point attached. It is not a runtime: llama.cpp has no architecture for this classification head and cannot serve it. The verified runtimes are NumPy for tier 1 and ONNX Runtime for tier 2.

Evaluation, and the split that makes it mean something

The train/test split is taken at tool family granularity, not at row granularity. Training and validation draw on fs, shell, mail, calendar, issues. The held-out split draws on browser, crm, storage, db -- every tool schema in it is one the model has never seen. The split is verified against the files on disk, not just asserted in code.

Measured for tier 1:

split	over-blocking	deviation detection
validation (seen families, unseen world)	2.9%	88.0%
held-out families (unseen tool schemas)	1.8%	79.3%

Over-blocking is the fraction of legitimate calls that were interrupted, and it is the headline number. A shield that interrupts real work gets switched off, and a switched-off shield detects nothing. Detection is reported second for that reason, not first.

Latency, timing a full check per call including encoding: 0.247 ms mean, 0.663 ms p95 over 400 single-threaded calls on arm64. Timing the model's arithmetic alone would understate what a deployment pays.

Intended use

Interlock sits between an agent's proposed tool call and its execution. Given the user's request, the proposed call and the tool's schema, it returns a verdict that a harness can act on: proceed, ask the user, or refuse. It is designed for agent runtimes where tool schemas are declared and where a confirmation prompt is a cheaper failure than an unwanted side effect.

from interlock.model import load_shield

shield = load_shield("artifacts")
# The request names one file. The call names the directory that contains it.
verdict = shield.check(
    "Archive the Q3 notes at /work/apollo/q3-notes.md",
    {"tool": "fs.delete_path", "arguments": {"path": "/work/apollo", "recursive": True}},
)
print(verdict.decision, verdict.deviation)   # block scope_expansion
for reason in verdict.reasons:
    print(" -", reason)

Nothing about that call reads as dangerous on its own: it is an ordinary delete, on an ordinary path, in the project the user named. It is wrong only relative to the request, which is the only place the difference exists.

A tool the model has never seen still gets a verdict rather than an exception, because an agent can propose anything:

verdict = shield.check(
    "Rotate the API key for the billing service",
    {"tool": "acme.rotate_key", "arguments": {"service": "billing"}},
)
print(verdict.decision, verdict.known_tool)   # confirm False

It is not a content moderation model, not a prompt-injection detector, and not a substitute for sandboxing or for asking the user.

Limitations

• Synthetic data. Every episode, tool schema and deviation in training is generated. The tool families model common operation shapes rather than any particular product's API, and no real service was contacted at any point. Performance on production agent traffic is unmeasured here.
• English only. Requests and tool descriptions are English. Nothing has been measured in any other language.
• One layer among several. Interlock reads the correspondence between a request and a call. It does not read the content of a payload for harm, does not sandbox anything, and cannot see effects that occur outside the tool call it is shown.
• It does not guarantee safety. An allow verdict means this shield found no boundary deviation, which is a weaker statement than "this call is safe". Deploy it beside content shields, sandboxing and human confirmation, not instead of them.
• Unseen tools cost accuracy. Detection falls from 88.0% to 79.3% on held-out tool families. The over-blocking rate holds, so the degradation shows up as missed deviations rather than as new interruptions, but it is real.
• The operating point is a choice. Thresholds were fitted against an explicit over-blocking budget on validation data. A deployment with a different tolerance for interruption should refit them rather than inherit these.

Files

file	what it is
interlock_linear.npz	tier-1 weights, the default serving path
policy.json	fitted decision thresholds
train_report_linear.json	measured metrics for tier 1
interlock.onnx	tier-2 graph, when the encoder has been trained
interlock-linear.gguf	tier-1 weight container, not a runtime
latency.json	measured per-call latency and peak RSS

Card generated 2026-08-30T01:30:45+00:00 for NagaYu/interlock.