SecJev-4B-Action-Monitor-C1

A SecJev-4B checkpoint fine-tuned to act as an action monitor for tool-using LLM agents: before a proposed tool call executes, the checkpoint scores whether the call is authorized by the user's request, given the tool outputs the agent has seen.

SecJev-4B-Action-Monitor-C1 is the Spot 2 (outgoing tool-call) half of the JEV defense evaluated in the jev-stageA research repository. The Spot 1 (incoming tool-output) half uses the unmodified upstream SecJev-4B checkpoint, mirrored at macmacmacmac/SecJev-4B. Used together, the two checkpoints form the secjev_cascade defense.

What changed relative to upstream SecJev-4B

Only adapter_model.safetensors, head.pt, and adapter_config.json differ. The LoRA adapter and pointer decision head were fine-tuned for one epoch (variant C1: lr 3e-5, no corpus replay, content view) on action-authorization questions built from recorded AgentDojo runs. Training used the workspace and travel suites only; banking and slack were held out for reporting. Full details: provenance.json in this repository. Upstream SecJev-4B v1.2.0 records are preserved under upstream-secjev-4b/ and upstream-provenance.json. This is a modified release; SecJev-4B-Action-Monitor-C1 is not the original SecJev-4B and must not be cited as the original SecJev-4B.

Run

The checkpoint is a Kev/SecJev checkpoint directory: LoRA adapter + decision head + tokenizer. The pinned Qwen3.5-4B-Base backbone and the Kev source code are fetched by the included bootstrap script (the backbone is distributed from the Qwen Hugging Face repository under Apache-2.0 and is not re-hosted here).

hf download macmacmacmac/SecJev-4B-Action-Monitor-C1 --local-dir SecJev-4B-Action-Monitor-C1
python -m pip install -r SecJev-4B-Action-Monitor-C1/requirements.txt
export SECJEV_WORK="$PWD/secjev-4b-work"
python SecJev-4B-Action-Monitor-C1/bootstrap.py --inference-only
python SecJev-4B-Action-Monitor-C1/infer.py --model ./SecJev-4B-Action-Monitor-C1 \
  --base "$SECJEV_WORK/base" --request SecJev-4B-Action-Monitor-C1/examples/certificate.json --device cuda

To run SecJev-4B-Action-Monitor-C1 as a live agent defense (stop or redact mode, alone or cascaded with the tool-output filter), use defenses.py and secjev_scorer.py from the jev-stageA repository and pass this directory as --secjev. The Kev checkpoint loader also accepts the Hub repo id directly (macmacmacmac/SecJev-4B-Action-Monitor-C1).

Reported results (read the caveats)

AgentDojo v1.2.2, banking+slack held-out suites, gpt-5.4-mini agent, bland-note attack set, live runs, threshold 0.35 tuned on workspace+travel:

Mode	Normal tasks	Hijacked	Tasks completed under attack
Undefended	97.3%	34.1%	78.3%
C1 monitor, stop	89.9%	1.6%	55.4%
C1 monitor, redact	90.5%	0.8%	75.5%
LLM judge (gpt-5.4-mini), redact	95.3%	0.0%	28.5%

Against a 10-round adaptive attacker (gpt-5.5, 100 banking+slack targets), C1 alone is weak: 58% hijacked in stop mode and 53% in redact mode, versus 64% undefended and 25% for the LLM judge. SecJev-4B-Action-Monitor-C1 is a research checkpoint, not a complete defense. The jev-stageA README reports the full results, including the decision rules SecJev-4B-Action-Monitor-C1 did not pass.

License and terms

The SecJev Use and Distribution Agreement v1.0 (LICENSE) applies, together with the upstream terms in UPSTREAM.md, SOURCES.md, and THIRD_PARTY_NOTICES.md (Kev and Qwen components are Apache-2.0). The upstream agreement's source-specific conditions, including ToN-IoT commercial-use limits in SecJev-Corpus, are not removed by this release.