ignite-scribe: models

Models for ignite-scribe, which turns the Windows desktop into redacted, structured state that an AI agent can read, served over MCP.

Folder	Model	Used by
scribe-guard/	prompt-injection classifier, release-05	ignite-scribe's secure mode

scribe-guard (release-05): a prompt-injection classifier for screen text

A binary classifier: does a piece of screen or file text contain an instruction aimed at an AI agent (a prompt injection)? It is BAAI/bge-small-en-v1.5 (33 M parameters) fine-tuned on window-sized screen text: emails, chats, terminals, code, web pages, UI lines and documents, with attacks planted in them and hard negatives around them. English only. In ignite-scribe it runs on the iGPU through DirectML, never on the CPU.

Labelling policy. An instruction to an AI agent planted in content is positive, wherever it sits. Text about prompt injection that quotes an attack is negative. So are instructions aimed at people ("Do not modify this file", "You are the reviewer for this pull request") and ordinary UI wording ("You are now connected").

Files

File	What
model.onnx	fp32; inputs input_ids, attention_mask, token_type_ids (int64, dynamic n x seq); output logits [n,2]
model_fp16.onnx	the same with fp16 weights and compute, fp32 inputs and outputs; what secure mode runs
tokenizer.json	bge-small WordPiece (lowercasing); add [CLS]/[SEP] yourself per window
config.json	BERT config, id2label {0 benign, 1 injection}, threshold, scribe_inference
results.json	the training-side numbers on the held-out sets, with breakdowns

Inference contract

• Text: NFKC, casefolded, invisible characters stripped.
• Windows: tokenize without special tokens, take windows of 510 body tokens with stride 256, and wrap each as [CLS] ... [SEP]; token_type_ids are all 0.
• Score: the maximum over windows of softmax(logits)[1]. Flag the text when the score >= threshold (0.992467, chosen on val only: the lowest threshold with val benign false positives <= 0.5% and hard negatives <= 2%). The threshold was never tuned on a test set.

Results

Run once at that threshold on an AMD Radeon 780M (DirectML, fp16). "Either" counts a flag from ignite-scribe's regex patterns or the classifier. False-positive rate in brackets. The targets were registered before any training.

Set	Release-01 (either)	Release-05 (either)	Target
BIPIA attacks planted in emails	42.8% (0%)	63.2% (0%)	>= 60%: met
BIPIA benign emails	0 of 50	0 of 50	<= 1 of 50: met
deepset test, English (38 attacks)	31.6%	39.5% (0%)	>= 60%: missed
Gandalf test	77.7%	88.4%	>= 90%: missed
UI lines (500, generated)	0%	0%	<= 0.5%: met
Hard negatives, classifier alone	(1.05%)	(0.10%)	<= 5%: met
BIPIA code and table contexts (transfer)	60.0% (3.0%)	65.3% (3.7%)	>= 50%, <= 5%: met

On a live Windows desktop (60 s, counts only), release-05 flagged 0 of 60 samples. Cost: about 54 ms per 512-token window on the 780M.

Limits: a tripwire, not a barrier

• About a third of attacks hidden in emails still get through, and most short, direct attacks (deepset).
• An attacker who writes against this model, or keeps trying, will get past it, as with every injection classifier. Treat a flag as a warning. The protection belongs in the agent: treat screen text as untrusted, and keep its permission checks.
• English only: it catches almost none of the German attacks in the deepset test.
• Most of the training and held-out data comes from generators in ignite-scribe's tools/injection_data.py. The public test sets are small: one row moves a deepset result by 2-3 points.

Training

• AdamW, fine-tuned in two stages on an AMD RX 7900 XTX:
  • release-04 from bge-small-en-v1.5;
  • release-05 from release-04 on the v6.1 data (lr 1.5e-5, 2 epochs, the checkpoint with the lowest val loss).
• A document's logit is the max over its windows (multiple-instance BCE), as at inference.
• The data (tools/injection_data.py in the GitHub repo):
  • every template and source pool is split 80/10/10 by item before anything is planted;
  • every row records its template ids;
  • the builder refuses to write if a held-out split rests on fewer than 6 templates of any pool;
  • near-copies of evaluation rows are dropped.

See scribe-guard/LICENSE.md for the sources and their licences.

Licence

AGPL-3.0-or-later, like ignite-scribe. The base model and the training sources keep their own licences and notices: see each folder's LICENSE.md. manifest.json lists every file with its sha256.