howwhenRaw/llm-for-sec-secjev

secjev-0.6b — Security-Perception Decision Model (Jev-style form)

A decision-form security-perception model: instead of autoregressively generating text, it performs a single forward pass and outputs a structured decision with probabilities — verdict / action / finding / confidence — over a 21-slot decision head, then renders it as a short action-conditioned template plus a VERDICT JSON line. Inspired by the Jev "System 1 / decision model" paradigm; head slots are initialized from lm_head rows (following the open JEV-27B recipe).

Why this exists

Our text-form 0.6B–8B safety-tuning arms were stuck on an over-refusal floor (ORR 7.6–13.7% across 13 arms, incl. aligned 8B). This model tests whether the ORR↔risk-discrimination coupling is an artifact of the text-generation form.

Result: the floor is a form artifact. In decision form the same backbone family reaches ORR 1.11% (r1, 300 steps) / 0.16% (r2, 1200 steps) — first touch of the 1% mission gate — while refusing 80/80 multi-turn jailbreak escalations, at 128 ms/decision (75× faster than 512-token generation).

Readings (frozen gates, zero instrument changes)

Gate	r1 (300 steps)	r2 (1200 steps)	Text-form history
v0.23 over-refusal (↓)	1.11%	0.16%	7.62–13.65% (floor band)
gen-suite ASR (↓)	79.17%	97.02%	78.57–80.36% (8B arms)
jailbreak tiered ASR (↓)	0.00% (80/80)	12.50%	28.75–77.50%
latency/decision (↓)	128 ms	125 ms	~10 s
structured contract (↑)	0.73	0.64	0.00 (all 8B text arms)

Known limitation (by design of the experiment): content-level risk coverage is a data-side gap independent of model form — gen-suite ASR stays ~79–97% across text 0.6B/8B and this decision form. The companion fix is adversarial-diversity training data, not another form change.

Usage

# render_eval.py ships with this repo (also handles the merged layout)
python render_eval.py --model <this-repo-clone> --smoke
# → verdict/action/finding/confidence per probe + rendered template text

# programmatic
from render_eval import DecisionModel, SecJevJudge
dm = DecisionModel("<path-to-repo>")          # loads merged weights + head.pt + slots.json
dec = dm.decide(input_ids, attention_mask)    # {'verdict':…, 'action':…, 'finding':…, 'confidence':…}
text = dm.render(dec)                         # template + VERDICT JSON

The decision vocabulary (21 slots):

• verdict (5): benign / risk / ambiguous / malformed / out-of-scope
• action (5): comply / refuse / clarify / deflect / escalate
• finding (8): insufficient-context / harmful-intent / dual-use / credential-risk / policy-violation / authorization-missing / scope-exceeded / no-finding
• confidence (3): high / medium / low

Training

• Backbone: internal Qwen3-0.6B lineage checkpoint (M-series priors; Apache-2.0 derivatives)
• LoRA r16 α32 all-linear + fp32 21-slot head (lm_head-row init), CE on the last prompt position, 300 steps (r1) / 1200 steps (r2), batch 8, lr 5e-5
• Data: 21,616 (prompt → decision) pairs distilled from our existing preference corpora — zero new annotation (m4 native VERDICT rows parsed directly; r3/r6 refusal/comply/suspension rows label-mapped)
• Full recipe: scripts/secjev/{convert,pretok,train_dec,render_eval}.py in the project repo

Layout

• repo root — merged self-contained model (model.safetensors, head.pt, slots.json, tokenizer) + render_eval.py
• adapter-r1/ — LoRA adapter + head for the 300-step reference arm (needs the internal base)
• adapter-r2/ — LoRA adapter + head for the 1200-step converged arm

License

Apache-2.0 (Qwen3-0.6B derivative). Research artifact; rendered templates are fixed strings, not free-form generations — do not use as a general chatbot.