LatentRisk-Probe

Two trained sparse linear probes for prompt harmfulness, using frozen Gemma 3 1B and Qwen3 0.6B activations. The repository contains actual readout weights, inference code, frozen labels, captured activations, text/control baselines and an independent serialized-weight audit.

The probes generalize beyond their training categories on these benchmarks, but falsely flag 43–46% of difficult benign requests. They are research instruments, not reliable safety filters. They do not establish covert intent, causal features or a model's eventual harmful response.

Cross-dataset results

XSTest: 450 prompts, including 250 safe and 200 unsafe requests. Threshold 0.5, selected without using XSTest.

Readout	Balanced accuracy	AUROC	Benign false-positive rate	Harmful false-negative rate
Gemma sparse probe	80.75%	0.8978	12.0%	26.5%
Qwen sparse probe	74.25%	0.8292	26.0%	25.5%
Character TF-IDF	59.85%	0.6260	32.8%	47.5%
Word TF-IDF	57.00%	0.5979	20.0%	66.0%
Length-only logistic	52.80%	0.5582	86.4%	8.0%

The fixed hidden-state-index-7 controls score 72.8% (Gemma) and 60.5% (Qwen) balanced accuracy. Shuffled-label controls score 48.45% and 47.9%; Gaussian-feature controls score 52.6% and 48.3%. The selected hidden states and regularization were chosen on validation only.

Failure regions and transfer

Suite	Gemma	Qwen
OR-Bench category holdout, balanced accuracy (222)	77.48%	85.59%
Difficult benign OR-Bench, false-positive rate (188)	45.74%	42.55%
SimpleSafetyTests, false-negative rate (100 unsafe)	20.0%	26.0%
Authored Portuguese stress test, balanced accuracy (24)	75.0%	100.0%

The Portuguese result is only 12 authored pairs, with no independent human review. It does not demonstrate broad multilingual generalization. Single-class suites have no AUROC or balanced-accuracy claim. Benchmark labels can be disputable, especially for hard benign/dual-use prompts.

Training and selection

Frozen OR-Bench benign/toxic prompts, length-matched within category: 816 balanced training examples, 122 validation examples from the held-out unethical category, and 222 held-out examples from privacy/illegal. Exact normalized-prompt deduplication spans all partitions. Near-duplicates may remain.

Independent tests: XSTest by Röttger et al., SimpleSafetyTests by Vidgen et al., 188 unused difficult benign OR-Bench prompts, and 24 authored Portuguese requests. Upstream CC-BY attribution, immutable revisions, hashes, split labels and preparation code are recorded in data/manifest.json and data/prepare_latent_probe.py. These benchmarks may overlap the backbones' pretraining; a fresh sealed evaluation remains outstanding.

Features are the final prompt-token hidden state at the assistant-generation boundary, with explicit cumulative attention-mask position IDs, left padding/truncation and a 512-token limit. No responses were generated. L1 logistic regression uses a training-fitted StandardScaler; hidden-state index and C∈{0.01,0.1,1} maximize validation AUROC. Text baselines fit only training text and select C on the same validation set.

Gemma: pinned base dcc83ea841ab6100d6b47a070329e1ba4cf78752, hidden-state index 9 (after transformer block 8), C=1, 138 nonzero features. Qwen: base c1899de289a04d12100db370d81485cdf75e47ca, hidden-state index 17 (after block 16), C=0.1, 62 nonzero features. Index 0 is the embedding state.

Serialized raw-feature readouts fold the training scaler into a double-precision weight/bias. An independent audit reconstructs every accuracy/Brier/log-loss from captured activations; tiny changes in training-set AUROC from numerically tied probabilities are bounded explicitly. No test threshold or layer tuning was applied after opening the independent suites.

Inference

Download probe.py, authenticate for the Gemma backbone if needed, then:

from probe import LatentRiskProbe

probe = LatentRiskProbe(backbone="gemma")  # Or backbone="qwen"; pin revision for frozen use.
scores = probe.score_texts([
    "How can I recognize a fraudulent message?",
    "How can I deceive someone into sending me money?",
])
print(scores)

Output is an uncalibrated benchmark risk score and a label at 0.5. It is not a probability of real intent. The base language model must be downloaded separately; the probe files are about 9 KB each. GPU reproduction: PyTorch2.6.0, Transformers4.57.1, NumPy2.1.3, scikit-learn1.6.1, Safetensors0.5.3, bfloat16/SDPA. CPU float32 is a different activation configuration from the reported GPU run.

Limits and licenses

This study has one split/seed, existing public benchmarks and a small authored language suite. It supplies no causal steering validation, covert-intent detector, response-level moderator, robustness guarantee or production refusal policy. Both missed harms and over-refusal remain substantial.

Gemma-associated components are subject to the included Gemma Terms and Section3.2 restrictions, including the Prohibited Use Policy. Qwen-associated readouts use Apache2.0. Benchmark text/labels retain their CC-BY4.0 terms. See LICENSE.md, GEMMA_TERMS.txt, GEMMA_PROHIBITED_USE_POLICY.txt, APACHE_LICENSE.txt and NOTICE. No backbone weights are redistributed.