a1800015531/ScamBrain-v0.2

ScamBrain v0.2 — Conversation Scam-Risk Engine (research preview)

Three LoRA adapters that turn Qwen3 models into a conversation scam-risk engine with two complementary signals:

Adapter	Base	Role
wm_est/	Qwen3-1.7B	Judgment model — outputs the risk contract (level / scam_type / mechanisms / recommended action)
wm_dyn_scam/	Qwen3-0.6B	Scam world model — next-turn predictor trained on scam dialogue (logP_scam)
wm_dyn_norm/	Qwen3-0.6B	Normal world model — next-turn predictor trained on benign dialogue (logP_norm)

The two 0.6B world models produce a token-level likelihood ratio (logP_scam − logP_norm) that scores how much a conversation "lives" in the scam world; wm_est produces the structured verdict.

Output contract (strict JSON):

{"scam_risk":{"level":"unknown|low|medium|high","scam_type":"..."},
 "mechanisms":["..."],
 "recommended_action":"...",
 "insufficient_evidence":false}

What this is — and is NOT

• ✅ A scam-risk warning prototype for dialogue and SMS, with early-warning behavior before explicit requests appear.
• ❌ NOT a polygraph. It cannot infer whether a person "is lying" and must never be used to output lie probabilities for individuals.
• ⚠️ Trained entirely on synthetic / public corpora. Performance on real phone calls is unvalidated — not for production decisions.

Evaluation (v0.2 report · 2,570 validation samples · 2026-10-31)

Judgment metrics (full-dialogue contract output):

Source	n	Level acc	Risk recall	FPR	Type acc
bothbosu_en	320	100.0%	100.0%	0.0%	100.0%
crosswoz_zh (benign control)	300	100.0%	–	0.0%	–
scamgen_hard_zh (cross-difficulty split)	1000	100.0%	100.0%	0.0%	99.7%
scamgen_zh	500	100.0%	100.0%	0.0%	100.0%
sms_en	450	94.4%	95.5%	1.6%	100.0%
Total	2,570	99.0%	99.5%	0.6%	–

Early warning (state tracking): benign-prefix FPR 0.2% (2/905); 15.3% of scam dialogues already flagged medium/high in rounds before the first explicit request appears; median advance warning 10 rounds.

Likelihood-ratio detection (dual-world): dialogue-level AUC 0.9999, EER 0.2%; recall 99.9% at FPR ≤ 1%.

Caveats: all sources are synthetic/public; scamgen shares its generator family with training data (hard split included as a cross-difficulty check); crosswoz is a travel-booking domain, not real calls.

Training data (all synthetic / public)

Source	Lang	License
ScamGen (Telescam) train + hard split	zh	CC BY 4.0
BothBosu scam-dialogue	en	Apache 2.0
UCI SMS Spam	en	CC BY 4.0
CrossWOZ train	zh	CC BY 4.0

Labels are weak supervision (rule-based mapping), not human annotation.

Usage (PEFT)

Merge each adapter with its own base — wm_est → Qwen3-1.7B, the two wm_dyn_* → Qwen3-0.6B:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "a1800015531/ScamBrain-v0.2", subfolder="wm_est")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")

For the dual-world score, run the same conversation through wm_dyn_scam and wm_dyn_norm (both on Qwen3-0.6B) and diff their per-token log-likelihoods.

Limitations & roadmap

• No real Chinese call data — the project's biggest gap; FPR conclusions do not transfer to real calls.
• Audio-deepfake branch (ASVspoof5): not yet trained.
• Next: learning-curve study (1k→5k dialogues), independent authorized test set.

License

Apache-2.0 (both base models are Apache-2.0). Commercial use permitted; polygraph-style claims are an explicit violation of the intended use.