jacobbeckdev/trigon-qwen3-4b-lms-safe

trigon qwen3-4b-lms-safe v1

A typed-decision model: send state and a map of typed questions (choice, yes/no, score), get one calibrated distribution per question in a single prefill pass. Served by the trigon gateway (docs/self-host.md).

What it is

• Backbone: Qwen/Qwen3-4B at 1cfa9a7208912126459214e8b04321603b3df60c (Apache-2.0), frozen; fetched from its own repository, never re-published here.
• This bundle: LoRA rank 16 and an answer readout (adapter.pt): each answer is scored as the backbone's own continuation, w * log p(answer) + residual, w = 0.500; build trigon-lm-score-qwen3-4b-qwen3-4b-lms-safe-s0+lm-score-v1.
• Trained: 1 epochs, lr 0.0001, seed 0, on cuda in 2.1 h.
• Licence: Apache-2.0 for this bundle. Every training corpus is green-tier (docs/data.md); share-alike and research-only corpora evaluate only and never train.

Trained on

Corpus	Cases	Calibration cases	Licence	Attribution
banking77	1,500	500	CC BY 4.0	Banking77 (Casanueva et al., 2020), PolyAI. CC BY 4.0. https://github.com/PolyAI-LDN/task-specific-datasets
goemotions	1,238	412	Apache-2.0	GoEmotions (Demszky et al., 2020), Google Research. Apache-2.0. https://github.com/google-research/google-research/tree/master/goemotions
helpsteer2	844	281	CC BY 4.0	HelpSteer2 (Wang et al., 2024), NVIDIA. CC BY 4.0. https://huggingface.co/datasets/nvidia/HelpSteer2
measuring_hate_speech	844	281	CC BY 4.0	Measuring Hate Speech (Kennedy et al., 2020; Sachdeva et al., 2022), UC Berkeley D-Lab. CC BY 4.0. https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech
hatexplain	1,107	368	MIT	HateXplain (Mathew et al., AAAI 2021), Punyajoy Saha and co-authors. MIT (repository LICENSE, Copyright (c) 2020 Punyajoy Saha); the authors' dataset card states CC BY 4.0. https://github.com/punyajoy/HateXplain at 01d742279dac
teacher-workflows	1,400	0	Apache-2.0	Generated and labelled by Qwen2.5-7B-Instruct (Qwen Team, Alibaba Cloud), Apache-2.0, huggingface.co/Qwen/Qwen2.5-7B-Instruct at a09a35458c70. Teacher labels buy coverage, not calibration.
teacher-local	1,120	0	Apache-2.0	Generated and labelled by Qwen3.6-35B-A3B (Qwen Team, Alibaba Cloud), Apache-2.0, huggingface.co/Qwen/Qwen3.6-35B-A3B, served by ollama; each record names the teacher and its ollama digest. Teacher labels buy coverage, not calibration.
mind2web-train	1,202	400	CC BY 4.0	Mind2Web (Deng et al., 2023), The Ohio State University. CC BY 4.0. https://huggingface.co/datasets/osunlp/Mind2Web
wanli	1,500	500	CC BY 4.0	WANLI (Liu et al., 2022), University of Washington and the Allen Institute for AI. CC BY 4.0. https://huggingface.co/datasets/alisawuffles/WANLI
jailbreak-train	1,199	399	Apache-2.0	jailbreak-classification (Jack Hao, 2023). Apache-2.0. https://huggingface.co/datasets/jackhhao/jailbreak-classification at 2f2ceeb39658
prompt-injections-train	410	136	Apache-2.0	prompt-injections (deepset, 2023). Apache-2.0. https://huggingface.co/datasets/deepset/prompt-injections at 4f61ecb038e9
aegis2-train	1,500	500	CC BY 4.0	Aegis 2.0 AI Content Safety Dataset (Ghosh et al., 2025), NVIDIA. CC BY 4.0. https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0 at d86bb8bedff5

Held out: corpora clinc150, boolq, circa. Every website in the web-action evaluation is excluded from mind2web-train.

Calibration was fitted on 4,140 held-out cases from banking77, goemotions, helpsteer2, measuring_hate_speech, hatexplain, mind2web-train, wanli, jailbreak-train, prompt-injections-train, aegis2-train, with option sets reshaped as training reshapes them. Teacher labels are never calibration evidence.

Evaluation

Generality (scripts/generality.py, 1,000 cases per task, fixed seed):

Task	Accuracy	Chance	ECE	ECE floor p95	Order agreement
banking77/declared	0.801	0.013	0.033	0.040
banking77/shift	0.852	0.020	0.034	0.035
banking77/renamed	0.836	0.020	0.071	0.031
clinc150	0.930	0.020	0.081	0.036
boolq	0.843	0.500	0.017	0.035
banking77/order	0.836	0.020	0.040	0.033	0.928

Web actions (scripts/webact.py, Mind2Web, websites unseen in training):

Steps	Operation	Element	Step success	ECE (floor p95)
600	0.893	0.692	0.622	0.065 (0.055)

Single seed: a measurement, not a certification (CLAUDE.md).

Use it

curl -LO <host>/models/bundles/qwen3-4b-lms-safe/v1/adapter.pt  # + SHA256SUMS files
sha256sum -c SHA256SUMS
trigon serve --backend torch --weights adapter.pt  # TRIGON_*_PATH for the calibrators