OpenSysOne
Inspired by Jev, TypeSafe.ai's System One model. Credit goes to the TypeSafe team for inspiring this project's exploration of structured decisions with probabilities. OpenSysOne is an independent experimental implementation; API compatibility does not establish Jev equivalence.
The completed 4B release scores a state, question and explicit candidate answers, returning probabilities over those choices. Training, separate calibration, final evaluation and local API verification completed on 17 September 2026.
Start with the model and reconstruction notes, results report, or publication guide. The calibrated artifact is model/model.pt. It contains custom OpenSysOne adapter/head weights and metadata. The pinned Qwen3-4B-Instruct-2507 base is required separately; this is not a standalone Transformers model or a standard PEFT adapter package.
Measured results
The full comparison uses the unchanged pretrained yes/no verifier, with a separate temperature fitted for each model. Intervals are paired 95% source-group bootstrap intervals for selected minus base accuracy.
| Evaluation | Decisions | Selected | Base verifier | Accuracy gain (95% interval) |
|---|---|---|---|---|
| Known-family test | 2,042 | 92.90% | 84.48% | +8.42 pp [6.85, 9.89] |
| Social IQA family holdout | 768 | 72.92% | 70.31% | +2.60 pp [0.13, 5.34] |
On a separate matched 320-decision profile, selected accuracy was 89.06%, versus 80.94% for the base verifier and 86.25% for a base model using one constrained answer-label token. The selected scorer was slower on all 12 profiled workloads: 1.11โ1.17ร the verifier latency and 2.18โ15.58ร the label baseline latency. These are warm, serial FP32 measurements on one GB10, not concurrent-serving throughput or comparisons with generated reasoning. See tables and charts.
Model and limits
The release uses rank-8 additive adapters and a scalar head: 16,517,633 trainable parameters. The selected expanded branch's step 0 retains the refinement parent's step-1,500 weights. The expanded branch's later step 159 was not selected; its post-selection diagnostic gains are reported separately.
One temperature, 1.745822, was fitted on 510 separate known-family calibration examples. Social IQA calibration remains limited: its top-label ECE is 8.30%. No general intelligence, Jev-level quality or universal calibration claim follows. Four-choice Banking77 is not the full 77-label task; benchmark grouping does not rule out base-model pretraining overlap. The 1,024-token inference limit includes the complete formatted candidate prompt, and longer inputs are rejected.
Files and provenance
- model/: calibrated artifact, hash and pinned base requirements.
- docs/: layout and reproduction guide.
- source/: complete committed project source, tests and usage guides.
- results/: final metrics, profiling report, tables and charts.
- archive/: index to preserved experiment history.
The original pointers remain authoritative: FINAL_MODEL.json identifies the calibrated release, PROFILE_RESULTS.json identifies verified profiling and wrap-up evidence, and CURRENT_SNAPSHOT.json identifies the earlier training backup. Historical payload paths and hashes are preserved.
Original OpenSysOne code, documentation, and adapter/head additions are licensed under Apache License 2.0; see NOTICE for attribution and scope. The required Qwen3-4B-Instruct-2507 base is separately distributed under Apache 2.0. Upstream model and dataset terms remain applicable; see the source's original data provenance and expanded data provenance. Base weights and credentials are excluded. Hosted Jev calls require separate authentication and were not exercised in this evaluation.
Model tree for andyshu/opensysone
Base model
Qwen/Qwen3-4B-Instruct-2507