Request access to Dwago
Dwago is released under the Dwago Evaluation License. Approved users may download the weights to evaluate, test and benchmark the model, or for non-commercial academic research. Any commercial or production use requires a separate paid license from Mohammedi Yunus; contact via the Hugging Face profile muhammediyunes. Requests are reviewed manually.
Log in or Sign Up to review the conditions and access this model content.
Dwago v6.2: a 30M-parameter decision model trained from scratch
Created by Mohammedi Yunus. Open weights under the Dwago Evaluation License: evaluation and non-commercial research only; commercial use requires a paid license (see License and commercial use).
Dwago answers questions about a piece of content (the state: plain text or an object with named fields) and returns calibrated probabilities. It supports three question types: choice (pick one of N described options), yes/no, and score (an ordinal rating). Every weight was trained from random initialisation in this project: its own tokenizer, an encoder pretrained on 8.58 billion tokens of decontaminated English Wikipedia and FineWeb-Edu, and multi-task supervised fine-tuning. No pretrained weights, external embeddings or pretrained tokenizers were used, and no other model's predictions were used as training targets (the only soft targets came from ensembles of Dwago's own from-scratch models).
Highlights
- Beats Jev on 8 of 17 Jev-published benchmarks with the release model Dwago v6.2 (9 of 17 with the code-review tool layer). Largest leads: Emotion (sysone) 89.00 vs 55.00, Emotion (dair-ai) 88.65 vs 58.65, Security: vulnerable-code labels 87.25 vs 71.50, Financial news topics 81.78 vs 66.99, DailyDialog emotion 79.29 vs 70.99.
- Beats Laya on 10 of 17 of the same benchmarks (2 ties), while being about 13ร smaller and about 15ร faster (1.77 ms vs 25.80 ms median per request on an RTX 3070).
- On Dwago's sealed final test suite, macro accuracy (DBpedia, SNLI, Banking77, BoolQ) is 84.54 vs Laya 66.12 (shipped) and 69.19 (adapted to these tasks).
- Code review: quality reaches 100.00 vs Jev 99.48 with Dwago's built-in code-review tool layer (static checks; disclosed as post-hoc, see โ ).
- Runs on a CPU: 4.77 ms median per request with the ONNX int8 export.
Benchmark results against Jev and Laya
Same inputs as Jev's public results; each benchmark's own (publisher) metric; every system evaluated once from a frozen checkpoint. Bold = above Jev.
| Benchmark | Jev 1.13.0 | Dwago v6.2 | Dwago v6.3 + tools* | Laya (shipped) |
|---|---|---|---|---|
| DailyDialog emotion | 70.99 | 79.29 | 75.74 | 61.37 |
| Emotion (dair-ai) | 58.65 | 88.65 | 87.50 | 58.80 |
| Financial news topics | 66.99 | 81.78 | 81.32 | 34.08 |
| Tweet topics | 79.33 | 79.21 | 80.80 | 63.20 |
| News topics (AG News) | 91.00 | 92.00 | 93.00 | 94.00 |
| Banking77 (12-way) | 90.63 | 94.79 | 97.92 | 80.21 |
| Emotion (sysone) | 55.00 | 89.00 | 89.00 | 55.00 |
| Guardrails | 96.67 | 83.33 | 81.67 | 88.33 |
| MNLI | 86.67 | 78.33 | 70.00 | 98.33 |
| Moderation | 98.89 | 83.33 | 82.22 | 83.33 |
| Multilingual intent | 100.00 | 40.00 | 32.00 | 36.00 |
| Support triage | 88.75 | 66.25 | 61.25 | 79.38 |
| Code review: correctness | 98.00 | 51.04 | 76.82 โ | 51.04 |
| Code review: quality | 99.48 | 67.45 | 100.00 โ | 54.43 |
| Security: vulnerable-code labels | 71.50 | 87.25 | 70.50 | 50.25 |
| Security: pair ranking | 89.00 | 95.00 | 89.50 | 47.50 |
| JevBench (public) | 86.58 | 44.16 | 43.29 | 54.55 |
| Benchmarks above Jev | โ | 8 of 17 | 9 of 17 | 3 of 17 |
* Dwago v6.3 + tools: the v6.3 network plus its built-in code-review tool layer (static analysis and sandboxed execution of the code under review). โ Post-hoc: the tool layer's rule reader was fixed after the clean one-time evaluation of v6.3 (correctness 50.00, quality 83.33); every round is kept in the records. No benchmark item was ever used for training or tuning.
Where Dwago is behind: multilingual intent (a 30M model across 51 languages), MNLI, support triage, moderation and guardrails, the general JevBench, and code-review correctness (which needs real program understanding). These are reported, not hidden.
Full report, Dwago's sealed final test suite, size and speed, and method: benchmark/README.md. Machine-readable results: benchmark/results/.
How Dwago was built
From an evaluation suite fixed before training, through tokenizer training, 8.58B-token pretraining on one RTX 3070 and free Kaggle GPUs, multi-task fine-tuning and self-distillation, to one-time evaluations recorded in hash-chained ledgers: benchmark/HOW_DWAGO_WAS_BUILT.md. The project was engineered with an AI coding agent (Claude Code) under the author's direction; the model's weights were all learned from random initialisation by these pipelines.
Quick start
from dwago_model import Dwago # dwago_model.py ships with the weights
model = Dwago.from_pretrained('.') # CUDA if available, else CPU (PyTorch)
model.predict({'message': 'Ich mรถchte heute Abend eine Pizza bestellen.'}, {
'intent': {'type': 'choice', 'instructions': 'What does the customer want?',
'criteria': {'takeaway_order': 'order takeaway food', 'transport_taxi': 'book a taxi', 'other': 'anything else'}},
'urgent': {'type': 'noul', 'instructions': 'Is there time pressure in `message`?'},
})
- Requirements:
torch,tokenizers,safetensors,numpy. - CPU deployment:
Dwago.from_pretrained('.', device='cpu', backend='onnx')uses the bundled ONNX export, which also needsonnxruntime.- The export quantizes the encoder's weight matrices to int8 and keeps attention and the decision head in fp32.
- It was adopted only after a rule fixed in advance: argmax agreement โฅ 99% and accuracy drop โค 0.2 points on development data. It reached 99.26% agreement over 26,097 decisions and +0.09 points.
Model
| Architecture | 10-layer pre-LN Transformer encoder, width 384, 6 heads, learned positions (512); schema-conditioned decision head |
| Parameters | 30,337,921 |
| Input | [CLS] question [SEP] state [SEP] (question โค 128 tokens, at most 512 positions in total). Each option description (โค 62 tokens) is encoded separately and cached. |
| Tokenizer | 24,576-token lower-cased BPE with word-start markers (trained on English; other scripts are covered character by character) |
| Layout markers | Line breaks and indentation, which the tokenizer would otherwise drop, are encoded as ยถ and ยฆ (config layout: true), so code and multi-line fields keep their structure. |
| Calibration | One temperature (1.567) fitted on development data. ECE 0.056 โ 0.0104 on dev_select and 0.054 โ 0.0085 on dev_report. |
| Precision | Weights fp32. GPU inference bf16 (CUDA graphs in the research engine); CPU inference fp32 or ONNX int8w. |
Training data
Pretraining:
- English Wikipedia 2023-11-01 and FineWeb-Edu sample-10BT.
- Decontaminated against every evaluation text before the tokenizer was trained: 13-gram overlap, exact short sentences, and evaluation article titles.
- Two of the v6 teacher's members additionally saw 0.4B tokens of Python code (CodeSearchNet, codeparrot-clean validation, APPS; mixed licenses as stated by the sources).
Supervised (training splits only).
- The v5 pool: DBpedia-14, SNLI, MultiNLI, SQuAD 2.0 answerability, Banking77, BoolQ, AG News and STS-B, plus schema augmentations of these rows.
- v6 tasks: the task types below, plus request-format variants and execution-labelled code review built from MBPP and HumanEval.
- v6.2 additions:
- multilingual intent, scenario, topic, banking and NLI data (MASSIVE in 51 languages, SIB-200, MINDS-14, XNLI);
- code-review and code-execution data generated by this project, labelled by running the code and by syntax-tree checks.
- Screening: every training row was checked against every evaluation input (exact text and 5-gram containment โฅ 0.9). Non-Latin text uses a Unicode-aware normaliser and character 5-grams. Matches were removed.
| Task | Source | License (as stated by the source) | Training rows |
|---|---|---|---|
| abuse_severity | google/civil_comments | CC0 1.0 | 52,605 |
| anli | facebook/anli | CC BY-NC 4.0 | 60,000 |
| arc | allenai/ai2_arc | CC BY-SA 4.0 | 2,747 |
| code_exec | generated by this project (assertion outcomes, executed) | this project | 13,357 |
| code_review2 | generated by this project (function families, executed) | this project | 50,000 |
| code_review3 | generated by this project (function families, executed) | this project | 96,000 |
| code_vulnerable | CyberNative/Code_Vulnerability_Security_DPO | Apache-2.0 | 7,230 |
| copa | super_glue/copa | BSD-2 | 319 |
| csqa | tau/commonsense_qa | MIT | 9,115 |
| daily_dialog | DailyDialog (roskoN mirror) | CC BY-NC-SA 4.0 | 28,586 |
| emotion | dair-ai/emotion | other (research/education) | 17,273 |
| fin_topic | zeroshot/twitter-financial-news-topic | MIT | 15,603 |
| fmt_field | format variant of abuse_severity | CC0 1.0 | 56,899 |
| fmt_nli | format variant of snli | release | 59,200 |
| glue_cola | nyu-mll/glue | GLUE (per-task terms) | 7,894 |
| glue_mrpc | nyu-mll/glue | GLUE (per-task terms) | 2,850 |
| glue_qnli | nyu-mll/glue | GLUE (per-task terms) | 59,384 |
| glue_qqp | nyu-mll/glue | GLUE (per-task terms) | 59,378 |
| glue_rte | nyu-mll/glue | GLUE (per-task terms) | 1,990 |
| harm_severity | PKU-Alignment/PKU-SafeRLHF | CC BY-NC 4.0 | 37,974 |
| hellaswag | Rowan/hellaswag | MIT | 39,176 |
| identity_attack | google/civil_comments | CC0 1.0 | 25,297 |
| insult | google/civil_comments | CC0 1.0 | 57,615 |
| jailbreak | jackhhao/jailbreak-classification | Apache-2.0 | 792 |
| massive_intent | MASSIVE (mteb mirror) | CC BY 4.0 | 59,980 |
| ml_bank | MINDS-14 | CC BY 4.0 | 8,145 |
| ml_intent | MASSIVE | CC BY 4.0 | 201,710 |
| ml_nli | XNLI | CC BY-NC 4.0 | 74,990 |
| ml_scenario | MASSIVE | CC BY 4.0 | 40,746 |
| ml_topic | SIB-200 | CC BY-SA 4.0 | 61,500 |
| multirc | super_glue/multirc | other | 4,045 |
| openbookqa | allenai/openbookqa | unknown | 4,206 |
| prompt_injection | deepset/prompt-injections | Apache-2.0 | 7,932 |
| sciq | allenai/sciq | CC BY-NC 3.0 | 11,073 |
| sensitive_data | ai4privacy/pii-masking-200k | other | 57,632 |
| spam | ucirvine/sms_spam | unknown | 4,512 |
| sst2 | stanfordnlp/sst2 | unknown | 57,348 |
| sst5 | SetFit/sst5 (SST-5) | unknown | 8,945 |
| support_intent | bitext customer support | CDLA-Sharing-1.0 | 22,280 |
| threat | google/civil_comments | CC0 1.0 | 7,633 |
| ticket_urgent | Tobi-Bueck/customer-support-tickets | CC BY-NC 4.0 | 11,132 |
| toxic | google/civil_comments | CC0 1.0 | 57,582 |
| tweet_topic | cardiffnlp/tweet_topic_single | other | 4,287 |
| wants_cancel | bitext customer support | CDLA-Sharing-1.0 | 22,280 |
| wants_refund | bitext customer support | CDLA-Sharing-1.0 | 22,280 |
| wic | super_glue/wic | CC BY-NC 4.0 | 4,827 |
| winogrande | allenai/winogrande | CC BY | 8,615 |
Data licenses. The training data includes sources under non-commercial, share-alike, research-only or unstated terms (for example CC BY-NC 4.0: ANLI, XNLI, PKU-SafeRLHF, WiC, ticket data; CC BY-NC-SA 4.0: DailyDialog; research/education-only: dair-ai/emotion). No dataset is redistributed here.
Evaluation
Detailed comparison with Jev 1.13.0 (confidence intervals, Holm test)
These are per-item comparisons on the exact public inputs, against Jev's recorded outputs from third-party studies (not new API calls).
Protocol (pre-registered in results/PREREGISTRATION-v6.md):
- Paired whole-group bootstrap (10,000 replicates) with one Holm family across all 17 comparisons.
- A win needs Holm-supported superiority and an estimated difference of at least 2 points.
- Each frozen system was evaluated once. No benchmark item was used for training, tuning or selection.
Dwago v6.2 has 5 wins.
| Benchmark | Jev 1.13.0 | Dwago v6.2 | ฮ | 95% CI | Holm | Dwago v6 |
|---|---|---|---|---|---|---|
| sysone emotion | 55.00 | 89.00 | +34.00 | [+24.0, +44.0] | win | 92.00 |
| Emotion (dair-ai test, 2,000) | 58.65 | 88.65 | +30.00 | [+27.7, +32.2] | win | 88.45 |
| Vulnerable code: label agreement | 71.50 | 87.25 | +15.75 | [+11.0, +20.5] | win | 86.25 |
| Financial news topics (4,117) | 66.99 | 81.78 | +14.79 | [+13.1, +16.5] | win | 82.49 |
| DailyDialog emotion (test, 7,740) | 70.99 | 79.29 | +8.29 | [+7.1, +9.6] | win | 79.25 |
| Vulnerable code: pair ranking | 89.00 | 95.00 | +6.00 | [+0.5, +11.5] | ahead (n.s.) | 96.00 |
| sysone Banking77 (12 intents) | 90.63 | 94.79 | +4.17 | [-2.1, +10.4] | ahead (n.s.) | 96.88 |
| sysone AG News | 91.00 | 92.00 | +1.00 | [-5.0, +7.0] | ahead (n.s.) | 91.00 |
| Tweet topics (test_2021, 1,693) | 79.33 | 79.21 | -0.12 | [-2.6, +2.4] | loss | 81.10 |
| sysone MNLI | 86.67 | 78.33 | -8.33 | [-21.7, +5.0] | loss | 70.00 |
| sysone guardrails | 96.67 | 83.33 | -13.33 | [-26.7, -3.3] | loss | 85.00 |
| sysone moderation | 98.89 | 83.33 | -15.56 | [-24.4, -7.8] | loss | 86.67 |
| sysone support triage | 88.75 | 66.25 | -22.50 | [-30.6, -14.4] | loss | 69.38 |
| Code review: quality rules | 99.48 | 67.45 | -32.03 | [-40.4, -23.5] | loss | 64.06 |
| JevBench (public 231) | 86.58 | 44.16 | -42.42 | [-53.3, -34.2] | loss | 45.45 |
| Code review: correctness rules | 98.00 | 51.04 | -46.96 | [-51.5, -42.5] | loss | 50.00 |
| sysone multilingual intent | 100.00 | 40.00 | -60.00 | [-80.0, -40.0] | loss | 12.00 |
All attempts are reported. The code-review benchmark was evaluated for three frozen systems: v6, v6.1 (a code-review round) and v6.2 (code-review correctness + multilingual). Correctness: 50.00 / 47.66 / 51.04; quality: 64.06 / 55.73 / 67.45. Multilingual intent: 12.00 / 20.00 / 40.00. Wins: 5 / 5 / 5. v6.2 was chosen on development data only (multilingual MASSIVE validation 35.5 โ 63.0; held-out code-correctness rules 44.7 โ 63.1) and replaced v6 as the release candidate by a rule fixed beforehand (at least as many wins). Choosing among frozen systems by benchmark results is disclosed as such.
Important caveats:
- Dwago is task-supervised: it was trained on the training splits of these task types, while Jev is a general model.
- The vulnerable-code benchmark was built from the same source dataset (CyberNative) as one of Dwago's training sets. Near-duplicates were removed, but the distributions are close.
- Code review is at chance on the benchmark's correctness rules, although every development measure improved. Real code reasoning is beyond this model.
- Several sysone suites are small (30โ100 decisions), so their intervals are wide.
On the frozen v5 final suite (descriptive)
This is the held-out suite used once per system. It was run for v6.2 after its release decision was fixed, and it shows the cost of breadth on the core tasks.
| System | DBpedia | SNLI | MNLI-m | MNLI-mm | Banking77 | BoolQ | AG News | Reserved schemas | STS-B ฯ | Primary macro |
|---|---|---|---|---|---|---|---|---|---|---|
| Dwago v6.2 | 98.88 | 83.41 | 73.36 | 73.99 | 89.45 | 66.42 | 91.53 | 88.04 | 0.752 | 81.72 |
| Dwago v6 | 98.96 | 83.61 | 74.04 | 74.54 | 90.55 | 65.63 | 91.95 | 86.46 | 0.751 | 82.19 |
| Dwago v5 | 99.09 | 84.21 | 74.37 | 75.59 | 91.46 | 66.61 | 51.47 | 79.35 | -0.034 | 82.80 |
| Dwago v1 | 96.98 | 65.55 | 43.67 | 43.35 | 85.55 | 60.64 | 30.21 | 59.86 | 0.447 | 61.57 |
Never-revealed primary macro: v6.2 vs v1 +20.16 pp (95% CI [+19.41, +20.92]); v6.2 vs v6 -0.47 pp ([-0.77, -0.16]). 52 results files verified against the ledger.
Latency
Batch size 1; raw JSON in, serialized JSON out; 2,500 timed requests per system on v1's request mix. This is local inference time and is not comparable to the response times of hosted APIs.
| System | Hardware | p50 ms | p95 ms | Requests/s |
|---|---|---|---|---|
| Dwago v6.2 | RTX 3070 (CUDA graphs) | 1.77 | 2.59 | 533 |
| Dwago v6 | RTX 3070 (CUDA graphs) | 1.76 | 2.47 | 540 |
| Dwago v1 | RTX 3070 (CUDA graphs) | 4.15 | 7.69 | 212 |
| Dwago v6.2 (ONNX int8w) | i5-14400F, 6 P-cores | 4.77 | 12.19 | 164 |
| Dwago v1 | i5-14400F, 6 P-cores | 4.00 | 5.81 | 232 |
Intended use and limitations
- Intended for research and prototyping of fast, local, structured decisions (routing, triage, tagging, moderation, intent and entailment-style checks) with human review.
- Not validated for consequential decisions (medical, legal, financial, employment) or safety-critical use. It is not a security scanner or a code reviewer.
- Weak spots:
- knowledge-heavy multiple choice;
- code review;
- non-Latin scripts, which are weaker than Latin-script languages (MASSIVE validation: Latin scripts 70.3%, other scripts 54.1%);
- long documents, truncated at 512 tokens.
- Accuracy on the core v5 tasks is slightly below v5, the cost of covering many tasks in 30M parameters.
- Calibration can degrade under distribution shift.
Reproducibility
results/ holds the pre-registrations (with every addendum), the frozen selection records, the append-only evaluation ledgers with the SHA-256 of every results file, all Jev comparisons, the development scores, the latency records and the data records. The training code is not published with this release; the pipeline is described step by step in the build write-up.
License and commercial use
- Released under the Dwago Evaluation License (LICENSE). Copyright (c) 2026 Mohammedi Yunus. Open weights, not open source.
- Allowed after access approval: evaluating, testing and benchmarking the model; non-commercial academic research, with credit to "Dwago by Mohammedi Yunus".
- Not allowed without a separate written license: any commercial or production use, redistribution of the weights or of modified versions, and using Dwago or its outputs to train other models.
- Commercial licenses (fees, royalties or revenue sharing) are available from Mohammedi Yunus: contact via the Hugging Face profile muhammediyunes.
- Some training datasets carry their own non-commercial or research-only terms (see Training data); this license grants no rights in them.
Citation
@misc{yunus2026dwago,
author = {Mohammedi Yunus},
title = {Dwago: a compact structured-decision model trained from scratch},
year = {2026}, howpublished = {\url{https://huggingface.co/muhammediyunes/dwago}}
}
- Downloads last month
- -