elythros-decider-0.6b

A small decision model for Apple Silicon. You give it some text (the state) and one or more typed questions, and it returns a probability for every allowed answer. It doesn't generate text, so it can't produce an answer outside the options you supply.

• Built on Qwen3-0.6B, trained and run entirely with MLX.
• Many questions, one state: the state is read once, then every question is scored in parallel on a copy of its KV cache, so questions can't influence each other.
• Question types: choice (2–255 options) and noul (yes/no). score (ordered levels) is experimental and wasn't trained explicitly.
• Calibrated: one temperature per question type, fitted on validation data.

This is an independent v0.1 research release. It is not affiliated with or endorsed by TypeSafe (makers of Jev) or Qwen. Read Limitations before relying on it.

Quickstart

pip install mlx mlx-lm huggingface_hub
hf download Elythros/elythros-decider-0.6b --local-dir elythros-decider-0.6b

import sys; sys.path.insert(0, "elythros-decider-0.6b")
from decider import Decider

d = Decider.load("elythros-decider-0.6b")
d.decide(
    state="My card was charged twice for one coffee and I'm furious. Please fix it today.",
    questions={
        "topic":   {"type": "choice", "instructions": "What is the ticket about?",
                    "options": ["billing", "shipping", "login", "other"]},
        "angry":   {"type": "noul", "instructions": "Is the customer angry?"},
        "urgency": {"type": "score", "instructions": "How urgent is it?",
                    "levels": ["low", "medium", "high"]},
    },
)
# {"topic": {"type": "choice", "choice": "billing", "probabilities": {...}, "confidence": ...},
#  "angry": {"type": "noul", "noul": <P(yes)>},
#  "urgency": {"type": "score", "choice": ..., "score": <expected level index>, ...}}

Pass calibrated=False to decide for the raw probabilities. python decider.py runs a demo.

How it works

State:\n{state}\n                                       <- encoded once
\nQuestion: {q}\nOptions: {opt1}<OPT> {opt2}<OPT> ...   <- one branch per question, batched

• <OPT> is a learned embedding inserted after every option (not a vocabulary token).
• Readout: the final hidden state at each <OPT> goes through RMSNorm and a linear head, giving one logit per option, then a softmax over that question's options.
• Yes/no questions are 2-option choices over yes/no.

Files

• model.safetensors: the backbone, with the LoRA fused in (bf16, standard mlx-lm Qwen3 format).
• decision_head.safetensors: the <OPT> embedding, readout norm and head.
• decider_config.json: prompt format and temperatures.
• decider.py: the whole inference code.

Latency (M4 Pro, 195-token state):

Questions × options	Latency
1 × 4	~48 ms
10 × 4	~85 ms
1 × 255	~270 ms

These timings were measured on the same architecture before release.

Training

• Starting point: frozen Qwen3-0.6B base weights.
• Trained parameters: LoRA (rank 16, all 28 layers) plus the <OPT> embedding and head, about 10M in total.
• Loss: cross-entropy over each question's options.
• Run: 5,000 steps at 2,048 tokens per step, AdamW at lr 2e-5 with warmup and cosine decay. Took about 2 hours on one M4 Pro (24 GB). The best checkpoint by validation NLL was at step 4,500.
• Option shuffling: option order is reshuffled every time a request is used, to avoid position bias.
• Calibration: temperature scaling on the validation splits, with choice T = 1.31 and yes/no T = 1.70.

Dataset	Decision taught	License
Banking77	77-way banking intent	CC-BY-4.0
CLINC150 ("plus")	151-way assistant intent + yes/no out-of-scope (out-of-scope oversampled to 20%)	CC-BY-3.0
MNLI (GLUE)	entailed / neutral / contradicted	see GLUE terms
SNLI	entailed / neutral / contradicted	CC-BY-SA-4.0
BoolQ	yes/no question about a passage	CC-BY-SA-3.0
SST-2	negative / positive	see dataset card
MMLU auxiliary train	4-way multiple choice	MIT

Some training sets carry share-alike licences (SNLI, BoolQ). The weights are released under Apache-2.0, like the Qwen3 base. Check that this suits your use.

Evaluation

All numbers below are for these released files, run with calibrated probabilities. ECE is expected calibration error: lower means the stated confidence is more honest.

1. Test splits of the training datasets (500 requests each, options shuffled)

Task	Zero-shot Qwen3-0.6B (acc)	Decision-2.0-Kai-0.6B (acc / ECE)	This model (acc / ECE)
Banking77	0.286	0.788 / 0.111	0.834 / 0.046
CLINC intent	0.330	0.738 / 0.079	0.838 / 0.080
CLINC out-of-scope (F1)	0.211	0.449	0.717
MNLI	0.386	0.740 / 0.186	0.844 / 0.070
SNLI	0.310	0.718 / 0.172	0.858 / 0.051
BoolQ	0.612	0.768 / 0.052	0.824 / 0.067
SST-2	0.794	0.856 / 0.159	0.904 / 0.022
MMLU	0.342	0.420 / 0.061	0.436 / 0.113
Mean accuracy (8 questions incl. CLINC out-of-scope accuracy)	0.410	0.724	0.802

Caveat: these are this model's training domains, and Kai was not trained on them as far as we know. That makes the comparison favourable to this model.

2. Held-out tasks this model never trained on (200 requests each)

Task	Chance	Decision-2.0-Kai-0.6B	mpuig/system-one-qwen3-0.6b	This model
Emotion (6-way)	0.17	0.520	0.495	0.440
TREC question type (6-way)	0.17	0.585	0.815	0.645
Subjectivity (yes/no)	0.50	0.690	0.595	0.560
Enron spam (yes/no)	0.50	0.495	0.515	0.510
Mean accuracy		0.573	0.605	0.539
Mean ECE		0.178	0.105	0.181

On unseen tasks this model is behind the two comparable open models. Spam detection is at chance for all three.

How the baselines were run:

• Kai: its official runtime, on the same requests.
• mpuig: its author's engine, on the same requests. It can't answer questions with more than 26 options.
• Task overlap: the held-out tasks were chosen to avoid this model's and mpuig's training sets. Kai's training data isn't public.

Limitations

• Yes/no questions on new tasks are weak. Only two yes/no tasks were in training. On unfamiliar yes/no questions the model often falls back to one answer: it said "yes" for 98.5% of subjectivity items. Test on your own data first.
• Specialised rather than general: strong on intent, NLI, sentiment and passage QA style decisions; weaker elsewhere.
• Knowledge-heavy questions: limited (MMLU 44%), as expected at 0.6B.
• Language and length: English only. Training requests were short (under ~1k tokens); long states are untested.
• Score questions: score is untrained and treated as a choice over the levels.
• Calibration: temperatures were fitted on in-domain validation data. Under domain shift, refit a temperature on about 100 labelled examples from your own workload.
• Fusion rounding: merging the LoRA into bf16 weights changes probabilities by about 0.006 on average versus the unfused checkpoint; about 1% of near-tie answers flip. The scores above are for the fused release.

License

Apache-2.0 (see LICENSE). Built on Qwen/Qwen3-0.6B (Apache-2.0). Training datasets are under their own licences (table above).