OpenDecider
Collection
Open decision models (System One): typed choice/score/yes-no questions, calibrated probabilities. pip install opendecider • 8 items • Updated
OpenDecider-small MLX 4-bit
OpenDecider-small, the 4B decision model, merged and
quantised to 4-bit for Apple Silicon with MLX: 2.1 GB instead of 8 GB, and 2.6 GB of
memory while answering. Ask typed questions (choice, score, noul) about any text or JSON and get a
calibrated probability for every option. Apache-2.0.
Smallest build: 2.6 GB, but it costs about 2 points on typed-decisions (0.651 vs 0.672). If you have the memory, use the 8-bit build.
Installation
pip install "opendecider[mlx]"
Quickstart
from opendecider import load
model = load("manjunathshiva/opendecider-small-mlx-4bit")
r = model.system_one(
"Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan.",
{"department": {"type": "choice", "instructions": "Which department should handle this?",
"criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages", "other": "everything else"}},
"churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"}})
print(r["answers"]["department"]["choice"], r["answers"]["churn_risk"]["noul"])
Same model, smaller: how close is it?
Scored with the benchmark harness on the same questions as the full-precision release:
| benchmark | OpenDecider-small (bf16, PyTorch) | MLX 4-bit | same top answer as bf16 |
|---|---|---|---|
| 200 general decisions | 0.735 | 0.745 | 376/400 |
| typed-decisions (2,000 decisions) | 0.672 | 0.651 | 1691/2000 |
Will it fit?
| Mac | memory used | latency, one question |
|---|---|---|
| Apple M4 Max, 64 GB | 2.6 GB | 65 ms (short question); 143 ms median on benchmark questions |
Any Apple Silicon Mac with 8 GB or more should run it (not every size tested).
Links
- Full-precision model and benchmarks: https://huggingface.co/manjunathshiva/opendecider-small
- GitHub: https://github.com/manjunathshiva/opendecider
- Collection: https://huggingface.co/collections/manjunathshiva/opendecider-6ab8c838909092518d50a9ea
Apache 2.0 · Base model Qwen3-4B-Instruct-2507 (Apache-2.0) · Manjunath Janardhan
- Downloads last month
- 20
Model size
4B params
Tensor type
U32
·
BF16 ·
Hardware compatibility
Log In to add your hardware
4-bit
Model tree for manjunathshiva/opendecider-small-mlx-4bit
Base model
Qwen/Qwen3-4B-Instruct-2507 Adapter
manjunathshiva/opendecider-small