sergolsub/mlx-d3-flash-9B

Decision3 Flash (MLX 8-bit)

8-bit MLX conversion of vllm-sr/d3-flash for Apple Silicon (mlx_lm).

Origin and license

• Original weights: https://huggingface.co/vllm-sr/d3-flash
• License: apache-2.0 — carried over from the upstream weights; if anything conflicts, the upstream repository's LICENSE prevails.
• Conversion: 8-bit quantization for MLX only; no retraining, no architecture changes.

Measured (Mac mini M4 Pro 48 GB, MLX 8-bit)

• Standard stand — load 4.41 s · peak RSS 9.96 GB
• Text 512 tokens (479 tok) — avg of 5 after 3 warm-up: binary 1317 ms · multichoice 1417 ms · score 1316 ms · peak RSS 9.98 GB
• Image 512x512 + text — avg of 5 after 3 warm-up: binary 2279 ms · multichoice 2290 ms · score 2276 ms · peak RSS 10.06 GB
• Box: not supported
• Quality after 8-bit quantization (our eval): accuracy 90.0% · noul 92.5% · choice 88.0% · score 88.0% · calibration PPL 1.843
• Decision Index 0.4.1: 54.21

Usage

from huggingface_hub import snapshot_download
path = snapshot_download("sergolsub/mlx-d3-flash-9B")  # weights + the loader .py
import sys; sys.path.insert(0, path)
from d3_mlx import D3
model = D3(path)
# one separate call per task type (each question is its own object):
print(model.decide(
    "I was charged twice for order #4411 and the app still shows it as unpaid.",
    {"refund": {"type": "noul", "instructions": "Does the customer want a refund?"}}
))
print(model.decide(
    "Order #7821 shipped on Monday but tracking has not updated for 5 days.",
    {
        "team": {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"billing": "Payments and refunds",
                               "shipping": "Delivery and tracking"}}
    }
))
print(model.decide(
    "Export failed: disk quota exceeded on the worker.",
    {
        "severity": {"type": "score", "instructions": "How severe is the impact?",
                      "criteria": ["cosmetic", "annoying", "blocks"]}
    }
))

Files

• *.safetensors — 8-bit weights (MLX layout)
• config.json, tokenizer*, chat_template.jinja — standard transformers/MLX set
• *_mlx.py / decision_config.json / joint_config.json — decision/joint-head inference glue (present when the model has a decision mode)

Quantization quality (KL vs bf16)

• Weights compare against the bf16 originals tensor by tensor (histogram KL over 512 bins, dequantized 8-bit vs source).
• Tensors compared: 759 · mean KL 2.17e-04 · max KL 5.49e-03 · nonzero 249
• Worst layer: language_model.layers.0.linear_attn.in_proj_a.weight (KL 5.49e-03)
• Full per-tensor table: kl_audit.json in this repository (tensor, KL, max abs err, RMSE).

Checksums (SHA-256)

SHA-256 of every file in this repository except this README itself: LFS digests pulled from the HF metadata, small files (config, tokenizer, …) hashed on demand — the safetensors shards included. Re-check any download with sha256sum <file> and compare.

File	SHA-256	Size
.gitattributes	34448b82c17d60fec9b65b1f093c115ddbaadc04beb1b0140b6bfed2e012a930	1,570 B
chat_template.jinja	a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715	7,756 B
config.json	bf30684e5eb3a6c61de99c6e6da1af758a4b0534820a835fd6ece17bf005b756	3,175 B
d3_mlx.py	7c837274db10ab3f7dd591efe2480494b1cdb03ec57435e231bb55a999db9601	40,757 B
decision_config.json	0269c72ac055d8149a7a778c477780af361f4c1cde3fd0e5ba9af791453ece18	5,502 B
kl_audit.json	172d214b88d0ef323d3b04465f2d1fe141766147b8c13c47f85c90c53c5c8f71	61,846 B
merges.txt	a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d	3 MB
model-00001-of-00004.safetensors	fd99efa421ea89b3236397ce5a77e933e3f8096376b9cdcbf6960f34710ff053	4.01 GB
model-00002-of-00004.safetensors	a57e68b9cce7f058fb80cbbe45c08529450bf86e73217b80a5f38e77e2768860	4.02 GB
model-00003-of-00004.safetensors	39b863e2649b587082df070d7b158833af9b35634539fffa158abcbf32f67742	1.32 GB
model.safetensors.index.json	a73ea7accfc01d793f19f4b4f32bd96427d71f515601faa43375b76050114894	115,836 B
preprocessor_config.json	27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516	390 B
readout.safetensors	8bbc9a9df4e1b7df85a80a1091977092b938d8d2d2869dfaeb8fd36063cf322f	4 MB
tokenizer.json	5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42	13 MB
tokenizer_config.json	316230d6a809701f4db5ea8f8fc862bc3a6f3229c937c174e674ff3ca0a64ac8	16,710 B
video_preprocessor_config.json	7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13	385 B
vocab.json	ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003	7 MB