sergolsub/mlx-d3-v3.1.0-27B

vLLM-SR d3 v3.1.0 (MLX 8-bit)

8-bit MLX conversion of vllm-sr/d3 for Apple Silicon (mlx_lm).

Origin and license

• Original weights: https://huggingface.co/vllm-sr/d3
• License: apache-2.0 — carried over from the upstream weights; if anything conflicts, the upstream repository's LICENSE prevails.
• Conversion: 8-bit quantization for MLX only; no retraining, no architecture changes.

Stream-converted from the upstream BF16 shards; LICENSE and NOTICE are uploaded together with the weights

Measured (Mac mini M4 Pro 48 GB, MLX 8-bit)

• Short decision — 3 warmup + median of 5: 1238 ms
• On disk 28.15 GB · load 7.08 s · peak resident memory 15.78 GB
• Decision latency: median 2141 ms · sustained 0.63 short decisions/s
• Quality after 8-bit quantization (our eval): accuracy 88.9% · noul 87.5% · choice 96.0% · score 84.0% · calibration PPL 1.981
• Decision Index 0.4.1: 62.5

Usage

from huggingface_hub import snapshot_download
path = snapshot_download("sergolsub/mlx-d3-v3.1.0-27B")  # weights + the loader .py
import sys; sys.path.insert(0, path)
from d3_mlx import D3
model = D3(path)
# one separate call per task type (each question is its own object):
print(model.decide(
    "I was charged twice for order #4411 and the app still shows it as unpaid.",
    {"refund": {"type": "noul", "instructions": "Does the customer want a refund?"}}
))
print(model.decide(
    "Order #7821 shipped on Monday but tracking has not updated for 5 days.",
    {
        "team": {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"billing": "Payments and refunds",
                               "shipping": "Delivery and tracking"}}
    }
))
print(model.decide(
    "Export failed: disk quota exceeded on the worker.",
    {
        "severity": {"type": "score", "instructions": "How severe is the impact?",
                      "criteria": ["cosmetic", "annoying", "blocks"]}
    }
))

Files

• *.safetensors — 8-bit weights (MLX layout)
• config.json, tokenizer*, chat_template.jinja — standard transformers/MLX set
• *_mlx.py / decision_config.json / joint_config.json — decision/joint-head inference glue (present when the model has a decision mode)

Quantization quality (KL vs bf16)

• Weights compare against the bf16 originals tensor by tensor (histogram KL over 512 bins, dequantized 8-bit vs source).
• Tensors compared: 1183 · mean KL 3.80e-04 · max KL 7.78e-03 · nonzero 581
• Worst layer: language_model.layers.0.linear_attn.in_proj_b.weight (KL 7.78e-03)
• Full per-tensor table: kl_audit.json in this repository (tensor, KL, max abs err, RMSE).

Checksums (SHA-256)

SHA-256 of every file in this repository except this README itself: LFS digests pulled from the HF metadata, small files (config, tokenizer, …) hashed on demand — the safetensors shards included. Re-check any download with sha256sum <file> and compare.

File	SHA-256	Size
.gitattributes	34448b82c17d60fec9b65b1f093c115ddbaadc04beb1b0140b6bfed2e012a930	1,570 B
LICENSE	c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4	11,357 B
NOTICE	61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469	100 B
chat_template.jinja	c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041	8,952 B
config.json	78015b33e5b42c8beb9d4d733b1902998094ffcfe49174fb64c1dd2b9f8668b3	4,032 B
d3_mlx.py	7c837274db10ab3f7dd591efe2480494b1cdb03ec57435e231bb55a999db9601	40,757 B
decision_config.json	6b80ca11bd6ba481df4b3c187db8a5d1983786563ca05e2edccb0bdac0d2fc4f	5,503 B
kl_audit.json	31179c80f3f96f123c3ba208fcf5a335614a6494a7ce3891e9d7201154252945	107,772 B
merges.txt	a9d356d7bdf1ef4949e3e748e95b8e10ad9d4e2e838eddc38a0a7b6b94d1db8d	3 MB
model-00001-of-00026.safetensors	6eca2bd8c872c6c6a89d09f53664dec97a6bd05b5df586e646a35d36aa17f776	1.35 GB
model-00002-of-00026.safetensors	b00cfdf2c0cae108512e78df47170c7d495a45165507ffd7f9b300c61fa8a603	1.08 GB
model-00003-of-00026.safetensors	e1ecaf13d5d38fd8c7eaad9014c1f1ba8825f2efcdee7ac3c2f85e3bdeb37df8	1.09 GB
model-00004-of-00026.safetensors	fe054b068e912dc7586f7f87895a5cd09640f260c9e5c93ae3c5e058bd040872	1.10 GB
model-00005-of-00026.safetensors	b0156842cf4021592941173c7ccf3a9fb2e1b77885815fa27a10409c8e88d5c9	1.12 GB
model-00006-of-00026.safetensors	752d8b3e258f4833a76e84fe08f7d81d8d9b3f47e33cb92a9afca20dc11df9dc	1.12 GB
model-00007-of-00026.safetensors	d6b08619971e00969993f65a7cca535fa898c1daee9d41e41f9c5620ef3c4172	1.10 GB
model-00008-of-00026.safetensors	3afbd5e8f5257bce11d2d98bb53d2c302fc02cad249cbae1a68399e17ac8efae	1.12 GB
model-00009-of-00026.safetensors	e18a5263f0b883bd1d9986a885e9821714fb342fba1916cae07b249aac82a5b6	1.08 GB
model-00010-of-00026.safetensors	5d74a10f5523092a447c390f86a699eea6189074336bbbc2208c3cf0eac0bb04	1.13 GB
model-00011-of-00026.safetensors	b68d89cf3f020b1d70b8c4d7a9567b1751c41472bde734b5d0e34fc6565cb5e1	1.12 GB
model-00012-of-00026.safetensors	f1c0175c2160dcff7c9caf08e13412bcea14c60e6079f6ad0e69cd77cce83291	1.12 GB
model-00013-of-00026.safetensors	05e720ae26b3da700e2198a6b0112261ded50bb8f448b0dd88430f20a3ed4c79	1.10 GB
model-00014-of-00026.safetensors	9c9a453f2aebdaf7878322491147e395d5673dc7836fd575ebf2529356efc22e	1.12 GB
model-00015-of-00026.safetensors	ebca6ba9c52ed91f1d7d24f87590f0441cabe0c33c5df27c2c6096bd985a108b	1.12 GB
model-00016-of-00026.safetensors	aeac27c4948fdebf09bd56affcea0d3b494fc8107468f88484e93e0d8cfa5080	1.10 GB
model-00017-of-00026.safetensors	02221537ccb86cb5781169abf07308d5a4c5fad1dc88e7b61786aba57fe281b0	1.08 GB
model-00018-of-00026.safetensors	a5a0e37ca41a77510b2ccac85ddbc23751334c8f1d28377da12d583adf9be27e	1.15 GB
model-00019-of-00026.safetensors	abda544429b1f46fd9128f822ac600feafd69080a91c63c17a5edcc4717e00c0	1.12 GB
model-00020-of-00026.safetensors	5a0b40460e3fb154dbaadaa9fc8d0c3f56ec9773054c6ac17fadf1678a2f0c9a	1.13 GB
model-00021-of-00026.safetensors	33eecd5b7ac81c98bc65a513cc0f94cc537972d0326b0660f341bb2bd001e1bd	1.08 GB
model-00022-of-00026.safetensors	a15a5566eb49b73297ebad998b711afeae417ce84d30c0b792f3c55a617910e1	1.11 GB
model-00023-of-00026.safetensors	03238986e77b685715eabd3a1c3bdbe7e4eba2b0e7cf3e566495c90269f5b86d	1.13 GB
model-00024-of-00026.safetensors	7d8fecff21c535d4562e584d4efa3e2c56812931ce7130efa11013503f046fe3	1.12 GB
model-00025-of-00026.safetensors	c56c72dd445431abed63a5d6adc5727c85d225d43768568bd7c0134069ef1ab3	992 MB
model.safetensors.index.json	d77d7a7c7ca4817ae9fe7ab82c8f8610a23de49fd0cb7350c6d081530429f09b	204,636 B
preprocessor_config.json	27225450ac9c6529872ee1924fcb0962ff5634834f817040f444118116f4e516	390 B
readout.safetensors	5319cbf3ccdc5d3ba348b318d7b54acfc3a861d3ff94b578e787852c8726055a	5 MB
tokenizer.json	0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3	13 MB
tokenizer_config.json	b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27	17,928 B
video_preprocessor_config.json	7768af27c1fafa9cc9011c1dc20067e03f8915e03b63504550e11d5066986d13	385 B
vocab.json	ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003	7 MB