sergolsub/mlx-clef-flash-9B

Cloudflare clef-flash (MLX 8-bit)

8-bit MLX conversion of Cloudflare/clef-flash for Apple Silicon (mlx_lm).

Origin and license

• Original weights: https://huggingface.co/Cloudflare/clef-flash
• License: apache-2.0 — carried over from the upstream weights; if anything conflicts, the upstream repository's LICENSE prevails.
• Conversion: 8-bit quantization for MLX only; no retraining, no architecture changes.

Measured (Mac mini M4 Pro 48 GB, MLX 8-bit)

• Standard stand — load 5.17 s · peak RSS 11.3 GB
• Text 512 tokens (522 tok) — avg of 5 after 3 warm-up: binary 1518 ms · multichoice 1625 ms · score 1669 ms · peak RSS 11.48 GB
• Image 512x512: not supported
• Box: not supported
• On disk 10.47 GB · load 3 s · peak resident memory 11.46 GB
• Decision latency: median 737 ms · sustained 2.15 short decisions/s
• Quality after 8-bit quantization (our eval): accuracy 83.0% (8-bit) vs 81.8% (bf16) · noul 87.2% vs 84.6% · choice 84.0% vs 84.0% · score 75.0% vs 75.0%
• Calibration PPL 2.349 (8-bit) vs 2.341 (bf16)
• Decision Index 0.4.1: 43.46

Usage

from huggingface_hub import snapshot_download
path = snapshot_download("sergolsub/mlx-clef-flash-9B")  # weights + the loader .py
import sys; sys.path.insert(0, path)
from clef_mlx import Clef
model = Clef(path)
# one separate call per task type (each question is its own object):
print(model.decide(
    "I was charged twice for order #4411 and the app still shows it as unpaid.",
    {"refund": {"type": "noul", "instructions": "Does the customer want a refund?"}}
))
print(model.decide(
    "Order #7821 shipped on Monday but tracking has not updated for 5 days.",
    {
        "team": {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"billing": "Payments and refunds",
                               "shipping": "Delivery and tracking"}}
    }
))
print(model.decide(
    "Export failed: disk quota exceeded on the worker.",
    {
        "severity": {"type": "score", "instructions": "How severe is the impact?",
                      "criteria": ["cosmetic", "annoying", "blocks"]}
    }
))

Files

• *.safetensors — 8-bit weights (MLX layout)
• config.json, tokenizer*, chat_template.jinja — standard transformers/MLX set
• *_mlx.py / decision_config.json / joint_config.json — decision/joint-head inference glue (present when the model has a decision mode)

Quantization quality (KL vs bf16)

• Weights compare against the bf16 originals tensor by tensor (histogram KL over 512 bins, dequantized 8-bit vs source).
• Tensors compared: 427 · mean KL 3.87e-04 · max KL 5.57e-03 · nonzero 249
• Worst layer: model.language_model.layers.0.linear_attn.in_proj_a.weight (KL 5.57e-03)
• Full per-tensor table: kl_audit.json in this repository (tensor, KL, max abs err, RMSE).

Checksums (SHA-256)

SHA-256 of every file in this repository except this README itself: LFS digests pulled from the HF metadata, small files (config, tokenizer, …) hashed on demand — the safetensors shards included. Re-check any download with sha256sum <file> and compare.

File	SHA-256	Size
.gitattributes	34448b82c17d60fec9b65b1f093c115ddbaadc04beb1b0140b6bfed2e012a930	1,570 B
LICENSE	bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a	11,544 B
chat_template.jinja	a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715	7,756 B
clef_mlx.py	607316fd13380bc8cda5109b6e9b8eefc00e7cef3aa7e916165c775edb701dee	15,495 B
config.json	a57825e295127bf61caa14db9214dfad58526bc6d95e10ba6fae13925893bb38	2,914 B
generation_config.json	45707f8467bf4e54e112c6095a1b2d1f2d63651cf86816344cfff21f02dda0d4	116 B
joint_head.safetensors	19cdcec8c81dc9212be320fff47462ab342fbc1278be4368fb3da71241cf5ba0	244 MB
joint_head_config.json	77efe959a38b5b17b241543e129e695f3c77465ece55a25985279bd8176279a0	119 B
joint_schema_model.py	0e304cf7c6500e8bb59bef7e2afd2c6373f82596dfb3b57d1aa93c175e2dc3a3	23,263 B
kl_audit.json	b65d028f970753cad058b8147791b4aee4d1807336ef9d320cc552d038b8d683	46,882 B
model-00001-of-00004.safetensors	655dc95988fadf4b2d8196cebb2755bfad5d3893309de177147fa94ad9bccc93	4.03 GB
model-00002-of-00004.safetensors	f4553aefd43ef0494c35c98e6176c7a3e00a69f0b951d6686c15d4029e7f5585	4.04 GB
model-00003-of-00004.safetensors	17353947634ffebbe779426a94ce219f7e02d66fec3eca8d0ec893906da19071	2.40 GB
model.safetensors.index.json	c7ec83de668c6e4de23544abf79bd945cfad9e8903768c071ae5fb92c20e8700	89,372 B
processor_config.json	d89ef49ce9cd37fbf510158e13c1ef063d9286411c1ec9049932dbe0487143b1	1,191 B
tokenizer.json	06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523	20 MB
tokenizer_config.json	91a08f825d370d085d692e04cf117cdd7faad7bf18e996f1e6031b6dab03db72	1,075 B