rapid-mlx/Ling-3.0-tiny-MLX-4bit

Run it on your Mac

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve ling-3.0-tiny-4bit
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'

Use it from OpenCode / Claude Code / Codex: rapid-mlx agents <name> --setup · Agent setup

Download the Desktop app · 8 GB unified memory minimum; allow extra headroom for long contexts.

Ling-3.0-tiny — MLX 4-bit

The first MLX conversion of inclusionAI/Ling-3.0-tiny: a 7.9B-total / 1.3B-active sparse-MoE reasoner (128 experts, top-8 + 1 shared) with a KDA + MLA hybrid attention stack and 131K context, MIT licensed.

4.2 GB at 4.507 bits/weight — it fits and runs on an 8 GB Apple Silicon Mac.

Quantization	4-bit, group size 64 (router kept 8-bit, short-conv weights fp)
Size on disk	4.2 GB
Context	131,072 tokens
Active parameters	1.3B per token
License	MIT (inherited from the base model)

Serve it

The bailing_hybrid architecture is not in upstream mlx-lm yet — this checkpoint is served by rapid-mlx, which ships a verified native implementation (reference parity 1.5e-6 against the official modeling code):

pip install -U rapid-mlx   # 0.12.10 or newer
rapid-mlx serve ling-3.0-tiny-4bit

You get an OpenAI-compatible server on localhost:8000 with reasoning (reasoning_content) and tool calling parsed natively — thinking is controlled with chat_template_kwargs: {"enable_thinking": true} or the model's detailed thinking on/off system-prompt switch.

Once mlx-lm gains native bailing_hybrid support, this checkpoint will load there unchanged.

Conversion provenance

Converted with mlx_lm.convert (quantize=True, q_bits=4, q_group_size=64) running rapid-mlx's vendored bailing_hybrid implementation (PR #1817), which was verified against the official modeling_bailing_moe_v3.py on identical random weights to a max logits deviation of 1.5e-6 (full prefill) / 1.9e-6 (token-by-token incremental) before conversion. End-to-end chat / reasoning / tool-call behaviour validated on an M2 Pro Mac mini.