ljupco/mlx-serve-macos-15-build-run

mlx-serve on macOS 15.x — build & run package (Qwen3.8-Flash-Next with MTP)

Proven 2026-09-05 on macOS 15.7.7, M2 Max 96 GB, Apple clang 17 (CommandLineTools 16.4, no full Xcode at start) → working mlx-serve 26.9.1-dev serving Qwen3.8-Flash-Next (qwen4_exp) 4-bit pack with native MTP at TG 27.43 t/s / PP 29.6 t/s.

Model pack source: https://huggingface.co/ddalcu/Qwen3.8-Flash-Next-MLX-Serve-4bit mlx-serve upstream: https://github.com/ddalcu/mlx-serve/ (site: mlxserve.com)

This directory is the reproduction package. AGENTS.md is the entry point: it explains why each step exists, lists the exact TODO order, the commands, the verification gate for every step, and every failure we hit with its fix.

Files:

• AGENTS.md — read this first; the TODO is the whole reproduction
• scripts/build-mlx-macos15.sh — MLX + mlx-c at deployment target 15.0 (NAX off)
• scripts/build-llama-b10472-macos15.sh — llama.cpp at mlx-serve's pinned tag b10472
• scripts/download-model.sh — fetch the 4-bit pack from HF
• scripts/verify-install.sh — every gate in one run
• scripts/run-mlxserve.sh — launch the server (MTP on, PLE streamed from disk)
• STATE-REPORT.md — full research state (llama.cpp PRs, alternatives, numbers)

Related (not in this package):

• $HOME/mlx-serve/ — the mlx-serve git clone we build from
• $HOME/mlx-serve-models/ddalcu/Qwen3.8-Flash-Next-MLX-Serve-4bit — the model pack (98 GiB, on NVME so the 30 GiB n-gram table streams from disk)

Acknowledgements

This package exists because of mlx-serve by David Dalcu — a remarkable piece of engineering. A native Zig server running MLX-format models on Apple Silicon with OpenAI-, Anthropic-, Responses- and Ollama-compatible APIs on one port, it ships a complete qwen4_exp engine — hyper-connections, gated delta-net layers, QSA sparse attention, batched decode, native MTP for the Qwen 3.5/3.6/3.8 family, and the n-gram PLE table designed to stream from disk (ngram_table.bin, mmap'd, host-gathered, never resident) — plus a published 4-bit pack and a 462-line engineering-gotchas log. Running a 176B-parameter model with speculative decoding on a MacBook was his work first; this package only documents how to get it onto macOS 15, where the official release binaries (built for macOS 26.2 / Metal 4.0) will not load. Thank you, David — profusely.

Also credited: kernelpool (the qwen4-exp ds4 branch that proved the Metal-graph port), AtomicChat (the AD-Q4_K_M quant), Unsloth (imatrix + MTP sidecar GGUFs), and the llama.cpp contributors behind PRs #27742 and #28015 (qwen4exp support and M2 Max Metal tuning).