ddalcu/MOSS-Image-SFT-MLX-Serve-8bit

MOSS-Image-SFT MLX-Serve 8-bit

8-bit pack of OpenMOSS-Team/MOSS-Image-SFT for mlx-serve: 14.4 GB on disk, about 21 GB of memory at 1024x1024. Renders match bf16 at the same seed, in-image text included.

Run it

Download MLX-Serve.app, open the Image tab, and pick MOSS-Image SFT 8-bit from the model menu. It downloads with a progress bar and generates in the same window. No terminal, nothing to configure.

Prefer Homebrew? It is a third-party tap, so tap it first:

brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve
brew trust ddalcu/mlx-serve
brew install --cask mlx-serve

Driving it from code instead? The app runs the server on http://localhost:11234:

curl localhost:11234/v1/images/generations -H 'Content-Type: application/json' \
  -d '{"model":"ddalcu/MOSS-Image-SFT-MLX-Serve-8bit","prompt":"a red fox in fresh snow","size":"1024x1024"}'

50 steps and guidance 3 when the request leaves them out (the authors' settings), with the negative branch skipping block 22. guidance_scale: 1 runs one forward per step. Prompts can be prose, tag lists, or the model's JSON layout (summary plus items with a bbox each). Text-to-image only. Needs an mlx-serve build newer than 26.10.3.

Measured (M4 Max, 128 GB)

Pack	1024x1024, 50 steps, CFG 3	Peak memory
bf16 (mlx-community/MOSS-Image-SFT-bf16)	358 s	32.9 GB
8-bit	403 s	21.0 GB

Per request with the model loaded; peak is MLX's own across the request with everything resident. Quantizing saves memory, not time.

mlx-serve

A native Zig server for Apple Silicon. No Python, no cloud, no Electron. One 9 MB binary.

• One server, every modality. Chat, images, video, music, speech with voice cloning, and 3D, all running natively on MLX.
• Points at what you already use. OpenAI- and Anthropic-compatible APIs on http://localhost:11234, so Claude Code, the OpenAI SDK, Continue, Cursor and Open WebUI just work.
• Any LLM, not just these. Every MLX model and every GGUF on Hugging Face, with speculative decoding built in.
• MLX-Serve.app included. Signed macOS menu-bar app: chat, agent mode with MCP tools, model downloads, and every generator above, no terminal needed.

mlxserve.com · GitHub

If it is useful to you, a star on GitHub genuinely helps.

What is in it

The release's own diffusers layout and key names, with the DiT block linears and every text-encoder linear (embed_tokens included) affine-quantized to 8-bit (group 64). Kept dense: the VAE, norms, and the DiT's small or shared linears. Dropped: the text encoder's vision tower. Built by tests/convert_moss_image_weights.py --bits 8 in the mlx-serve repo.

Apache-2.0, same as the base model. The original model and weights are by the OpenMOSS team.