FractalSurfer/MOSS-Music-8B-Thinking-mlx-5bit

MOSS-Music-8B-Thinking, MLX 5-bit

OpenMOSS-Team/MOSS-Music-8B-Thinking converted to MLX and quantized to 5 bits, for a Mac with 16 GB of memory. It listens to music and answers questions about it in text. A community conversion: the model is the OpenMOSS Team's, and so is the credit.

It was made to be the local "ears" of Trainspotting Suite, where it hears every track sixteen bars at a time and fills in a form (who is singing and how, what the synth sounds like, the words, the mood). There, python -m djkit ears fetch downloads it and nothing else needs setting up.

Other builds: at 8 bits, mlx-community/MOSS-Music-8B-Thinking-8bit (10.2 GB, peaks at 11.1 GB: for 24 GB and up); and the Instruct checkpoint at 8 and 5 bits.

Use

With mlx-audio 0.5.7 or later, on Apple Silicon:

from mlx_audio.stt import load

model = load("FractalSurfer/MOSS-Music-8B-Thinking-mlx-5bit")
print(model.generate("clip.wav", prompt="What instruments are playing, and what is the mood?").text)

It thinks before it answers, inside <think>...</think>, which mlx-audio strips. To have it answer straight away (2-3x faster, and no worse at filling in a form), open its answer with an empty block: end the prompt you build with <|im_start|>assistant\n<think>\n\n</think>\n\n. Trainspotting does that (djkit/ears.py), and everything measured below was measured that way.

Sample it the way Qwen3, its language model, asks: temperature 0.7, top-p 0.8, top-k 20. At temperature 0 it repeats itself on anything long. Keep clips short (about 30 seconds) and questions narrow.

How it was converted

hf download OpenMOSS-Team/MOSS-Music-8B-Thinking --local-dir MOSS-Music-8B-Thinking      # commit 2ce89998
python -m mlx_audio.stt.models.moss_music.convert --hf-path MOSS-Music-8B-Thinking \
    --mlx-path MOSS-Music-8B-Thinking-mlx-5bit --quantize --q-bits 5 --q-group-size 64

with mlx 0.31.2 and mlx-audio 0.5.7. Nothing else was changed.

• Quantized to 5 bits (affine, group size 64): the Qwen3 language model, its token embeddings and lm_head.
• Left in bf16: the audio encoder, the audio adapter and the three DeepStack mergers. What they produce goes straight into the language model's input, and together they are under 2 GB.

Measured

On an M4 Max (64 GB), putting Trainspotting's listening-pass questions to it: a 30-second clip, a long system prompt (cached after the first clip) and a JSON form of about 230 tokens to fill in. Four clips, eight times each.

On disk	7.4 GB
Peak memory (MLX)	8.2 GB
Load	4 s
Per question	9 s (28 tokens a second; the machine was busy, and the same test has run a third faster on it idle)
Answers that were valid JSON as written	32 of 32
Full-mix answers with every field of the form	16 of 16
Questions that got stuck repeating a lyric and were asked again	3 in 32

The 8-bit build linked above, on the same questions: 32 of 32, 16 of 16, and 1 in 32. The Instruct checkpoint at 5 bits: 17 of 32 valid as written (31 after mending its brackets), 14 of 16, and 4 in 32. That is the reason this one is Trainspotting's default.

What was learned on the way, for anyone choosing a size:

• 5 bits cost nothing that this test could see against 8. 4 bits did, on the Instruct checkpoint: it misnamed or invented a field in a quarter of its answers and left a field out of nearly every full-mix form. That is why this is not a 4-bit build.
• Every size occasionally loops on a lyric that really does repeat. Cut the answer when it does and ask again.
• With thinking off it will still write words for a voice that isn't there when given an instrument stem and asked about voices. Ask an isolated stem only about what is in it.

Not measured: anything about a 16 GB machine other than the peak above, and accuracy on the benchmarks the base model's card reports.

Licence

Apache-2.0, the base model's (its card: "Models in MOSS-Music are licensed under the Apache License 2.0"). A copy is in LICENSE. The changes made to the original are the format conversion and quantization described above.

@misc{mossmusic2026,
  title  = {MOSS-Music Technical Report},
  author = {OpenMOSS Team},
  year   = {2026},
  howpublished = {\url{https://github.com/OpenMOSS/MOSS-Music}}
}