FractalSurfer/MOSS-Music-8B-Instruct-mlx-8bit

MOSS-Music-8B-Instruct, MLX 8-bit

OpenMOSS-Team/MOSS-Music-8B-Instruct converted to MLX and quantized to 8 bits, for a Mac with 24 GB of memory or more. It listens to music and answers questions about it in text. A community conversion: the model is the OpenMOSS Team's, and so is the credit.

It was made to be the local "ears" of Trainspotting Suite, where it hears every track sixteen bars at a time and fills in a form (who is singing and how, what the synth sounds like, the words, the mood). There, python -m djkit ears fetch downloads it and nothing else needs setting up.

The other size: 5-bit (7.4 GB, peaks at 8.2 GB: for a 16 GB Mac). For filling in a strict form, the Thinking checkpoint with its thinking switched off was the tidier of the two: MOSS-Music-8B-Thinking, MLX 5-bit.

Use

With mlx-audio 0.5.7 or later, on Apple Silicon:

from mlx_audio.stt import load

model = load("FractalSurfer/MOSS-Music-8B-Instruct-mlx-8bit")
print(model.generate("clip.wav", prompt="What instruments are playing, and what is the mood?").text)

Sample it the way Qwen3, its language model, asks: temperature 0.7, top-p 0.8, top-k 20. At temperature 0 it repeats itself on anything long. Keep clips short (about 30 seconds) and questions narrow.

How it was converted

hf download OpenMOSS-Team/MOSS-Music-8B-Instruct --local-dir MOSS-Music-8B-Instruct      # commit fce7f830
python -m mlx_audio.stt.models.moss_music.convert --hf-path MOSS-Music-8B-Instruct \
    --mlx-path MOSS-Music-8B-Instruct-mlx-8bit --quantize --q-bits 8 --q-group-size 64

with mlx 0.31.2 and mlx-audio 0.5.7, then model_repo in config.json corrected to name the Instruct checkpoint (the converter writes its default). Nothing else was changed.

• Quantized to 8 bits (affine, group size 64): the Qwen3 language model, its token embeddings and lm_head.
• Left in bf16: the audio encoder, the audio adapter and the three DeepStack mergers. What they produce goes straight into the language model's input, and together they are under 2 GB.

Measured

On an M4 Max (64 GB), putting Trainspotting's listening-pass questions to it: a 30-second clip, a long system prompt (cached after the first clip) and a JSON form of about 230 tokens to fill in. Four clips, eight times each.

On disk	10.4 GB
Peak memory (MLX)	11.3 GB
Load	4 s
Per question	6 to 7 s (27 to 40 tokens a second, depending on what else the machine was doing)
Answers that were usable JSON	29 of 32 after bracket repair (18 as written); the rest were asked again
Full-mix answers with every field of the form	13 of 16
Questions that got stuck repeating a lyric and were asked again	1 in 32

What was learned on the way, for anyone choosing a size:

• 8, 6 and 5 bits were close on this test: the same field names, the same dropped brackets (the full-precision model, run through PyTorch, wrote 4 of 8 answers with one too, so that is the model and not the quantization). The 5-bit build got stuck a little more often: 4 questions in 32 against 1.
• 4 bits fell apart. It misnamed or invented a field in a quarter of its answers, left a field out of nearly every full-mix form and got stuck repeating itself in a quarter of the questions. That is why there is no 4-bit build here.
• Every size occasionally loops on a lyric that really does repeat. Cut the answer when it does and ask again.
• This is the Instruct checkpoint: do not open its answer with an empty <think> block, as one does to stop the Thinking checkpoint thinking. Given one, it got stuck in 3 questions of 12, against 1 of 32 without.

Not measured: anything about a 16 GB machine other than the peak above, and accuracy on the benchmarks the base model's card reports.

Licence

Apache-2.0, the base model's (its card: "Models in MOSS-Music are licensed under the Apache License 2.0"). A copy is in LICENSE. The changes made to the original are the format conversion and quantization described above.

@misc{mossmusic2026,
  title  = {MOSS-Music Technical Report},
  author = {OpenMOSS Team},
  year   = {2026},
  howpublished = {\url{https://github.com/OpenMOSS/MOSS-Music}}
}