MOSS-Transcribe-Diarize (MLX, 8-bit decoder)

8-bit MLX conversion of OpenMOSS-Team/MOSS-Transcribe-Diarize (revision 704aa4a9c304e8520be88901e0d1960158ef5b15, Apache-2.0), used by the mac-toolbox app on Apple silicon.

What was converted

The Qwen3 text decoder and the shared embedding are quantized to 8-bit (affine, group size 64).
The Whisper encoder and the VQ adaptor stay in BF16 (excluded by the model's own model_quant_predicate).
Weights are saved after mlx-audio's sanitize, so config.json carries quantization: {"group_size": 64, "bits": 8, "mode": "affine"}.

Converted with mlx-audio 0.6.1 (mlx 0.32.4):

from mlx_audio.convert import convert

convert(
    "OpenMOSS-Team/MOSS-Transcribe-Diarize",
    "MOSS-Transcribe-Diarize-mlx-q8",
    quantize=True,
    q_group_size=64,
    q_bits=8,
    revision="704aa4a9c304e8520be88901e0d1960158ef5b15",
)

Check

On AliMeeting R8009 cut into six ~5-minute windows, greedy decoding with the official Chinese prompt gives byte-identical raw output to loading the original BF16 weights and quantizing them in memory with the same settings (6/6 windows).

Files

File	SHA-256
`added_tokens.json`	`c0284b582e14987fbd3d5a2cb2bd139084371ed9acbae488829a1c900833c680`
`chat_template.jinja`	`8641466a16b184ebaf7c4903391e607cfd532ab937e81a64120628ab79d827f4`
`config.json`	`0371f3ef44fc4fc10a0f8debe5adadcfa2238c6264eec1aa0fa8754d414a6a68`
`configuration_moss_transcribe_diarize.py`	`b4d12b0f4609af69b61c2fe3aa5fbaf476af22278369e4540745bc47d1d37892`
`generation_config.json`	`e53a4b3ce4f944230cf1ca8fed0c42f4ff0d8c1443eaf98b5315d987334dd9e4`
`merges.txt`	`8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5`
`model.safetensors`	`6a1909445bfbde657b641456adf11a1287895217dcd79363be517173b9a287a9`
`model.safetensors.index.json`	`0946d5defb824aa93894eaf538cf8be50c7b35882689f469db42b5ee93fd411d`
`modeling_moss_transcribe_diarize.py`	`a01da90fe1f7cb88942b8c56f443e7b4ecd307ed4b4c356ac08d7503dd7422c1`
`preprocessor_config.json`	`ba2e601484abc80f4cded977f9a4fd4a53175b7d35c2f2511f0cfc3a32ad2499`
`processing_moss_transcribe_diarize.py`	`6f228d22d9379e2f6a6830b18ce7336b22da8267547e96e65545d871d7f48766`
`processor_config.json`	`a978c2dd54a65b576c3dae4b654fe9bcbac1184c6db2df0afb2c90fcdc872ae7`
`special_tokens_map.json`	`76862e765266b85aa9459767e33cbaf13970f327a0e88d1c65846c2ddd3a1ecd`
`tokenizer.json`	`bcf03774334462d6e34b5005cb11120a62275f146ee2953e68731ecdbce84fbb`
`tokenizer_config.json`	`61d04c96104177240688396655ae3f7cf38ce2ea036db867a1d2b6883e27c3d5`
`vocab.json`	`ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910`

Downloads last month: 23

Safetensors

Model size

0.9B params

Tensor type

U32

BF16

MLX

Hardware compatibility

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Maxkuo/MOSS-Transcribe-Diarize-mlx-q8

Base model

OpenMOSS-Team/MOSS-Transcribe-Diarize

Quantized

(14)

this model