bouroo/flint-mlx-mxfp4

Flint 40B-A2.5B, MXFP4 for MLX

MXFP4 quantization of RecursiveSI/flint for Apple Silicon via MLX.

Base	RecursiveSI/flint (BF16, 79.6 GB)
Format	MLX mxfp4, group size 32, 4 bit
Size	see the file list (about 20 GB)
Kept BF16	mlp.gate routers, embed_tokens, lm_head, all norms, softmax_offset
Experts	256 routed, unpruned, stacked into switch_mlp.*
Architecture	39.8B total / ~2.5B active, 49 layers (2 dense + 47 MoE), sliding window 513

Sampling

Mirror the upstream card: temperature 1.0, top_p 0.95, top_k 20. generation_config.json already carries these.

Reasoning tokens

Reasoning counts toward max_tokens. A coding prompt that thinks for 900 tokens needs max_tokens above that or the reply returns with finish_reason: "length" and empty content.

Template note

chat_template.jinja here is the upstream template with one change: the replace("\x00", "") filter is written with a literal NUL byte instead of the \x00 escape. Engines that reject backslash escapes in templates (LM Studio / MLX, error Unexpected escaped character: x) cannot render the upstream file at all. Output is identical on engines that accept both forms.

Known limitation

The tokenizer has no Thai (or other non-Latin script) vocabulary: 0 of 150144 vocab entries contain a codepoint in U+0E00..U+0E7F. Thai input byte-fragments into mojibake and the model degenerates. This is inherited from the base checkpoint, not introduced by the quantization.

Conversion

Expert weights were stacked per layer from the sharded BF16 checkpoint and quantized one layer at a time; nothing was fine-tuned, and no calibration data was used.