Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit

MLX 4-bit quantisation of mistralai/Mistral-Small-3.1-24B-Instruct-2503, converted for use on Apple Silicon via mlx-vlm.

Source model

• Repository: mistralai/Mistral-Small-3.1-24B-Instruct-2503
• Release: 2025-03
• Family: mistral
• Origin: eu
• Languages / coverage: 24+ languages incl. all major EU languages; vision-language model (image + text input)
• License: apache-2.0 (inherited)

Notes from upstream

Mistral AI (France). Multimodal: mistral3 architecture with a Pixtral vision encoder. Converted with mlx-vlm (not mlx-lm) so the language model, vision tower and multi_modal_projector are quantised consistently with current mlx-vlm loaders. Older community 4-bit uploads that predate the affine quantisation mode fail to load under mlx-vlm 0.5.0.

Conversion details

• Tool: mlx-vlm 0.5.0
• Quantisation: 4-bit (defaults from mlx_vlm.convert)
• Converted on: 2026-05-22

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("luiscalisto/Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit")
config = load_config("luiscalisto/Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit")

image = ["path/to/image.jpg"]
prompt = apply_chat_template(processor, config, "Describe this image.", num_images=len(image))
print(generate(model, processor, prompt, image, max_tokens=128, verbose=False))

License and attribution

This is a quantised redistribution of mistralai/Mistral-Small-3.1-24B-Instruct-2503. The original model and its license terms (apache-2.0) carry through unchanged. Please cite the upstream authors when using this model. See the source repository for the authoritative model card and citation.

Conversion provenance

Produced by llm-mlx-conversions, a small utility for publishing community MLX 4-bit quants of open-weight LLMs.