AutomatosX/AX-Devstral-Small-2-24B-Instruct-2512-MLX-OptiQ-4bit

AX Devstral Small 2 24B Instruct 2512 MLX OptiQ 4-bit

Parameter count: 24,011,361,280 logical inference parameters (approximately 24.01B). 4-bit names the low precision tier; it is not a 4B model-size claim. The count includes the bundled BF16 vision tower.

The logical count is derived from every tensor shape in the pinned BF16 source and independently reconstructed from the packed MLX artifact. It intentionally does not treat packed storage elements—or an inconsistent upstream index override—as the architecture size.

This is an MLX OptiQ mixed-precision release of a multimodal agentic coding and software-engineering model for Apple Silicon. It is intended for Apple Silicon and retains image-and-text support through an indexed BF16 vision sidecar.

Try this model locally with AX Engine.

Attribution and changes

AutomatosX did not train the base model. AutomatosX produced this OptiQ conversion from mlx-community/Devstral-Small-2-24B-Instruct-2512-bf16 at immutable revision 5cc5ba993ad5220ec4cbac1ab7126cd80189094a with mlx-optiq 0.4.2.

The conversion used exact calibration-driven per-layer KL sensitivity against the BF16 reference: 24 sequences of 512 tokens from the six-domain OptiQ mix, 4/8-bit candidates, group size 64, and a target of 5.0 bits per weight. The target and achieved BPW values describe OptiQ's measured language allocation, not the complete download. The Mistral tokenizer config also sets fix_mistral_regex: true, following the runtime warning for correct token splitting. No model training or benchmark results are claimed by AutomatosX.

The source model card is preserved as UPSTREAM_README.md; exact conversion inputs, allocation, validation, and hashes are recorded in ax_provenance.json, optiq_metadata.json, and optiq_sensitivity.json.

The source index mapping matched the actual shards; it was rebuilt from the Safetensors headers to add verified logical-count metadata. The verbatim upstream and rebuilt source indexes are preserved as UPSTREAM_MODEL_SAFETENSORS_INDEX.json and REBUILT_SOURCE_MODEL_SAFETENSORS_INDEX.json.

The pinned MLX source config contains max_position_embeddings: 393,216, while Mistral's official model card documents 262,144 tokens. This release preserves the source config value in text_config and keeps generation_config.max_length at 262,144. Treat 262,144 tokens as the documented operational context unless you have validated a longer window yourself.

Package details

Property	Value
Format	MLX Safetensors
Quantization	OptiQ mixed 4/8-bit, group size 64
Measured allocation	112 components at 4-bit; 169 at 8-bit
Allocation target / achieved BPW	5.0 / 5.1053
Sensitivity reference	BF16
Calibration	24 × 512-token, six-domain mix
Allocation guards	2-component block floor; 3-block low-bit run limit
Vision tower	Bundled BF16 sidecar
Source companions	processor_config.json, params.json, tekken.json, CHAT_SYSTEM_PROMPT.txt, VIBE_SYSTEM_PROMPT.txt
Official documented context	262,144 tokens
Source config max_position_embeddings	393,216 tokens
Logical parameters	24,011,361,280 (24.01B)
Original upstream index total_parameters	Not provided
Download size	Approximately 17.7 GB
Intended hardware	Apple Silicon

Measured allocation counts OptiQ's sensitivity-selected language components only. Forced high-bit tensors outside that set (notably language_model.model.embed_tokens at 8-bit) appear in config.json and can make a naive config recount differ by one 8-bit entry.

Achieved BPW is above target because allocation guards (first/last protection, block floor, and low-bit run limit) keep additional language components at 8-bit.

The download size also includes the BF16 vision sidecar, protected or unmeasured high-precision components, quantization scales/biases, Safetensors headers, tokenizer assets (including tekken.json when bundled), processor config, and documentation. It therefore should not be estimated by multiplying the logical parameter count by the achieved BPW.

Recommended system prompts

This package includes upstream agent prompts from the pinned BF16 source:

• CHAT_SYSTEM_PROMPT.txt — general Le Chat-style assistant
• VIBE_SYSTEM_PROMPT.txt — Mistral Vibe coding-agent wrapper

Pass the relevant file contents as the system message for agentic or coding use.

Run with MLX-VLM

pip install -U mlx-vlm

python -m mlx_vlm.generate \
  --model AutomatosX/AX-Devstral-Small-2-24B-Instruct-2512-MLX-OptiQ-4bit \
  --max-tokens 100 \
  --temperature 0 \
  --prompt "Describe this image." \
  --image <path_to_image>

Download and serve with AX Engine

hf download AutomatosX/AX-Devstral-Small-2-24B-Instruct-2512-MLX-OptiQ-4bit \
  --local-dir ./AX-Devstral-Small-2-24B-Instruct-2512-MLX-OptiQ-4bit

ax-engine doctor \
  --mlx-model-artifacts-dir ./AX-Devstral-Small-2-24B-Instruct-2512-MLX-OptiQ-4bit

ax-engine serve ./AX-Devstral-Small-2-24B-Instruct-2512-MLX-OptiQ-4bit --port 31418

Validation

Validated on macOS arm64 with MLX 0.32.0, MLX-LM 0.31.3, MLX-VLM 0.6.6, mlx-optiq 0.4.2, and AX Engine 6.11.0:

• all JSON and Safetensors index mappings: passed
• stock MLX-LM text load/generation: READY
• MLX-VLM image load/generation through the BF16 sidecar: passed
• AX native manifest validation and artifact doctor: ready
• Hub logical parameter override: 24,011,361,280

Quantization can change model quality. Evaluate the model on your own tasks, especially long-context, coding, multilingual, tool-use, and vision workloads.

License

Apache License 2.0. See LICENSE, the mistralai/Devstral-Small-2-24B-Instruct-2512 model card, and UPSTREAM_README.md for limitations and responsible-use guidance.