pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit

Mistral-Small-3.2-24B-OpenThoughts-Distill-v1 (MLX 4bit)

A LoRA fine-tune of Mistral Small 3.2 24B Instruct on the OpenThoughts-114k reasoning dataset, intended to strengthen structured chain-of-thought reasoning patterns. Released in the MLX 4-bit format for Apple Silicon.

GGUF quantizations will be released as a separate companion repository — see the Roadmap below.

Model Details

Base model	mistralai/Mistral-Small-3.2-24B-Instruct-2506, via the mlx-community/...-4bit MLX quantization
Parameters	23.57 B total · 4.62 M trainable LoRA (0.020%)
Format	MLX, 4-bit quantized
Architecture	Mistral3ForConditionalGeneration (vision-capable base; this fine-tune targets the text backbone only)
Languages	multilingual (inherited from base: en, fr, de, es, it, pt, ja, ko, …)
License	Apache 2.0

Training

LoRA fine-tune with mlx-lm on an Apple M4 Pro (64 GB unified memory).

Hyperparameter	Value
Method	LoRA
Trainable layers	Last 4
Iterations	1500
Batch size	1
Gradient accumulation steps	2 (effective batch size 2)
Learning rate	1e-5
Max sequence length	2048 tokens
Gradient checkpointing	Yes
Mask prompt	Yes (only assistant turns contribute to the loss)
Peak memory	~17.7 GB
Hardware	Apple M4 Pro, 64 GB unified memory

Reproducible training command

mlx_lm.lora \
  --model mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit \
  --train --data ./data --iters 1500 \
  --batch-size 1 --grad-accumulation-steps 2 \
  --num-layers 4 --grad-checkpoint --mask-prompt \
  --learning-rate 1e-5 --max-seq-length 2048 \
  --save-every 250 --steps-per-eval 250 \
  --adapter-path ./adapters

After training, the resulting LoRA adapter was merged into the base via mlx_lm.fuse.

Dataset

open-thoughts/OpenThoughts-114k — 113,957 reasoning trace examples, generated by DeepSeek-R1, released under Apache 2.0.

The ShareGPT-style schema (system + conversations[from/value]) was converted to mlx-lm's chat-template-compatible {"messages": […]} JSONL format:

Split	Examples
Train	113,457
Validation	500

Caveat — Sequence Truncation

OpenThoughts contains very long reasoning chains (single examples up to ~23k tokens). To fit the 64 GB Mac memory budget, max_seq_length was set to 2048. As a result, the model is trained on the first 2048 tokens of every example and the long tails of longer reasoning chains were dropped.

Practical implication: the model has been exposed to reasoning structures up to 2048 tokens. When generating responses substantially longer than that on its own, quality may degrade earlier than the chosen max_tokens cap. A v2 with a higher sequence length is on the roadmap.

Usage (MLX)

from mlx_lm import load, generate

model, tokenizer = load(
    "pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit"
)

messages = [
    {"role": "user",
     "content": "Three friends share 24 cookies equally. "
                "One friend gives 3 cookies to each of the others. "
                "How many does each friend have now? "
                "Reason step by step."}
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(
    model, tokenizer, prompt=prompt, max_tokens=1024, verbose=True
)

CLI:

mlx_lm.generate \
  --model pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit \
  --prompt "Your reasoning question here" \
  --max-tokens 1024

Reasoning Style — Observed Differences

This v1 release is honestly framed as a stylistic distillation, not a capability leap. The goal was to transfer R1-style explicit-reasoning patterns onto a strong base. Side-by-side probes on classic reasoning prompts (cookie-share math, bat-and-ball Cognitive Reflection Test, sister-brother counting trap) gave the following picture:

Pattern transfer (intended outcome — present)

The fine-tuned model exhibits R1-signature self-verification language that the base model does not, e.g.:

"Let me rephrase the information to make sure I get it right…" "Wait, let me reconsider…" "Actually, let me check…"

This is the visible signature of the OpenThoughts-114k (R1-generated) reasoning data.

Capability gain (honest note — not demonstrated)

On the probes above, the base Mistral Small 3.2 24B Instruct already produces correct, structured, didactic answers — including a spontaneous "Common Pitfall" warning on the bat-and-ball trap. This v1 fine-tune did not measurably improve accuracy on those particular probes.

Likely contributing factors:

• The base is already strong at structured reasoning out-of-the-box.
• Training was on a 4-bit quantized base (gradients against quantized weights).
• The 2048-token training truncation means the model has been exposed to starts of long reasoning chains but not to their convergence. On long traps (sister-brother), the fine-tune showed more self-questioning loops but did not converge faster than the base.

What this release is useful for

• An MLX-format, Apache-2.0, openly-licensed-chain model that visibly exhibits R1-style explicit reasoning on Apple Silicon.
• A baseline for our own iteration: a v2 trained on a higher-precision base, longer sequence length, and more iterations is on the roadmap and is where a measurable capability shift is expected.

License Chain

Component	License
Base model (Mistral Small 3.2 24B Instruct)	Apache 2.0
4-bit MLX quantization (mlx-community)	Apache 2.0
Training dataset (OpenThoughts-114k)	Apache 2.0
This LoRA adapter + merged model	Apache 2.0

The entire supply chain is Apache 2.0. No proprietary LLM outputs (e.g. OpenAI, Anthropic) were used either in dataset generation or in training.

Roadmap

• GGUF quantizations (at minimum Q4_K_M and Q8_0) released as a sibling …-GGUF repository
• v2 with higher sequence length and more iterations once a fp16/bf16-capable host is available
• Public evaluation on standard reasoning benchmarks (GSM8K, MATH, MMLU-Pro)

Limitations

• Sequence truncation during training at 2048 tokens may cause earlier-than-ideal stopping behaviour on very long self-generated reasoning chains.
• 4-bit quantized base before LoRA: gradients are computed against the quantized weights. Quality is solid for distillation but a fp16/bf16-based v2 would likely be stronger.
• Language coverage: the training dataset is predominantly English; multilingual reasoning quality may vary by language.
• Multimodality: the base architecture is Mistral3 (vision + text). This LoRA is purely text-only training; vision capabilities are inherited unchanged from the base and have not been fine-tuned here.

Attribution & Acknowledgements

• Mistral AI — for releasing an excellent multilingual base model under Apache 2.0.
• OpenThoughts team — for assembling and releasing the high-quality reasoning distillation dataset.
• DeepSeek — for the R1 model whose generations form the reasoning traces inside OpenThoughts-114k.
• mlx-community — for the MLX 4-bit quantization of the base model used as the LoRA host.
• Apple ML Research — for mlx-lm, which made training on a 64 GB Mac feasible.

Citation

@misc{pnoid_mistral_openthoughts_distill_v1_mlx_4bit_2026,
  author       = {pnoid},
  title        = {Mistral-Small-3.2-24B-OpenThoughts-Distill-v1 (MLX 4-bit)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit}}
}

When using this model, please also cite the base model and the dataset (see their respective model/dataset cards for canonical citation formats).

---

Trained June 2026 on an Apple M4 Pro (64 GB unified memory).