barrontang/SmolLM2-135M-mlx-fp16

SmolLM2-135M MLX FP16

MLX FP16 conversion of the base pretrained, not instruction-tuned, SmolLM2-135M model. Published by barrontang; this is not an official mlx-community release.

Conversion

The local original HF safetensors were exported to an FP16 GGUF with llama.cpp's HF converter, passing the original config explicitly to preserve the RoPE base of 100000. The original tokenizer JSON was embedded in the GGUF to retain Unicode normalization. The GGUF was then converted with gguf2mlx:

python -m gguf2mlx --input SmolLM2-135M-f16.gguf \
  --output SmolLM2-135M-mlx-fp16 --dtype float16

Conversion uses local fixes for tied Llama embeddings and inverse GGUF Q/K rotary projection ordering. No additional low-bit quantization was applied. FP16 conversion does not preserve the entire BF16 numeric range.

Usage

from mlx_lm import generate, load
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("barrontang/SmolLM2-135M-mlx-fp16")
print(generate(model, tokenizer, prompt="Gravity is",
               max_tokens=64, sampler=make_sampler(0)))

Requires Apple Silicon and MLX-LM. Use completion prompts: this base model is not a chat assistant.

Validation

Tested locally on Apple Silicon on 2026-10-19:

• All 272 parameter tensors exactly match the original local model cast to FP16.
• Token IDs and all finite logits exactly match for six English, code, whitespace, and Unicode test prompts.
• Two deterministic 32-token generations exactly match the FP16 reference.
• 116 focused converter, architecture, and native integration tests passed.

See validation_results.json for raw prompts, generation outputs, and the source weight SHA-256. The reference is the original HF model loaded with MLX-LM and cast to FP16, not an independent Transformers inference engine. These are conversion parity/smoke tests, not a broad quality benchmark. The small base model can repeat text or produce incorrect statements.

Attribution and license

Original model: HuggingFaceTB/SmolLM2-135M, Apache-2.0. See the original model card for training data, evaluation, and limitations.