sahilchachra/LFM2.5-VL-3B-MXFP4

LFM2.5-VL-3B — MLX MXFP4

MLX MXFP4 (4-bit microscaling float) quantization of LiquidAI/LFM2.5-VL-3B, Liquid AI's efficient vision-language model: a hybrid convolution/attention LFM2.5 text backbone (≈2.6B) paired with a SigLIP2 NaFlex vision encoder (≈400M). Runs on Apple Silicon via mlx-vlm. Stays image-text-to-text — the vision tower and multimodal projector are kept in bf16, so only the text backbone is quantized.

Precision	MXFP4 (E2M1 + E8M0 shared scale, group size 32)
Bits per weight	5.855 bpw
On-disk size	2.3 GB (2 shards)
Quantized	LFM2.5 text backbone (incl. tied embeddings)
Kept in bf16	SigLIP2 vision tower + multimodal projector + short-conv kernels
Runtime footprint	≈2.2 GB weights + KV cache (tested on a 24 GB M-series Mac)

This is the smallest build — ideal for on-device / edge use. For maximum fidelity, the MXFP8 build (3.6 GB) is also available.

Verification

Quantized with mlx_lm.quantize_model (mode mxfp4, group 32), keeping the vision path in bf16. LFM2.5-VL is an efficient instruct model (not a heavy reasoner), so the reference is the model's own bf16 behaviour, generated on the same machine with deterministic greedy decoding.

Text — vs bf16 reference (greedy):

Prompt	bf16	MXFP4
17 + 28 (number only)	45	45	✅
256 / 4 (number only)	64	64	✅
capital of France	Paris	Paris	✅
largest planet	Jupiter	Jupiter	✅
three primary colors	Red, Blue, Yellow	Red, Blue, Yellow	✅
describe a sunrise (open-ended)	"…horizon in shades of gold and pink."	"…sky in hues of gold and pink."	≈

5/6 exact match to bf16 on the deterministic prompts; the only divergence is the open-ended creative sentence, which stays coherent and on-theme (expected for free-form generation). Mean character-level similarity to bf16: 0.976.

Vision — image discrimination (bf16 vision path, through the quantized backbone):

Image	Question	Answer
solid red	main color?	Red	✅
solid blue	main color?	Blue	✅
green circle	shape and color?	"a green circle"	✅

Colour and shape are read correctly — the vision path is live, not ignored. A larger 1024×640 two-region image (left red / right green, exercising the multi-tile + thumbnail split) is also read correctly: "the left half is red, and the right half is green."

Usage (mlx-vlm)

pip install -U mlx-vlm   # needs the lfm2_vl architecture (>= 0.6.12)

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("sahilchachra/LFM2.5-VL-3B-MXFP4")
config = model.config

# text
prompt = apply_chat_template(processor, config, "What is the capital of France?")
print(generate(model, processor, prompt, max_tokens=64, verbose=True))

# image
prompt = apply_chat_template(processor, config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image="photo.jpg", max_tokens=128, verbose=True))

Recommended sampling (from the base model card): temperature=0.2, top_k=50, repetition_penalty=1.0.

Run in LM Studio

Loads and runs in LM Studio (tested on 0.4.20, mlx-llm runtime) with both text and image working — the lfm2_vl architecture is recognized, the model indexes cleanly (format: mlx metadata is present), and the ChatML template runs as-is. Verified: text prompts return correct answers, and image prompts sent via the OpenAI-compatible API are read correctly (solid-red → "Red", solid-blue → "Blue"). It indexes and runs as shipped from this repo — no additional changes needed on top of what the repo already contains.

Notes & limitations

• Compatibility fix baked into this repo. On current mlx-vlm 0.6.12 the lfm2_vl config defaults are set for the earlier (smaller) LFM2-VL sizes, so the base LFM2.5-VL-3B config cannot load as-is: block_ff_dim falls back to 6656 and the model fails with a feed-forward shape error, and image_token_index falls back to 396 so every image request raises "Image features and image tokens do not match". This repo's config.json adds both fields explicitly (block_ff_dim=10752, image_token_index=124907) — redundant, correct aliases of the base config's intermediate_size / image_token_id that change nothing about the weights. If you hit those errors loading the base repo, this is the fixed one.
• tie_word_embeddings=true: the token embedding is shared with the output projection and is quantized at 4-bit. Verified correct above; for the most numeric-precision-sensitive workloads the MXFP8 build has more headroom.
• Inherits all capabilities and limitations of the base model. See the original model card.
• Quantized by @sahilchachra with MLX. Original model © Liquid AI, released under the LFM1.0 license (see LICENSE).