LFM2.5-VL-3B-ffn8192

A surgically compressed derivative of LiquidAI/LFM2.5-VL-3B: FFN width pruned 10752 → 8192 (−24%) + LoRA distillation recovery, BF16.

	Size	vs original
Original BF16	6.25 GB	—
This checkpoint	5.30 GB	−15%

What was done (3-step pipeline, all reproducible)

1. Wanda channel pruning: FFN intermediate 10752 → 8192 (uniform across all 30 layers), joint swiglu importance (w1/w3 rows + w2 cols, one shared mask) from multimodal calibration (wikitext-2 + COCO), equal text/image weight.
2. Distillation recovery: r32 LoRA on all FFN linears trained to match the base model's distribution (KL over the base's top-64 logits + small NLL anchor) on 2048 rows disjoint from evaluation data (two low-LR passes).
3. Evaluation: our statistical integrity harness (paired NLL, truncated teacher KL, argmax agreements, generation chrF) vs the base.

Quality vs base (quick suite, our harness)

Axis	Value
text PPL ratio (e^ΔNLL)	0.83 (≤1.05 gate: PASS)
mm PPL ratio	0.80 (≤1.10 gate: PASS)
text kld_t64	0.161 (base = 0)
mm kld_t64	0.173
generation chrF (text/mm)	0.50 / 0.54
integrity index (record-only)	42.7 / 100

Semantics preserved; phrasing compresses (verified live in vLLM: same answers on image and text prompts, tool-calling intact — see the companion repos and the pipeline repo for the full reports).

Load

from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained(
    "konic-labs/LFM2.5-VL-3B-ffn8192", dtype="bfloat16")

License & provenance

• Derivative of LiquidAI/LFM2.5-VL-3B (LFM Open License v1.0 — license text retained; modifications per this card).
• Pipeline and full reports: https://github.com/konic-labs/lfm25-vl-3b-lab
• Siblings: LFM2.5-VL-3B-ffn7168, LFM2.5-VL-3B-ffn8192-int4.