DJLougen/LFM2.5-VL-3B-DSpark-MLX-4bit

LFM2.5-VL-3B-DSpark MLX drafter (4-bit)

What

An MLX-native conversion of Liquid AI's four-layer LFM2.5-VL-3B-DSpark DFlash drafter. Draft transformer linears (four draft layers and the fc projection) use affine 4-bit quantization with group size 64; the Markov and confidence heads and normalization weights remain BF16. This is a speculative-decoding drafter, not a standalone model. Model file: model.safetensors (251,454,064 bytes), SHA-256 431fb254296f4da77598376906e219fa105cbe2bafcf111773658c38718f6687.

Upstream source: LiquidAI/LFM2.5-VL-3B-DSpark, pinned at revision af77e9306a26e8625fde74d2a3051ab6d21bd955. Conversion and benchmark code: DJLougen/lfm25-vl-3b.

Why

The original DSpark checkpoint uses an SGLang-oriented layout. This conversion provides MLX-native tensors and config metadata for the MLX-VLM DFlash runtime while reducing the draft checkpoint size.

Why it matters

DFlash uses the drafter to propose tokens and the target to verify them. At greedy temperature zero, the target's verified output is authoritative. Quantizing only draft linears trades drafter size and proposal quality; the speed effect depends on the target precision and workload.

Results

Single-machine measurement on an Apple M3 Max with 36 GB unified memory, Python 3.12, mlx 0.32.2, mlx-vlm 0.7.3, and transformers 5.17.0. Short-prompt screen: three instructions against the same image, greedy generation, two repetitions per prompt/configuration. The run with the lowest end-to-end wall time is retained per prompt/configuration, then throughput and ratios are averaged across prompts; best-of-two selection may be optimistic. results/sweep.json contains the selected rows only, not both repetitions.

Target precision	Draft block	Target-only decode (mean)	With this drafter (mean)	Decode speed ratio	End-to-end ratio	Accepted tokens/round
BF16	8	43.6 tok/s	121.7 tok/s	2.79x	2.31x	4.09
8-bit	4	74.9 tok/s	79.8 tok/s	1.08x	1.06x	2.92

The selected draft output for each prompt matched its target-only text. These local three-prompt results are not a vendor benchmark or a guaranteed gain. With the 8-bit target, the measured end-to-end difference was small; target-only may be preferable. Benchmark your own prompts before enabling speculation.

How to use

Install compatible Apple-Silicon MLX and MLX-VLM builds, then download this drafter and the target checkpoint. For the requested 8-bit target, the best tested setting was block 4:

python -m mlx_vlm.generate \
  --model LiquidAI/LFM2.5-VL-3B-MLX-8bit \
  --draft-model DJLougen/LFM2.5-VL-3B-DSpark-MLX-4bit \
  --draft-block-size 4 --temperature 0 \
  --image image.jpg --prompt "Describe this image." --max-tokens 128

The drafter requires mlx-vlm >= 0.7.2; the results above used mlx-vlm==0.7.3. DFlash speculation in that runtime is greedy-only; set temperature to zero. To compare against target-only, remove the --draft-model and --draft-block-size arguments.

Known issues

• This is an MLX conversion for the LFM2.5-VL-3B target family, not a standalone or general-purpose Qwen3 model.
• The 8-bit-target speed difference was small and measured on only three short prompts; no robust gain is established.
• The tested 4-bit draft can be slower than the 8-bit draft or target-only at different block sizes.
• The long-context sweep used the BF16 drafter, not this 4-bit drafter, and showed speculation slower than target-only at 32k and 64k book-context tokens. Long-context performance for this 4-bit draft is unestablished.
• No SGLang runtime or upstream benchmark result is claimed.

License

The draft weights and upstream-derived model artifacts are distributed under Liquid AI's LFM Open License v1.0, following the upstream DSpark repository. The linked source-code repository has a separate Apache-2.0 code license; it does not apply to these model weights.