Instructions to use BennyDaBall/LFM3-1.7B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use BennyDaBall/LFM3-1.7B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="BennyDaBall/LFM3-1.7B-NVFP4", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("BennyDaBall/LFM3-1.7B-NVFP4", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use BennyDaBall/LFM3-1.7B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "BennyDaBall/LFM3-1.7B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BennyDaBall/LFM3-1.7B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/BennyDaBall/LFM3-1.7B-NVFP4

SGLang

How to use BennyDaBall/LFM3-1.7B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "BennyDaBall/LFM3-1.7B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BennyDaBall/LFM3-1.7B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "BennyDaBall/LFM3-1.7B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BennyDaBall/LFM3-1.7B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use BennyDaBall/LFM3-1.7B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/BennyDaBall/LFM3-1.7B-NVFP4
```

LFM3-1.7B NVFP4 (Schur Replay)

Modified weights notice: this repo contains a quantized (NVFP4) derivative of LiquidAI/LFM3-1.7B. The feed-forward weights were re-quantized by BennyDaBall; everything else is Liquid AI's original BF16. Not affiliated with or endorsed by Liquid AI.

Same layer map, same file format and same speed as Liquid's official LFM3-1.7B-NVFP4. Better 4-bit weights: 1.36x closer to BF16 under W4A4, 1.87x on the weights alone. For llama.cpp / LM Studio, see the GGUF twin: BennyDaBall/LFM3-1.7B-NVFP4-GGUF (1.28 GB, 2.03x closer to BF16 than a converted Liquid NVFP4, runs on stock llama.cpp).

What changed vs Liquid's NVFP4

	Liquid NVFP4	This repo
Layer map	NVFP4 W4A4 on language feed-forward weights + activations; attention, convs, embeddings, head, norms BF16	Identical (Liquid's map, cited from their card)
Weight solver	Round-to-nearest (ModelOpt default; we checked: weight error identical to plain RTN)	GPTQ + Schur Replay block-scale selection + Four Over Six
Activations	Static per-tensor input_scale, dynamic per-16 block scales	Same scheme, recalibrated on our set
Format	ModelOpt NVFP4 safetensors	Same (drop-in)

Schur Replay (arXiv 2609.36654): inside the GPTQ loop, each 16-weight block tries a window of E4M3 scale codes plus the Four Over Six map-to-4 scale (arXiv 2512.02010). Every candidate replays the block's GPTQ updates and is scored by its real GPTQ loss (the Schur complement of the not-yet-quantized columns). Each row keeps its best. Hessians come from 599k calibration tokens (tool calls, chat, reasoning, raw text). The chat, tool-call and reasoning answers are the BF16 model's own generations.

KLD vs BF16 (lower is better)

Gold = LiquidAI/LFM3-1.7B BF16 (transformers). Held-out eval: 150 sequences, 124,790 scored tokens (every model-written span: tool calls, chat, reasoning, plus raw text). NVFP4 math reproduced exactly as the W4A4 kernels compute it.

W4A4 (vLLM / SGLang path)	Mean KLD	Median	p99	Top-1 agree	Tool	Chat	Reasoning	Raw
Liquid NVFP4	0.0628	0.0187	0.584	90.3%	0.0353	0.0707	0.0378	0.0870
This repo	0.0461	0.0133	0.441	91.6%	0.0233	0.0523	0.0262	0.0663

Weight-only view (W4A16)	Mean KLD	Top-1 agree
Liquid NVFP4	0.0332	92.9%
This repo	0.0178	94.8%

Speed and memory (one RTX 5090, same sitting)

Same kernels (CUTLASS SM120 block-scaled FP4 GEMMs, 48 per step), same 1.86 GiB of weights. Single stream, 512 tokens, vLLM 0.32.0 + LFM3 PR overlay, KV pool 1 GiB:

Checkpoint	Decode tok/s	Prefill tok/s (2.3k prompt)	Weights	Peak VRAM
BF16	290	33,555	3.36 GiB	5,424 MiB
Liquid NVFP4	379	54,685	1.86 GiB	3,945 MiB
This repo	402	53,368	1.86 GiB	4,001 MiB

SGLang (PR 46858): BF16 345, Liquid NVFP4 513, this repo 457 tok/s decode. The ours-vs-Liquid gap flips sign between runtimes (FlashInfer/CUTLASS autotuner tile picks); summed GPU kernel time differs by ~2%, so treat the two NVFP4 files as the same speed.

Usage

Recommended sampling (Liquid's card): temperature 0.1, top_k 50, repetition_penalty 1.1 (math/STEM: temperature 1.0, top_p 0.95, top_k 20). Thinking is on by default; tool calls use Liquid's XML format.

LFM3 is not in a vLLM or SGLang release yet (Nov 1, 2026). It works today with vLLM v0.32.0 + vllm-project/vllm PR 64959, or SGLang nightly + sgl-project/sglang PR 46858, exactly like Liquid's NVFP4:

vllm serve BennyDaBall/LFM3-1.7B-NVFP4 --trust-remote-code
python -m sglang.launch_server --model-path BennyDaBall/LFM3-1.7B-NVFP4 --reasoning-parser auto --tool-call-parser auto

License

LFM Open License v1.0 (copy in LICENSE, unchanged). Key terms, quoted from it:

"You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications" (Section 4)

"The rights granted under this License for Commercial Use are conditioned upon You or Your Legal Entity not exceeding the Threshold." (Section 5(a)), where the Threshold is "annual revenue of 10 million United States dollars ($10,000,000) or more." (Section 1)

Free for any use, commercial included, while you (and your corporate group) make under $10M a year; above that, commercial use needs a license from Liquid AI.

Credits

Model: Liquid AI, LFM3-1.7B. NVFP4 layer map from Liquid's own LFM3-1.7B-NVFP4.
Methods: GPTQ (Frantar et al. 2022), Four Over Six (Cook et al., arXiv 2512.02010), Schur Replay (arXiv 2609.36654).
Quantization: BennyDaBall.

Downloads last month: -

Safetensors

Model size

1B params

Tensor type

BF16

Model tree for BennyDaBall/LFM3-1.7B-NVFP4

Base model

LiquidAI/LFM3-1.7B-Base

Finetuned

LiquidAI/LFM3-1.7B

Quantized

(86)

this model