Instructions to use kingjones777/LFM2.5-2.6B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kingjones777/LFM2.5-2.6B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kingjones777/LFM2.5-2.6B-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kingjones777/LFM2.5-2.6B-NVFP4")
model = AutoModelForCausalLM.from_pretrained("kingjones777/LFM2.5-2.6B-NVFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kingjones777/LFM2.5-2.6B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kingjones777/LFM2.5-2.6B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/LFM2.5-2.6B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kingjones777/LFM2.5-2.6B-NVFP4

SGLang

How to use kingjones777/LFM2.5-2.6B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kingjones777/LFM2.5-2.6B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/LFM2.5-2.6B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kingjones777/LFM2.5-2.6B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/LFM2.5-2.6B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kingjones777/LFM2.5-2.6B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/kingjones777/LFM2.5-2.6B-NVFP4
```

LFM2.5-2.6B-NVFP4

NVFP4 (W4A4) quantisation of LiquidAI/LFM2.5-2.6B, built and verified on NVIDIA GB10 Blackwell (sm_121a).

✅ The first NVFP4 build of any LFM2.5 checkpoint. Checked 2026-08-22: the only other FP4-family LFM2.5 build is mlx-community/LFM2.5-2.6B-mxfp4, which is MXFP4 for Apple MLX — a different format and a different runtime. Repository-content comparison only.

This is real W4A4, not weight-only

A weight-only NVFP4 export will not serve: SGLang's _is_fp4a4_nvfp4 returns False when input_quant is None, and there is no weight-only float4 scheme in the stack. Real W4A4 requires a calibrated input_global_scale per quantised Linear. This export has them:

tensor suffix	count
`weight_packed`	122
`weight_scale`	122
`weight_global_scale`	122
`input_global_scale`	122

format: nvfp4-pack-quantized      quant_method: compressed-tensors
weights:            4-bit float, tensor_group, group_size 16, static
input_activations:  4-bit float, tensor_group, group_size 16, dynamic="local"

The file


architecture	`Lfm2ForCausalLM`
layers	30 (22 conv + 8 full-attention)
size	2,442,951,412 bytes (2.28 GiB) (from 5.1 GB bf16 — 2.2×)
quantised Linears	122

Ignore list — the LFM2 hybrid conv path is protected. Quantised: attention projections and SwiGLU MLPs. Left at bf16: all conv blocks (conv.in_proj / conv.conv / conv.out_proj, including 3-D kernels), every norm, and the tied embeddings. Quantising recurrent/conv state is the classic way to destroy this family.

Measured throughput

	decode
this NVFP4 build	69.8 t/s
bf16 control, same box + method	33.94 t/s

2.06× bf16, measured — not estimated.

Sample output

At the manufacturer-recommended sampling from Liquid's own generation_config.json (temperature 0.1, top_k 50, repetition_penalty 1.1). See the greedy note below.

the state of a quantum system. The wave function is not directly observable, but its probability density can be measured.

Let me provide an example with a simple harmonic oscillator: the time-dependent Schrödinger equation for a 1D harmonic oscillator…

⚠️ Use the recommended sampling, not bare greedy

At temperature=0 this base model falls into repetition loops. That is not a quantisation artifact: a bf16 control run on the identical prompt loops the same way, emitting meta-tokens like "(Repeat this sentence twice)". Liquid's own generation_config.json ships temperature 0.1, top_k 50, repetition_penalty 1.1, do_sample true — plain greedy is outside the recommended configuration. At those settings all five test prompts were coherent.

Requirements

Needs a runtime with compressed-tensors NVFP4 W4A4 support on Blackwell — SGLang v0.5.18-cu130 was used here; vLLM nightly also registers Lfm2ForCausalLM.

python3 -m sglang.launch_server --model-path <this-repo> \
  --host 0.0.0.0 --port 30000 --tp-size 1 --mem-fraction-static 0.45

Provenance

llmcompressor oneshot with QuantizationModifier(targets="Linear", scheme="NVFP4") — not model_free_ptq, which cannot calibrate activations and can only emit the unservable weight-only form. Calibration: HuggingFaceH4/ultrachat_200k (train_sft) via the model's own chat template. Source revision a334ee78cd38458bb71eda24109ac42dcec1309d. Licence inherited from the base model.

Downloads last month: 141

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for kingjones777/LFM2.5-2.6B-NVFP4

Base model

LiquidAI/LFM2.5-2.6B-Base

Finetuned

LiquidAI/LFM2.5-2.6B

Quantized

(67)

this model