Instructions to use kingjones777/LFM2.5-1.2B-Instruct-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kingjones777/LFM2.5-1.2B-Instruct-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kingjones777/LFM2.5-1.2B-Instruct-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kingjones777/LFM2.5-1.2B-Instruct-NVFP4")
model = AutoModelForCausalLM.from_pretrained("kingjones777/LFM2.5-1.2B-Instruct-NVFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kingjones777/LFM2.5-1.2B-Instruct-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kingjones777/LFM2.5-1.2B-Instruct-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/LFM2.5-1.2B-Instruct-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kingjones777/LFM2.5-1.2B-Instruct-NVFP4

SGLang

How to use kingjones777/LFM2.5-1.2B-Instruct-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kingjones777/LFM2.5-1.2B-Instruct-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/LFM2.5-1.2B-Instruct-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kingjones777/LFM2.5-1.2B-Instruct-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kingjones777/LFM2.5-1.2B-Instruct-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kingjones777/LFM2.5-1.2B-Instruct-NVFP4 with Docker Model Runner:
```
docker model run hf.co/kingjones777/LFM2.5-1.2B-Instruct-NVFP4
```

LFM2.5-1.2B-Instruct-NVFP4

NVFP4 (W4A4) quantisation of LiquidAI/LFM2.5-1.2B-Instruct, built and verified on NVIDIA GB10 Blackwell (sm_121a).

✅ The first NVFP4 build of any LFM2.5 checkpoint. Checked 2026-08-22: the only other FP4-family LFM2.5 build is mlx-community/LFM2.5-2.6B-mxfp4, which is MXFP4 for Apple MLX — a different format and a different runtime. Repository-content comparison only.

This is real W4A4, not weight-only

A weight-only NVFP4 export will not serve: SGLang's _is_fp4a4_nvfp4 returns False when input_quant is None, and there is no weight-only float4 scheme in the stack. Real W4A4 requires a calibrated input_global_scale per quantised Linear. This export has them:

tensor suffix	count
`weight_packed`	72
`weight_scale`	72
`weight_global_scale`	72
`input_global_scale`	72

format: nvfp4-pack-quantized      quant_method: compressed-tensors
weights:            4-bit float, tensor_group, group_size 16, static
input_activations:  4-bit float, tensor_group, group_size 16, dynamic="local"

The file


architecture	`Lfm2ForCausalLM`
layers	16 (10 conv + 6 full-attention)
size	1,097,399,803 bytes (1.02 GiB) (from 2.2 GB bf16 — 2.2×)
quantised Linears	72

Ignore list — the LFM2 hybrid conv path is protected. Quantised: attention projections and SwiGLU MLPs. Left at bf16: all conv blocks (conv.in_proj / conv.conv / conv.out_proj, including 3-D kernels), every norm, and the tied embeddings. Quantising recurrent/conv state is the classic way to destroy this family.

Measured throughput

	decode
this NVFP4 build	135.8 t/s

Sample output

/v1/completions, temperature 0, run twice — byte-identical both runs.

mathematical achievements is the Babylonian tablet known as the Yongkash tablet, which dates back to 2200 BC. The ancient Egyptians also made significant contributions to mathematics, particularly in the field of geometry. The Rhind Mathematical Papyrus, dating from around 1550 BC, is another important ancient mathematical text.

Requirements

Needs a runtime with compressed-tensors NVFP4 W4A4 support on Blackwell — SGLang v0.5.18-cu130 was used here; vLLM nightly also registers Lfm2ForCausalLM.

python3 -m sglang.launch_server --model-path <this-repo> \
  --host 0.0.0.0 --port 30000 --tp-size 1 --mem-fraction-static 0.45

Provenance

llmcompressor oneshot with QuantizationModifier(targets="Linear", scheme="NVFP4") — not model_free_ptq, which cannot calibrate activations and can only emit the unservable weight-only form. Calibration: HuggingFaceH4/ultrachat_200k (train_sft) via the model's own chat template. Source revision df58c174f05ff733f83f8cae10ea9298224c8006. Licence inherited from the base model.

Downloads last month: 139

Safetensors

Model size

1B params

Tensor type

BF16

Model tree for kingjones777/LFM2.5-1.2B-Instruct-NVFP4

Base model

LiquidAI/LFM2.5-1.2B-Base

Finetuned

LiquidAI/LFM2.5-1.2B-Instruct

Quantized

(82)

this model