Instructions to use LiquidAI/LFM2.5-1.2B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiquidAI/LFM2.5-1.2B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM2.5-1.2B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-1.2B-Instruct")
model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-1.2B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LiquidAI/LFM2.5-1.2B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiquidAI/LFM2.5-1.2B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-1.2B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiquidAI/LFM2.5-1.2B-Instruct

SGLang

How to use LiquidAI/LFM2.5-1.2B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LiquidAI/LFM2.5-1.2B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-1.2B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LiquidAI/LFM2.5-1.2B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-1.2B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LiquidAI/LFM2.5-1.2B-Instruct with Docker Model Runner:
```
docker model run hf.co/LiquidAI/LFM2.5-1.2B-Instruct
```

Fix chat template: render assistant tool_calls history

#19

by Cloudbusting - opened 5 days ago

base: refs/heads/main

←

from: refs/pr/19

Discussion Files changed

+31

-3

Cloudbusting

5 days ago

Thanks for LFM2.5. The 1.2B is a remarkably capable tool caller for its size, and we have enjoyed working with it.

Defect: the shipped message loop renders content but never renders a past assistant turn's tool_calls. Standard OpenAI-compatible clients replay history as an assistant tool_calls array followed by role: tool results. The current template therefore emits an empty assistant turn, and the model cannot see the actions it already took.

Measured effect: in seeded 4-step and 6-step runs with the official Q4_K_M GGUF, temperature 0.1, top_k 50, and llama.cpp, the shipped template scored ordered 0/6 and exact 0/6. With history rendered, the fixed 4-step runs scored ordered 3/3 and exact 0/3 because the model inserted one unrequested note call each run. The fixed 6-step runs restored coherent history but still fabricated the final read, so they scored ordered 0/3 and exact 0/3. This patch restores memory. It does not claim to fix the model's extra call or fabricated final.

Fix: port the format_arg_value and tool-call rendering macros already shipped by LFM2.5-8B-A1B into this template's message loop. A past tool-call turn then renders in the model family's native form:

<|im_start|>assistant
<|tool_call_start|>[get_weather(city='Paris')]<|tool_call_end|><|im_end|>

The change also tolerates content: null or missing content on pure tool-call turns. Everything outside the tool-call path stays byte-identical. The included regression test covers the defect, fixed rendering, tool-free byte equality, multimodal content, null content, hostile argument values, and malformed call shapes against the pinned 8B-family behavior.

The live route doctor's verdict changes from a template-history failure to healthy with the fix alone. The strict exact suite can still fail on this small model's own extra call, and the evidence reports that rather than hiding it.

GGUF users receive the fix through a re-converted GGUF or a runtime override such as llama-server --jinja --chat-template-file <this file>. We used the runtime override for the live proof.

Happy to adjust formatting, macro placement, or null-content handling to match your conventions.

Evidence, per-run ledgers, wire excerpts, regression tests, and reproductions:
https://github.com/graphometer/droplet/tree/main/public/evidence

Disclosure: prepared with heavy AI assistance; every claim above comes from recorded runs that can be inspected. Not affiliated with, endorsed by, or connected to Liquid AI.

Fix chat template: render assistant tool_calls history64951137

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Cannot merge

This branch has merge conflicts in the following files:

chat_template.jinja

· Sign up or log in to comment