Instructions to use LiquidAI/LFM2.5-8B-A1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiquidAI/LFM2.5-8B-A1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM2.5-8B-A1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-8B-A1B")
model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-8B-A1B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LiquidAI/LFM2.5-8B-A1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiquidAI/LFM2.5-8B-A1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-8B-A1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiquidAI/LFM2.5-8B-A1B

SGLang

How to use LiquidAI/LFM2.5-8B-A1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LiquidAI/LFM2.5-8B-A1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-8B-A1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LiquidAI/LFM2.5-8B-A1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-8B-A1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LiquidAI/LFM2.5-8B-A1B with Docker Model Runner:
```
docker model run hf.co/LiquidAI/LFM2.5-8B-A1B
```

Tool call sometimes emitted as plain JSON in `content` (missing `<|tool_call_start|>` after `</think>`) — llama.cpp + pi agent

#36

by hyge666 - opened Sep 20

Discussion

hyge666

Sep 20

Summary

When using a pi coding-agent (with 4 tools: read/bash/edit/write) via llama.cpp,
the model occasionally finishes the <think> phase and then outputs the tool call as raw JSON text inside content, instead of using the expected <|tool_call_start|>[bash(...)]<|tool_call_end|> format.

In these cases:

llama.cpp’s lazy grammar never triggers,
the response ends with finish_reason: stop,
tool_calls is empty,
the agent loop halts.

Environment

llama.cpp b10964's llama-server
Try with these chat templates. This happens on both template.
1. GGUF included chat template.
2. new template from this PR.
Model: LFM2.5-8B-A1B, GGUF (q8_0, get from here)
Sampling: temperature 0.2, top_k 80, top_p 0.95, repeat_penalty 1.05
Client: pi (VS Code extension), OpenAI-compatible API

Observed behavior

The token immediately after </think> seems to determine the outcome:

turn	token after `</think>`	next token	result
OK (tool call)	`"\n"` (id 207)	`<\|tool_call_start\|>` (124905)	proper `tool_calls` emitted
NG	`"\n\n"` (id 8)	`{`	JSON emitted in `content`, `finish_reason: stop`

For NG cases, the server logs:

Grammar still awaiting trigger after token 8

The final answer also begins with "\n\n", suggesting the model switched into “answer mode” instead of “tool-call mode”.

The JSON varies but resembles terminal-agent formats, e.g.:

{"bash": {"command": "head -50 ./Sources/Program.cs\n", "timeout": 0}}
{"analysis": "...", "plan": "...", "commands": [{"keystrokes": "find . -maxdepth 1 ...\n", "timeout": 0.5}], ...}
{"keystrokes": "cd X && ls -la\n", "duration": 0.1}

Frequency

In my tests, 2 out of 4 tool-call turns failed.
Simple requests ("list files") usually work, but multi-step ones (list *.cs → read → summarize) fail often, typically on the 1st or 2nd tool turn.

What I tried (no effect)

System prompt reminders ("respond with tool calls only", "do not write JSON")

Questions

Is this a known behavior? Is there a recommended system prompt or harness setup for stable reasoning + tool calls in agent loops?
Does the training data cover the pattern of <think> → reasoning → immediate tool call inside multi-step agent loops?

I can attach full verbose logs and the rendered prompts if useful.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment