LFM2.5-1.2B-Thinking — LiteRT-LM

LiquidAI/LFM2.5-1.2B-Thinking converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime (requires litert-lm ≥ 0.14 / a recent AI Edge Gallery). Sibling of litert-community/LFM2.5-1.2B-Instruct.

LFM2.5-1.2B-Thinking is the reasoning variant of Liquid AI's hybrid conv-attention flagship: it works problems inside <think>…</think> before answering. The bundle declares the thought channel in its metadata, so LiteRT-LM ≥ 0.14 streams the reasoning on a separate thought channel — your app can show or hide it natively, and past thinking is stripped from multi-turn context automatically.

File	Recipe	Size	GSM8K (n=100)
LFM2.5-1.2B-Thinking_int8.litertlm	int8 dynamic (linears + embedding; convs float)	1.24 GB	77% (bf16 reference: 81%)
LFM2.5-1.2B-Thinking_int4.litertlm	int4 blockwise-32 + OCTAV linears, int8 embedding, convs float	736 MB	72%

Context (KV cache)	4096 max
Backend	CPU (the hybrid conv graph is not supported by current GPU delegates)
Template	bundled — full chat template + thought channel (<think>/</think>)
Base model	LiquidAI/LFM2.5-1.2B-Thinking (LFM Open License v1.0)

Quality

GSM8K (greedy, 0-shot CoT, max-tokens 2048 — a thinking model needs the budget, n=100, same harness for all rows): PyTorch bf16 81% · LiteRT int8 77% (−4pt) · int4 72% (−9pt). Both files pass an 8-question sanity gate (7/8, zero degenerate); the reasoning stream arrives on the thought channel and the final answer follows cleanly after </think>.

Usage

litert-lm run ./LFM2.5-1.2B-Thinking_int8.litertlm --prompt "A train travels 60 km in 45 minutes. What is its average speed in km/h?"

Give it a generous token budget (≥2048) — a reasoning model truncated mid-thought produces no final answer. The reasoning arrives on the thought channel; the final answer arrives on the main text channel after </think>.

Run on Android

Install a recent Google AI Edge Gallery, import this repo (or adb push a file and use local import: menu → Models → “+” → From local model file), select the CPU backend, set max tokens high (2048–4096), and chat.

Speed

Same architecture as the Instruct sibling: on a Mac M4 Max CPU the int8 file decodes at ~100 tok/s; on a Pixel 8a int8 decodes at ~19 tok/s and int4 at ~31 tok/s (int4 is ~1.7× faster on phone-class memory bandwidth as well as 41% smaller). First device load compiles the graph and can take about a minute; later loads are instant.

Conversion notes

Converted with released litert-torch 0.9.1 with the same exporter fix as the Instruct sibling: the stock LFM2 short-conv block saves its conv state from padded prefill columns, corrupting the first generated token of nearly every reply; the fix derives the chunk's valid length from the attention mask in-graph and gathers the state from the last valid columns (verified token-identical to an exact per-token reference loop). Multi-length prefill signatures (1–1024). Quantization: export-time int8 including convs, or post-hoc int4 on linears only — post-hoc conv quantization breaks generation.

License and changes

Distributed under the LFM Open License v1.0 (see LICENSE, inherited from the base model). Note the license's commercial-use limitation for organizations above US$10M annual revenue. Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged unmodified; exporter conv-state fix as described in Conversion notes. This repository is a community conversion and is not affiliated with Liquid AI.