LFM2.5-1.2B-JP — LiteRT-LM

LiquidAI/LFM2.5-1.2B-JP converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime (requires litert-lm ≥ 0.14 / a recent AI Edge Gallery). Sibling of litert-community/LFM2.5-1.2B-Instruct.

LFM2.5-1.2B-JP is the Japanese-optimized chat variant of Liquid AI's hybrid conv-attention flagship — a 1.2B model tuned for natural Japanese conversation that runs fully on-device at ~100 tok/s on a laptop CPU.

日本語に最適化された 1.2B オンデバイスモデルです。 スマホの CPU だけで動作し、ネット接続なしで自然な日本語チャットができます。AI Edge Gallery にインポートするだけで試せます。

File	Recipe	Size	GSM8K (n=100)
LFM2.5-1.2B-JP_int8.litertlm	int8 dynamic (linears + embedding; convs float)	1.24 GB	65% (bf16 reference: 63%)
LFM2.5-1.2B-JP_int4.litertlm	int4 blockwise-32 + OCTAV linears, int8 embedding, convs float	736 MB	55%

Context (KV cache)	4096 max
Backend	CPU (the hybrid conv graph is not supported by current GPU delegates)
Template	bundled — full chat template (tool-calling supported)
Base model	LiquidAI/LFM2.5-1.2B-JP (LFM Open License v1.0)

Quality

GSM8K (English, greedy, 0-shot CoT, max-tokens 1024, n=100, same harness for all rows): PyTorch bf16 63% · LiteRT int8 65% (full parity, +2pt within noise) · int4 55%. English math undersells a Japanese-optimized tune — it is reported for quantization-fidelity transparency, not as this model's headline; both files pass an 8-question sanity gate (7/8, zero degenerate) and Japanese conversation quality was verified by inspection. Note the int8 file quantizes linears only (convs stay float): quantizing this tune's convs costs ~9pt, unlike the Instruct sibling where conv-int8 is free.

Usage

litert-lm run ./LFM2.5-1.2B-JP_int8.litertlm --prompt "海について一文で書いてください。"

Run on Android

Install a recent Google AI Edge Gallery, import this repo (or adb push a file and use local import: menu → Models → “+” → From local model file), select the CPU backend, and chat — in Japanese.

Speed

Same architecture as the Instruct sibling: on a Mac M4 Max CPU the int8 file decodes at ~100 tok/s; on a Pixel 8a int8 decodes at ~19 tok/s and int4 at ~31 tok/s (int4 is ~1.7× faster on phone-class memory bandwidth as well as 41% smaller). First device load compiles the graph and can take about a minute; later loads are instant.

Conversion notes

Converted with released litert-torch 0.9.1 with the same exporter fix as the Instruct sibling: the stock LFM2 short-conv block saves its conv state from padded prefill columns, corrupting the first generated token of nearly every reply; the fix derives the chunk's valid length from the attention mask in-graph and gathers the state from the last valid columns (verified token-identical to an exact per-token reference loop). Multi-length prefill signatures (1–1024). Quantization: export-time int8 including convs, or post-hoc int4 on linears only — post-hoc conv quantization breaks generation.

License and changes

Distributed under the LFM Open License v1.0 (see LICENSE, inherited from the base model). Note the license's commercial-use limitation for organizations above US$10M annual revenue. Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged unmodified; exporter conv-state fix as described in Conversion notes. This repository is a community conversion and is not affiliated with Liquid AI.