LFM2-2.6B — LiteRT-LM

LiquidAI/LFM2-2.6B converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime. Tested on litert-lm 0.18.0 on a Mac; the phone rows ran the prebuilt Android binary named below. On litert-lm 0.17.1 the int8 file answered "What is the capital of France?" as if no question had been asked (one CPU run; the int4 file was not tried on 0.17.1).

Liquid AI suggests LFM2 for agentic tasks, data extraction, RAG, creative writing and multi-turn conversations, not for knowledge-intensive tasks or programming, and recommends fine-tuning it on narrow use cases. LFM2-2.6B has 2,569,272,320 parameters in 30 layers (22 convolution + 8 attention); Liquid AI lists English, Arabic, Chinese, French, German, Japanese, Korean and Spanish and states a 32,768-token context (4096 in these files). Per Liquid AI, it is the only LFM2 model that uses dynamic hybrid reasoning, a trace between <think> and </think> for complex or multilingual prompts.

File	Recipe	Size	sha256
LFM2-2.6B_int8.litertlm	int8 dynamic on linears + int8 embedding, convs float	2.72 GB	3dda920ce47e5ef2597c1a43732daeecc5d4b9b91fe29599cb9eaafb874b29e8
LFM2-2.6B_int4.litertlm	int4 blockwise-32 + OCTAV on linears, int8 lm_head + embedding	1.53 GB	01c77fbd925f593014636573b024c162c6b910c4c1372cea010d54955e6942f8

On GSM8K (n=100) and IFEval-lite (n=120) the two files score within two items of the bf16 model (table in Accuracy). The int4 file is 43.8 % smaller. Time for a 1024-token prefill plus 256 decoded tokens at context 2048, from the rates in Performance:

Seconds	Mac M4 Max	Galaxy S26	Pixel 8a
int4 GPU	1.98	10.70	22.11
int8 GPU	2.45	13.47	25.07
int8 CPU	9.10	18.17	50.17
int4 CPU	10.75	23.54	68.48

So int4 is the faster file on every GPU and int8 on every CPU. Not measured on iOS: the int8 file's one weights section is 2,722,843,088 bytes (2.54 GiB), and the LFM2.5-2.6B card reports an int8 file with a 2.87 GB weights section running on an iPhone 17 Pro only when the app carries the extended-virtual-addressing and increased-memory entitlements.

Correctness

8-question check on the Mac M4 Max (the questions with "Answer briefly." appended, --temperature 0, one litert-lm run process each): int8 7/8 on CPU and GPU, int4 8/8 on CPU and 7/8 on GPU, bf16 (transformers) 7/8.

Both files' 12 signatures were fully delegated on the three GPUs measured. On both phones, both files answered "What is the capital of France? Answer in one sentence." on GPU and CPU: "The capital of France is Paris."

Accuracy

GSM8K: test questions 0–99, prompted for step-by-step reasoning and a final #### <number> line. IFEval-lite: the first 120 google/IFEval rows with mechanically checkable instructions, prompt-level strict. Greedy, at most 2048 output tokens; litert-lm on the Mac GPU, a fresh process per item. Scores are comparable only within this table, not with published IFEval scores or Liquid AI's GSM8K 82.41 (a different setup).

Configuration	GSM8K (n=100)	IFEval-lite (n=120)
bf16 reference (transformers, MPS)	88	85
int8, litert-lm	89	85
int4, litert-lm	87	87

Scores exclude the reasoning trace: no GSM8K reply had one, and the one IFEval-lite prompt that traced per row passed on its reply.

Usage

litert-lm run ./LFM2-2.6B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./LFM2-2.6B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm benchmark ./LFM2-2.6B_int8.litertlm -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend cpu

Both run commands, and the int4 file, listed the two dates with no chat marker such as <|im_end|> and no [thought] block. Without --backend, run uses the CPU; without --cache, it writes a compiled-weights cache next to the model (on CPU 2.58 GB for int8, 1.51 GB for int4). For Android, see android-gpu.md.

The bundle's default sampler is temperature 0.1, top-k 50, top-p 1.0; Liquid AI recommends temperature 0.3, min_p 0.15 and repetition_penalty 1.05, and the bundle has no min_p or repetition penalty. run uses the bundle's sampler unless you pass --top-k, --top-p, --temperature or --seed; --top-k 1 gives greedy output.

The chat template is byte-identical to LiteRT-LM's models/lfm2/chat_template.jinja at commit 86a7e6d0, with its stops <|im_end|> and <|endoftext|> and start token <|startoftext|>. A trace between <think> and </think> goes to the thought channel, printed by the CLI between [thought] and [/thought]. Traces appeared for "What is the capital of France?" (int8, CPU and GPU) and 1 of 120 IFEval-lite prompts per row, none for GSM8K questions, checks, phone prompts or the commands above. --thinking-budget 0 and 32 did not suppress the trace on that prompt: on the GPU the int8 output was byte-identical to the run without the flag, and on the CPU each run still printed a [thought] block.

Tool calling is declared (model type lfm2): both files returned a parsed get_weather call (Python API, CPU, greedy) and, with automatic tool calling, used its result. Without tools the template renders the same text as Liquid AI's (four conversations checked); with tools it omits Liquid AI's <|tool_list_start|> / <|tool_list_end|> and <|tool_response_start|> / <|tool_response_end|> markers, wraps the tool result in [ … ], and quotes call arguments with double quotes instead of single quotes.

Engine reuse (int8 file, Python API, CPU, greedy; 3 runs on a shared Engine and 3 on fresh Engines; the two conversations had different system prompts and the same three questions): after an Engine served one conversation, the next conversation's two factual answers ("Paris"; "391" for 17 × 23) were the same in the 3 shared-Engine runs and the 3 fresh-Engine runs; the third answer, a self-description, varied in wording, as it did between the fresh runs. The open issue LiteRT-LM #3165 reports the runtime reusing the recurrent state across conversations that share prompt tokens; the probe's two conversations started with different system prompts. Open a new Engine for each conversation.

Performance

Mac Studio M4 Max, litert-lm 0.18.0: the benchmark command above with --backend cpu or gpu (1024-token prefill, 256-token decode, context 2048), one warmup, then the mean of 3 iterations; GPU is WebGPU on Metal. Init is the first iteration's engine init.

Mac M4 Max	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init (s)
int8 GPU	2466.00	125.63	0.4232	2.6537
int4 GPU	2452.56	163.77	0.4236	3.1914
int8 CPU	399.41	39.18	2.5896	30.2006
int4 CPU	185.00	49.07	5.5556	8.2622

Phones, Android 16: Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno 840, MemTotal 10.9 GiB) and Pixel 8a (Tensor G3, Mali-G715, MemTotal 7.4 GiB), the same workload through the prebuilt litert_lm_advanced_main (sha256 adac974b…, 40,382,152 bytes) published 2026-09-18 in LiteRT-LM's public binary bucket (binaries/latest/android_arm64/litert_lm/), with --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true. Rates are the mean of iterations 2 and 3 (GPU: LITERT_CL). Init cold = the first process after the compiled-program caches were removed, warm = the benchmark process; peak = the largest of the benchmark process's three Peak system ram usage lines.

Galaxy S26	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init cold / warm (s)	Peak (MB)
int8 GPU	1122.5	20.39	0.965	9.05 / 5.28	784.4
int4 GPU	682.3	27.84	1.54	9.99 / 6.45	1030.3
int8 CPU	322.6	17.07	3.26	3.10 / 0.22	4158.3
int4 CPU	83.0	22.85	12.38	5.76 / 4.71	3108.7

The clocks lowered during these runs: the GPU clock ceiling fell from 1300 MHz to 902 MHz (int8) and 826 MHz (int4) inside the benchmark, and in the CPU runs the CPU clock was lowered in 24 of 25 (int8) and 34 of 35 (int4) samples, taken about 2 s apart.

Pixel 8a	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init cold / warm (s)	Peak (MB)
int8 GPU	504.4	11.11	2.12	116.2 / 12.55	1654.3
int4 GPU	171.9	15.85	6.02	24.53 / 12.97	1510.7
int8 CPU	87.3	6.66	11.93	12.24 / 0.73	3647.4
int4 CPU	30.3	7.38	33.94	13.18 / 10.02	2602.4

In the CPU runs the CPU clock was lowered in 40 of 48 (int8) and 63 of 66 (int4) samples, taken about 3 s apart; the sampler reads only Adreno GPU clocks, so no Pixel 8a GPU clock was recorded. In the int8 GPU benchmark process the lowest MemAvailable sample was 419,100 kB. Not run on the NPU.

Conversion notes

• Source and stack: LiquidAI/LFM2-2.6B at revision 36ed799f; litert-torch 0.9.4, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0; packed by the litert-lm 0.18.0 CLI. lfm_work/convert_lfm2_26b.sh in hf-to-litertlm rebuilt both files, all four sections byte-identical (only the packer's uuid and timestamp differ).
• Order: unquantized export (11 prefill signatures, 1024 down to 1, plus decode, over a 4096-position KV cache) → post-hoc int8 and int4 quantization → int4 zero-scale fix (746,432 zero block scales set to their tensor's smallest nonzero scale, block payload zeroed; every scale normal on re-scan) → metadata repack with LiteRT-LM's lfm2 configuration.
• Template: Liquid AI's chat template fails at render on litert-lm 0.18.0 (map has no method named get), so the bundle carries LiteRT-LM's lfm2 template, which reads message content as a list of parts.

License and changes

Distributed under the LFM Open License v1.0 inherited from the base model (LICENSE); its section 5 conditions commercial use on an annual revenue below 10 million US dollars. Changes from the original work: exported to the LiteRT-LM format and quantized to int8 and int4 as described above; chat template and stop tokens taken from LiteRT-LM's models/lfm2 configuration, with max_num_tokens changed from 32768 to 4096. This conversion was made and is published independently of Liquid AI.