LFM3-1.7B-CoreML

Core ML conversion of LiquidAI/LFM3-1.7B (stateful KV cache + short-conv state, one model for prefill and decode). Converted from the original weights; not an official Liquid AI release.

File	Weights	Size
LFM3-1.7B-int4-s2048.mlpackage	int4, block size 32, symmetric	0.9 GB
LFM3-1.7B-fp16-s2048.mlpackage	fp16	3.2 GB

Requires iOS 18 / macOS 15 or later (ct.StateType). Context window of this build: 2048 tokens (the original model supports 256K; rebuild with conversion/convert.py <S> for a different length).

Interface

• Inputs: input_ids (1, T) int32 with 1 <= T <= 128; position (1,) int32 = index of input_ids[0, 0].
• Output: logits (1, 128000) fp16, for the last input token only.
• States (35 buffers, created by make_state()): 13 short-conv states and K/V caches for the 11 attention layers.
• Prefill: feed the prompt in chunks of up to 128 tokens, advancing position. Decode: T = 1.
• Tokenization and the chat template are those of the original model (tokenizer.json, chat_template.jinja).

import coremltools as ct, numpy as np
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained(".")
ml = ct.models.MLModel("LFM3-1.7B-int4-s2048.mlpackage", compute_units=ct.ComputeUnit.CPU_AND_GPU)
ids = tok.apply_chat_template([{"role": "user", "content": "日本の首都は？"}],
                              add_generation_prompt=True, return_tensors="np", return_dict=True)["input_ids"].astype(np.int32)
state = ml.make_state()
out = ml.predict({"input_ids": ids, "position": np.array([0], np.int32)}, state)
pos = ids.shape[1]
for _ in range(64):
    nxt = int(out["logits"][0].argmax())
    print(tok.decode([nxt]), end="", flush=True)
    out = ml.predict({"input_ids": np.array([[nxt]], np.int32), "position": np.array([pos], np.int32)}, state)
    pos += 1

Verification

Against the original model in fp32 (Hugging Face transformers), greedy decoding, 2 prompts x 32 tokens, macOS, Apple M2 Pro, CPU_AND_GPU:

Variant	Logit cosine (prompt end)	Greedy tokens identical to original	Decode speed
fp16	0.999999	32/32 on both prompts	22 tok/s
int4	0.975 - 0.983	20/32 and 32/32 (outputs remain coherent)	20 - 28 tok/s

Not measured: Neural Engine (CPU_AND_NE) execution, iPhone speed/memory, long-context (>256 tokens) behavior (sliding-window layers are implemented through masking and are covered only by the short prompts above), and benchmark scores of the converted models. The decode speed is from an unoptimized attention implementation (full-length masked attention over the cache); expect room for improvement.

License

This is a Derivative Work of LFM3-1.7B and is distributed under the LFM Open License v1.0 (see LICENSE). Notably, commercial use is not licensed for legal entities with annual revenue of USD 10M or more. Modifications: the model was converted to Core ML format, reimplemented as a stateful graph, and (int4) weight-quantized.

Conversion

conversion/ contains the PyTorch re-implementation (lfm3_stateful.py), the conversion and quantization scripts, and the verification scripts. Built with coremltools 9.0, torch 2.7.1, transformers 5.21.