LFM2.5-Encoder-350M-Prompt-Router — LiteRT

LiquidAI/LFM2.5-Encoder-350M-Prompt-Router converted to LiteRT (.tflite) for on-device inference. Zero-shot prompt routing: define your routing lanes as free text and the model scores the whole prompt against every lane in one CPU pass (demo Space).

File	Recipe	Size
LFM2.5-Encoder-350M-Prompt-Router_wi8fc.tflite	int8 dynamic-range (linears + embedding, convs float)	365 MB	mobile + desktop (iPhone-verified bit-exact, 145 ms)
LFM2.5-Encoder-350M-Prompt-Router_fp16.tflite	fp16 weights, float compute	713 MB	desktop — phone memory limits (XNNPACK per-signature fp32 unpacking)

Signatures

route_128 / route_512 (S = 128 / 512, batch 1, right-padded, up to 8 lane slots):

Input	Shape
input_ids	int32 [1, S]	prompt tokens: Categories:\n- <lane 1>\n- <lane 2>…\n\nText:\n<prompt>
attention_mask	int32 [1, S]	1 = token, 0 = pad
text_pool	float32 [1, 1, S]	mean-pool weights over the prompt's text tokens (1/n each)
category_pool	float32 [1, 8, S]	row r = mean-pool weights over lane r's tokens; unused lane rows all-zero

Output: logits float32 [1, 8]. Softmax over the first N (real) lanes only — all-zero pool rows produce a constant bias logit that must be ignored.

The pool matrices are built host-side from tokenizer character offsets, exactly like the base repo's route() helper:

import numpy as np
from tokenizers import Tokenizer

def build_inputs(text, lanes, tok, S=512):
    body = "\n".join(f"- {r}" for r in lanes)
    prefix = f"Categories:\n{body}\n\nText:\n"
    enc = tok.encode(prefix + text)
    ids, offs = enc.ids, enc.offsets
    x = np.zeros((1, S), np.int32); m = np.zeros((1, S), np.int32)
    x[0, :len(ids)] = ids; m[0, :len(ids)] = 1
    tp = np.zeros((1, 1, S), np.float32)
    ti = [i for i, (a, b) in enumerate(offs) if b > len(prefix) and a != b]
    tp[0, 0, ti] = 1 / len(ti)
    cp = np.zeros((1, 8, S), np.float32)
    pos = len("Categories:\n")
    for r, lane in enumerate(lanes):
        a, b = pos + 2, pos + 2 + len(lane); pos = b + 1
        idx = [i for i, (ta, tb) in enumerate(offs) if ta < b and tb > a and ta != tb]
        cp[0, r, idx] = 1 / len(idx)
    return {"input_ids": x, "attention_mask": m, "text_pool": tp, "category_pool": cp}

Verification

Task-level parity vs the PyTorch reference (demo prompt, 4 lanes): fp32, fp16 and int8 all reproduce the reference lane probabilities to 4 decimal places ("coding question" 0.838). On an iPhone 17 Pro the int8 file reproduces the desktop outputs bit-exactly (cosine 1.000000, max diff 0.0) at 145 ms per route_512 pass (6 threads, XNNPACK).

License

LFM Open License v1.0 (see LICENSE, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted Derivative Works of LiquidAI/LFM2.5-Encoder-350M-Prompt-Router with modification notices per Section 4; all credit for the model to Liquid AI.