LFM2.5-Encoder-350M-Policy-Linter-ONNX

ONNX export of LiquidAI/LFM2.5-Encoder-350M-Policy-Linter, quantized to run fully in the browser through transformers.js. No inference server: the weights are fetched once, cached, and every forward pass happens in the tab.

A zero-shot policy linter. Rules are ordinary prose supplied at call time. One pass scores every token against every rule, so adding a rule costs a few tokens rather than another model.

All credit for the model itself goes to Liquid AI. This repository contains only a re-export; the weights are unchanged apart from quantization, and the original LFM Open License v1.0 applies.

Try it in your browser → — no install, no API key.

Tooling, demo and the export pipeline: https://github.com/kucukkanat/lfm-encoders

Files

dtype	File	Size
q8	onnx/model_quantized.onnx	357 MB
q4	onnx/model_q4.onnx	449 MB

The graph takes input_ids + attention_mask, is dynamic in batch and sequence, and returns token_proj and rule_proj.

Usage

import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";

const id = "kucukkanat/LFM2.5-Encoder-350M-Policy-Linter-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });

const { input_ids } = tokenizer("some text");
const out = await model({
  input_ids,
  attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
});

PreTrainedModel rather than AutoModel is deliberate: this is a plain "feed the named inputs, read the named outputs" session, not one of transformers.js's built-in architectures.

Prompt format

Both projection towers expect one string laid out exactly like this — the model was trained on it and the character arithmetic that locates each label depends on it byte for byte:

Policy:
- label one
- label two

Text:
<the text>

token_proj is the query tower and rule_proj the key tower, both 256-d and emitted per token. Pool the tokens covering each label to get its vector. Pooling after projecting is exact rather than an approximation: both towers are affine, and an affine map commutes with a mean — which is what keeps one static graph usable for any number of labels.

Scoring is a dot product scaled by 1/sqrt(256) plus a bias, through a sigmoid per (token, rule) pair.

@lfm-encoder/tasks implements all of this, including the character-offset reconstruction transformers.js does not provide.

Accuracy

Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a final probability.

dtype	max Δ	mean Δ	threshold flips (6 cases)
fp32	8.4e-4	3.4e-4	0
q8	0.5241	0.2328	3
q4	0.3698	0.1818	3

Notes

• q8 is smaller on disk but uses more browser RAM than fp32 and runs slower: onnxruntime's WASM kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys download size, not speed or memory.
• Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until reloaded.
• fp16 / q4f16 are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and every hidden state collapses to zeros.