LFM2.5-ColBERT-350M — Core ML (Apple Neural Engine)

Core ML conversions of LiquidAI/LFM2.5-ColBERT-350M for on-device late-interaction (ColBERT / MaxSim) retrieval on Apple silicon.

This repository contains only the converted, ready-to-run .mlmodelc files and does not modify the model's behavior — the weights, architecture, and outputs come entirely from the base model. All credit for the model belongs to Liquid AI.

Base model: LiquidAI/LFM2.5-ColBERT-350M License: LFM Open License v1.0 (lfm1.0) — inherited from the base model (see License)

---

What this is

The base model is a multilingual late-interaction retriever: it produces one 128-dimensional, L2-normalized vector per token, and scores query↔document with MaxSim (Σ_q max_d (q · d)). This gives higher accuracy and better generalization than single-vector embeddings, at the cost of a larger index.

These compiled Core ML models let you run the encoder on the Apple Neural Engine (ANE) / GPU / CPU on iPhone, iPad, and Mac, without Python or PyTorch at inference time.

Exported graph: embed → 16 hybrid layers (10 short-conv + 6 GQA attention) → Dense(1024→128, no bias) → per-token L2 norm.

• Inputs: input_ids (1, seq) int32, attention_mask (1, seq) int32
• Output: token_embeddings (1, seq, 128) — per-token, L2-normalized

Because Core ML on the ANE prefers static shapes, the encoder is split into two fixed-length packages (matching the base model's limits: query = 32 tokens, document = 512 tokens).

---

Files

File	Role	Seq	Precision	Size
LFM25-ColBERT-query-6bit.mlmodelc	Query encoder	32	6-bit palettized	350 MB
LFM25-ColBERT-doc-6bit.mlmodelc	Document encoder	512	6-bit palettized	350 MB

These are compiled Core ML models (.mlmodelc) — load them directly, no compile step needed.

---

Accuracy (converted vs. PyTorch fp32 reference)

Per-token cosine similarity over valid tokens, and end-to-end MaxSim ranking vs. the official PyLate pipeline:

Precision	Query min-cos	Doc min-cos	MaxSim ranking
fp16	1.0000	0.9975	exact match
6-bit	0.9983	0.9973	exact match

Neural Engine residency

Model	ComputeUnit.ALL	CPU_AND_NE
Document (512)	96% ANE	~100% ANE
Query (32)	GPU-preferred by the planner	~99% ANE

All ops are ANE-compatible. To pin the query model to the Neural Engine, load it with .cpuAndNeuralEngine compute units (see below). Measure real latency with Xcode Instruments on your target device.

---

Usage

Pre / post-processing (do this in your host app — it is not in the graph)

1. Prefixes: prepend "[Q] " to queries and "[D] " to documents before tokenizing.
2. Query expansion: pad/expand queries to 32 tokens; keep all 32 query vectors for MaxSim.
3. Documents: drop padding positions (attention_mask == 0) from the output before scoring.
4. Scoring: MaxSim = Σ_q max_d (q · d) over the L2-normalized token vectors.

Use the base model's tokenizer (LiquidAI/LFM2.5-ColBERT-350M) to produce input_ids / attention_mask.

Swift

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine        // pin to the Neural Engine

let docModel = try MLModel(
    contentsOf: URL(fileURLWithPath: "LFM25-ColBERT-doc-6bit.mlmodelc"),
    configuration: config)

// input_ids / attention_mask: MLMultiArray shape [1, 512], Int32
let out = try docModel.prediction(from: inputs)
// out.featureValue(for: "token_embeddings") -> [1, 512, 128]

Python (coremltools)

import numpy as np, coremltools as ct

# .mlmodelc is a compiled model -> use CompiledMLModel
m = ct.models.CompiledMLModel("LFM25-ColBERT-doc-6bit.mlmodelc",
                              compute_units=ct.ComputeUnit.CPU_AND_NE)
out = m.predict({
    "input_ids":      ids.astype(np.int32),        # (1, 512)
    "attention_mask": mask.astype(np.int32),       # (1, 512)
})
doc_vecs = out["token_embeddings"][0]              # (512, 128), L2-normalized

# MaxSim against query vectors q (Q, 128):
scores = (q @ doc_vecs[doc_mask.astype(bool)].T).max(axis=1).sum()

---

Model details (from the base model)

• Parameters: ~353M
• Layers: 17 (10 convolutional + 6 attention + 1 dense)
• Backbone: LFM2.5-350M-Base (bidirectional), hidden 1024
• Output: 128-dim vector per token, L2-normalized; scored with MaxSim
• Max sequence length: document 512, query 32
• Languages (11): English, Spanish, German, French, Italian, Portuguese, Arabic, Swedish, Norwegian, Japanese, Korean

See the base model card for training data, intended use, evaluation, and limitations.

---

Conversion notes

• Exported from the official model via coremltools (mlprogram, fp16; minimum_deployment_target = iOS18).
• Compression: post-training grouped-LUT k-means palettization (per_grouped_channel, group_size = 16); embeddings kept at fp16.
• No fine-tuning or weight changes beyond quantization — outputs match the base model within the tolerances above.

---

License

This model is a conversion of LiquidAI/LFM2.5-ColBERT-350M and is distributed under the same LFM Open License v1.0 (lfm1.0). Your use of these files is subject to that license. See the license on the base model repository: https://huggingface.co/LiquidAI/LFM2.5-ColBERT-350M/blob/main/LICENSE.

All rights to the underlying model belong to Liquid AI. This repository only redistributes a hardware-optimized (Core ML) form of their released weights.

Citation

@article{liquidai2025lfm2,
  title={LFM2 Technical Report},
  author={Liquid AI},
  journal={arXiv preprint arXiv:2511.23404},
  year={2025}
}

Acknowledgements

Model by Liquid AI. Core ML conversion for on-device Apple Neural Engine inference.