aneforge/reranker

Reranking on the Apple Neural Engine (via ANEForge)

ANEForge runs computation on the Apple Neural Engine (ANE) directly, without CoreML. Its CrossEncoder loads a BERT-family cross-encoder / reranker from the Hub by repo id and runs the transformer on the engine, matching the sentence_transformers.CrossEncoder API.

This is a usage card, not a re-hosted model: it points at the upstream weights and shows how to run them on the ANE.

Install

pip install aneforge

Apple Silicon, macOS 14+.

Use

from aneforge.sentence_transformers import CrossEncoder

ce = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")   # any BERT-family cross-encoder
query = "How many people live in Berlin?"
passages = [
    "Berlin has about 3.85 million residents.",
    "Paris is the capital of France.",
]
scores = ce.predict([(query, p) for p in passages])   # higher = more relevant; transformer on the ANE
ranked = sorted(zip(scores, passages), reverse=True)

Measured

On an M5 Pro, cross-encoder/ms-marco-MiniLM-L-6-v2 scores a (query, passage) pair in ~0.8 ms, matching the Hugging Face reference ranking (relerr 5e-4).

Scope

BERT-family cross-encoders with a pooler + classifier head (e.g. cross-encoder/ms-marco-MiniLM-L-6-v2, -L-12-v2). RoBERTa/XLM-R rerankers (bge-reranker) are a work in progress -- see the repo issues.

Why the ANE

The ANE is the fixed-function accelerator on every recent Apple device. In production it is reachable only through CoreML, which can silently fall back to CPU/GPU; ANEForge compiles the transformer to a single ANE program and dispatches it through the same daemon and kernel-driver stack Apple's own frameworks use.

Links

• Code: https://github.com/sbryngelson/ANEForge
• Package: https://pypi.org/project/aneforge/
• Paper: https://arxiv.org/abs/2606.17090

Cite

Bryngelson, S. H. ANEForge: Python for direct computation on the Apple Neural Engine. arXiv:2606.17090 (2026).