aneforge/sentence-embeddings

Sentence embeddings on the Apple Neural Engine (via ANEForge)

ANEForge runs computation on the Apple Neural Engine (ANE) directly, without CoreML. Its SentenceTransformer drop-in loads any sentence-transformers model from the Hub by repo id and runs the encoder on the engine, matching the sentence_transformers API.

This is a usage card, not a re-hosted model: it points at the upstream weights and shows how to run them on the ANE.

Install

pip install aneforge          # numpy-only core; the dispatch shim builds on first use

Requires Apple Silicon under macOS 14+. import aneforge works anywhere; compiling and dispatching to the ANE needs the hardware.

Use (drop-in for sentence_transformers.SentenceTransformer)

from aneforge.sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")   # any Hub repo id
emb = model.encode(["a sentence on the Neural Engine", "another one"],
                   normalize_embeddings=True)
print(emb.shape)   # (2, 384)

int8=True streams int8 weights (cosine ~0.9999 vs fp16). A model whose modules.json declares a Normalize module is always L2-normalised, as with upstream sentence-transformers.

Why the ANE

The ANE is the fixed-function accelerator on every recent Apple device. In production it is reachable only through CoreML, which treats it as a schedulable option that can silently fall back to CPU/GPU. ANEForge compiles a lazy tensor graph into a single ANE program and dispatches it through the same daemon and kernel-driver stack Apple's own frameworks use, so the encoder runs on the engine deterministically and off the CPU/GPU.

Links

• Code: https://github.com/sbryngelson/ANEForge
• Package: https://pypi.org/project/aneforge/
• Paper: https://arxiv.org/abs/2606.17090

Citation

@article{bryngelson2026aneforge,
  title   = {ANEForge: Python for direct computation on the Apple Neural Engine},
  author  = {Bryngelson, Spencer H.},
  journal = {arXiv preprint arXiv:2606.17090},
  year    = {2026},
  doi     = {10.48550/arXiv.2606.17090}
}