wegotomars/d3-nano-CoreAI-ANE

d3-nano on Apple Neural Engine

Core AI conversion of vllm-sr/d3-nano for typed yes/no, choice, and score decisions over text and images, using tinkerbox-ane.

This model has 2B parameters, for a larger 4B parameter model from the same family see wegotomars/d3-mini.

d3-nano accepts prompts of up to 32,768 tokens. The Gated DeltaNet and full-attention layers and the vision tower run fully on the ANE. Tokenization, image preprocessing, position tables, token-embedding lookup, and answer readout run on the CPU while your GPU stays idle and cool 😎❄️.

Requires macOS 27, Apple Silicon M4 or newer, and Python 3.12 or 3.13.

Usage

Install tinkerbox-ane>=2.1.0. Loading the model downloads only this Mac's bundle: its checkpoint folder carries the upstream code, tokenizer and configuration, and the token embeddings and vision position table the CPU uses.

import tinkerbox_ane

with tinkerbox_ane.load("d3-nano") as model:
    result = model.system_one("I was charged twice. Please refund me today.", {
        "refund": {"type": "noul", "instructions": "Is this a refund request?"},
    })
    print(result)

Images can accompany text, or be used alone:

from PIL import Image
import tinkerbox_ane

with tinkerbox_ane.load("d3-nano") as model, Image.open("receipt.jpg") as image:
    result = model.system_one(None, {
        "receipt": {"type": "noul", "instructions": "Is this a receipt?"},
    }, images=[image])
    print(result)

d3's full-attention layers let every prompt token see the question, so questions cannot share a pass. A call with several questions runs one pass per question, in order, and warns with SequentialQuestionsWarning. Each result's usage reports its tokens, Neural Engine time, and counted FLOPs.

Benchmarks

Measured October 21, 2026 on Apple M5 Pro (h17s, 24 GiB), macOS 27.0.1, one question, warm.

Metric	Result
Top-answer agreement with FP32 text reference	99.5%
Absolute probability drift, mean / maximum	0.0027 / 0.087
Official BF16 drift from FP32, mean / maximum	0.0011 / 0.025
91-token text prompt	31 ms
252-token text prompt	64 ms
508-token text prompt	137 ms
4,092-token text prompt	1.19 s
640 × 480 image prompt	0.42 s
ANE energy, 91-token prompt	121 mJ

The one disagreement is a 320-token review whose top two scores differ by 0.01. Short prompts drift about twice as much as official BF16, which keeps the Gated DeltaNet recurrence in FP32, prompts over 512 tokens drift as little as BF16.

Upstream checkpoint: vllm-sr/d3-nano@462efc366229ea268cf4eed3738b3fc4415e4817. Weights, upstream code, and converted assets retain the Apache License 2.0. See NOTICE.