wegotomars/d3-mini-CoreAI-ANE

d3-mini on Apple Neural Engine

Core AI conversion of vllm-sr/d3-mini for typed yes/no, choice, and score decisions over text and images, using tinkerbox-ane.

This model has 4B parameters, for a samller 2B parameter model from the same family see wegotomars/d3-nano.

d3-mini accepts prompts of up to 32,768 tokens. The Gated DeltaNet and full-attention layers and the vision tower run fully on the ANE. Tokenization, image preprocessing, position tables, token-embedding lookup, and answer readout run on the CPU while your GPU stays idle and cool 😎❄️.

Requires macOS 27, Apple Silicon M4 or newer, and Python 3.12 or 3.13.

Usage

Install tinkerbox-ane>=2.1.0. Loading the model downloads only this Mac's bundle: its checkpoint folder carries the upstream code, tokenizer and configuration, and the token embeddings and vision position table the CPU uses.

import tinkerbox_ane

with tinkerbox_ane.load("d3-mini") as model:
    result = model.system_one("I was charged twice. Please refund me today.", {
        "refund": {"type": "noul", "instructions": "Is this a refund request?"},
    })
    print(result)

Images can accompany text, or be used alone:

from PIL import Image
import tinkerbox_ane

with tinkerbox_ane.load("d3-mini") as model, Image.open("receipt.jpg") as image:
    result = model.system_one(None, {
        "receipt": {"type": "noul", "instructions": "Is this a receipt?"},
    }, images=[image])
    print(result)

d3's full-attention layers let every prompt token see the question, so questions cannot share a pass. A call with several questions runs one pass per question, in order, and warns with SequentialQuestionsWarning. Each result's usage reports its tokens, Neural Engine time, and counted FLOPs.

Benchmarks

Measured October 21, 2026 on Apple M5 Pro (h17s, 24 GiB), macOS 27.0.1, one question, warm.

Metric	Result
Top-answer agreement with FP32 text reference	100%
Absolute probability drift, mean / maximum	0.0008 / 0.025
Official BF16 drift from FP32, mean / maximum	0.0006 / 0.018
91-token text prompt	85 ms
250-token text prompt	171 ms
506-token text prompt	354 ms
4,090-token text prompt	3.07 s
7,998 token text prompt	8.23 s
26,063-token text prompt	51.4 s
640 × 480 image prompt	0.64 s
ANE energy, 91-token prompt	317 mJ

Upstream checkpoint: vllm-sr/d3-mini@bd0090dd758d32ba59ef5c749910a8e87fc4391f. Weights, upstream code, and converted assets retain the Apache License 2.0. See NOTICE.