D3 Lite Core AI

This is hugging-mac's independent text-only conversion of the vLLM Semantic Router D3 Lite checkpoint. The conversion uses the original BF16 text weights and FP32 decision readout. The graph is stateless and computes candidate scores in one forward pass.

Contract Value
Source revision a8d25a91ef75552803f7009d25b7bd435991abd9
Runtime Apple Core AI
Capacity 512 prompt tokens, batch size 1
Inputs input_ids and attention_mask as int32 [1, 512], position_ids as int32 [4, 1, 512]
Output 255 FP32 candidate logits in the order of the codes in decision_config.json
Graph bytes 1,510,776,860
Graph SHA-256 c9117d0cfdbd6fae885bda3011c222afd4ac5871a3c58ce6d65ecd525de658ea

Use the original chat template and candidate codes. Pad on the left and start the first real position at zero. Apply the original temperature and softmax over the requested candidates. The Python adapter rejects inputs that exceed capacity and requests containing images or video.

Verification

Check Result
Original PyTorch CPU top-choice agreement 82/82 synthetic prompts
Largest absolute candidate probability difference 0.015833
Prompts with a different top choice none
Synthetic bilingual four-way voice routing 6/8 for Core AI, 6/8 for the original
Native Swift smoke test Passed, max probability error 0.000000 against Python Core AI
Mac runtime Apple M5, macOS 27.2.1
Observed Mac load 88.465 s including tokenizer and graph load
Median repeat inference on Mac 109.60 ms including tokenization and output processing

The synthetic voice cases check interrupt, discard, queue, and continue listening. They are a small diagnostic set. Physical iPhone loading, memory and latency remain unmeasured.

Run

Download this repository as a snapshot to bundle on Apple Silicon with Core AI available. Install the pinned Python dependencies in an isolated environment.

python -m pip install -r bundle/requirements-lock.txt
PYTHONPATH=bundle python - <<'PYTHON'
import asyncio
from coreai_d3 import D3CoreAI

async def main():
    model = await D3CoreAI.load("bundle")
    result = await model.system_one(
        state="The customer was charged twice for the same order.",
        questions={
            "route": {
                "type": "choice",
                "instructions": "Which team should help?",
                "criteria": {"billing": "Payments", "shipping": "Delivery"},
            },
        },
    )
    print(result)

asyncio.run(main())
PYTHON

Reproduce from the pinned source and re-run both comparison paths:

hf download vllm-sr/d3-lite --revision a8d25a91ef75552803f7009d25b7bd435991abd9 --local-dir upstream
python bundle/convert.py --source upstream --output rebuilt
python bundle/validate.py native --bundle rebuilt
python bundle/validate.py reference --source upstream --bundle rebuilt
xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 bundle/verify_coreai.swift -o verify-coreai
./verify-coreai rebuilt

The validation scripts come from this release. MODEL_MANIFEST.json records the original checkpoint inventory. SHA256SUMS covers this converted release. See NOTICE for attribution.

Downloads last month
42
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for hugging-mac/d3-lite-CoreAI

Finetuned
vllm-sr/d3-lite
Finetuned
(32)
this model