D3 Lite Core AI
This is hugging-mac's independent text-only conversion of the vLLM Semantic Router D3 Lite checkpoint. The conversion uses the original BF16 text weights and FP32 decision readout. The graph is stateless and computes candidate scores in one forward pass.
| Contract | Value |
|---|---|
| Source revision | a8d25a91ef75552803f7009d25b7bd435991abd9 |
| Runtime | Apple Core AI |
| Capacity | 512 prompt tokens, batch size 1 |
| Inputs | input_ids and attention_mask as int32 [1, 512], position_ids as int32 [4, 1, 512] |
| Output | 255 FP32 candidate logits in the order of the codes in decision_config.json |
| Graph bytes | 1,510,776,860 |
| Graph SHA-256 | c9117d0cfdbd6fae885bda3011c222afd4ac5871a3c58ce6d65ecd525de658ea |
Use the original chat template and candidate codes. Pad on the left and start the first real position at zero. Apply the original temperature and softmax over the requested candidates. The Python adapter rejects inputs that exceed capacity and requests containing images or video.
Verification
| Check | Result |
|---|---|
| Original PyTorch CPU top-choice agreement | 82/82 synthetic prompts |
| Largest absolute candidate probability difference | 0.015833 |
| Prompts with a different top choice | none |
| Synthetic bilingual four-way voice routing | 6/8 for Core AI, 6/8 for the original |
| Native Swift smoke test | Passed, max probability error 0.000000 against Python Core AI |
| Mac runtime | Apple M5, macOS 27.2.1 |
| Observed Mac load | 88.465 s including tokenizer and graph load |
| Median repeat inference on Mac | 109.60 ms including tokenization and output processing |
The synthetic voice cases check interrupt, discard, queue, and continue listening. They are a small diagnostic set. Physical iPhone loading, memory and latency remain unmeasured.
Run
Download this repository as a snapshot to bundle on Apple Silicon with Core AI available.
Install the pinned Python dependencies in an isolated environment.
python -m pip install -r bundle/requirements-lock.txt
PYTHONPATH=bundle python - <<'PYTHON'
import asyncio
from coreai_d3 import D3CoreAI
async def main():
model = await D3CoreAI.load("bundle")
result = await model.system_one(
state="The customer was charged twice for the same order.",
questions={
"route": {
"type": "choice",
"instructions": "Which team should help?",
"criteria": {"billing": "Payments", "shipping": "Delivery"},
},
},
)
print(result)
asyncio.run(main())
PYTHON
Reproduce from the pinned source and re-run both comparison paths:
hf download vllm-sr/d3-lite --revision a8d25a91ef75552803f7009d25b7bd435991abd9 --local-dir upstream
python bundle/convert.py --source upstream --output rebuilt
python bundle/validate.py native --bundle rebuilt
python bundle/validate.py reference --source upstream --bundle rebuilt
xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 bundle/verify_coreai.swift -o verify-coreai
./verify-coreai rebuilt
The validation scripts come from this release. MODEL_MANIFEST.json records the original
checkpoint inventory. SHA256SUMS covers this converted release. See NOTICE for attribution.
- Downloads last month
- 42