hugging-mac/d3-edge-CoreAI

d3-edge Core AI — text Decision model

Locally converted Apple Core AI .aimodel from vllm-sr/d3-edge, published by hugging-mac. This is a text-only, experimental conversion, not a fine-tuned full-duplex classifier. It outputs choice probabilities, yes/no probabilities, or ordered-level scores without generating text.

Conversion works on the tested Mac; full-duplex business accuracy is not sufficient for autonomous production routing. All failures are published. No iPhone, microphone, live ASR, audio interruption, or physical-device acceptance is claimed.

Contract

• Source revision: bee5e872f75b9c5d1405767bf244a6948b95c0af.
• Original BF16 text backbone and FP32 255-way readout; no quantization or training.
• Full-attention layers remain noncausal; DeltaNet layers remain causal.
• 512 tokens total, including state, instructions, candidate descriptions and chat template. Overlength requests fail explicitly; no truncation.
• One independent question per forward pass; up to 16 questions per system_one call, processed sequentially. Up to 255 candidate codes, normally limited earlier by prompt capacity.
• Images and video are explicitly rejected; the original vision encoder is omitted.
• Batch size 1; no persistent recurrent/KV state between decisions. Padding, attention mask and last-valid-token pooling are handled by the adapter.
• Probability values are relative to supplied candidates, not calibrated routing confidence. Do not interpret the largest probability as a guaranteed safety or correctness score.
• Neural inference uses Core AI. Tokenization, question formatting and response assembly run on the host. Original PyTorch weights are not needed to run the bundle.

Run

Tested on Apple M5 (16 GiB), macOS 27.2, Python 3.13.13. Exact dependencies are in requirements-lock.txt; primary versions are in requirements.txt. The converter warns that PyTorch 2.14 is newer than its validated range (<=2.13); the tests here were run with the recorded versions.

hf download hugging-mac/d3-edge-CoreAI --local-dir d3-edge-CoreAI
cd d3-edge-CoreAI
python3.13 -m venv .venv
source .venv/bin/activate
pip install -r requirements-lock.txt
python smoke_package.py

import asyncio
from coreai_d3 import D3CoreAI

async def main():
    model = await D3CoreAI.load(".")
    result = await model.system_one(
        state="The assistant is speaking. The user says: Stop talking now.",
        questions={"action": {
            "type": "choice",
            "instructions": "Which action best describes the user request?",
            "criteria": {
                "interrupt": "Stop the assistant speaking now",
                "discard": "Ignore background noise",
                "enqueue": "Handle a new question later",
            },
        }},
    )
    print(result)

asyncio.run(main())

This short demonstration is not the full product policy. An input such as “嗯” can be acknowledgment or a valid answer depending on context. duplex_cases.py defines the tested policy and labeled scenarios; evaluate_duplex.py contains both exploratory prompt variants.

One model instance admits one native inference at a time and rejects overlap rather than silently queuing. The caller owns the task and must await completion/cancellation before replacement. Cancellation is cooperative and does not promise immediate hardware interruption. The adapter does not execute audio actions or own the application's utterance queue. A failure in a multi-question call raises an exception rather than returning a fabricated decision.

Conversion fidelity

Compared with the pinned original BF16 runtime on CPU:

Suite	Same selected option	Largest absolute probability difference
27 general text + 48 long-policy full-duplex cases	75/75	0.0188751
Short Chinese prompt, same 48 business cases	48/48	0.0230880
Short English prompt, same 48 business cases	47/48	0.0135900

These are 171 prompt-case comparisons, not 171 independent business scenarios. One near-tie changed choice: in the English prompt, a user's “不要” answering “是否删除草稿？” changed from enqueue (reference) to discard (Core AI). Source probabilities were approximately [0.0685, 0.4646, 0.4669]; converted probabilities were [0.0717, 0.4680, 0.4603] in interrupt/discard/enqueue order. Strict numerical equivalence is not claimed.

• Native Swift Core AI loading and inference passed. Maximum probability difference from Python Core AI on the supplied fixture: 5.96e-8.
• Offline package-only smoke passed without source weights.
• Exact 512-token native input produces finite output; 513-token input is rejected before inference.
• Oversized text, media, empty options, invalid question type, invalid score levels and busy admission are rejected.
• Repeated input produced identical probabilities in five baseline repeats.

Raw records: comparison.json, prompt_reference.json, runtime.json, package_smoke_isolated.json, swift-smoke.json.

Full-duplex speech-text evaluation / 全双工业务测试

48 synthetic Chinese cases cover stop/cancel, correction, urgent changes, acknowledgment, negation, deferred requests, idle-state questions and short answers, paired contexts, and known echo/noise provenance. Expected labels were frozen before evaluating prompts. Noise/echo provenance is provided metadata, not acoustic detection performed by the model. All ASR inputs are final transcripts; partial-ASR and endpoint timing are not evaluated.

Prompt	Core AI correct	Original model correct	False interrupt	False discard	Missed interrupt
Long Chinese policy + JSON state	24/48	24/48	0	16	9
Short Chinese policy + natural-text state	34/48	34/48	11	3	0
Short English policy + natural-text state	35/48	36/48	1	4	6

False interrupt means choosing interrupt for a discard/enqueue expectation. False discard means losing an interrupt/enqueue expectation. Missed interrupt means choosing another class for an interrupt expectation; these categories can overlap.

中文结论：转换成功，但这组业务样例最佳仅 35/48（72.9%）。中文简短提示词对打断样例召回较好，却出现 11 次误打断；英文简短提示词误打断较少，却漏掉 6 次打断。当前模型不适合作为全双工对话的唯一控制依据。显式停止、回声/噪声过滤、输入是否最终确认等确定性规则仍应由业务系统负责。需要独立真实语音转写数据上的验证或专项训练，不能把本轮调提示词结果当作独立测试集表现。

No prompt variant is selected as a production default. Both shorter variants were developed after viewing the baseline and evaluated on the same frozen cases; results are exploratory, not held-out accuracy. Fixtures and full confusion matrices are published in duplex_cases.py and prompt_experiments.json.

Performance and resources

On this Apple M5 Mac, default specialization (CPU/GPU/Neural Engine placement not verified):

• Initial cold load including specialization: 58.81 s; original validation process peak RSS 6.13 GiB, including compilation and Python dependencies.
• Cached load in isolated package smoke: 2.20 s.
• 20 warm single-question end-to-end calls: median 55.5 ms, p95 57.6 ms. Includes tokenization and host response formatting; excludes ASR, networking and speech cancellation.
• Final resident process RSS 1.32 GiB; cached-load process peak RSS 2.08 GiB.
• A separate run concurrent with CPU reference evaluation had median 115.4 ms and p95 405.6 ms, including a 1.70 s outlier. Timing is load-dependent; no hard real-time guarantee.

The artifact is roughly 945 MiB. These Mac measurements do not establish iPhone performance or energy efficiency. No simulator result is substituted for physical-device testing.

Reproduce

hf download vllm-sr/d3-edge --revision bee5e872f75b9c5d1405767bf244a6948b95c0af --local-dir upstream
python convert.py --source upstream --output rebuilt --length 512 --revision bee5e872f75b9c5d1405767bf244a6948b95c0af
# convert.py writes the graph and manifest; copy tokenizer/helper files into rebuilt before runtime use.
python validate.py native --bundle . --source upstream --evidence evidence
python validate.py reference --bundle . --source upstream --evidence evidence
python evaluate_duplex.py --bundle . --output evidence/prompt_experiments.json
python compare_duplex.py --source upstream --native-results evidence/prompt_experiments.json --output evidence/prompt_reference.json

Native Swift smoke with Xcode 27:

xcrun swiftc -parse-as-library -target arm64-apple-macos27.0 verify_coreai.swift -o verify-coreai
./verify-coreai .

swift_fixture.json is a synthetic, pretokenized fixture; it is not a Swift tokenizer implementation. The graph takes int32 input_ids [1,512], attention_mask [1,512], and position_ids [4,1,512], and returns FP32 readout logits. Use the output name in manifest.json, not a hard-coded compiler-generated name. This Swift test does not integrate a provider into MobileHarness.

Provenance and license

Original model: vLLM Semantic Router Team, based on Qwen/Qwen3.5-0.8B, Apache-2.0. Conversion recipe and adapter adapted from Max Farrell's conversion, also Apache-2.0. New local conversion, text-only adaptation, validation and release assembly by hugging-mac. Independent community work, not an official Apple or vLLM release.

LICENSE, NOTICE, UPSTREAM_NOTICE, UPSTREAM_MANIFEST.json and source.json preserve attribution and source identity. SHA256SUMS covers the released files. No user audio, account token, original model-weight duplicate, cache, or device identifier is included.