JohnP1/d1a-e4b-it-coreml-4bit

D1A-E4B-it v0.6 · Core ML int4 (text, ≤ 512 tokens)

A small decision model for Apple devices: give it a document and a few questions, and it picks the answers. Each question is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no generated text, so every answer is a valid option.

This is the Core ML int4 conversion of JohnP1/d1a-e4b-it v0.6. It is text only, for documents up to 512 tokens, with int4 weights (per block of 32): 2.1 GB of chunks plus 2.3 GB of embedding tables. The conversion is the one used for the E2B and E2B-it Core ML builds, unchanged.

Status: read before use

• Answers move at 4 bits. Mac check on an Apple M1 Max: 28 real requests of up to 512 tokens (42 questions), against the fp32 PyTorch rebuild of the 8-bit export it was converted from.
  • 3 of 42 answers change (7.1%), all three on clear-cut questions (where the reference's top answer leads by more than 0.05); the largest probability change is 0.455.
  • For comparison, D1A-E2B-it's int4 Core ML build changes 0 of 42.
• Speed on the Mac's GPU (CPU_AND_GPU): p50 6.6 s, p95 9.2 s per request; 469 s to load (first, compiled).
• Not verified on iPhone. No int8 Core ML build is planned for E4B-it; for answers that must not move, use the MLX 8-bit build below.
• Not included: photo, voice and video (their encoders are separate models, not converted), and inputs longer than 512 tokens.

For decisions that matter, prefer the MLX 8-bit build JohnP1/d1a-e4b-it-mlx-q8 (1 of 202 demo answers and 3 of 276 labeler answers differ from the served model).

<bos>: encode with it

The Gemma 4 E4B-it tokenizer adds no <bos>, but this model was trained with one at the start of every record. d1a from jonpol01/d1a#261 on puts it back (bos_token_id 2; d1a_config.json records leading_ids: [2]). An app that encodes requests itself must prepend id 2 before right-padding to 512.

Run it

pip install coremltools numpy torch transformers safetensors "d1a @ git+https://github.com/jonpol01/d1a"
hf download JohnP1/d1a-e4b-it-coreml-4bit --revision v0.6 --local-dir d1a-e4b-it-coreml-4bit
python d1a-e4b-it-coreml-4bit/reference/run.py d1a-e4b-it-coreml-4bit request.json

request.json is a System One request, for example:

{"state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
 "questions": {"team": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"returns": null, "shipping": null, "billing": null}},
               "urgent": {"type": "noul", "instructions": "Does this need a reply today?"}}}

The first run compiles the chunks into coreml/.compiled/ (several minutes); later runs load them.

Files

Path	What
coreml/L512_<a>-<b>.mlpackage	7 chunks of the 42-layer text model, int4 weights (per block of 32)
embeddings.safetensors	the token and per-layer embedding tables (MLX affine-quantized, from JohnP1/d1a-e4b-it-mlx-q8@v0.6), read row by row on the host
head.safetensors, d1a_config.json	the pointer head, the temperature (1.55), the <bos>, pad and delimiter ids
config.json, tokenizer.json, tokenizer_config.json	the Gemma 4 E4B-it model config and tokenizer
io.json	each chunk's input names
reference/	Python reference: run.py (one request), workers.py (one process per chunk), d1a_coreml.py (host-side embeddings and masks), pipeline.py (all chunks in one process)

macOS 15 / iOS 18 or later.

License

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). D1A is built on Kev by Jared Palmer (Apache-2.0).