FluidInference/decision-1.0-lex-coreml

decision-lex Core ML

FP16 Core ML conversion of llm-semantic-router/Decision-1.0-Lex-0.6B at revision ee8e74d912fca8328a353c11d174b44da3f91781. The complete native model has 571,909,635 unique parameters and three trained decision paths. choice.mlpackage, noul.mlpackage, and score.mlpackage jointly implement those paths; calling only Choice is not the full native model.

This is an initial fixed-shape release. Supported shapes are listed below. Longer requests or larger candidate sets must be rejected or served by a later variant; silently truncating them changes the model. The source model's public input budget is 1,024 tokens. conversion/run_coreml.py supplies the pinned System One renderer, tokenizer, candidate masks, softmax, and output mapping.

Decision path	Tokens	Candidate slots	FP16 package MB	Validated source requests	Max probability error vs native
choice	128	3	643.8	2	0.000516415
noul	128	2	643.8	2	0.00113606
score	128	3	643.8	2	0.00046593

All three installed packages total 1931.4 MB. The three graphs each carry an embedding copy; this exceeds the unique native parameter payload. Package size does not include Core ML compilation.

Each package was compared against the complete pinned native model on the source repository's real decision and System One examples. This small parity check establishes conversion behavior on those inputs; it is not an official Decision Index score or a broad quality evaluation.

The tracker lists an estimated 307.8M encoder for this entry. This release contains about 572M unique native parameters, including all trained paths and heads. Its exact historical serving adapter is not publicly verified, so the tracker score is not attributed to this Core ML build.

The Decision contributions are Apache 2.0. The inherited tokenizer has additional Gemma-origin terms. See LICENSE, NOTICE, LICENSING_STATUS.md, DISTRIBUTION_TERMS.md, and LICENSES/ included here. The tokenizer and source runtime were not retrained or modified.

Conversion code, source locks, fixed real fixtures, and reports are in conversion/ and reports/. Source artifacts remain at the upstream model link above.

uv sync --project conversion
uv run --project conversion python conversion/run_coreml.py --repo-root . --request-json conversion/upstream-system-one.json

Repository: https://huggingface.co/FluidInference/decision-1.0-lex-coreml

Optional embedding-only W8 packages

These packages quantize only the token embedding to per-channel int8; the encoder and heads remain FP16. They are size options validated on two pinned real requests per typed path, not a Decision Index score or a measured speedup. Use --embedding-w8-kinds with conversion/run_coreml.py to select the listed paths; other paths keep FP16.

Path	Package MB	Max probability error vs native
noul	448.0	0.00323045
score	448.0	0.00101513

Lex Choice remains FP16: both symmetric and asymmetric embedding W8 changed one pinned real Choice decision.

Forced CPU+ANE placed 926/938 executable operations on ANE in the profiled W8 path, with int32 CPU boundaries. This is operation placement, not a runtime percentage. Exact package hashes and validation reports are in reports/.