drbh/Qwen3.8-27B-CoreAI-H8

Qwen3.8-27B Core AI H8

Prebuilt Apple Core AI bundles for coreai-rs. The model body uses INT4 linear quantization and the output head uses INT8 quantization. Every bundle is the same weights; they differ only in the context bound compiled into the graph. The runtime allocates KV state on demand up to that bound, at 64 KiB per token, so pick the largest bound whose full allocation fits your machine next to the 17.6 GB model.

Folder	Context bound (tokens)	KV state at full context
repo root	4096	0.3 GB
c8k/	8192	0.5 GB
c32k/	32768	2.1 GB
c64k/	65536	4.3 GB
c128k/	131072	8.6 GB

The bundles target Apple silicon with macOS 27 and Xcode 27. Download and run one through the repository's Qwen/Pi server:

make serve                      # c8k by default
make serve QWEN_CONTEXT=c32k MAX_CONTEXT_TOKENS=32768

Prefill runs near 85 tokens/s on an M3 Max, so a 32k-token prompt takes over six minutes before the first token.

Source model: Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.