Model Card for QwenPaw-Flash-9B (Core ML, Apple Neural Engine)
A compiled Core ML build of QwenPaw-Flash-9B, packaged to run entirely on device on the Apple Neural Engine (ANE). It targets macOS (Apple silicon), needs no network at inference time, and streams tokens locally with the weights quantized to 6 bits.
This repository holds the compiled model set only, not training or original weights. It is a self-contained artifact that a companion on-device runtime loads to drive generation. Nothing here calls out to a server.
Model Details
Model Description
QwenPaw-Flash-9B shares the Qwen3.5 hybrid design: of its 32 layers, 24 are Gated DeltaNet linear-attention (state-space) blocks and 8 are full softmax-attention blocks, interleaved "three linear, one attention". The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall. It is the largest of the 0.8B, 2B, and 4B builds, with the same vocabulary and a deeper, wider network. Unlike the smaller builds it does not tie its input and output embeddings, so it ships a separate output-projection matrix. This build converts the upstream model to Core ML and palettizes the weights to 6 bits for on-device use.
- Developed by: AgentScope / QwenPaw authors (base model); Core ML conversion by leok7v
- Model type: Hybrid Gated DeltaNet + attention causal language model, converted to Core ML (mlprogram)
- Language(s): English and the languages of the base model
- License: Apache 2.0 (inherited from the base model)
- Converted from model: agentscope-ai/QwenPaw-Flash-9B
Model Sources
- Repository (this build): https://huggingface.co/leok7v/QwenPaw-Flash-9B-coreml
- Base model: agentscope-ai/QwenPaw-Flash-9B (Apache 2.0)
Uses
Direct Use
On-device assistants and chat, drafting, and summarization, in offline or privacy-sensitive settings where prompts must not leave the device. At 9B parameters it is the most capable of these builds but also the heaviest: it is intended for Apple silicon Macs with ample memory, not phones. The set is driven by an on-device runtime that runs prompt ingestion (prefill) as a batched pass on the Neural Engine and generates tokens one at a time, carrying the linear-attention state forward and reading the attention layers from a paged key/value cache.
Out-of-Scope Use
Not for tasks requiring verified factual accuracy, and not a basis for legal, medical, financial, or safety decisions. Not intended for high-stakes automated decisions without human review.
Bias, Risks, and Limitations
A language model predicts plausible text; it does not verify facts and has no understanding of truth. Output can be confident and still be wrong, incomplete, or biased, and 6-bit quantization can widen small errors relative to the base model. The base model's biases and limitations carry over; see the base model card.
Recommendations
Treat every response critically and check important output against a trusted source. This model defaults to emitting a "thinking" prefix; preserve that per-model default when integrating a chat template.
How to Get Started with the Model
The repository contains a set of compiled .mlmodelc programs plus the
tokenizer. It is designed to be loaded by an on-device runtime that implements
paged key/value attention on the host between the Neural Engine trunk programs;
it is not a single .mlpackage you call with one prediction. The included
chat_template.jinja documents the chat format (roles, the "thinking" prefix,
tool-call rendering).
Technical Specifications
Model Architecture and Objective
| Property | Value |
|---|---|
| Objective | Autoregressive next-token prediction |
| Hidden size | 4096 |
| Layers | 32 (24 Gated DeltaNet + 8 attention) |
| Attention heads | 16 query / 4 key-value (grouped-query) |
| Vocabulary | 248320 |
| Context length | up to 262144 tokens |
| Tied embeddings | no (separate output projection) |
| Weight format | 6-bit palettized (per-grouped-channel k-means) |
| Activations | fp16 |
Files
mf0of9.mlmodelc...mf8of9.mlmodelc- the transformer trunk, cut into 9 programs around the attention layers; each is a multifunction bundle whose "prefill" (batched) and "decode" (single-token) graphs share one quantized weight blob, so the weights are resident once.head.mlmodelc- the vocabulary matrices as one bundle of three functions: the token-embedding matrix (shared by the single-token and batched embed lookups, one quantized copy) and a separate output-projection (lm_head) matrix, since 9B does not tie them.tile.mlmodelc,tile_prefill.mlmodelc,tile_carry.mlmodelc- weightless flash-attention kernels for the paged key/value cache.tokenizer.json,tokenizer_config.json,chat_template.jinja- the byte-level BPE tokenizer and chat template.
Compute Infrastructure
- Hardware: Apple silicon Mac with a Neural Engine.
- Software: macOS 15+. The first launch compiles the programs for the Neural Engine once; the OS caches that build, so later launches are fast.
Citation
Please cite the upstream QwenPaw-Flash-9B model. See the base model card at agentscope-ai/QwenPaw-Flash-9B for the canonical reference and license.
Model Card Contact
leok7v on Hugging Face.