massif/Qwen3-1.7B-CoreAI-GPU-INT4-4K-h16p

Qwen3-1.7B Core AI — GPU INT4, Dynamic KV, 4K, h16p

This repository contains a complete, downloadable Apple Core AI resource directory for Qwen3-1.7B. It is a community conversion, not an official Qwen or Apple release.

The conversion recipe, sanitized Xcode physical-device benchmark apps, deterministic scoring tools, and aggregate W8/ANE comparison evidence are published in the companion GitHub reproduction repository.

• 4-bit blockwise weight compression
• FP16 growing/dynamic KV cache
• GPU-preferred Core AI execution
• 4,096-token context
• compiled for iOS 27, h16p / A17 Pro
• verified on a physical iPhone 15 Pro

The repository also publishes the frozen protocols and aggregate evidence used to compare this artifact with the public W8 / static-KV / ANE reference.

Decision summary

The quality result is deliberately reported with uncertainty rather than as a single marketing score.

Criterion	W8 / static KV / ANE	INT4 / dynamic KV / GPU	Interpretation
CMRC2018 EM, frozen 300	59.33	57.67	W8 point estimate +1.67
CMRC2018 F1, frozen 300	81.70	81.42	W8 point estimate +0.28
(EM + F1) / 2	70.52	69.54	W8 point estimate +0.97
Identical predictions	—	187 / 300	62.3% exactly identical
Compiled bundle	1,714.4 MiB	924.6 MiB	INT4 is 46.1% smaller
Peak process RSS	2,865.0 MiB	1,995.3 MiB	INT4 is 30.4% lower

For the paired quality comparison, the W8-minus-INT4 F1 difference had a 95% bootstrap interval of -2.44 to +3.00 points and a paired win/loss sign-test p = 0.614. The EM interval was -3.00 to +6.33 points with p = 0.576. At 300 examples, this experiment does not establish a statistically detectable quality difference. It also does not prove equivalence outside this task.

The frozen protocol also contained a strict heuristic acceptance rule: INT4 could trail W8 by no more than 1.0 point on both EM and F1. INT4 passed that rule on F1 (-0.28) but missed it on EM (-1.67). Therefore this model card does not claim quality parity or a passed non-inferiority gate, despite the statistically inconclusive paired result.

Speed is workload-dependent:

Controlled physical-device workload	W8 / ANE	INT4 / GPU	Winner
161 input / 60 output, total	3.259 s	2.592 s	INT4 by 20.5%
161 input / 60 output, TTFT	0.431 s	1.221 s	W8 by 2.83x
3,790 input / 10 output, total	6.116 s	13.717 s	W8 by 2.24x
120 input / 256 output, total	13.292 s	7.159 s	INT4 by 46.1%
120 input / 256 output, visible decode	19.615 tok/s	38.487 tok/s	INT4 by 1.96x

On the separate 300-question CMRC run, where every question used a fresh session and produced a short answer, W8 averaged 1.025 s/question and INT4 averaged 5.942 s/question. That end-to-end task favors W8's faster prefill; it is not a contradiction with INT4's higher sustained decode throughput.

Consider this INT4 artifact when install size, memory headroom, or sustained generation is the binding constraint. Prefer the W8/ANE reference when the slightly higher quality point estimate, time-to-first-token, long-prompt prefill, or many short-answer requests dominate. Run your own frozen product task before making a shipping decision.

中文摘要

这是 Qwen3-1.7B 的完整 Core AI INT4/GPU 资源目录，已在 iPhone 15 Pro 真机 运行。与同条件 W8/ANE 版本相比，INT4 编译包小 46.1%，峰值进程内存低 30.4%，持续生成约快 1.96 倍；W8 的首 token 和长文本预填明显更快。

质量方面，冻结的 CMRC2018 validation 300 条测试中，W8 为 EM 59.33 / F1 81.70，INT4 为 EM 57.67 / F1 81.42。当前样本没有证明 两者存在统计显著的质量差异，但 INT4 的 EM 差值 -1.67 没有通过预设的 -1.0 分启发式门槛，所以不能据此宣称质量等价或闸门通过。完整中文报告见 evidence/QUALITY_COMPARISON.zh-CN.md。

Included resource directory

.
├── metadata.json
├── tokenizer/
└── qwen3_1_7b_dynamic_int4_4k-h16p-gpu.aimodelc/

Download the repository without Git LFS setup:

hf download massif/Qwen3-1.7B-CoreAI-GPU-INT4-4K-h16p \
  --local-dir Qwen3-1.7B-CoreAI-GPU-INT4-4K-h16p

The directory containing metadata.json, tokenizer/, and .aimodelc/ is the URL passed to CoreAILanguageModel(resourcesAt:).

Swift usage

The following is a minimal shape of the API used during validation. It requires Apple's coreai-models packages and the corresponding iOS 27 / Xcode 27 SDK surface.

import CoreAILanguageModels
import FoundationModels

let model = try await CoreAILanguageModel(
    resourcesAt: resourcesURL,
    mode: .eager,
    kvCacheStrategy: .auto
)

let session = LanguageModelSession(
    model: model,
    instructions: "Answer accurately and concisely."
)

let stream = session.streamResponse(
    to: "Explain why the sky is blue.",
    options: GenerationOptions(
        temperature: 0,
        maximumResponseTokens: 256
    )
)

for try await snapshot in stream {
    print(snapshot.content)
}

kvCacheStrategy: .auto selected the growing KV path during physical-device validation. Core AI verbose logs reported the coreai-pipelined engine and GPU execution for this asset.

No-thinking benchmark contract

The CMRC quality comparison used Qwen's official chat-template control enable_thinking=false. A /no_think string in the prompt was not treated as an equivalent control. Without the template context, some INT4 answers were generated inside an unclosed reasoning block and appeared empty to the client.

The pinned public coreai-models runtime did not expose this option through the high-level generation API, so the isolated benchmark passed it to tokenizer.applyChatTemplate(..., additionalContext:). The exact benchmark-only diff and integration guidance are in runtime/NO_THINKING.md. Production integrations should expose this as a per-request option rather than hard-code it globally.

Artifact facts

Property	Value
Base model	Qwen/Qwen3-1.7B
Base revision	70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
Conversion source	Apple coreai-models
Conversion commit	85e2f2d8c6433825e64219a32693ed1a4aed519f
Compiler	coreai-build 3600.75.3
Weight compression	block-32 INT4, symmetric with clipping
INT4 storage elements	1,730,174,976
Blockwise shift/scale ops	141
Compute / KV dtype	FP16 / FP16
Context	4,096 tokens
KV behavior	growing/dynamic
Platform	iOS 27
Architecture	h16p
Supported chip reported by inspector	A17 Pro
Preferred compute	GPU
Compiled bundle logical size	969,503,754 bytes / 924.6 MiB

This is an architecture-specific compiled asset. It is not claimed to run on other iPhone chips or OS versions. Recompile for a different target rather than renaming the architecture directory.

Physical-device validation

Validation was performed on:

• iPhone 15 Pro (iPhone16,1, h16p)
• iOS 27.0
• Xcode 27 Beta
• fresh, separately signed test applications—not the existing host app

The feasibility gate confirmed dynamic input shapes, growing FP16 KV, model load, token generation, and a 4K-context boundary on device. See evidence/FEASIBILITY_RESULT.md.

Quality evaluation

Protocol

• Dataset: hfl/cmrc2018, validation split
• Frozen subset: 300 unique contexts, stratified into 100 short, 100 medium, and 100 long examples
• Frozen sample SHA-256: 9fb9d896c96fc2ef0fa9961a5b65d2ac5b8bd09744f717a0a0ecb9f6c9fe05ff
• Same tokenizer, chat template, prompt, sample order, and 128-token budget
• Temperature 0; fresh session for each example
• enable_thinking=false at chat-template application time
• Both variants completed 300/300 examples with 0 runtime errors and 0 maximum-token hits
• Paired bootstrap: 10,000 resamples

Results by context length

Stratum	W8 EM / F1	INT4 EM / F1
Short	60.00 / 82.17	54.00 / 78.68
Medium	64.00 / 84.09	62.00 / 84.79
Long	54.00 / 78.84	57.00 / 80.80

The direction is not monotonic: W8 scored higher on short contexts, while INT4's F1 point estimate was higher on medium and long contexts. This is another reason not to turn the aggregate 0.28-point F1 gap into a blanket precision claim.

The complete English report is evidence/QUALITY_COMPARISON.md; the machine- readable aggregate is evidence/quality-comparison.json; and the frozen procedure is evidence/QUALITY_BENCHMARK_PROTOCOL.md.

Scoring compatibility

The official CMRC metric structure was reproduced: exact match plus mixed-token longest-common-subsequence F1. The original scorer is Python 2; this run used a Python 3 implementation with NLTK Treebank word tokenization and no sentence splitting. CMRC answers are short single-line spans, but byte-for-byte parity with the original Python 2 environment is not claimed.

Only aggregate metrics and evidence hashes are redistributed here, not CMRC examples or full prediction files. The dataset remains subject to its own license.

Speed and memory evaluation

The controlled speed run used the same device, tokenizer, template, prompts, temperature, fresh-session policy, one warm-up, and three accepted samples per workload. Fixed-width unique tags prevented repeat-input cache contamination. Values reported above are medians.

See evidence/SPEED_BENCHMARK_PROTOCOL.md and evidence/SPEED_BENCHMARK_RESULT.md for raw accepted samples, ranges, exclusions, and memory semantics.

Limitations

• Quality was measured on 300 CMRC2018 extractive-QA examples, not the complete dataset and not a general language-model benchmark.
• No public bookmark-analysis, structured-output, safety, or multilingual product benchmark is claimed.
• The W8 and INT4 quality point estimates are statistically inconclusive at this sample size; absence of significance is not proof of equivalence.
• INT4 missed the frozen protocol's heuristic EM tolerance by 0.67 point; no quality-parity or non-inferiority claim is made.
• Power draw, battery cost, sustained thermal throttling, background execution, and Jetsam behavior were not measured in this release.
• Load time is sensitive to OS file and driver caches. Observed load values are documented, but no universal cold-load ratio is claimed.
• Compatibility has only been validated on h16p / A17 Pro with iOS 27.

Reproducibility and integrity

• evidence/artifact-summary.json records the exact revisions, compiler, architecture, compression, and principal hashes.
• SHA256SUMS covers every published file except the manifest itself.
• Raw per-example predictions are not redistributed, but their SHA-256 hashes are recorded in the quality reports so the original run can be audited.

License and attribution

The base model is Qwen3-1.7B and is distributed under Apache License 2.0. This repository includes the license and conversion notices. See NOTICE.md.