neonforestmist/smolgpt-fables-coreml

SmolGPT-Fables Core ML v1

This repository is the separate Core ML companion to SmolGPT-Fables. It is for native Apple apps that want private, on-device story generation. The main language-model repository contains the standard Transformers and GGUF variants; this page contains only the Apple runtime package and its tokenizer assets.

Download the Core ML model

Asset	Size	Use
SmolGPT-Fables-v1-CoreML-INT4.mlpackage	0.96 GB	Core ML on iOS 18+ or macOS 15+
Tokenizer files	3.4 MB	Required to turn story prompts into tokens

The package uses INT4 weights with FP16 activations and a stateful key/value cache. It has a 2,048-token context window and a 49,152-token vocabulary.

For a complete reference implementation, see SmolGPT-Fables-iOS.

Core ML validation

These are the published conversion and numerical smoke-test results from coreml/coreml_manifest.json:

Check	Result
Core ML conversion	Passed
INT4 compression	Passed
Source SHA-256 verification	Passed
Stateful prediction smoke test	Passed
FP16 / INT4 cosine similarity	0.9529
Matching top token	Yes
Top-5 token overlap	4 / 5
Output shape	[1, 1, 49,152]

The companion package is a format conversion of the 1.71B-parameter model. The quality and story-behavior results are reported on the main SmolGPT-Fables model card, not silently presented as a separate Core ML benchmark.

Use it in a Swift app

1. Download the package and tokenizer files from this repository.
2. Verify the package with coreml/coreml-checksums.sha256.
3. Compile the .mlpackage on the device with MLModel.compileModel.
4. Load it with MLModelConfiguration.computeUnits = .cpuAndGPU for the stateful decoder, then cache the compiled model for offline generation.

The iOS reference app shows the complete download, checksum, compilation, and generation flow.

Runtime contract

• inputIds: int32 [1, query_length], with query_length from 1 to 2,048
• causalMask: float16 [1, 1, query_length, end_step]
• logits: float16 [1, 1, 49,152]
• Stateful caches: keyCache and valueCache
• Minimum deployment targets: iOS 18 and macOS 15

Relationship to the main model

The standard model and compact GGUF model are published at neonforestmist/smolgpt-fables. That card contains the story-generation metrics, training-data description, prompt examples, and model limitations. This repository deliberately keeps the Core ML distribution and Apple-specific instructions separate.

License

Apache-2.0. Review the SmolLM2-1.7B-Instruct model card alongside the main SmolGPT-Fables card when assessing intended use and limits.