CommuniDev/sd-turbo-coreml

SD-Turbo — Core ML (6-bit, for iPhone)

Powered by Stability AI. This is a converted copy of stabilityai/sd-turbo, created by Stability AI: a fast, distilled version of Stable Diffusion 2.1 that makes a 512 × 512 picture in 1–4 steps.

CommuniDev LLC did not train or fine-tune this model. We only changed its file format so it can run on an iPhone. It is published so that our iOS app CommuniDream can download it on first use and turn dream journal entries into pictures, entirely on the device.

What we changed

Converted with Apple's ml-stable-diffusion converter (torch2coreml, commit 5a170d2) from the fp16 safetensors at sd-turbo revision b261bac6. No retraining.

• Core ML, attention SPLIT_EINSUM_V2 (Neural Engine layout), 512 × 512 images.
• UNet and text encoder: 6-bit palettized weights (k-means). The text encoder's 77 × 77 causal attention mask holds -inf and is left uncompressed.
• VAE decoder: fp16. The VAE encoder is not included (text-to-image only).
• Safety checker: the CompVis Stable Diffusion safety checker in Apple's Core ML conversion, unmodified.

Modified files: TextEncoder.mlmodelc, Unet.mlmodelc and VAEDecoder.mlmodelc are converted, modified forms of the original checkpoint. Details are in NOTICE.

Files

Path	Size
compiled/Unet.mlmodelc	623 MB
compiled/TextEncoder.mlmodelc	304 MB
compiled/VAEDecoder.mlmodelc	95 MB
compiled/SafetyChecker.mlmodelc	580 MB
compiled/vocab.json, compiled/merges.txt	1.3 MB

SHA256SUMS lists a SHA-256 hash for every file. Check it with shasum -a 256 -c SHA256SUMS.

How it is used

With Apple's Swift StableDiffusionPipeline: DPM-Solver++ (linspace spacing, which starts at t = 999), 2–4 steps, guidance 1.0–2.0, seed-reproducible. At guidance 1.0 the negative prompt has no effect. The safety checker must stay on (disableSafety: false); CommuniDream refuses to run without it.

Limitations

• 512 × 512 only (the shape is fixed at conversion).
• Distilled for speed: less detail than Stable Diffusion 2.1 at 20+ steps, and weaker style control.
• It can produce wrong, biased or disturbing pictures, and text in pictures is unreliable.
• The safety checker can miss unsafe pictures and can block harmless ones.

Licence

TextEncoder, Unet and VAEDecoder are under the Stability AI Community License (LICENSE-stability-ai-community.md, word for word): use must follow Stability AI's Acceptable Use Policy, and commercial use requires registration with Stability AI. SafetyChecker is under CreativeML Open RAIL-M (LICENSE-creativeml-openrail-m.txt). See LICENSE.md and NOTICE.

Citation

@article{sauer2023adversarial,
  title   = {Adversarial Diffusion Distillation},
  author  = {Sauer, Axel and Lorenz, Dominik and Blattmann, Andreas and Rombach, Robin},
  journal = {arXiv preprint arXiv:2311.17042},
  year    = {2023}
}