coreml-sdxl-vae-encoder-768

A Core ML VAE encoder for Stable Diffusion XL base 1.0 at 768×768 — for img2img (SDEdit) with Apple's hosted iOS package apple/coreml-stable-diffusion-xl-base-ios, which ships no VAE encoder. Apple's only hosted SDXL encoder (macOS repo, compiled/VAEEncoder.mlmodelc) is locked to a fixed 1024×1024 input and cannot be used with the 768² chunked-UNet iOS package. To our knowledge no 768×768 SDXL Core ML encoder was publicly hosted anywhere before this repo.

What's here

compiled/VAEEncoder.mlmodelc — 5 files, ~137 MB total, Float32:

File	Bytes	SHA-256
metadata.json	2,709	ce09ee7cb00a9c86b067ce10e6cf00e8f54420e6af74e397f78c2f9ee8e5d50d
model.mil	135,577	d960ebc1ed8e752e9b29dc9dd704e718b827bf096402d967b90a19587cf7884b
coremldata.bin	934	905f6cbf6aa87d23eb78929a486cd3c3c0e039f1b8d4128eea3e49c367a99975
weights/weight.bin	136,668,992	03990d804c1e5d164d58d6692f53ae980dddc0090ef45bc8f1ea735b6ba384cb
analytics/coremldata.bin	243	a0a2a39c54545e0383cf491ed558de736f992e7959574180ef1a21f896c8800d

• Input: x, Float32 [1, 3, 768, 768] (fixed), normalized to [-1, 1]
• Output: latent, Float32 [1, 8, 96, 96] (4 mean + 4 logvar channels)
• Spec 7 (iOS 16+), generated with coremltools 9.0

How it was made

python -m python_coreml_stable_diffusion.torch2coreml \
  --convert-vae-encoder --xl-version \
  --model-version stabilityai/stable-diffusion-xl-base-1.0 \
  --latent-h 96 --latent-w 96 \
  --compute-unit CPU_AND_NE \
  -o <dir>
xcrun coremlcompiler compile <dir>/Stable_Diffusion_version_..._vae_encoder.mlpackage <dir>

(The latent size is 96×96; pixel size is latent × 8 = 768², matching the iOS package's native UNet resolution.)

Why Float32 (important)

At Float16 the SDXL VAE encoder returns all-NaN latents on CPU and GPU — its GroupNorm accumulation overflows fp16 (verified against Apple's own hosted 1024² encoder, which behaves identically; only the Neural Engine path survives fp16, since the ANE accumulates in fp32 internally). ml-stable-diffusion loads VAE components with .cpuAndGPU (StableDiffusionXL+Resources.swift hardcodes it), so an fp16 encoder breaks img2img on any device that doesn't route it to the ANE. Float32 is also what Apple's own converter produces for SDXL VAE components (--xl-version).

Usage

Copy (or download) VAEEncoder.mlmodelc into the model directory of the SDXL iOS package, next to VAEDecoder.mlmodelc. Set PipelineConfiguration.startingImage + strength < 1 for img2img; scale the source image to 768×768 first (the encoder rejects every other size).

Converted and verified by the Slowpokes Coloring project, September 2026. License follows Stability AI's OpenRAIL++-M (SDXL base 1.0).