zeromodels/stable-diffusion-v1-3

See our collection for all Stable Diffusion 1.x checkpoints.

Run Stable Diffusion with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/stable-diffusion-v1-3

Paper: High-Resolution Image Synthesis with Latent Diffusion Models (arXiv:2112.10752) | HF Papers

Pure-Keras 3 conversion of CompVis/stable-diffusion-v1-3 for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX. The whole text-to-image model ships as one container: the UNet denoiser, the VAE and the CLIP ViT-L/14 text encoder in a single model.weights.h5 (1.07B parameters, 3.97 GB), plus zm_config.json (the three component configs, the checkpoint's PNDMScheduler schedule and the default generation settings) and the CLIP tokenizer as tokenizer.json. Weights are stored in float32, exactly as released.

For model details, intended use and limitations, see the upstream model card.

Architecture

Component	zeromodels class	Details
Denoiser	UNet2DConditionModel	(320, 640, 1280, 1280) channels, 2 ResNet blocks per level, 8-head cross-attention on the 768-d text context, 64x64x4 latent
Autoencoder	AutoencoderKL	(128, 256, 512, 512) channels, x8 spatial compression to 4 latent channels, scaling_factor 0.18215
Text encoder	CLIPTextModel (ViT-L/14)	768-d, 12 layers, 12 heads, 77 tokens
Scheduler	PNDMScheduler	scaled_linear betas 0.00085 to 0.012 over 1000 steps; DDIM / Euler / Euler-ancestral are drop-in

Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.stable_diffusion import StableDiffusionTextToImage, StableDiffusionTokenizer

model = StableDiffusionTextToImage.from_weights("zeromodels/stable-diffusion-v1-3")
tokenizer = StableDiffusionTokenizer.from_weights("zeromodels/stable-diffusion-v1-3")

inputs = tokenizer("a photograph of an astronaut riding a horse")
images = model.generate(**inputs, num_inference_steps=50, guidance_scale=7.5, seed=0)
Image.fromarray(images[0]).save("astronaut.png")  # (512, 512, 3) uint8

generate takes the tokenizer's input_ids (batch them for several prompts), an optional negative_input_ids (tokenize the negative prompt), num_inference_steps, guidance_scale, a seed, or explicit latents of shape (batch, 64, 64, 4) for results that are identical across backends.

Load any Stable Diffusion 1.x checkpoint the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Training
stable-diffusion-v1-1	zeromodels/stable-diffusion-v1-1	237k steps at 256px on laion2B-en, then 194k steps at 512px on laion-high-resolution
stable-diffusion-v1-2	zeromodels/stable-diffusion-v1-2	v1-1 + 515k steps at 512px on laion-aesthetics v2 5+
stable-diffusion-v1-3	zeromodels/stable-diffusion-v1-3	v1-2 + 195k steps at 512px, 10% text-conditioning dropout (classifier-free guidance)
stable-diffusion-v1-4	zeromodels/stable-diffusion-v1-4	v1-2 + 225k steps at 512px, 10% text-conditioning dropout (classifier-free guidance)
stable-diffusion-v1-5	zeromodels/stable-diffusion-v1-5	v1-2 + 595k steps at 512px, 10% text-conditioning dropout (classifier-free guidance)

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• The graphs are built for 512px. Pass unet_sample_size=96, vae_sample_size=768 to from_weights to build for 768px (the weights are resolution-independent).
• Swap the sampler any time: model.scheduler = EulerDiscreteScheduler.from_config(model.config.scheduler_config) (zeromodels.base.base_scheduler).
• StableDiffusionModel.from_weights(...) loads the same repo as the bare container (UNet / VAE / text encoder as .unet / .vae / .text_encoder) without the generation loop.
• On-the-fly hf: conversion is not supported for diffusion models; the five checkpoints are hosted here, converted once.
• Parity: at 512px, fp32, PNDM 50 steps, the generated image matches diffusers' to within 1 uint8 level (99.4% of pixels identical), and the UNet / VAE / text encoder outputs to ~1e-6.
• See the Stable Diffusion docs.

License

The weights are redistributed under the CreativeML OpenRAIL-M license of the upstream checkpoint, including its use-based restrictions (Attachment A). By using them you agree to those terms.

Special Thanks

Thank you to the CompVis group at LMU Munich, Runway and Stability AI for training and releasing Stable Diffusion, and to the Hugging Face diffusers team, whose implementation this port was verified against.