CompVis

/

stable-diffusion

like 933

Which note book are you trying to run? If you're using diffusers I think the from_pretrained() function will download and cache all the components you need (which is more than one file).

It's a Stable Diffusion notebook - very recent, but I lost track of the name and link. I'll look around and see if I can sort it, Thanks much.

Heya, this is the Notebook you were referring to @dhmiller . https://colab.research.google.com/github/deforum/stable-diffusion/blob/main/Deforum_Stable_Diffusion.ipynb

Would love some help here on the same topic please! Thanks a bunch

you can specify which folder you want the ckpt file downloaded to with

from diffusers import StableDiffusionPipeline
model_id = "CompVis/stable-diffusion-v1-4"
pipe = StableDiffusionPipeline.from_pretrained(model_id, cache_dir="path/to/folder/")

Hello, any suggestions on how can I create my own .ckpt file as a custom model? For example, I want to upload my images to a model and merge it with the stable diffusion model. How should I create my custom model?

Hey @hamzagorgulu + others with this question

1. For Modifying the model, have a look here https://github.com/CompVis/stable-diffusion/tree/main/ldm/models

This model is quite complicated and contains lots of submodules (diffusion/, autoencoder.py, transformer.py etc.). But seeing what the authors have done is a great guide to starting out.

Look for classes that inhering from nn.Module e.g. ldm/modules/encoders/modules.py:class ClassEmbedder(nn.Module):

Also looking at the paper will explain some of the model decisions: what the authors did and why

I know that's a lot, but there is so much to learn by just studying why each line exists in that code folder!

2. You need to train this model somehow. Here I am not sure what's the best way to proceed. The paper gives some details on this, particularly section 4.2. Image Generation with Latent Diffusion

3. After training, you can Use this page to safe the model state (ckpt): https://pytorch.org/tutorials/recipes/recipes/saving_and_loading_a_general_checkpoint.html

Edit: Also this is an amazing video to give you intuition: https://youtu.be/1CIpzeNxIhU (Computerphile)

Never stop learning (unless you're overfitting)!

Ariel