Lesson 10: Deep Learning Foundations to Stable Diffusion, 2022
Channel: Jeremy Howard
Date: 2022-10-20

(All lesson resources are available at http://course.fast.ai.) This lesson creates a complete Diffusers pipeline from the underlying components: the VAE, unet, scheduler, and tokeniser. By putting them together manually, this gives you the flexibility to fully customise every aspect of the inference process.

We also discuss three important new papers that have been released in the last week, which improve inference performance by over 10x, and allow any photo to be “edited” by just describing what the new picture should show.

The second half of the lesson begins the “from the foundations” stage of the course, developing a basic matrix class and random number generator from scratch, as well as discussing the use of iterators in Python.

You can discuss this lesson, and access links to all notebooks and resources from it, at this forum topic: https://forums.fast.ai/t/lesson-10-part-2-preview/101337

Additional Links:
- Progressive Distillation for Fast Sampling of Diffusion Models - https://arxiv.org/abs/2202.00512
- On Distillation of Guided Diffusion Models - https://arxiv.org/abs/2210.03142
- Imagic: Text-Based Real Image Editing with Diffusion Models - https://arxiv.org/abs/2210.09276

0:00 - Introduction
0:35 - Showing student’s work over the past week.
6:04 - Recap Lesson 9
12:55 - Explaining “Progressive Distillation for Fast Sampling of Diffusion Models” & “On Distillation of Guided Diffusion Models”
26:53 - Explaining “Imagic: Text-Based Real Image Editing with Diffusion Models”
33:53 - Stable diffusion pipeline code walkthrough
41:19 - Scaling random noise to ensure variance
50:21 - Recommended homework for the week
53:42 - What are the foundations of stable diffusion? Notebook deep dive
1:06:30 - Numpy arrays and PyTorch Tensors from scratch
1:28:28 - History of tensor programming
1:37:00 - Random numbers from scratch
1:42:41 - Important tip on random numbers via process forking

Thanks to fmussari for the transcript, and to Raymond-Wu (on forums.fast.ai) for the timestamps.