Lesson 12: Deep Learning Foundations to Stable Diffusion
Channel: Jeremy Howard
Date: 2023-04-04

(All lesson resources are available at http://course.fast.ai.) In this lesson, we start by discussing the CLIP Interrogator, a Hugging Face Spaces Gradio app that generates text prompts for creating CLIP embeddings. We then dive back into matrix multiplication, using Einstein summation notation and torch.einsum to simplify code and improve performance. We explore GPU acceleration with CUDA and Numba, demonstrating how to write a kernel function for matrix multiplication and launch it on the GPU.

Next up we exercise our tensor programming skills by implementing mean shift clustering, a technique for identifying clusters within a dataset. We create synthetic data, explain the mean shift algorithm, and introduce the Gaussian kernel for penalizing distant points. We implement the mean shift clustering algorithm using PyTorch and discuss the importance of tensor manipulation operations for efficient GPU programming.

Finally, we optimize the mean shift algorithm using PyTorch and GPUs, demonstrating how to calculate weights, multiply matrices, and sum up points to obtain new data points. We explore the impact of changing batch sizes on performance and encourage viewers to research other clustering algorithms.

The lesson concludes with an introduction to calculus, focusing on derivatives and the calculus of infinitesimals.

0:00 - Introduction
0:15 - CLIP Interrogator & how it works
10:52 - Matrix multiplication refresher
11:59 - Einstein summation
18:34 - Matrix multiplication put on to the GPU
33:31 - Clustering (Meanshift)
37:05 - Create Synthetic Centroids
41:47 - Mean shift algorithm
47:37 - Plotting gaussian kernels
53:33 - Calculating distances between points
57:42 - Calculating distances between points (illustrated)
1:04:25 - Getting the weights and weighted average of all the points
1:11:53 - Matplotlib animations
1:15:34 - Accelerating our work by putting it on the GPU
1:37:33 - Calculus refresher

Thank you Raymond-Wu and laith.zumot on forums.fast.ai for creating the timestamps, and to fmussari for the transcript.