{ "cells": [ { "cell_type": "markdown", "source": [ "##Part 2: Fine-tuning Stable Diffusion" ], "metadata": { "id": "NsIPHcm-4hQ5" } }, { "cell_type": "markdown", "source": [ "To train stable diffusion on our images, we can use a technique called fine-tuning, which involves taking a pre-trained stable diffusion model and training it further on a new dataset. In this way, we can leverage the knowledge already learned by the pre-trained model and adapt it to the new dataset, allowing us to achieve better performance with less training time. In this context, we will be fine-tuning a stable diffusion model on our own images, which will enable us to generate new images that are similar in style and content to our original dataset. Once we have fine-tuned the model, we will be able to use it for inference, generating new images on demand." ], "metadata": { "id": "2cPpUHOA4nbB" } }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "d0Qw8JuZJSRQ", "outputId": "0de784b4-8cf5-4e4a-9c47-82e707f47189" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Cloning into 'diffusers'...\n", "remote: Enumerating objects: 27034, done.\u001b[K\n", "remote: Counting objects: 100% (606/606), done.\u001b[K\n", "remote: Compressing objects: 100% (290/290), done.\u001b[K\n", "remote: Total 27034 (delta 371), reused 475 (delta 291), pack-reused 26428\u001b[K\n", "Receiving objects: 100% (27034/27034), 18.46 MiB | 35.59 MiB/s, done.\n", "Resolving deltas: 100% (19684/19684), done.\n", "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Processing ./diffusers\n", " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", "Collecting importlib-metadata (from diffusers==0.18.0.dev0)\n", " Downloading importlib_metadata-6.6.0-py3-none-any.whl (22 kB)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from diffusers==0.18.0.dev0) (3.12.0)\n", "Collecting huggingface-hub>=0.13.2 (from diffusers==0.18.0.dev0)\n", " Downloading huggingface_hub-0.15.1-py3-none-any.whl (236 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m236.8/236.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from diffusers==0.18.0.dev0) (1.22.4)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from diffusers==0.18.0.dev0) (2022.10.31)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from diffusers==0.18.0.dev0) (2.27.1)\n", "Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from diffusers==0.18.0.dev0) (8.4.0)\n", "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.2->diffusers==0.18.0.dev0) (2023.4.0)\n", "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.2->diffusers==0.18.0.dev0) (4.65.0)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.2->diffusers==0.18.0.dev0) (6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.2->diffusers==0.18.0.dev0) (4.5.0)\n", "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.2->diffusers==0.18.0.dev0) (23.1)\n", "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata->diffusers==0.18.0.dev0) (3.15.0)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.18.0.dev0) (1.26.15)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.18.0.dev0) (2022.12.7)\n", "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.18.0.dev0) (2.0.12)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->diffusers==0.18.0.dev0) (3.4)\n", "Building wheels for collected packages: diffusers\n", " Building wheel for diffusers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for diffusers: filename=diffusers-0.18.0.dev0-py3-none-any.whl size=1081907 sha256=3bd6bab6e9ed9a71d373098faaef32e86ccc8001d38939cc4e3290c61cb5327b\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-rz2ca4ml/wheels/95/c5/3b/e1b4269f8a2584de57e75f949a185b48fc4144e9a91fc9965a\n", "Successfully built diffusers\n", "Installing collected packages: importlib-metadata, huggingface-hub, diffusers\n", "Successfully installed diffusers-0.18.0.dev0 huggingface-hub-0.15.1 importlib-metadata-6.6.0\n", "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Collecting accelerate>=0.16.0 (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 1))\n", " Downloading accelerate-0.20.3-py3-none-any.whl (227 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m227.6/227.6 kB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 2)) (0.15.2+cu118)\n", "Collecting transformers>=4.25.1 (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 3))\n", " Downloading transformers-4.30.1-py3-none-any.whl (7.2 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.2/7.2 MB\u001b[0m \u001b[31m73.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting datasets (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading datasets-2.12.0-py3-none-any.whl (474 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m474.6/474.6 kB\u001b[0m \u001b[31m38.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting ftfy (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 5))\n", " Downloading ftfy-6.1.1-py3-none-any.whl (53 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.1/53.1 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (2.12.2)\n", "Collecting tensorboard (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 6))\n", " Downloading tensorboard-2.13.0-py3-none-any.whl (5.6 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.6/5.6 MB\u001b[0m \u001b[31m107.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: Jinja2 in /usr/local/lib/python3.10/dist-packages (from -r /content/diffusers/examples/text_to_image/requirements.txt (line 7)) (3.1.2)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (1.22.4)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (23.1)\n", "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (5.9.5)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (6.0)\n", "Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (2.0.1+cu118)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from torchvision->-r /content/diffusers/examples/text_to_image/requirements.txt (line 2)) (2.27.1)\n", "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision->-r /content/diffusers/examples/text_to_image/requirements.txt (line 2)) (8.4.0)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (3.12.0)\n", "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (4.5.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (1.11.1)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (3.1)\n", "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (2.0.0)\n", "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (3.25.2)\n", "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (16.0.5)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.14.1 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->-r /content/diffusers/examples/text_to_image/requirements.txt (line 3)) (0.15.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->-r /content/diffusers/examples/text_to_image/requirements.txt (line 3)) (2022.10.31)\n", "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers>=4.25.1->-r /content/diffusers/examples/text_to_image/requirements.txt (line 3))\n", " Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m97.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting safetensors>=0.3.1 (from transformers>=4.25.1->-r /content/diffusers/examples/text_to_image/requirements.txt (line 3))\n", " Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m62.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers>=4.25.1->-r /content/diffusers/examples/text_to_image/requirements.txt (line 3)) (4.65.0)\n", "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (9.0.0)\n", "Collecting dill<0.3.7,>=0.3.0 (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading dill-0.3.6-py3-none-any.whl (110 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m110.5/110.5 kB\u001b[0m \u001b[31m13.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (1.5.3)\n", "Collecting xxhash (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading xxhash-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.5/212.5 kB\u001b[0m \u001b[31m23.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting multiprocess (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading multiprocess-0.70.14-py310-none-any.whl (134 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.3/134.3 kB\u001b[0m \u001b[31m16.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (2023.4.0)\n", "Collecting aiohttp (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading aiohttp-3.8.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m56.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting responses<0.19 (from datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n", "Requirement already satisfied: wcwidth>=0.2.5 in /usr/local/lib/python3.10/dist-packages (from ftfy->-r /content/diffusers/examples/text_to_image/requirements.txt (line 5)) (0.2.6)\n", "Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (1.4.0)\n", "Requirement already satisfied: grpcio>=1.48.2 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (1.54.0)\n", "Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (2.17.3)\n", "Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (1.0.0)\n", "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (3.4.3)\n", "Requirement already satisfied: protobuf>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (3.20.3)\n", "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (67.7.2)\n", "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (0.7.0)\n", "Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (2.3.0)\n", "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (0.40.0)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from Jinja2->-r /content/diffusers/examples/text_to_image/requirements.txt (line 7)) (2.1.2)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (23.1.0)\n", "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (2.0.12)\n", "Collecting multidict<7.0,>=4.5 (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.5/114.5 kB\u001b[0m \u001b[31m13.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n", "Collecting yarl<2.0,>=1.0 (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading yarl-1.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (268 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m27.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting frozenlist>=1.1.1 (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m149.6/149.6 kB\u001b[0m \u001b[31m16.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting aiosignal>=1.1.2 (from aiohttp->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4))\n", " Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n", "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (5.3.0)\n", "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (0.3.0)\n", "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (1.16.0)\n", "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (4.9)\n", "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (1.3.1)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision->-r /content/diffusers/examples/text_to_image/requirements.txt (line 2)) (1.26.15)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision->-r /content/diffusers/examples/text_to_image/requirements.txt (line 2)) (2022.12.7)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision->-r /content/diffusers/examples/text_to_image/requirements.txt (line 2)) (3.4)\n", "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets->-r /content/diffusers/examples/text_to_image/requirements.txt (line 4)) (2022.7.1)\n", "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (0.5.0)\n", "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard->-r /content/diffusers/examples/text_to_image/requirements.txt (line 6)) (3.2.2)\n", "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.6.0->accelerate>=0.16.0->-r /content/diffusers/examples/text_to_image/requirements.txt (line 1)) (1.3.0)\n", "Installing collected packages: tokenizers, safetensors, xxhash, multidict, ftfy, frozenlist, dill, async-timeout, yarl, responses, multiprocess, aiosignal, transformers, aiohttp, tensorboard, datasets, accelerate\n", " Attempting uninstall: tensorboard\n", " Found existing installation: tensorboard 2.12.2\n", " Uninstalling tensorboard-2.12.2:\n", " Successfully uninstalled tensorboard-2.12.2\n", "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", "tensorflow 2.12.0 requires tensorboard<2.13,>=2.12, but you have tensorboard 2.13.0 which is incompatible.\u001b[0m\u001b[31m\n", "\u001b[0mSuccessfully installed accelerate-0.20.3 aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 datasets-2.12.0 dill-0.3.6 frozenlist-1.3.3 ftfy-6.1.1 multidict-6.0.4 multiprocess-0.70.14 responses-0.18.0 safetensors-0.3.1 tensorboard-2.13.0 tokenizers-0.13.3 transformers-4.30.1 xxhash-3.2.0 yarl-1.9.2\n" ] } ], "source": [ "!cd /content/\n", "!git clone https://github.com/huggingface/diffusers.git\n", "!pip install ./diffusers\n", "!pip install -U -r /content/diffusers/examples/text_to_image/requirements.txt" ] }, { "cell_type": "markdown", "source": [ "Let's configure HuggingFace Accelerate, a platform for allowing us to automatically configure our system to be able to run our training script. You can learn more about Accelerate [here](https://huggingface.co/docs/accelerate/index)." ], "metadata": { "id": "KIoTn0Fs5V_r" } }, { "cell_type": "code", "source": [ "!nvidia-smi" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "VtJ3R-bfuVDL", "outputId": "bc403096-3e3c-4089-ca4d-4eccc43d7992" }, "execution_count": 2, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Sat Jun 10 19:26:39 2023 \n", "+-----------------------------------------------------------------------------+\n", "| NVIDIA-SMI 525.85.12 Driver Version: 525.85.12 CUDA Version: 12.0 |\n", "|-------------------------------+----------------------+----------------------+\n", "| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n", "| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n", "| | | MIG M. |\n", "|===============================+======================+======================|\n", "| 0 NVIDIA A100-SXM... Off | 00000000:00:04.0 Off | 0 |\n", "| N/A 32C P0 47W / 400W | 0MiB / 40960MiB | 0% Default |\n", "| | | Disabled |\n", "+-------------------------------+----------------------+----------------------+\n", " \n", "+-----------------------------------------------------------------------------+\n", "| Processes: |\n", "| GPU GI CI PID Type Process name GPU Memory |\n", "| ID ID Usage |\n", "|=============================================================================|\n", "| No running processes found |\n", "+-----------------------------------------------------------------------------+\n" ] } ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "luS0NPCIJaO0", "outputId": "190c4de6-4a85-4d75-cf14-a0e8d4cb1bc7" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "2023-06-10 19:27:16.685383: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n", "accelerate configuration saved at /root/.cache/huggingface/accelerate/default_config.yaml\n" ] } ], "source": [ "!accelerate config default --mixed_precision fp16" ] }, { "cell_type": "markdown", "source": [ "Time to configure our environment variables. For this, we want to tell the script our model name, dataset name, and where we would like it to output the model. We will be automatically pushing the model directly to HuggingFace Hub which will require us to login to our HuggingFace account using the token provided through your HuggingFace account settings." ], "metadata": { "id": "m4tfw7v15zh3" } }, { "cell_type": "code", "execution_count": 4, "metadata": { "id": "PUjY_wzXJskm" }, "outputs": [], "source": [ "import os\n", "\n", "os.environ['MODEL_NAME'] = f'CompVis/stable-diffusion-v1-2'\n", "os.environ['DATASET_NAME'] = f'Ali-fb/martin_valen_dataset'\n", "os.environ['OUTPUT_DIR'] = f'sd_martin_valen-model-v1-2_400_demo'" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 145, "referenced_widgets": [ "d54cb0183efe4c4ab977899bc9755750", "99d944a1c7884aa4b6237a360d5ea7f2", "0d0f0f552b5f449dbde6e483e5953c9a", "37fc2b65dcab4f97809cf035d6b6b96a", "0f4fd25cb05f402f9809db1144fc415e", "49f2adcbcc2b4556a397e37f36e0152f", "e41ad54534774ee386074df12ba5e309", "db914811e9a14f0b96e47d85e24819c4", "3c4b959d80b242a98d5ac8765599b98e", "4146401933af4e13bf1287e07098540e", "ba1025f337b54ad1a91c53bda08448ff", "8777198d0dd8433380211108e17284de", "4b6269e18b974847a230dfe220585000", "998e694b790749eebda1ea63333a5d28", "a614cacfd49f4debbc937ee1bbba49d0", "8864f54ba2484fe58840f91034558aa5", "8648354eec0f4791b81d4d330e119a2d", "aa6d986da8724d2396f925223f9b6ef3", "c27aa3b3bc7a4e3aabf4327d46d427d2", "87627dbffe9a4281a00cd6d4d2641d1a", "bbc00fcdca5f4392b5e96e1ef4700bba", "f6147f5093e04fc19c69aba9045e648a", "4692b437067840a3bfea97dfcfa30450", "6fe3326e281844b99a625d1dd1ec9a21", "f7f1ae0882cb466da36e9c997dc3ef75", "bb7e2c9e8a3b4415811b59eb1979399a", "5d0a532112d446c0a352ee6d635adb36", "849c28a484b94c51bab706479d58b465", "5efecc49a2b74db5a62f4a0790862c47", "6b8d5a16ebe94eaaa88bf3d608c4b302", "7e2d80c3ad414e8fa93893a4f5362b9b", "d93bf4ed81d143a0b509b3bea8348213" ] }, "id": "Uif9_QRzJuby", "outputId": "96c48483-cdb1-4a17-efbb-40e773a74273" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "VBox(children=(HTML(value='