Generate a QRCode with StableDiffusion and ControlNet
Tags: python, stablediffusion, controlnet, generativeai
Published: 2023-07-04 19:08:39

As you may know last month, people were passionate about generating QR codes with Stable Diffusion.

https://arstechnica.com/information-technology/2023/06/redditor-creates-working-anime-qr-codes-using-stable-diffusion/

Actually, I tried to generate some QR codes with Automatic1111.
It was fun but at the same time, it was kind of hard since the success rate to generate a scannable QR code was almost 50% 🥲

![bonsai](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xx36j9g7wh9n1qp9xu2v.png)

![building](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kq57710vkf2rogayyf7m.png)

![robot](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/slnv4kc9kih7kgn3wxm0.png)

Then I found an interesting modal for generating QR codes on Hugging Face.

https://huggingface.co/DionTimmer/controlnet_qrcode-control_v1p_sd15

In this post, I will introduce a way to generate a QR code with controlnet_qrcode-control_v1p_sd15 and v2_1.
This post will use Google Colab (free plan).

### generate a QR code
First, we need to generate a QR code.
You can generate a QR code with python package(I use that when I need to create multiple QR codes).
But for this case, we won't need to create many QR codes so I recommend you to use this, https://keremerkan.net/qr-code-and-2d-code-generator/


### install packages
```python
!pip install diffusers transformers accelerate torch xformers
```

### import packages
```python
import torch
from PIL import Image
from diffusers import StableDiffusionControlNetImg2ImgPipeline, ControlNetModel, DDIMScheduler
from diffusers.utils import load_image
```

### load pre_trainned model `controlnet_qrcode-control` and `stable diffusion` model
If you want to try v1.5, please comment out the bottom 2 line and remove the comment from the lines for v1.5. 
```python
# v1.5
# controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcodecode-control_v1p_sd15", torch_dtype=torch.float16)
# pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, safety_checker=None, torch_dtype=torch.float16)

# v2.1
controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcode-control_v11p_sd21", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", controlnet=controlnet, safety_checker=None, torch_dtype=torch.float16)

pipe.enable_xformers_memory_efficient_attention()
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
pipe.enable_model_cpu_offload()
```

### generate a qr code
For this step, what you need to do is the followings.
1. Upload the QR code you generated to your Google Colab folder
2. Get the source image / the source image URL If you want an image that is uploaded somewhere, you can use the image URL directly The image must be square.
**The image must be square.**
3. Set prompt and negative prompt
4. Run (it will take 1-2min)

```python
def resize_image(input_image: Image, resolution: int):
  input_image = input_image.convert("RGB")
  W, H = input_image.size
  k = float(resolution) / min(H, W)
  H *= k
  W *= k
  H = int(round(H / 64.0)) * 64
  W = int(round(W / 64.0)) * 64
  img = input_image.resize((W, H), resample=Image.LANCZOS)
  return img

orriginal_qr_code_image = load_image('/content/code_with_block.png')
# img_path = 'https://images.squarespace-cdn.com/content/v1/59413d96e6f2e1c6837c7ecd/1536503659130-R84NUPOY4QPQTEGCTSAI/15fe1e62172035.5a87280d713e4.png'
img_path = load_image('/content/test.png')
init_image = load_image(img_path)
condition_image = resize_image(orriginal_qr_code_image, 768)
init_image = resize_image(init_image, 768)
generator = torch.manual_seed(123121231)
image = pipe(prompt="8k, 3d, ((photo-realistic)), bonsai, Japanese style",
             negative_prompt="ugly, disfigured, low quality, blurry, nsfw, worst quality, illustration, drawing",
             image=init_image,
             control_image=condition_image,
             width=768,
             height=768,
             guidance_scale=20,
             controlnet_conditioning_scale=2.5,
             generator=generator,
             strength=0.9,
             num_inference_steps=150,
            )

image.images[0]

```


![bonsa-ish](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y3gpn75m7oq27hdvyuaq.png)


Google Colab code
https://colab.research.google.com/drive/1D2V-5hebvOeHJOOK19l0Bujqh4MlcLTl?usp=sharing

Right now, I'm trying Gradio to add the UI for the above script.