Package: sdqrcode
Stable Diffusion QR Codealpha version, expect breaking changescall diffusers pipeline orAutomatic1111 webuiapi to generate qrcodes.tldrimportsdqrcodesd_qr_images,generator=sdqrcode.init_and_generate_sd_qrcode(config="default_diffusers")EngineColabDiffusersAutomatic1111UpdatesJune 25:The diffusers version has been added to the packageJune 23:a colab with a pure diffusers version without automatic1111 dependencie is now available. It will be added to the package soonMotivationThere is multiple methodes availables to generate ai qr code with differents controlnets models and params. Some parameters might works better with some stable diffusion checkpoints and it's a pain to find somethings that works consistanly.
This repo aims to easily try and evaluate differents methods, models, params and share them with a simple config fileHow it worksThe idea is to use controlnet to guide the generation:an image is generated  based on the prompt for a few stepscontrolnet is activated for some steps to add the qrcode on the generating imagecontrolnet is deactivated to blend the qrcode and the imageWith this method, small modifications of theweight,startandendparameters can have huge impacts on the generation.Exempleclick to expand, cherry picked, will add more results laterInstallpip install sdqrcode # Automatic1111 engine
#or
pip install sdqrcode[diffusers] # Diffusers engine

pip install git+https://github.com/huggingface/diffusers # require dev version of diffusersUsage Diffusersimportsdqrcode# init with a default configgenerator=sdqrcode.init(config="default_diffusers")# or with a custom configgenerator=sdqrcode.init(config="/path/to/config.yaml")# or you can also set custom config params (base model, controlnet models, steps, ...)generator=sdqrcode.init(config="default_diffusers",model_name_or_path="Lykon/DreamShaper")# Then you can generate according to the configimages=generator.generate_sd_qrcode()# or with some custom parameters (you can't set the models at this stage)images=generator.generate_sd_qrcode(prompt="A beautiful minecraft landscape",steps=30,cfg_scale=7,width=768,height=768,seed=-1,controlnet_weights=[0.35,0.65],# [weight_cn_1, weight_cn_2, ...]controlnet_startstops=[(0,1),(0.35,0.7)],# [(start_cn_1, end_cn_1), ... ]. (0.35, 0.7) means apply CN after 35% of total steps until 70% of total stepsqrcode_text="https://koll.ai",qrcode_error_correction="high",qrcode_box_size=10,qrcode_border=4,qrcode_fill_color="black",qrcode_back_color="white",)Usage Automatic1111importsdqrcode# Use an auto config and define the auto_* params in init to use Automatic1111 backendgenerator=sdqrcode.init(config_name_or_path="default_auto",auto_api_hostname="auto_hostname",auto_api_port=7860,auto_api_https=True,auto_api_username="auto_user",auto_api_password="auto_pass")# Then you can generate like the diffusers versionimages=generator.generate_sd_qrcode()# get available modelsgenerator.engine.api.util_get_model_names()# get available controlnet modulesgenerator.engine.api.controlnet_module_list()# get available controlnet modelsgenerator.engine.api.controlnet_model_list()Get default configsimportsdqrcodesdqrcode.CONFIGS()Config FileThis lib uses a yaml file to describe the qrcode generation process. You can change any parameters to experiment. Exemple:global:mode:txt2imgprompt:"abeautifulminecraftlandscape,lightsandshadows"negative_prompt:"ugly"model_name_or_path:"SG161222/Realistic_Vision_V2.0"steps:20scheduler_name:Euler acfg_scale:7width:768height:768seed:-1batch_size:1input_image:qrcode# img2img mode onlydenoising_strength:0.7# img2img mode onlycontrolnet_units:brightness:model:ioclab/control_v1p_sd15_brightnesscn_input_image:qrcodemodule:none#not implemented yet for diffusersweight:0.35start:0.0end:1.0tile:model:lllyasviel/control_v11f1e_sd15_tilemodule:none#not implemented yet for diffuserscn_input_image:qrcodeweight:0.5start:0.35end:0.70qrcode:text:"https://koll.ai"error_correction:high# [low, medium, quart, high]box_size:10border:4fill_color:blackback_color:whiteglobalmode: txt2img or img2img (str)prompt: the prompt to use (str)negative_prompt: the negative prompt to use (str)model_name_or_path: stable diffusion checkpoint to use (str)for diffusers, you can use the model name or local pathfor automatic1111, you should use one of the installed models (see utils methods above to get the list)steps: the number of steps (int)scheduler_name: the scheduler to use (str)DDIM,Euler,Euler a,LMS,DPM2 Karras,DPM2 a Karras,Heun,DDPM,UniPC,PNDM,DEI,DPM++ SDE,DPM++ 2S a,DPM++ 2M,DPM++ SDE Karras,DPM++ 2S a Karras,DPM++ 2M Karrascfg_scale: the cfg scale (float)width: the width of the output image (int)height: the height of the output image (int)seed: the seed to use, -1 for random (int)batch_size: the batch size (int)input_image: local path or url of the input image, orqrcodeimg2img only (str)denoising_strength: the denoising strength, img2img only (float)controlnet_units: the controlnet units to use The unit name (tile, brightness, in above exemple) is used for better readability and does not impact the generationmodel: the controlnet model to use (str)for diffusers, you can use the model name or local pathfor automatic1111, you should choose from the available webui controlnet modelsmodule: the controlnet module to use (str)for diffusers, not available yetfor automatic1111, you should choose from the available webui controlnet modulescn_input_image: (str) can bepath or url of the input image to use for the controlnetqrcodeto use the qrcode as input imageweight: the weight of the controlnet (float)start: when the controlnet starts applying, in fract of total steps (ex: 0.35 means "start after 35% of total steps are done") (float)end: when the controlnet stops applying, in fract of total steps (ex: 0.7 means "end after 70% of total steps are done") (float)qrcode: the qrcode parameterstext: the text to encode (str)error_correction: the error correction level (str)box_size: the box size (int)border: the border size (int)fill_color: the fill color (str)back_color: the back color (str)Available configs:defaultThis method seem to be the best for me, I use it with the modelrealistic_visionV2.
It usesControlnet BrightnessandControlnet TileHere are my firsts thoughts:CN brightness should be left as isYou can play with CN tile parameters to get an image more or less "grid like"Controlnet modelsThere are multiple controlnet models that can be used:Controlnet Tile: This CN takes an image as input and guide the generation toward this image and to increase the details. We can use a qr code as input.Controlnet Brightness:https://huggingface.co/ioclab/control_v1p_sd15_brightnesshttps://huggingface.co/ViscoseBean/control_v1p_sd15_brightnessControlnet QR codehttps://huggingface.co/DionTimmer/controlnet_qrcode-control_v1p_sd15https://huggingface.co/DionTimmer/controlnet_qrcode-control_v11p_sd21https://huggingface.co/models?search=qrcodeTodosadd img2img for diffusersallow to set the sampler (diffusers)allow to set the seed (diffusers)allow to set the model in the config (auto)add more configsallow to set the config without having the file in local pathmore teststry to install the webui in demo colabadd diffusers backendadd docsallow to change modelsContribPlease don't hesitate to submit a PR to improve the code or submit a configOther projectsYou can checkoutour websiteto discover more of our projects such as:Seg2Sat: Controlnet model to generate aerial picturesPoetGPT: generate beautiful poems and lyrics with AIThisSCPDoesNotExist: generate custom SCP entities