Package: sdkit
sdkitsdkit(stablediffusionkit) is an easy-to-use library for using Stable Diffusion in your AI Art projects. It is fast, feature-packed, and memory-efficient.New: Stable Diffusion XL, ControlNets, LoRAs and Embeddings are now supported!This is a community project, so please feel free to contribute (and use in your project)!Why?The goal is to let you be productive quickly (at your AI art project), so it bundles Stable Diffusion along with commonly-used features (like ControlNets, LoRAs, Textual Inversion Embeddings, GFPGAN and CodeFormer for face restoration, RealESRGAN for upscaling, k-samplers, support for loading custom VAEs, NSFW filter etc).Advanced features include a model-downloader (with a database of commonly used models), support for running in parallel on multiple GPUs, auto-scanning for malicious models etc.Full list of featuresInstallationTested with Python 3.8. Supports Windows, Linux and Mac.Windows/Linux:pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu116Runpip install sdkitMac:Runpip install sdkitExampleLocal modelA simple example for generating an image from a Stable Diffusion model file (already present on the disk):importsdkitfromsdkit.modelsimportload_modelfromsdkit.generateimportgenerate_imagesfromsdkit.utilsimportlogcontext=sdkit.Context()# set the path to the model file on the disk (.ckpt or .safetensors file)context.model_paths['stable-diffusion']='D:\\path\\to\\512-base-ema.ckpt'load_model(context,'stable-diffusion')# generate the imageimages=generate_images(context,prompt='Photograph of an astronaut riding a horse',seed=42,width=512,height=512)# save the imageimages[0].save("image.png")# images is a list of PIL.Imagelog.info("Generated images!")Auto-download a known modelA simple example for automatically downloading a known Stable Diffusion model file:importsdkitfromsdkit.modelsimportdownload_models,resolve_downloaded_model_path,load_modelfromsdkit.generateimportgenerate_imagesfromsdkit.utilsimportsave_imagescontext=sdkit.Context()download_models(context,models={'stable-diffusion':'1.5-pruned-emaonly'})# downloads the known "SD 1.5-pruned-emaonly" modelcontext.model_paths['stable-diffusion']=resolve_downloaded_model_path(context,'stable-diffusion','1.5-pruned-emaonly')load_model(context,'stable-diffusion')images=generate_images(context,prompt='Photograph of an astronaut riding a horse',seed=42,width=512,height=512)save_images(images,dir_path='D:\\path\\to\\images\\directory')Please see the list ofexamples, to learn how to use the other features (like filters, VAE, ControlNet, Embeddings, LoRA, memory optimizations, running on multiple GPUs etc).APIPlease see theAPI Referencepage for a detailed summary.Broadly, the API contains 5 modules:sdkit.models# load/unloading models, downloading known models, scanning modelssdkit.generate# generating imagessdkit.filter# face restoration, upscalingsdkit.train# model merge, and (in the future) more training methodssdkit.utilsAnd asdkit.Contextobject is passed around, which encapsulates the data related to the runtime (e.g.deviceandvram_optimizations) as well as references to the loaded model files and paths.Contextis a thread-local object.Models DBClick here to see thelist of known models.sdkit includes a database of known models and their configurations. This lets you download a known model with a single line of code. You can customize where it saves the downloaded model.Additionally, sdkit will attempt to automatically determine the configuration for a given model (when loading from disk). For e.g. if an SD 2.1 model is being loaded, sdkit will automatically know to usefp32forattn_precision. If an SD 2.0 v-type model is being loaded, sdkit will automatically know to use thev2-inference-v.yamlconfiguration. It does this by matching the quick-hash of the given model file, with the list of known quick-hashes.For models that don't match a known hash (e.g. custom models), or if you want to override the config file, you can set the path to the config file incontext.model_paths. e.g.context.model_paths['stable-diffusion'] = 'path/to/config.yaml'FAQDoes it have all the cool features?It has a lot of features! It was born out of a popular Stable Diffusion UI, splitting out the battle-tested core engine intosdkit.Features include:SD 2.1, SDXL, ControlNet, LoRAs, Embeddings, txt2img, img2img, inpainting, NSFW filter, multiple GPU support, Mac Support, GFPGAN and CodeFormer (fix faces), RealESRGAN (upscale), 16 samplers (including k-samplers and UniPC), custom VAE, low-memory optimizations, model merging, safetensor support, picklescan, etc.Click here to see the full list of features.📢 We're looking to add support forLycoris,AMD,Pix2Pix, andoutpainting. We'd love code contributions for these!Is it fast?It is pretty fast, and close to the fastest. For the same image,sdkittook 5.5 seconds, whileautomatic1111webui took 4.95 seconds. 📢 We're looking for code contributions to makesdkiteven faster!xformersis supported experimentally, which will makesdkiteven faster.Details of the benchmark:Windows 11, NVIDIA 3060 12GB, 512x512 image, sd-v1-4.ckpt, euler_a sampler, number of steps: 25, seed 42, guidance scale 7.5.No xformers. No VRAM optimizations for low-memory usage.Time takenIterations/secPeak VRAM usagesdkit5.5 sec6.0 it/s5.1 GBautomatic1111webui4.95 sec6.15 it/s5.1 GBDoes it work on lower-end GPUs, or without GPUs?Yes. It works on NVIDIA/Mac GPUs with atleast 2GB of VRAM. For PCs without a compatible GPU, it can run entirely on the CPU. Running on the CPU will beveryslow, but atleast you'll be able to try it out!📢 We don't support AMD yet on Windows (it'll run in CPU-mode, or in Linux), but we're looking for code contributions for AMD support!Why not just use diffusers?You can certainly use diffusers.sdkitis infact usingdiffusersinternally, so you can think ofsdkitas a convenient API and a collection of tools, focused on Stable Diffusion projects.sdkit:is a simple, lightweight toolkit for Stable Diffusion projects.natively includes frequently-used projects like GFPGAN, CodeFormer and RealESRGAN.works with the popular.ckptand.safetensorsmodel format.includes memory optimizations for low-end GPUs.built-in support for running on multiple GPUs.can download models from any server.auto-scans for malicious models.includes 16 samplers (including k-samplers).born out of the needs of the new Stable Diffusion AI Art scene, starting Aug 2022.Who is using sdkit?Easy Diffusion (cmdr2 UI)for Stable Diffusion.Arthemy AIIf your project is using sdkit, you can add it to this list. Please feel free to open a pull request (or let us know at ourDiscord community).ContributingWe'd love to accept code contributions. Please feel free to drop by ourDiscord community!📢 We're looking for code contributions for these features (or anything else you'd like to work on):Lycoris.Outpainting.Pix2Pix.AMD support.If you'd like to set up a developer version on your PC (to contribute code changes), please followthese instructions.Instructions for running automated tests:Running Tests.CreditsStable Diffusion:https://github.com/Stability-AI/stablediffusionCodeFormer:https://github.com/sczhou/CodeFormer(license:https://github.com/sczhou/CodeFormer/blob/master/LICENSE)GFPGAN:https://github.com/TencentARC/GFPGANRealESRGAN:https://github.com/xinntao/Real-ESRGANk-diffusion:https://github.com/crowsonkb/k-diffusionCode contributors and artists on Easy Diffusion (cmdr2 UI):https://github.com/easydiffusion/easydiffusionand Discord (https://discord.com/invite/u9yhsFmEkB)Lots of contributors on the internetDisclaimerThe authors of this project are not responsible for any content generated using this project.The license of this software forbids you from sharing any content that:Violates any laws.Produces any harm to a person or persons.Disseminates (spreads) any personal information that would be meant for harm.Spreads misinformation.Target vulnerable groups.For the full list of restrictions please readthe License. You agree to these terms by using this software.