Package: onediffusion
🖼️ OneDiffusionOneDiffusion is an open-source one-stop shop for facilitating the deployment of any diffusion models in production. It caters specifically to the needs of diffusion models, supporting both pretrained and fine-tuned diffusion models with LoRA adapters.Key features include:🌐Broad compatibility: Support both pretrained and LoRA-adapted diffusion models, providing flexibility in choosing and deploying the appropriate model for various image generation tasks.💪Optimized performance and scalability: Automatically select the best optimizations like half-precision weights or xFormers to achieve best inference speed out of the box.⌛️Dynamic LoRA adapter loading: Dynamically load and unload LoRA adapters on every request, providing greater adaptability and ensuring the models remain responsive to changing inputs and conditions.🍱First-class support for BentoML: Seamless integration with theBentoMLecosystem, allowing you to build Bentos and push them toBentoCloud.OneDiffusion is designed for AI application developers who require a robust and flexible platform for deploying diffusion models in production. The platform offers tools and features to fine-tune, serve, deploy, and monitor these models effectively, streamlining the end-to-end workflow for diffusion model deployment.Supported modelsCurrently, OneDiffusion supports the following models:Stable Diffusion v1.4, v1.5 and v2.0Stable Diffusion XL v1.0More models (for example, ControlNet and DeepFloyd IF) will be added soon.Get startedPrerequisitesYou have installed Python 3.8 (or later) andpip.Install OneDiffusionInstall OneDiffusion by usingpipas follows:pipinstallonediffusionTo verify the installation, run:$onediffusion-h

Usage:onediffusion[OPTIONS]COMMAND[ARGS]...██████╗███╗██╗███████╗██████╗██╗███████╗███████╗██╗██╗███████╗██╗██████╗███╗██╗██╔═══██╗████╗██║██╔════╝██╔══██╗██║██╔════╝██╔════╝██║██║██╔════╝██║██╔═══██╗████╗██║██║██║██╔██╗██║█████╗██║██║██║█████╗█████╗██║██║███████╗██║██║██║██╔██╗██║██║██║██║╚██╗██║██╔══╝██║██║██║██╔══╝██╔══╝██║██║╚════██║██║██║██║██║╚██╗██║╚██████╔╝██║╚████║███████╗██████╔╝██║██║██║╚██████╔╝███████║██║╚██████╔╝██║╚████║╚═════╝╚═╝╚═══╝╚══════╝╚═════╝╚═╝╚═╝╚═╝╚═════╝╚══════╝╚═╝╚═════╝╚═╝╚═══╝Anopenplatformforoperatingdiffusionmodelsinproduction.Fine-tune,serve,deploy,andmonitoranydiffusionmodelswithease.Options:-v,--versionShowtheversionandexit.-h,--helpShowthismessageandexit.

Commands:buildPackageagivenmodelintoaBento.downloadSetupdiffusionmodelsinteractively.startStartanydiffusionmodelsasaRESTserver.Start a diffusion serverOneDiffusion allows you to quickly spin up any diffusion models. To start a server, run:onediffusionstartstable-diffusionThis starts a server athttp://0.0.0.0:3000/. You can interact with it by visiting the web UI or send a request viacurl.curl-X'POST'\'http://0.0.0.0:3000/text2img'\-H'accept: image/jpeg'\-H'Content-Type: application/json'\--outputoutput.jpg\-d'{"prompt": "a bento box","negative_prompt": null,"height": 768,"width": 768,"num_inference_steps": 50,"guidance_scale": 7.5,"eta": 0}'By default, OneDiffusion usesstabilityai/stable-diffusion-2to start the server. To use a specific model version, add the--model-idoption as below:onediffusionstartstable-diffusion--model-idrunwayml/stable-diffusion-v1-5To specify another pipeline, use the--pipelineoption as below. Theimg2imgpipeline allows you to modify images based on a given prompt and image.onediffusionstartstable-diffusion--pipeline"img2img"OneDiffusion downloads the models to the BentoML local Model Store if they have not been registered before. To view your models, install BentoML first withpip install bentomland then run:$bentomlmodelslist

TagModuleSizeCreationTime
pt-sd-stabilityai--stable-diffusion-2:1e128c8891e52218b74cde8f26dbfc701cb99d79bentoml.diffusers4.81GiB2023-08-1617:52:33
pt-sdxl-stabilityai--stable-diffusion-xl-base-1.0:bf714989e22c57ddc1c453bf74dab4521acb81d8bentoml.diffusers13.24GiB2023-08-1616:09:01Start a Stable Diffusion XL serverOneDiffusion also supports running Stable Diffusion XL 1.0, the most advanced development in the Stable Diffusion text-to-image suite of models launched by Stability AI. To start an XL server, simply run:onediffusionstartstable-diffusion-xlIt downloads the model automatically if it does not exist locally. Options such as--model-idare also supported. For more information, runonediffusion start stable-diffusion-xl --help.Similarly, visithttp://0.0.0.0:3000/or send a request viacurlto interact with the XL server. Example prompt:{
  "prompt": "the scene is a picturesque environment with beautiful flowers and trees. In the center, there is a small cat. The cat is shown with its chin being scratched. It is crouched down peacefully. The cat's eyes are filled with excitement and satisfaction as it uses its small paws to hold onto the food, emitting a content purring sound.",
  "negative_prompt": null,
  "height": 1024,
  "width": 1024,
  "num_inference_steps": 50,
  "guidance_scale": 7.5,
  "eta": 0
}Example output:Add LoRA weightsLow-Rank Adaptation (LoRA) is a training method to fine-tune models without the need to retrain all parameters. You can add LoRA weights to your diffusion models for specific data needs.Add the--lora-weightsoption as below:onediffusionstartstable-diffusion-xl--lora-weights"/path/to/lora-weights.safetensors"Alternatively, dynamically load LoRA weights by adding thelora_weightsfield:{
  "prompt": "the scene is a picturesque environment with beautiful flowers and trees. In the center, there is a small cat. The cat is shown with its chin being scratched. It is crouched down peacefully. The cat's eyes are filled with excitement and satisfaction as it uses its small paws to hold onto the food, emitting a content purring sound.",
  "negative_prompt": null,
  "height": 1024,
  "width": 1024,
  "num_inference_steps": 50,
  "guidance_scale": 7.5,
  "eta": 0,
  "lora_weights": "/path/to/lora-weights.safetensors"
}By specifying the path of LoRA weights at runtime, you can influence model outputs dynamically. Even with identical prompts, the application of different LoRA weights can yield vastly different results. Example output (oil painting vs. pixel):Download a modelIf you want to download a diffusion model without starting a server, use theonediffusion downloadcommand. For example:onediffusion download stable-diffusion --model-id "CompVis/stable-diffusion-v1-4"Create a BentoML RunnerYou can create a BentoML Runner withdiffusers_simple.stable_diffusion.create_runner(), which downloads the model specified automatically if it does not exist locally.importbentoml# Create a Runner for a Stable Diffusion modelrunner=bentoml.diffusers_simple.stable_diffusion.create_runner("CompVis/stable-diffusion-v1-4")# Create a Runner for a Stable Diffusion XL modelrunner_xl=bentoml.diffusers_simple.stable_diffusion_xl.create_runner("stabilityai/stable-diffusion-xl-base-1.0")You can then wrap the Runner into a BentoML Service. See theBentoML documentationfor more details.Build a BentoABentoin BentoML is a deployable artifact with all the source code, models, data files, and dependency configurations. You can build a Bento for a supported diffusion model directly by runningonediffusion build.# Build a Bento with a Stable Diffusion modelonediffusionbuildstable-diffusion# Build a Bento with a Stable Diffusion XL modelonediffusionbuildstable-diffusion-xlTo specify the model to be packaged into the Bento, use--model-id. Otherwise, OneDiffusion packages the default model into the Bento. If the model does not exist locally, OneDiffusion downloads the model automatically. In addition, the pipeline to use can also be specified through--pipeline. By default, OneDiffusion uses thetext2imagepipeline.To package LoRA weights into the Bento, use the--lora-diroption to specify the directory where LoRA files are stored. These files can be dynamically loaded to the model when deployed with Docker or BentoCloud to create task-specific images.onediffusionbuildstable-diffusion-xl--lora-dir"/path/to/lorafiles/dir/"If you only have a single LoRA file to use, run the following instead:onediffusionbuildstable-diffusion-xl--lora-weights"/path/to/lorafile"Each Bento has aBENTO_TAGcontaining both the Bento name and the version. To customize it, specify--nameand--versionoptions.onediffusionbuildstable-diffusion-xl--namesdxl--versionv1Once your Bento is ready, log in toBentoCloudand run the following command to push the Bento.bentomlpushBENTO_TAGAlternatively, create a Docker image by containerizing the Bento with the following command. You can retrieve theBENTO_TAGby runningbentoml list.bentomlcontainerizeBENTO_TAGYou can then deploy the image to any Docker-compatible environments.RoadmapWe are working to improve OneDiffusion in the following ways and invite anyone who is interested in the project to participate 🤝.Support more models, such as ControlNet and DeepFloyd IFSupport more pipelines, such as inpaintingAdd a Python API client to interact with diffusion modelsImplement advanced optimization like AITemplateOffer a unified fine-tuning training APIContributionWe weclome contributions of all kinds to the OneDiffusion project! Check out the following resources to start your OneDiffusion journey and stay tuned for more announcements about OneDiffusion and BentoML.Submit a pull request or create an issue in theOneDiffusion GitHub repository.Join theBentoML community on Slack.Follow us onTwitterandLinkedin.