library_name: diffusers
base_model: segmind/Segmind-Vega
tags:
- lora
- text-to-image
license: apache-2.0
inference: false
Segmind-VegaRT - Latent Consistency Model Segmind-Vega
Fused model by gfodor
Try real-time inference here VegaRT demo⚡
API for Segmind-VegaRT
Segmind-VegaRT a distilled consistency adapter for Segmind-Vega that allows to reduce the number of inference steps to only between 2 - 8 steps.
Latent Consistency Model (LCM) LoRA was proposed in LCM-LoRA: A universal Stable-Diffusion Acceleration Module by Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu et al.
Image comparison (Segmind-VegaRT vs SDXL-Turbo)
Speed comparison (Segmind-VegaRT vs SDXL-Turbo) on A100 80GB
Model | Params / M |
---|---|
lcm-lora-sdv1-5 | 67.5 |
Segmind-VegaRT | 119 |
lcm-lora-sdxl | 197 |
Usage
LCM-LoRA is supported in 🤗 Hugging Face Diffusers library from version v0.23.0 onwards. To run the model, first
install the latest version of the Diffusers library as well as peft
, accelerate
and transformers
.
audio dataset from the Hugging Face Hub:
pip install --upgrade pip
pip install --upgrade diffusers transformers accelerate peft
Text-to-Image
Let's load the base model segmind/Segmind-Vega
first. Next, the scheduler needs to be changed to LCMScheduler
and we can reduce the number of inference steps to just 2 to 8 steps.
Please make sure to either disable guidance_scale
or use values between 1.0 and 2.0.
import torch
from diffusers import LCMScheduler, AutoPipelineForText2Image
model_id = "segmind/Segmind-Vega"
adapter_id = "segmind/Segmind-VegaRT"
pipe = AutoPipelineForText2Image.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.to("cuda")
# load and fuse lcm lora
pipe.load_lora_weights(adapter_id)
pipe.fuse_lora()
prompt = "Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"
# disable guidance_scale by passing 0
image = pipe(prompt=prompt, num_inference_steps=4, guidance_scale=0).images[0]