Running with diffusers?
Is there a simple example of how to use this with SDXL and diffusers? Here is some starter code (that does not work) for img2img with turbo:
from diffusers import AutoPipelineForImage2Image
from diffusers.utils import load_image
import torch
SDXL_PATH = "./sdxl-turbo/"
LORA_PATH = "./layer_xl_transparent_attn.safetensors"
pipe = AutoPipelineForImage2Image.from_pretrained(SDXL_PATH, torch_dtype=torch.float16, variant="fp16")
pipe.load_lora_weights(LORA_PATH)
pipe.to("cuda")
init_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png").resize((512, 512))
prompt = "cat wizard, detailed, fantasy, cute, adorable, 8k"
image = pipe(prompt, image=init_image, num_inference_steps=2, strength=0.5, guidance_scale=0.0).images[0]
SDXL_PATH = "./sdxl-turbo/" is stabilityai/sdxl-turbo ???
yes via git clone https://huggingface.co/stabilityai/sdxl-turbo
yes via
git clone https://huggingface.co/stabilityai/sdxl-turbo
error:
File "/data/miniconda3/envs/ai_outside/lib/python3.10/site-packages/diffusers/loaders/lora.py", line 1250, in load_lora_weights
self.load_lora_into_unet(
File "/data/miniconda3/envs/ai_outside/lib/python3.10/site-packages/diffusers/loaders/lora.py", line 453, in load_lora_into_unet
lora_config_kwargs = get_peft_kwargs(rank, network_alphas, state_dict, is_unet=True)
File "/data/miniconda3/envs/ai_outside/lib/python3.10/site-packages/diffusers/utils/peft_utils.py", line 144, in get_peft_kwargs
r = lora_alpha = list(rank_dict.values())[0]
IndexError: list index out of range