File size: 2,115 Bytes
100e821 df5839e 100e821 df5839e b9090f5 7f3f245 b9090f5 7f3f245 b9090f5 7f3f245 b9090f5 df5839e 7f3f245 b9090f5 7f3f245 b9090f5 7f3f245 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
license: mit
base_model: stabilityai/stable-diffusion-xl-base-1.0
tags:
- stable-diffusion
- stable-diffusion-diffusers
- text-to-image
- diffusers
- lora
inference: true
---
# sdxl-wrong-lora
A LoRA for SDXL 1.0 Base which improves output image quality after loading it and using `wrong` as a negative prompt during inference.
Benefits of using this LoRA:
- Higher color saturation and vibrance
- Higher detail in textures/fabrics
- Higher sharpness for blurry/background objects
- Better at anatomically-correct hands
- Less likely to have random artifacts
- Appears to allow the model to follow the input prompt with a more expected behavior
## Usage
The LoRA can be loaded using `load_lora_weights` like any other LoRA in `diffusers`:
```py
import torch
from diffusers import DiffusionPipeline, AutoencoderKL
vae = AutoencoderKL.from_pretrained(
"madebyollin/sdxl-vae-fp16-fix",
torch_dtype=torch.float16
)
base = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
vae=vae,
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True
)
base.load_lora_weights("minimaxir/sdxl-wrong-lora")
_ = base.to("cuda")
```
During inference, use `wrong` as the negative prompt.
## Examples
Left is the base model output (no LoRA) + refiner, right is base + LoRA and refiner. The generations use the same seed.
## Methodology
The methodology and motivation for creating this LoRA is similar to my [wrong SD 2.0 textual inversion embedding](https://huggingface.co/minimaxir/wrong_embedding_sd_2_0) by training on a balanced variety of undesirable outputs, except trained as a LoRA since textual inversion on SDXL is complicated. The base images were generated from SDXL itself, with some prompt weighting to emphasize undesirable attributes for test images.
## Notes
- The intuitive way to think about how this LoRA works
- It's possible to use `not wrong` in the prompt itself but in testing it has no effect.
- You can use other negative prompts in conjunction with the `wrong` prompt but you may want to weight them appropriately.
|