File size: 2,441 Bytes
7c1d359 d859e69 7c1d359 d779908 7c1d359 b49a5ff 7c1d359 b49a5ff 7c1d359 d779908 7c1d359 d779908 7c1d359 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 |
---
base_model: stabilityai/stable-diffusion-xl-base-1.0
library_name: diffusers
license: cc0-1.0
tags:
- stable-diffusion-xl
- stable-diffusion-xl-diffusers
- text-to-image
- diffusers
- diffusers-training
- lora
inference: true
---
<!-- This model card has been generated automatically according to the information the training script had access to. You
should probably proofread and complete it, then remove this comment. -->
# LoRA fine-tuning - jonathandinu/sdxl-metamorphosis
These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on illustrations from [Maria Sibylla Merian’s Metamorphosis Insectorum Surinamensium (1705)](https://huggingface.co/datasets/jonathandinu/merian-metamorphosis).
LoRA for the text encoder was enabled: False.
Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.
## Intended uses & limitations
### How to use
#### text2img
```python
from diffusers import DiffusionPipeline, AutoencoderKL, utils
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", vae=vae, torch_dtype=torch.float16, variant="fp16")
pipeline.to("cuda")
pipeline.load_lora_weights("jonathandinu/sdxl-metamorphosis-lora", weight_name="pytorch_lora_weights.safetensors")
pipeline(
prompt="an astronaut in the jungle",
num_inference_steps=30,
generator=torch.manual_seed(1)
).images[0]
```
#### img2img
```python
from diffusers import AutoPipelineForImage2Image
url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/img2img-sdxl-init.png"
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
pipeline = AutoPipelineForImage2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", vae=vae, torch_dtype=torch.float16, variant="fp16")
pipeline.to("cuda")
pipeline.load_lora_weights("jonathandinu/sdxl-metamorphosis-lora", weight_name="pytorch_lora_weights.safetensors")
pipeline(
prompt="an astronaut in the jungle",
image=init_image,
num_inference_steps=30,
generator=torch.manual_seed(1),
strength=0.7
).images[0]
```
#### Limitations and bias
[TODO: provide examples of latent issues and potential remediations]
## Training details
[TODO: describe the data used to train the model] |