File size: 4,323 Bytes
03049f3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
---
license: openrail
language:
- en
tags:
- stable-diffusion
- stable-diffusion-diffusers
- stable-diffusion-xl
- lora
- diffusers
base_model: stabilityai/stable-diffusion-xl-base-1.0
datasets:
- frank-chieng/chinese_architecture_siheyuan
library_name: diffusers
inference:
parameter:
negative_prompt:
widget:
- text: >-
siheyuan, chinese traditional architecture, perfectly shaded, morning lighting, medium closeup, mystical setting, during the day
example_title: example1 siheyuan
- text: >-
siheyuan, chinese modern architecture, perfectly shaded, night lighting, medium closeup, mystical setting, during the day
example_title: example2 siheyuan
pipeline_tag: text-to-image
---
## Overview
**Architecture Lora Chinese Style** is a lora training model with sdxl1.0 base model, latent text-to-image diffusion model. The model has been fine-tuned using a learning rate of `1e-5` over 3000 total steps with a batch size of 4 on a curated dataset of superior-quality chinese building style images. This model is derived from Stable Diffusion XL 1.0.
- Use it with 🧨 [`diffusers`](https://huggingface.co/docs/diffusers/index)
- Use it with the [`ComfyUI`](https://github.com/comfyanonymous/ComfyUI) **(recommended)**
-
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** [FrankChieng](https://github.com/frankchieng)
- **Model type:** Diffusion-based text-to-image generative model
- **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/LICENSE-MODEL)
- **Finetuned from model [optional]:** [Stable Diffusion XL 1.0 base](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
<hr>
## How to Use:
- Download `Lora model` [here](https://huggingface.co/frank-chieng/sdxl_lora_architecture_siheyuan/resolve/main/sdxl_lora_architecture_siheyuan.safetensors), the model is in `.safetensors` format.
- You need to use include siheyuan prompt in natural language, then you will get realistic result image
- You can use any generic negative prompt or use the following suggested negative prompt to guide the model towards high aesthetic generationse:
```
low quality, low resolution,watermark, mark, nsfw, lowres, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark
```
- And, the following should also be prepended to prompts to get high aesthetic results:
```
masterpiece, best quality
```
<hr>
## 🧨 Diffusers
Make sure to upgrade diffusers to >= 0.18.2:
```
pip install diffusers --upgrade
```
In addition make sure to install `transformers`, `safetensors`, `accelerate` as well as the invisible watermark:
```
pip install invisible_watermark transformers accelerate safetensors
```
Running the pipeline (if you don't swap the scheduler it will run with the default **EulerDiscreteScheduler** in this example we are swapping it to **EulerAncestralDiscreteScheduler**:
```py
pip install -q --upgrade diffusers invisible_watermark transformers accelerate safetensors
pip install huggingface_hub
from huggingface_hub import notebook_login
notebook_login()
import torch
from torch import autocast
from diffusers import StableDiffusionXLPipeline, EulerAncestralDiscreteScheduler
base_model_id = "stabilityai/stable-diffusion-xl-base-1.0"
lora_model = "frank-chieng/sdxl_lora_architecture_siheyuan"
pipe = StableDiffusionXLPipeline.from_pretrained(
base_model_id,
torch_dtype=torch.float16,
use_safetensors=True,
)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.load_lora_weights(lora_model, weight_name="sdxl_lora_architecture_siheyuan.safetensors")
pipe.to('cuda')
prompt = "siheyuan, chinese modern architecture, perfectly shaded, night lighting, medium closeup, mystical setting, during the day"
negative_prompt = "watermark"
image = pipe(
prompt,
negative_prompt=negative_prompt,
width=1024,
height=1024,
guidance_scale=7,
target_size=(1024,1024),
original_size=(4096,4096),
num_inference_steps=28
).images[0]
image.save("chinese_siheyuan.png")
```
<hr>
## Limitation
This model inherit Stable Diffusion XL 1.0 [limitation](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0#limitations) |