Commit
•
f2a2ae5
1
Parent(s):
7cac5a7
Delete README.md
Browse files
README.md
DELETED
@@ -1,129 +0,0 @@
|
|
1 |
-
---
|
2 |
-
inference: true
|
3 |
-
tags:
|
4 |
-
- stable-diffusion
|
5 |
-
- stable-diffusion-diffusers
|
6 |
-
- text-to-image
|
7 |
-
license: creativeml-openrail-m
|
8 |
-
---
|
9 |
-
|
10 |
-
## Please Note!
|
11 |
-
|
12 |
-
This model is NOT the 19.2M images Characters Model on TrinArt, but an improved version of the original Trin-sama Twitter bot model. This model is intended to retain the original SD's aesthetics as much as possible while nudging the model to anime/manga style.
|
13 |
-
|
14 |
-
Other TrinArt models can be found at:
|
15 |
-
|
16 |
-
https://huggingface.co/naclbit/trinart_derrida_characters_v2_stable_diffusion
|
17 |
-
|
18 |
-
https://huggingface.co/naclbit/trinart_characters_19.2m_stable_diffusion_v1
|
19 |
-
|
20 |
-
|
21 |
-
## Diffusers
|
22 |
-
|
23 |
-
The model has been ported to `diffusers` by [ayan4m1](https://huggingface.co/ayan4m1)
|
24 |
-
and can easily be run from one of the branches:
|
25 |
-
- `revision="diffusers-60k"` for the checkpoint trained on 60,000 steps,
|
26 |
-
- `revision="diffusers-95k"` for the checkpoint trained on 95,000 steps,
|
27 |
-
- `revision="diffusers-115k"` for the checkpoint trained on 115,000 steps.
|
28 |
-
|
29 |
-
For more information, please have a look at [the "Three flavors" section](#three-flavors).
|
30 |
-
|
31 |
-
## Gradio
|
32 |
-
|
33 |
-
We also support a [Gradio](https://github.com/gradio-app/gradio) web ui with diffusers to run inside a colab notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1RWvik_C7nViiR9bNsu3fvMR3STx6RvDx?usp=sharing)
|
34 |
-
|
35 |
-
|
36 |
-
### Example Text2Image
|
37 |
-
|
38 |
-
```python
|
39 |
-
# !pip install diffusers==0.3.0
|
40 |
-
from diffusers import StableDiffusionPipeline
|
41 |
-
|
42 |
-
# using the 60,000 steps checkpoint
|
43 |
-
pipe = StableDiffusionPipeline.from_pretrained("naclbit/trinart_stable_diffusion_v2", revision="diffusers-60k")
|
44 |
-
pipe.to("cuda")
|
45 |
-
|
46 |
-
image = pipe("A magical dragon flying in front of the Himalaya in manga style").images[0]
|
47 |
-
image
|
48 |
-
```
|
49 |
-
|
50 |
-
![dragon](https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/a_magical_dragon_himalaya.png)
|
51 |
-
|
52 |
-
If you want to run the pipeline faster or on a different hardware, please have a look at the [optimization docs](https://huggingface.co/docs/diffusers/optimization/fp16).
|
53 |
-
|
54 |
-
### Example Image2Image
|
55 |
-
|
56 |
-
```python
|
57 |
-
# !pip install diffusers==0.3.0
|
58 |
-
from diffusers import StableDiffusionImg2ImgPipeline
|
59 |
-
import requests
|
60 |
-
from PIL import Image
|
61 |
-
from io import BytesIO
|
62 |
-
|
63 |
-
url = "https://scitechdaily.com/images/Dog-Park.jpg"
|
64 |
-
|
65 |
-
response = requests.get(url)
|
66 |
-
init_image = Image.open(BytesIO(response.content)).convert("RGB")
|
67 |
-
init_image = init_image.resize((768, 512))
|
68 |
-
|
69 |
-
# using the 115,000 steps checkpoint
|
70 |
-
pipe = StableDiffusionImg2ImgPipeline.from_pretrained("naclbit/trinart_stable_diffusion_v2", revision="diffusers-115k")
|
71 |
-
pipe.to("cuda")
|
72 |
-
|
73 |
-
images = pipe(prompt="Manga drawing of Brad Pitt", init_image=init_image, strength=0.75, guidance_scale=7.5).images
|
74 |
-
image
|
75 |
-
```
|
76 |
-
|
77 |
-
If you want to run the pipeline faster or on a different hardware, please have a look at the [optimization docs](https://huggingface.co/docs/diffusers/optimization/fp16).
|
78 |
-
|
79 |
-
|
80 |
-
## Stable Diffusion TrinArt/Trin-sama AI finetune v2
|
81 |
-
|
82 |
-
trinart_stable_diffusion is a SD model finetuned by about 40,000 assorted high resolution manga/anime-style pictures for 8 epochs. This is the same model running on Twitter bot @trinsama (https://twitter.com/trinsama)
|
83 |
-
|
84 |
-
Twitterボット「とりんさまAI」@trinsama (https://twitter.com/trinsama) で使用しているSDのファインチューン済モデルです。一定のルールで選別された約4万枚のアニメ・マンガスタイルの高解像度画像を用いて約8エポックの訓練を行いました。
|
85 |
-
|
86 |
-
## Version 2
|
87 |
-
|
88 |
-
V2 checkpoint uses dropouts, 10,000 more images and a new tagging strategy and trained longer to improve results while retaining the original aesthetics.
|
89 |
-
|
90 |
-
バージョン2は画像を1万枚追加したほか、ドロップアウトの適用、タグ付けの改善とより長いトレーニング時間により、SDのスタイルを保ったまま出力内容の改善を目指しています。
|
91 |
-
|
92 |
-
## Three flavors
|
93 |
-
|
94 |
-
Step 115000/95000 checkpoints were trained further, but you may use step 60000 checkpoint instead if style nudging is too much.
|
95 |
-
|
96 |
-
ステップ115000/95000のチェックポイントでスタイルが変わりすぎると感じる場合は、ステップ60000のチェックポイントを使用してみてください。
|
97 |
-
|
98 |
-
#### img2img
|
99 |
-
|
100 |
-
If you want to run **latent-diffusion**'s stock ddim img2img script with this model, **use_ema** must be set to False.
|
101 |
-
|
102 |
-
**latent-diffusion** のscriptsフォルダに入っているddim img2imgをこのモデルで動かす場合、use_emaはFalseにする必要があります。
|
103 |
-
|
104 |
-
#### Hardware
|
105 |
-
|
106 |
-
- 8xNVIDIA A100 40GB
|
107 |
-
|
108 |
-
#### Training Info
|
109 |
-
|
110 |
-
- Custom dataset loader with augmentations: XFlip, center crop and aspect-ratio locked scaling
|
111 |
-
- LR: 1.0e-5
|
112 |
-
- 10% dropouts
|
113 |
-
|
114 |
-
#### Examples
|
115 |
-
|
116 |
-
Each images were diffused using K. Crowson's k-lms (from k-diffusion repo) method for 50 steps.
|
117 |
-
|
118 |
-
![examples](https://pbs.twimg.com/media/FbPO12-VUAAf2CJ?format=jpg&name=900x900)
|
119 |
-
![examples](https://pbs.twimg.com/media/FbPO65cUIAAga8k?format=jpg&name=900x900)
|
120 |
-
![examples](https://pbs.twimg.com/media/FbPO_QuVsAAG6xE?format=png&name=900x900)
|
121 |
-
|
122 |
-
#### Credits
|
123 |
-
|
124 |
-
- Sta, AI Novelist Dev (https://ai-novel.com/) @ Bit192, Inc.
|
125 |
-
- Stable Diffusion - Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bjorn
|
126 |
-
|
127 |
-
#### License
|
128 |
-
|
129 |
-
CreativeML OpenRAIL-M
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|