File size: 8,579 Bytes

---
language: '-en'
license: other
tags:
- stable-diffusion
- text-to-image
- safetensors
datasets: nakayama/DeDeDeDataset
---
# ご利用の際は下記のライセンス内容を十分にご確認ください。
HonyararaMixはアニメ調の人物を出力しやすいように調整されたStable Diffusion 2.1 解像度768対応版モデルです。    
ベースモデルの[Illuminati Diffusion 1.0](https://huggingface.co/IlluminatiAI/Illuminati_Diffusion_v1.0)へ[Waifu Diffusion 1.5 Beta](https://huggingface.co/waifu-diffusion/wd-1-5-beta) 、[Waifu Diffusion 1.5 Beta2](https://huggingface.co/waifu-diffusion/wd-1-5-beta2)、[Replicant-V1.0](https://huggingface.co/gsdf/Replicant-V1.0)、[Stable Diffusion 2.1](https://huggingface.co/stabilityai/stable-diffusion-2-1)を用いマージしたものをベースに    
[DeDeDeDataset](https://huggingface.co/datasets/nakayama/DeDeDeDataset)による学習などを行い調整したものです。

旧HonyararaMixにはライセンス上の問題があったため、それらを解決したモデルになります。

Prompt/Bad Promptについては、ベースモデルとなった二種のモデルが用意した文書の内容、およびTI Embeddingがこのモデルでも作用します。    
WDについては[こちら](https://cafeai.notion.site/WD-1-5-232cfa9443fc4661b469e7d1df8b7dd8)を参照。Illuminati DiffusionについてはNfixerが割とよく効く印象です。

# 例
<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image01.png" style="max-width:1000px;" width="75%"/>

```
best quality, masterpiece, great nature and secret garden with anime 1girl with twintail green hair and green eye and beautiful iris and (((from behind)))
Negative prompt: nfixer,3d, flat shading, flat shadow, bad anatomy, bad hands, missing fingers, extra digit, fewer digits looking at viewer
Steps: 28, Sampler: DDIM, CFG scale: 9, Seed: 43662843, Size: 768x512, Denoising strength: 0.7, Clip skip: 2, ENSD: 31337, Hires upscale: 1.5, Hires steps: 25, Hires upscaler: Latent
```

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image02.png" style="max-width:1000px;" width="75%"/>

```
newest, masterpiece, exceptional, anime game cg of cute cute cute 1girl sitting on a bench in harajuku park with detailed wavy pink hair pink and detailed yellow hair
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 28, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 3241266003, Size: 768x1152, Clip skip: 2, ENSD: 31337
```

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image03.png" style="max-width:1000px;" width="75%"/>

```
newest, masterpiece, exceptional, anime game cg of kawaii samurai 1girl, fighting form and (((intricate hands))), fist and dynamic angle with detailed pink hair and detailed yellow eye and beautiful iris in japanese background
Negative prompt: nfixer, 3d, flat shading, flat shadow, ((bad hands, missing fingers, extra digit, fewer digits))
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 3429333130, Size: 768x1152, Clip skip: 2, ENSD: 31337
```

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image04.png" style="max-width:1000px;" width="75%"/>

```
((masterpiece, best quality)),best aesthetic, 1girl, solo, black skirt, blue eyes, guitar, headphones, instrument, long hair, music, one side up, pink hair, playing guiter, pleated skirt, black shirt, indoors
Negative prompt: nfixer,missing finger, extra digits, fewer digits,((mutated hands and fingers)), 3d, flat shadow
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3730425302, Size: 1152x768, Clip skip: 2, ENSD: 31337
```

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image05.png" style="max-width:1000px;" width="75%"/>

```
best quality, masterpiece, exceptional, 1girl sitting in room ground wearing white school uniform and mini skirt and wavy detailed pink hair pink and detailed yellow eye yellow, large chest,(((look from viewer,smile)))
Negative prompt: nfixernext, 3d, flat shading, flat shadow
Steps: 25, Sampler: DDIM, CFG scale: 7, Seed: 2337880991, Size: 512x768, Denoising strength: 0.7, Clip skip: 2, ENSD: 31337, Hires upscale: 1.5, Hires steps: 25, Hires upscaler: Latent
```

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image06.png" style="max-width:1000px;" width="75%"/>

```
newest, masterpiece, exceptional, anime game cg of teenage 1boy wizard bust shot casting fire magic spell in New York City, detailed eye and beautiful iris,picturesque, golden hour, dynamic pose
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 28, Sampler: Euler a, CFG scale: 7.5, Seed: 1414427875, Size: 768x1152, Clip skip: 2, ENSD: 31337
```

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image07.png" style="max-width:1000px;" width="75%"/>

```
newest, masterpiece, exceptional, anime game cg of samurai old man, fighting pose, dynamic angle, fist with detailed pink hair and detailed yellow eye and beautiful iris in japanese background
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 28, Sampler: DDIM, CFG scale: 9, Seed: 403559970, Size: 768x1152, Clip skip: 2, ENSD: 31337
```

# マージ・学習手順について
1. Illuminati Diffusion 1.0にWaifu Diffusion 1.5 Beta1を差分マージする    
| Interpolation Method | Primary Model | Secondary Model | Tertiary Model | Merge Name |
| --- | --- | --- | --- | --- |
| Add Difference @ 0.6 | Illuminati Diffusion 1.0 | Waifu Diffusion 1.5 Beta1 | Stable Diffusion v2-1_768-ema-pruned | HonyararaMix_pre1 |

2. HonyararaMix_pre1に対し、[自前で用意した他Diffusion Modelの出力からなる素材画像](https://huggingface.co/datasets/nakayama/DeDeDeDataset)にて学習させる    
用意の際に利用したサービス/モデルは、SD2.1、Novel AI、WD1.3/1.4、CoolJapan Diffusion 2.1、Dreamlike Photoreal 2.0。    
使用したソフトは[kohya版Fine Tuning](https://github.com/kohya-ss/sd-scripts)。    
総数は30000程、flipしたものと合わせてlearning rateは5e-6、以下の設定で学習させ、    
試行錯誤の後にepoch 2相当のものをHonyararaMix_pre2とした。    

```
accelerate launch --num_cpu_threads_per_process 1 fine_tune.py
  --pretrained_model_name_or_path=Honyarara_pre1.safetensors
  --in_json meta_lat.json --train_data_dir=train_data
  --output_dir=fine_tuned
  --shuffle_caption
  --train_batch_size=4
  --learning_rate=5e-6
  --max_train_steps=60000
  --v2
  --v_parameterization
  --use_8bit_adam
  --xformers
  --mixed_precision=bf16
  --save_every_n_epochs=2
  --save_precision=float
  --max_token_length=150
  --seed=42
```

3. Honyarara_pre1にHonyarara_pre2を加重平均でマージする    
| Interpolation Method | Primary Model | Secondary Model | Merge Name |
| --- | --- | --- | --- |
| Weighted Sum @ 0.5 | HonyararaMix_pre1 | HonyararaMix_pre2 | HonyararaMix_pre3 |

4. HonyararaMix_pre3にIlluminati Diffusion 1.0を階層マージする    
使用マージ用スクリプトは[Super Merger](https://github.com/hako-mikan/sd-webui-supermerger)。    
| Model: A | Model: B | Weight | Merge Name |
| --- | --- | --- | --- |
| HonyararaMix_pre3 | Illuminati Diffusion 1.0 | 0.5,0.45,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0 | HonyararaMix_pre4|

5. HonyararaMix_pre4にWaifu Diffusion 1.5 Beta2 Aestheticを差分マージする    
| Interpolation Method | Primary Model | Secondary Model | Tertiary Model | Merge Name |
| --- | --- | --- | --- | --- |
| Add Difference @ 0.5 | HonyararaMix_pre4 | WD 1.5 Beta 2 Aesthetic | WD 1.5 Beta 2 | HonyararaMix_pre5 |

6. HonyararaMix_pre5にReplicant-V1.0を加重平均でマージする    
| Interpolation Method | Primary Model | Secondary Model | Merge Name |
| --- | --- | --- | --- |
| Weighted Sum @ 0.5 | HonyararaMix_pre5 | Replicant-V1.0 | HonyararaMix A |

7. HonyararaMix AにHonyararaMix_pre4を加重平均でマージする    
| Interpolation Method | Primary Model | Secondary Model | Merge Name |
| --- | --- | --- | --- |
| Weighted Sum @ 0.5 | HonyararaMix A | HonyararaMix_pre4 | HonyararaMix B |

# ライセンスについて

 Fair AI Public License 1.0-SD (https://freedevproject.org/faipl-1.0-sd/)