File size: 7,019 Bytes
2b754f2 29a790a 2b754f2 29a790a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 |
---
license: creativeml-openrail-m
tags:
- text-to-image
- stable-diffusion
- anime
- aiart
---
This model is trained on 33 different concepts from Bofuri: I Don't Want to Get Hurt, so I'll Max Out My Defense (防振り: 痛いのは嫌なので防御力に極振りしたいと思います。).
Here are some examples generations.
### Example Generations
Prompt: `BoMaple uniform BoSally unfirom, yuri, in classroom, 4K wallpaper, beautiful eyes`
Prompt: `2girls, BoMay BoYui, yuri, half body, floating in the sky, cloud, sparkling eyes, 4K wallpaer, anime coloring, official art`
Prompt: `BoKanade casting magic, 4K wallpaper, outdoors`
(Negative is mostly variations of: `bad hands, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry`)
### Usage
The model is shared in both diffuser safetensors format. Intermediatet checkpoints are also shared in ckpt format in the directory `checkpoints`.
### Concepts
The 33 concepts are listed in `concept_list` and demonstrated below.
Expect bad results for `BoMaple sheep form` and non-human concepts. Espeically the model clearly does not understand the anatomy of syrup.
For `BoKasumi sarashi` adding `bandages` seems to help. For `BoMaple pajama` you can add `stripe` for more similarity to the ones appearing in anime.
The remaining concepts should go through smoothly.
#### Prompt format
During training the concept names are put at the beginning of the image separated only by spaces, but not doing so seems to work as well.
Put `aniscreen` after the concept names would reinfoce the anime style.
Mixint two concepts is fairly doable as demonstrated above.
However expect weird blending to happen most of the time starting from three concepts.
This is partially because this model is not trained too much on multi-concept scenes.
Below is roughly the best we can get after multiply tries (there is still clothe blending).
Prompt: `(BoMaple black armor) BoSally turtleneck BoKasumi, 3girls, 4K wallpaper, ahoge, black hair, brown hair, outdoors, long hair`
### More Generations
Prompt: BoMaple black armors aniscreen, 1girl solo, Hydra in the sky, light purple eyes, 4K wallpaper
Prompt: BoMaple black armors near small turtle syrup, sitting with knees up on rock looking at viewer, turtle shell, beautiful hand in glove, in front of trees , outdoors, close-up, 4K wallpaper
Prompt: BoMaple pajama stripe, sitting on bed with barefoot, in girl's room, detailed and fancy background, sparkling purple eyes, hand on bed, 4K wallpaper
Prompt: BoFrederica, cowboy shot, in rubble ruins, ((under blue sky)), cinematic angle, dynamic pose, oblique angle, 4K wallpaer, anime coloring, official art
Prompt: Turtle Syrup Fox Oboro next to each other simple background white background, animals
Failures are of course unavoidable
Finally, you can always get different styles via model merging
### Dataset Description
The dataset is prepared via the workflow detailed here: https://github.com/cyber-meow/anime_screenshot_pipeline
It contains 27031 images with the following composition
- 7752 bofuri images mainly composed of screenshots from the first season and of the first three episods of the second season
- 19279 regularization images which intend to be as various as possible while being in anime style (i.e. no photorealistic image is used)
Note that the model is trained with a specific weighting scheme to balance between different concepts so that every image does not weight equally.
After applying the per-image repeat we get around 20 images per epoch.
### Training
Training is done with [EveryDream2](https://github.com/victorchall/EveryDream2trainer) trainer using [JosephusCheung/ACertainty](https://huggingface.co/JosephusCheung/ACertainty) as base model.
I use the following configuration thanks to the suggestion of 金Goldkoron
- resolution 512
- cosine learning rate scheduler, lr 2.5e-6
- batch size 4
- conditional dropout 0.05
- change beta scheduler from `scaler_linear` to `linear` in `config.json` of the scheduler of the model
The released model is trained for 57751 steps, but among the provided checkpoints all the three starting from 34172 steps seem to work reasonably well. |