breakcore2 commited on
Commit
6d23e1e
1 Parent(s): 8ca92f0

add amber lora

Browse files
README.md CHANGED
@@ -1,3 +1,18 @@
1
  ---
2
  license: creativeml-openrail-m
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: creativeml-openrail-m
3
  ---
4
+
5
+ # Loras
6
+ You will need to install this extension https://github.com/kohya-ss/sd-webui-additional-networks in order to use it in the web ui. Follow the "How to use" section on that page.
7
+
8
+ ## Characters
9
+ ### Amber (Genshin Impact)
10
+ [Download .safetensors](characters\amber)
11
+
12
+ [Example Images and Prompts](https://gist.github.com/breakcore2/0b2fcae494c5f38a22ef59486ec55682)
13
+
14
+ Relevant full character prompt:
15
+ ```
16
+ amber \(genshin impact\), brown eyes, brown hair, long hair, red hair ribbon, brown short shorts, leather gloves, pouch, thighhighs, white boots, medium breasts, cleavage cutout, shrug \(clothing\)
17
+ ```
18
+ ![amber (genshin impact)](characters/amber/extras/example.png)
characters/amber/amber (genshin impact) flexible.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b1e7a42952dc0c3793fe52c8534779df5f50d0ae10f509c52c7febd6c9b2163
3
+ size 151073665
characters/amber/amber (genshin impact) strict.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c37a5fd40b6bb53182e65c33355a144cc16971f274c787d379e1a6533b16288b
3
+ size 151073665
characters/amber/extras/example.png ADDED
characters/amber/recommended prompts.txt ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ flexible for custom outfits:
2
+ amber \(genshin impact\), brown eyes, brown hair, long hair, red hair ribbon,
3
+ strict or flexible for original outfit:
4
+ amber \(genshin impact\), brown eyes, brown hair, long hair, red hair ribbon, brown short shorts, leather gloves, pouch, thighhighs, white boots, medium breasts, cleavage cutout, shrug \(clothing\)
5
+ flexible can further summon her goggles with:
6
+ (googgles on head:1.1) or (goggles around neck:1.1) if you want them, put them in negative if you don't (helps remove brown blobs around neck in flexible version).
characters/amber/training notes.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Training Parameters
2
+ ### flexible:
3
+ dataset construction information was lost. Likely resulted in around 600 to 1.4k steps.
4
+ ```
5
+ $learning_rate = 3e-5
6
+ $lr_warmup_ratio = 0.10
7
+ $train_batch_size = 4
8
+ $num_epochs = 1
9
+ $save_every_n_epochs=1
10
+ $scheduler="constant_with_warmup"
11
+ $network_dim=128
12
+ ```
13
+
14
+ ### strict:
15
+ Used three categories of images: full body original outfit, original outfight thigh up, alternative costume images.
16
+ Explicitly prune images that included goggles around the neck because previous training attempts resulted in models where a brown suggestion of a goggles would show up often even when not prompted for.
17
+ ```
18
+ amber full body: 10 repeats * 31 images = 310
19
+ good amber: 30 repeats * 7 images = 210
20
+ amber generic: 8 repeats * 35 images = 280
21
+
22
+ Total images with repeats: 800
23
+ Max training steps 800 / 4 * 5 = 1000
24
+ Warmup steps: 100, Batch size: 4, Epochs: 5
25
+ last epoch was chosen for this model.
26
+ ```
27
+ ```
28
+ $learning_rate = 7e-5
29
+ $lr_warmup_ratio = 0.10
30
+ $train_batch_size = 4
31
+ $num_epochs = 5
32
+ $save_every_n_epochs=1
33
+ $scheduler="cosine_with_restarts"
34
+ $network_dim=128
35
+ ```
36
+
37
+ ## Points of failure:
38
+ Overfitting of the gold pattern on her stomach area was a common issue accross all models when trying to train enough to capture original outfit.
39
+ Underfitting of the shoes was also common. Usually the gold trim is wrong, the red inner sock portion would be missing or the shoe would be entirely red or brown or the wrong length.
40
+
41
+ Goggles affected image quality a lot because they would leave traces of brown smudges in some generations when it was not specified. This is likely fixable with longer training period, more examples and lower training rates. For the strict model, we chose to remove images that included goggles around the neck to avoid it.
42
+
43
+ ## Considerations for the future:
44
+ Consider crops for the legs so it can learn the shoes better since the details around that is one of the things it gets wrong the most. Quite often we will reach a point where all the other details are trained in well but the shoes will come out the wrong color (likely because the word boot is associated with brown boots).
45
+
46
+ Many art pieces do not agree on the finer details of the gold trim of the boots and thus the model cannot learn something accurate. One possible solution is to generate many images including the boots and chosing the ones with a more correct pattern to be used as training images for future revisions.
47
+
48
+ ## Learnings:
49
+ You can very quickly learn a character's likeness by overfitting with a high training rate like 1e-4 and training only on original outfit images. As little as 200 steps with batch size 4 was enough.
50
+ Learning rates below 2e-5 were ineffective at learning all the required details (if your goal is to finish training in under 30 minutes).
51
+ Learning rate scheduler is not something really worth changing other than for using cosine with restarts and many epochs. It is usually better to just change learning rate or adjust the training data.