okingjo commited on
Commit
6bb896c
1 Parent(s): 4472870

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md CHANGED
@@ -7,6 +7,7 @@ I will share most of my LORA model with multiple identifier here. by saying "mul
7
  Not only will the LORA model will be post here, training setups and tips will also be shared.
8
  I`m still in the state of learning, so any comments/feedbacks are welcom!
9
 
 
10
  # some tips during trainig
11
  ## general idea
12
  The way to have multiple identifiers within one Lora is by using captioning. Since most auto captioning of an anime character starts with "1girl/boy", the second prompt will be used as the triggering word, i.e. the prompt to let the AI to excite the desired costume.
@@ -16,6 +17,7 @@ I`m trying to find a way to optimize the captioning of identifiers. I`ll write d
16
  * It looks like that the more different the trigger prompts are, the better. The exception are my first multi-id Lora "ningguang"
17
  * The format of promt are different in SD webUI and the captioning file, e.g. the " "and "_". Currently I`m using the webUI format in captioning, and so far so good.
18
 
 
19
  # Charater from Genshin
20
  ## Ningguang/凝光
21
  ### Brief intro
@@ -51,3 +53,40 @@ After phase 1, choose the one with the best result (a little bit underfitting, n
51
  ![sample3](https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/38d47c4a-6ba5-4925-5a56-e8701856a100/width=640)
52
  ![sample4](https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/b60aa7f4-6f63-46fb-381f-05b11f4afe00/width=640)
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  Not only will the LORA model will be post here, training setups and tips will also be shared.
8
  I`m still in the state of learning, so any comments/feedbacks are welcom!
9
 
10
+
11
  # some tips during trainig
12
  ## general idea
13
  The way to have multiple identifiers within one Lora is by using captioning. Since most auto captioning of an anime character starts with "1girl/boy", the second prompt will be used as the triggering word, i.e. the prompt to let the AI to excite the desired costume.
 
17
  * It looks like that the more different the trigger prompts are, the better. The exception are my first multi-id Lora "ningguang"
18
  * The format of promt are different in SD webUI and the captioning file, e.g. the " "and "_". Currently I`m using the webUI format in captioning, and so far so good.
19
 
20
+
21
  # Charater from Genshin
22
  ## Ningguang/凝光
23
  ### Brief intro
 
53
  ![sample3](https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/38d47c4a-6ba5-4925-5a56-e8701856a100/width=640)
54
  ![sample4](https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/b60aa7f4-6f63-46fb-381f-05b11f4afe00/width=640)
55
 
56
+
57
+ # Charater from Honkai impact
58
+ ## Elysia/爱莉希雅
59
+ ### Brief intro
60
+ LORA of Elysia, with 4 costumes in game. civitAI page [Download](https://civitai.com/models/14616)
61
+
62
+ ### Training dataset
63
+ #### Default costume/Miss Pink Elf
64
+ 70 images in total, in folder "14_Elysia (miss pink elf) 1girl"
65
+ * 20 normal illustrations, non-nude
66
+ * 30 normal 360 3D model snapshots
67
+ #### Default herrscher costume/herrscher of human:ego
68
+ 93 images in total, in folder "11_elysia (herrscher of humanego) 1girl"
69
+ * 33 normal illustrations, included few NSFW, not completely nude though
70
+ * 60 normal 360 3D model snapshots
71
+ #### Miad costume
72
+ 62 images in total, in folder “16_Elysia-maid 1girl”
73
+ * 33 normal illustrations, non-nude
74
+ * 30 normal 360 3D model snapshots
75
+ #### swimsuit
76
+ 74 images in total, in folder “14_elysia-swimsuit 1girl”
77
+ * 14 normal illustrations, non-nude
78
+ * 60 normal 360 3D model snapshots
79
+ In addition, I have also included 12 images with non-official costumes in a new folder "10_Elysia 1girl"
80
+
81
+ ### Captioning
82
+ WD14 captioning instead of the deepdanbooru caption was used, since the former one will not crop/resize the images. Threshold are usually set to 0.75-0.8. since I don't like to have a very long and sometimes inaccurate caption for my training data. After captionin is done, I added "elysia \\(miss pink elf\\) \\(honkai impact\\)", "elysia \\(herrscher of human:ego\\) \\(honkai impact\\)", "Elysia-maid", "Elysia-swimsuit" and "1girl, elysia \\(honkai impact\\)" to the captioning respectively as identifiers.
83
+
84
+ ### Training setup
85
+ Trained with Kohya_SS stable diffusion trainer Base model was [Anything V3.0 full](https://huggingface.co/Linaqruf/anything-v3.0/blob/main/anything-v3-fp32-pruned.safetensors) Trainig process consist of two phases. The first one with default parameters of:
86
+ * learning_rate: 0.0001
87
+ * text_encoder_lr: 5e-5
88
+ * unet_lr: 0.0001 and 4 epoch,
89
+ After phase 1, choose the one with the best result (a little bit underfitting, no over fitting, and the two costume are seperated), which is the 6th one. Then trained with 1/10 of the original LR for another 8 epochs.
90
+
91
+ ### Result
92
+ ![sample1](https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/9a699f92-b026-4efb-9714-6d6e2675f400/width=1280/174757)