okingjo commited on
Commit
9de44f3
1 Parent(s): 4ccbf0e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +45 -0
README.md CHANGED
@@ -1,3 +1,48 @@
1
  ---
2
  license: creativeml-openrail-m
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: creativeml-openrail-m
3
  ---
4
+ # Okingjo's Multi-identifier LORAs
5
+
6
+ I will share most of my LORA model with multiple identifier here. by saying "multiple", one charater with two or more costumes are stored within the model.
7
+ Not only will the LORA model will be post here, training setups and tips will also be shared.
8
+ I`m still in the state of learning, so any comments/feedbacks are welcom!
9
+
10
+ # some tips during trainig
11
+ ## general idea
12
+ The way to have multiple identifiers within one Lora is by using captioning. Since most auto captioning of an anime character starts with "1girl/boy", the second prompt will be used as the triggering word, i.e. the prompt to let the AI to excite the desired costume.
13
+ Captioning the images properly and put them into different folders respectively, just as the Kohya_SS SD trainier docs says, and you are good to go.
14
+ ## captioning tips
15
+ I`m trying to find a way to optimize the captioning of identifiers. I`ll write down some of my findings here as a record.
16
+ * It looks like that the more different the trigger prompts are, the better. The exception are my first multi-id Lora "ningguang"
17
+ * The format of promt are different in SD webUI and the captioning file, e.g. the " "and "_". Currently I`m using the webUI format in captioning, and so far so good.
18
+
19
+ # Charater from Genshin
20
+ ## Ningguang/凝光
21
+ ### Brief intro
22
+ LORA of Ningguang, with two costumes in game. civitAI page [Download](https://civitai.com/models/8546/ningguang)
23
+
24
+ ### Training dataset
25
+ #### Default costume
26
+ 72 images in total, in folder "30_Ningguang"
27
+ * 36 normal illustrations
28
+ * 15 normal 360 3D model snapshots
29
+ * 5 nude illustrations
30
+ * 16 nude 360 3D model snapshots
31
+ #### Orchid costume
32
+ 43 images in total, in folder "20_NingguangOrchid"
33
+ * 11 normal illustrations
34
+ * 15 normal 360 3D model snapshots
35
+ * 2 nude illustrations
36
+ * 15 nude 360 3D model snapshots
37
+
38
+ ### Captioning
39
+ WD14 captioning instead of the deepdanbooru caption was used, since the former one will not crop/resize the images. Threshold are usually set to 0.75-0.8. since I don't like to have a very long and sometimes inaccurate caption for my training data. After captionin is done, I added "ningguang \ \(genshin impact\ \)" after "1girl" to every caption file of the default costume, and "ningguang \ \(orchid's evening gown\ \) \ \(genshin impact\ \)" to the orchid costume. Some of the caption files were empty so I have to manually type the words.
40
+
41
+ ### Training setup
42
+ Trained with Kohya_SS stable diffusion trainer Base model was [Anything V3.0 full](https://huggingface.co/Linaqruf/anything-v3.0/blob/main/anything-v3-fp32-pruned.safetensors) Trainig process consist of two phases. The first one with default parameters of:
43
+ * learning_rate: 0.0001
44
+ * text_encoder_lr: 5e-5
45
+ * unet_lr: 0.0001 and 6 epoch,
46
+ After phase 1, choose the one with the best result (a little bit underfitting, no over fitting, and the two costume are seperated), which is the 6th one. Then trained with 1/10 of the original LR for another 7 epochs.
47
+
48
+ v