Create v2.1.md

shinymas/release_notes/v2.1.md

@@ -0,0 +1,201 @@
+# v2.1 & v2.0
+
+[Other versions](https://huggingface.co/gustproof/sd-models/tree/main/shinymas)
+
+A LoRA based on 966 Shiny Colors cards, recognizing the 26 characters to varying degrees.
+
+Resumed from [deresute-v1.3-epoch16](https://huggingface.co/gustproof/sd-models/blob/main/deresute/checkpoints/deresute-v1.3-000016.safetensors) by accident instead of training from scratch, 
+retaining surprisingly much of the knowledge of Cinderella Girls characters. 
+This suggests that it's likely the model's capacity is nowhere close to being reached.
+
+This also serves as a demo of an extremely inefficient style transfer method: train the characters, and then train on lots of unrelated styled images.
+
+## Usage
+The style itself doesn't have a trigger word. For the characters, below are the character tags and respective top 20 related tags.
+
+```
+Sakuragi Mano,
+brown eyes, smile, 1girl, open mouth, brown hair, looking at viewer, solo, bangs, light brown hair, outdoors, skirt, holding, medium hair, blush, shirt, dutch angle, breasts, hair ornament, multiple girls, blonde hair
+
+Kazano Hiori,
+black hair, mole, mole under mouth, long hair, looking at viewer, smile, 1girl, solo, bangs, open mouth, ponytail, purple eyes, multiple girls, blonde hair, skirt, dress, hair ornament, breasts, outdoors, holding
+
+Hachimiya Meguru,
+blonde hair, blue eyes, smile, long hair, looking at viewer, hair ornament, 1girl, open mouth, solo, breasts, ahoge, hairclip, twintails, multiple girls, shirt, outdoors, bangs, black hair, skirt, large breasts
+
+Osaki Tenka,
+long hair, brown hair, yellow eyes, bangs, looking at viewer, 1girl, hair between eyes, smile, solo, open mouth, holding, blush, long sleeves, dress, multiple girls, frills, brown eyes, braid, sisters, skirt
+
+Morino Rinze,
+red eyes, black hair, bangs, short hair, 1girl, looking at viewer, solo, smile, blue hair, long sleeves, hair ornament, holding, hair bun, outdoors, multiple girls, skirt, shirt, open mouth, flower, long hair
+
+Tsukioka Kogane,
+long hair, brown hair, bangs, looking at viewer, breasts, brown eyes, large breasts, 1girl, smile, solo, open mouth, blush, skirt, jewelry, bow, shirt, ribbon, dress, holding, outdoors
+
+Arisugawa Natsuha,
+long hair, smile, bangs, orange hair, looking at viewer, breasts, jewelry, 1girl, solo, earrings, black eyes, open mouth, red hair, shirt, multiple girls, skirt, brown hair, jacket, ahoge, outdoors
+
+Kuwayama Chiyuki,
+brown hair, long hair, smile, looking at viewer, brown eyes, bangs, breasts, 1girl, ahoge, solo, braid, holding, open mouth, jewelry, blush, dress, skirt, multiple girls, large breasts, shirt
+
+Osaki Amana,
+brown hair, long hair, yellow eyes, smile, bangs, looking at viewer, open mouth, 1girl, solo, jewelry, breasts, multiple girls, blush, frills, dress, holding, collarbone, sitting, skirt, bare shoulders
+
+Saijo Juri,
+blonde hair, short hair, purple eyes, 1girl, looking at viewer, bangs, solo, smile, jacket, jewelry, outdoors, shirt, shorts, holding, multiple girls, open mouth, dutch angle, skirt, breasts, brown hair
+
+Komiya Kaho,
+red hair, long hair, red eyes, smile, open mouth, bangs, looking at viewer, 1girl, solo, ahoge, jacket, skirt, multiple girls, shirt, holding, shorts, brown hair, outdoors, hair between eyes, gloves
+
+Tanaka Mamimi,
+purple hair, purple eyes, bangs, diagonal bangs, looking at viewer, jewelry, 1girl, twintails, solo, earrings, long hair, smile, nail polish, holding, shirt, skirt, choker, open mouth, breasts, jacket
+
+Yukoku Kiriko,
+long hair, grey hair, purple eyes, bangs, looking at viewer, twintails, dress, smile, 1girl, solo, multiple girls, open mouth, long sleeves, holding, frills, brown hair, breasts, shirt, black hair, bandaid
+
+Mitsumine Yuika,
+black hair, long hair, smile, looking at viewer, 1girl, blue eyes, solo, twintails, glasses, open mouth, hat, jacket, brown hair, skirt, bangs, jewelry, holding, outdoors, multiple girls, shirt
+
+Sonoda Chiyoko,
+brown hair, looking at viewer, twintails, 1girl, solo, red eyes, smile, bangs, hair bun, double bun, holding, skirt, food, long hair, open mouth, multiple girls, jewelry, shirt, breasts, one eye closed
+
+Serizawa Asahi,
+blue eyes, short hair, 1girl, bangs, looking at viewer, grey hair, solo, open mouth, outdoors, smile, jacket, shirt, skirt, shorts, long sleeves, jewelry, shoes, sky, holding, brown hair
+
+Shirase Sakuya,
+black hair, long hair, yellow eyes, smile, bangs, breasts, jewelry, ponytail, hair between eyes, large breasts, looking at viewer, 1girl, solo, high ponytail, cleavage, earrings, multiple girls, jacket, open mouth, necklace
+
+Izumi Mei,
+long hair, blonde hair, brown hair, multicolored hair, gradient hair, gyaru, black eyes, breasts, looking at viewer, jewelry, 1girl, earrings, large breasts, solo, smile, tan, cleavage, jacket, dark skin, multiple girls
+
+Mayuzumi Fuyuko,
+black hair, long hair, bangs, looking at viewer, brown eyes, blunt bangs, smile, 1girl, skirt, solo, two side up, open mouth, shirt, jewelry, breasts, long sleeves, multiple girls, multicolored hair, holding, sitting
+
+Asakura Toru,
+short hair, looking at viewer, 1girl, solo, jewelry, multicolored hair, gradient hair, bangs, blue hair, earrings, ahoge, brown hair, outdoors, jacket, hair ornament, dutch angle, blue eyes, dress, multiple girls, smile
+
+Higuchi Madoka,
+short hair, mole under eye, mole, brown hair, purple eyes, hair ornament, hairclip, 1girl, solo, looking at viewer, bangs, skirt, jacket, long sleeves, outdoors, multiple girls, black hair, shirt, nail polish, dutch angle
+
+Ichikawa Hinana,
+brown hair, looking at viewer, brown eyes, long hair, bangs, 1girl, solo, skirt, open mouth, hair bun, smile, ribbon, dutch angle, parted bangs, hair ribbon, dress, jewelry, hair ornament, single hair bun, shirt
+
+Fukumaru Koito,
+black hair, purple eyes, twintails, 1girl, looking at viewer, solo, fangs, open mouth, skin fangs, skirt, hair ornament, ribbon, shirt, indoors, short hair, dutch angle, hair ribbon, long sleeves, multiple girls, brown hair
+
+Nanakusa Nichika,
+looking at viewer, 1girl, brown hair, green hair, solo, short hair, smile, bangs, brown eyes, multiple girls, long hair, jewelry, open mouth, jacket, shirt, outdoors, holding, earrings, gloves, dress
+
+Aketa Mikoto,
+multicolored hair, brown hair, gradient hair, jewelry, looking at viewer, jacket, 1girl, blonde hair, smile, earrings, brown eyes, solo, multiple girls, 2girls, gloves, nail polish, green hair, shirt, dress, breasts
+
+Ikaruga Luca,
+1girl, multicolored hair, black hair, yellow eyes, looking at viewer, solo, blonde hair, bangs, skirt, streaked hair, midriff, crop top, two-tone hair, medium hair, jacket, gloves, shirt, fishnets, breasts, off shoulder
+```
+
+As a bonus, [Cinderella Girls](https://civitai.com/models/23476/) prompts mostly work as well.
+
+## Training info
+
+Training info
+Dataset:
+* 996 Shiny Colors cards collected from [シャニマスDB](https://imassc.gamedbs.jp/)
+
+Tags generated with [SmilingWolf/wd-v1-4-convnextv2-tagger-v2](https://huggingface.co/SmilingWolf/wd-v1-4-convnextv2-tagger-v2) with a threshold of 0.35. No balancing and tag cleaning. No distinction between Produce and Support cards.
+
+Inspired by [a 2022 paper](https://openreview.net/pdf?id=Uad23IcIEs), a custom LR scheduler was used:
+```
+SequentialLR(optimizer, [
+    LinearLR(optimizer, 0.1, total_iters=steps_per_epoch),
+    CosineAnnealingWarmRestarts(optimizer, steps_per_epoch*2, T_mult=2)
+], [steps_per_epoch])
+```
+
+Training cost: ~12 T4-hours (in addition to v1.1 and v1.2 costs)
+
+
+Data distribution:
+```
+            character  cards
+1       Sakuragi Mano     52
+2        Kazano Hiori     49
+3    Hachimiya Meguru     47
+4         Osaki Tenka     46
+5        Morino Rinze     45
+6     Tsukioka Kogane     44
+7   Arisugawa Natsuha     44
+8    Kuwayama Chiyuki     44
+9         Osaki Amana     43
+10         Saijo Juri     43
+11        Komiya Kaho     43
+12      Tanaka Mamimi     42
+13      Yukoku Kiriko     42
+14    Mitsumine Yuika     40
+15     Sonoda Chiyoko     40
+16     Serizawa Asahi     39
+17     Shirase Sakuya     38
+18          Izumi Mei     35
+19    Mayuzumi Fuyuko     34
+20       Asakura Toru     28
+21     Higuchi Madoka     28
+22    Ichikawa Hinana     26
+23     Fukumaru Koito     25
+24   Nanakusa Nichika     23
+25       Aketa Mikoto     22
+26       Ikaruga Luca      4
+```
+
+
+Training config (v2.0):
+```
+[model_arguments]
+v2 = false
+v_parameterization = false
+pretrained_model_name_or_path = "animefull-final-pruned.ckpt"
+
+[additional_network_arguments]
+no_metadata = false
+network_module = "lycoris.kohya"
+network_dim = 16
+network_alpha = 1
+network_args = [ "conv_dim=0", "algo=loha",]
+network_train_unet_only = false
+network_train_text_encoder_only = false
+cache_latents = false
+
+[optimizer_arguments]
+optimizer_type = "AdamW8bit"
+learning_rate = 0.001
+max_grad_norm = 1.0
+lr_scheduler = "custom"
+
+[dataset_arguments]
+debug_dataset = false
+# batch size: 4 x 2
+# resolution: 640
+
+[training_arguments]
+output_name = "shinymas-v2.0"
+save_precision = "fp16"
+save_every_n_epochs = 1
+max_token_length = 225
+mem_eff_attn = false
+xformers = true
+max_train_epochs = 31
+max_data_loader_n_workers = 0
+persistent_data_loader_workers = false
+gradient_checkpointing = false
+gradient_accumulation_steps = 1
+mixed_precision = "fp16"
+clip_skip = 2
+log_prefix = "shinymas-v2.0"
+lowram = true
+
+[sample_prompt_arguments]
+
+[saving_arguments]
+save_model_as = "safetensors"
+```
+
+v2.1 is tuned from v2.0 with resolution raised to 768, batch size and learning rate halved.
+