Create release_notes/v1.0.md

grimoire/release_notes/v1.0.md

@@ -0,0 +1,349 @@
+# v1.0
+
+[Other versions](https://huggingface.co/gustproof/sd-models/tree/main/grimoire)
+
+A LoRA based on 4267 [Grimoire A: Private Grimoire Magic Academy](https://grimoire.applibot.co.jp/) cards, recognizing 60+ characters to varying degrees.
+
+A request by Civit user themonky007. The model is far from complete but there's currently no plans to further improve it. [The dataset](https://huggingface.co/datasets/gustproof/sd-data/blob/main/grimoire_cards_notext_denoised.zip) is available for those willing to take on the task.
+
+Lesson of the day: suitable learning rates can differ significantly between datasets (this uses 2e-4 while the Idolmaster models use 1e-3).
+
+[An undertrained version (v0.1)](https://huggingface.co/gustproof/sd-models/blob/main/grimoire/checkpoints/grimoire-v0.1.safetensors) with card text is also available. Unlike v1.0, v0.1 uses Japanese names with spaces (e.g. `朝比奈 龍季`) as character tags. Interestingly, it can produce the correct Romaji initials for out-of-distribution characters quite reliably.
+
+## Usage
+The style itself doesn't have a trigger word. For the characters, below are the character tags and respective top 20 related tags.
+
+```
+Minami Tomoka,
+brown hair, short hair, blush, brown eyes, 1girl, ahoge, smile, looking at viewer, bow, solo, open mouth, hair bow, skirt, :d, outdoors, ribbon, dress, necktie, sky, thighhighs
+
+Nobara Hime,
+blonde hair, flower, long hair, 1girl, hair ornament, blue eyes, hair flower, blush, solo, rose, drill hair, looking at viewer, smile, open mouth, frills, bow, red flower, red rose, dress, skirt
+
+Kannagi Rei,
+1girl, long hair, brown hair, solo, brown eyes, looking at viewer, blush, hair between eyes, smile, skirt, outdoors, japanese clothes, holding, open mouth, black hair, yellow eyes, bangs, long sleeves, weapon, day
+
+Shinonome Aira,
+long hair, 1girl, looking at viewer, solo, red eyes, blush, smile, open mouth, fang, dress, multicolored hair, grey hair, frills, purple hair, thighhighs, hat, :d, hair between eyes, outdoors, long sleeves
+
+Fuyuki Noel,
+1girl, blonde hair, green eyes, solo, braid, blush, looking at viewer, bow, open mouth, smile, short hair, hair bow, outdoors, twin braids, skirt, :d, ribbon, sky, day, long hair
+
+Kishida Natsumi,
+1girl, long hair, brown hair, twintails, solo, brown eyes, blush, ahoge, open mouth, looking at viewer, smile, hair ornament, ribbon, camera, hair between eyes, holding, fang, skirt, red hair, :d
+
+Fuyuki Eve,
+blonde hair, green eyes, long hair, 1girl, solo, hairband, blush, looking at viewer, skirt, open mouth, smile, dress, gloves, outdoors, long sleeves, holding, shirt, flower, sitting, thighhighs
+
+Minazuki Fuko,
+blonde hair, blue eyes, long hair, 1girl, braid, solo, twintails, ribbon, hair ribbon, blush, looking at viewer, thighhighs, smile, skirt, open mouth, gloves, necktie, very long hair, school uniform, hat
+
+Rurikawa Akiho,
+brown hair, blush, 1girl, brown eyes, solo, open mouth, looking at viewer, ribbon, smile, skirt, hair ribbon, bow, :d, short hair, dress, twintails, outdoors, socks, long hair, day
+
+Tachibana Ui,
+long hair, 1girl, purple eyes, braid, solo, looking at viewer, antenna hair, ribbon, blonde hair, skirt, hair between eyes, bow, hair ribbon, blush, hair bow, sitting, pantyhose, dress, outdoors, flower
+
+Momose momo,
+pink hair, 1girl, solo, blush, ahoge, hair ornament, looking at viewer, smile, side ponytail, open mouth, short hair, skirt, brown eyes, long hair, :d, flower, shirt, holding, pink eyes, outdoors
+
+Moriya Tsukuyo,
+green eyes, 1girl, long hair, solo, blush, ribbon, looking at viewer, aqua hair, hair ribbon, open mouth, green hair, two side up, skirt, thighhighs, smile, hair ornament, long sleeves, holding, outdoors, twintails
+
+Tateno Nozomi,
+1girl, long hair, solo, black hair, purple eyes, blush, open mouth, looking at viewer, smile, skirt, purple hair, thighhighs, sitting, black eyes, gloves, very long hair, white gloves, boots, outdoors, striped
+
+Hattori Azusa,
+green hair, 1girl, green eyes, long hair, solo, looking at viewer, blush, smile, open mouth, scarf, hair ornament, gloves, pantyhose, outdoors, japanese clothes, ninja, fingerless gloves, shorts, red scarf, sparkle
+
+Minase Kaoruko,
+long hair, purple eyes, 1girl, black hair, breasts, solo, looking at viewer, smile, mole under eye, mole, ponytail, hair between eyes, blush, open mouth, very long hair, large breasts, skirt, outdoors, day, long sleeves
+
+Otonashi ritsu,
+short hair, 1girl, solo, blush, brown hair, smile, yellow eyes, open mouth, brown eyes, looking at viewer, fingerless gloves, gloves, red hair, shorts, thighhighs, skirt, necktie, shirt, bangs, :d
+
+Yusa Naruko,
+short hair, 1girl, purple eyes, solo, black hair, smile, looking at viewer, blush, shirt, purple hair, pants, jewelry, open mouth, holding, outdoors, skirt, gloves, white shirt, long sleeves, sky
+
+Shirafuji Konoha,
+long hair, blue hair, 1girl, green eyes, hair ornament, blush, heart, solo, looking at viewer, smile, heart hair ornament, open mouth, mole under eye, mole, hairclip, skirt, thighhighs, outdoors, shirt, very long hair
+
+Takanashi Miyu,
+short hair, 1girl, yellow eyes, solo, looking at viewer, smile, grey hair, blush, open mouth, flower, outdoors, hair between eyes, hairband, breasts, :d, shirt, day, brown eyes, gloves, long sleeves
+
+Narumi Jun,
+blue hair, short hair, 1girl, solo, blush, looking at viewer, smile, open mouth, yellow eyes, earrings, jewelry, green eyes, breasts, skirt, outdoors, necktie, shorts, sitting, shirt, school uniform
+
+Shiina Yukari,
+purple hair, glasses, 1girl, long hair, solo, braid, smile, brown eyes, looking at viewer, open mouth, breasts, blush, bow, twin braids, skirt, very long hair, outdoors, dress, holding, flower
+
+Wagatsuma Asari,
+blue hair, blush, blue eyes, short hair, solo, looking at viewer, 1girl, open mouth, smile, thighhighs, ahoge, bow, 1boy, hat, skirt, male focus, white thighhighs, otoko no ko, frills, crossdressing
+
+Nakazuki Sara,
+brown hair, 1girl, brown eyes, ahoge, blush, solo, open mouth, long hair, bow, smile, animal ears, dress, thighhighs, looking at viewer, frills, outdoors, :d, rabbit ears, gloves, hair ornament
+
+Sumeragi Ayaka,
+pink hair, 1girl, blush, looking at viewer, smile, blue eyes, solo, open mouth, short hair, hair ornament, skirt, jewelry, long hair, earrings, bangs, purple eyes, star, :d, holding, gloves
+
+Tsukimiya Sana,
+blue eyes, 1girl, grey hair, solo, blush, looking at viewer, short hair, long hair, smile, dress, holding, breasts, long sleeves, open mouth, apron, bangs, indoors, shirt, gloves, outdoors
+
+Kusunogi Arisu,
+long hair, blue eyes, blonde hair, multiple girls, 2girls, blush, bow, open mouth, hair bow, dress, red eyes, smile, twintails, grey hair, ribbon, looking at viewer, very long hair, blue hair, hat, skirt
+
+Kurusu Homura,
+black hair, 1girl, solo, short hair, looking at viewer, blush, red eyes, shorts, open mouth, bangs, hood, outdoors, shirt, skirt, gloves, navel, jewelry, fingerless gloves, belt, short shorts
+
+Mamiya Chika,
+brown hair, 1girl, brown eyes, blush, long hair, solo, bow, hair bow, looking at viewer, open mouth, smile, jewelry, ponytail, breasts, skirt, medium breasts, sitting, navel, outdoors, side ponytail
+
+Reisen Aoi,
+1girl, brown hair, solo, blush, long hair, hair ornament, flower, single hair bun, smile, hair bun, looking at viewer, open mouth, hair flower, bangs, blue eyes, japanese clothes, aqua eyes, skirt, holding, sidelocks
+
+Emilia Bloomfield,
+long hair, 1girl, solo, green eyes, blonde hair, blush, looking at viewer, smile, open mouth, ribbon, skirt, outdoors, hair ribbon, gloves, sky, holding, day, :d, bow, shirt
+
+Kirizuka Moegi,
+black hair, long hair, 1girl, blush, braid, solo, looking at viewer, twintails, ribbon, hair ribbon, twin braids, open mouth, pink eyes, very long hair, red eyes, skirt, long sleeves, bangs, hair between eyes, smile
+
+Koshimizu Sophia,
+blonde hair, 1girl, blue eyes, long hair, solo, blush, open mouth, smile, bow, hair rings, looking at viewer, hair bow, twintails, :d, skirt, outdoors, ribbon, day, gloves, shirt
+
+Mary Williams,
+blonde hair, blue eyes, 1girl, solo, long hair, breasts, smile, looking at viewer, navel, midriff, jacket, weapon, open mouth, outdoors, shorts, gun, belt, medium breasts, blush, shirt
+
+Hikawa Saki,
+blue hair, long hair, 1girl, brown eyes, blush, solo, hair ornament, yellow eyes, open mouth, hairclip, side ponytail, looking at viewer, skirt, shirt, blue skirt, long sleeves, breasts, outdoors, necktie, white shirt
+
+Nanjo Ren,
+purple eyes, 1girl, blush, solo, black hair, long hair, looking at viewer, smile, open mouth, bangs, purple hair, skirt, japanese clothes, holding, :d, ribbon, hair ornament, shirt, outdoors, day
+
+Ebina Ayase,
+long hair, 1girl, green eyes, solo, pink hair, blush, smile, looking at viewer, braid, open mouth, breasts, outdoors, skirt, shirt, dress, :d, day, gloves, sitting, flower
+
+Yuki Sena,
+green hair, glasses, short hair, 1girl, solo, green eyes, looking at viewer, open mouth, yellow eyes, semi-rimless eyewear, skirt, smile, blush, outdoors, hat, breasts, pantyhose, hair ornament, black-framed eyewear, necktie
+
+Charlotte Dior,
+blonde hair, long hair, 1girl, blue eyes, solo, blush, looking at viewer, breasts, open mouth, large breasts, smile, holding, dress, very long hair, long sleeves, nun, cross, habit, skirt, shirt
+
+Saibara Yueko,
+purple hair, short hair, 1girl, solo, purple eyes, blush, braid, open mouth, hair ornament, smile, looking at viewer, skirt, pantyhose, outdoors, black pantyhose, necktie, :3, jewelry, star, sky
+
+Kazeyari Mina,
+1girl, solo, eyepatch, long hair, grey hair, looking at viewer, open mouth, blue eyes, hair ornament, blush, smile, cape, skirt, two side up, thighhighs, bow, aqua eyes, long sleeves, hair between eyes, black thighhighs
+
+Takeda Torachiyo,
+blonde hair, long hair, 1girl, solo, looking at viewer, smile, brown eyes, breasts, yellow eyes, gloves, medium breasts, midriff, skirt, fingerless gloves, open mouth, navel, blush, black gloves, jacket, hair between eyes
+
+Nabatame Tsukasa,
+short hair, 1girl, solo, purple eyes, blue hair, smile, looking at viewer, bandages, navel, shorts, midriff, breasts, green hair, ahoge, gloves, skirt, open mouth, pink eyes, outdoors, sitting
+
+Rurikawa Haruno,
+brown hair, long hair, 1girl, solo, blush, ribbon, red eyes, open mouth, smile, hair ribbon, earrings, looking at viewer, jewelry, breasts, brown eyes, skirt, long sleeves, jacket, shirt, heart
+
+Tokisaka Chitose,
+black hair, blue eyes, 1girl, long hair, solo, looking at viewer, smile, hair between eyes, blush, open mouth, skirt, breasts, sky, very long hair, long sleeves, sitting, bangs, medium breasts, dress, thighhighs
+
+Futami Kokoro,
+pink hair, 1girl, short hair, solo, purple eyes, blush, open mouth, looking at viewer, skirt, thighhighs, smile, sky, gloves, blue skirt, outdoors, school uniform, black thighhighs, fingerless gloves, closed eyes, breasts
+
+Yonamise Rina,
+purple hair, 1girl, blue eyes, solo, short hair, blush, open mouth, looking at viewer, smile, :d, breasts, bandaid, skirt, shirt, bandaid on face, outdoors, water, sky, midriff, navel
+
+Ellen Amedick,
+red hair, 1girl, yellow eyes, solo, eyepatch, looking at viewer, breasts, smile, long hair, blush, gloves, short hair, braid, outdoors, pants, weapon, holding, white gloves, flower, brown eyes
+
+SHIshido yuki,
+purple eyes, short hair, blue hair, 1girl, solo, glasses, looking at viewer, thighhighs, open mouth, skirt, sitting, headphones, jacket, smile, hood, dress, school uniform, multiple girls, breasts, holding
+
+Asahina Tatsuki,
+blue eyes, long hair, brown hair, 1girl, solo, blush, ahoge, looking at viewer, shirt, smile, open mouth, jewelry, hair between eyes, breasts, pants, outdoors, bangs, skirt, bracelet, belt
+
+Yukishiro Mashiro,
+1girl, long hair, blue hair, solo, blue eyes, looking at viewer, smile, breasts, blush, very long hair, dress, open mouth, jewelry, skirt, cleavage, medium breasts, hair between eyes, earrings, outdoors, braid
+
+Soma Rena,
+1girl, solo, green eyes, open mouth, fang, blush, short hair, smile, grey hair, looking at viewer, dark skin, animal ears, outdoors, dark-skinned female, :d, breasts, long hair, sky, day, barefoot
+
+Lee Xiaolian,
+green eyes, green hair, 1girl, blush, hair bun, solo, smile, open mouth, looking at viewer, bun cover, dress, double bun, short hair, single hair bun, chinese clothes, holding, long hair, shorts, outdoors, food
+
+Qiao Mingling,
+long hair, braid, 1girl, blue eyes, solo, blush, smile, open mouth, pink hair, looking at viewer, red hair, single braid, ahoge, chinese clothes, fang, dress, :d, china dress, ribbon, gloves
+
+Kisaragi Sora,
+1girl, solo, long hair, green eyes, red hair, goggles, blush, goggles on head, gloves, open mouth, looking at viewer, skirt, thighhighs, smile, ahoge, breasts, jewelry, shirt, black thighhighs, earrings
+
+Hasekura Toko,
+1girl, long hair, solo, black hair, brown eyes, ponytail, looking at viewer, blush, very long hair, brown hair, open mouth, breasts, japanese clothes, skirt, smile, bangs, weapon, flower, outdoors, hair ornament
+
+Jinguji Hatsune,
+long hair, blue eyes, 1girl, solo, blush, brown hair, looking at viewer, smile, open mouth, hairband, ponytail, skirt, red hair, frills, ribbon, very long hair, thighhighs, :d, long sleeves, bow
+
+Yayoi Ruqa,
+braid, 1girl, solo, orange hair, smile, green eyes, open mouth, blush, looking at viewer, bird, breasts, yellow eyes, medium breasts, long hair, outdoors, hair ornament, gloves, cleavage, skirt, boots
+
+Satonaka Karin,
+brown eyes, brown hair, 1girl, solo, blush, short hair, open mouth, looking at viewer, smile, braid, flower, breasts, freckles, sky, outdoors, twin braids, :d, hair ornament, bow, skirt
+
+Madokano Marika,
+black hair, green hair, multicolored hair, short hair, 1girl, looking at viewer, green eyes, two-tone hair, solo, open mouth, smile, breasts, blush, outdoors, blue eyes, jacket, mask, sky, shorts, hat
+
+Urajiro Nana,
+black hair, 1girl, long hair, blush, solo, red eyes, ahoge, open mouth, looking at viewer, smile, skirt, braid, bow, bangs, hair bow, :d, brown eyes, outdoors, holding, ribbon
+
+Matsushima Michiru,
+short hair, 1girl, brown hair, green eyes, solo, open mouth, blush, looking at viewer, smile, hairband, skirt, :d, shirt, jacket, hair ornament, holding, indoors, gloves, outdoors, breasts
+
+Inukawa Nene,
+1girl, pink hair, long hair, solo, blush, hair ornament, bell, red eyes, smile, looking at viewer, open mouth, ribbon, pink eyes, fang, very long hair, jingle bell, dress, hair bell, hair between eyes, :d
+
+Nanayoshi Chihiro,
+short hair, solo, 1girl, blush, bow, brown eyes, green hair, open mouth, smile, hair bow, looking at viewer, hair ornament, heart, blonde hair, skirt, red bow, :d, school uniform, sky, outdoors
+
+Lizette,
+black hair, solo, purple eyes, 1girl, multicolored hair, hair between eyes, two-tone hair, looking at viewer, white hair, long hair, 1boy, blush, bangs, school uniform, skirt, necktie, short hair, male focus, indoors, breasts
+
+Uno,
+1girl, smile, pink hair, twintails, long hair, solo, grey eyes, looking at viewer, hair ornament, blush, open mouth, holding, dress, thighhighs, :d, confetti, multicolored hair, striped, stuffed toy, one eye closed
+```
+
+## Training info
+
+Dataset:
+* [4267 Grimoire cards](https://huggingface.co/datasets/gustproof/sd-data/blob/main/grimoire_cards_notext_denoised.zip) extracted from [official](https://play.google.com/store/apps/details?id=jp.co.applibot.grimoireOfflineMain.android) [offline](https://play.google.com/store/apps/details?id=jp.co.applibot.grimoireOfflineRose.android) [apps](https://play.google.com/store/apps/details?id=jp.co.applibot.grimoireOfflineSunflower.android) with text removed (characters labeled by themonky007)
+
+Tags generated with [SmilingWolf/wd-v1-4-convnextv2-tagger-v2](https://huggingface.co/SmilingWolf/wd-v1-4-convnextv2-tagger-v2) with a threshold of 0.35. No balancing and tag cleaning. Images were processed with [RealESRGAN_x4plus_anime_6B](https://github.com/xinntao/Real-ESRGAN/blob/master/docs/anime_model.md).
+
+Inspired by [a 2022 paper](https://openreview.net/pdf?id=Uad23IcIEs), a custom LR scheduler was used:
+```
+SequentialLR(optimizer, [
+    LinearLR(optimizer, 0.1, total_iters=steps_per_epoch),
+    CosineAnnealingWarmRestarts(optimizer, steps_per_epoch*2, T_mult=2)
+], [steps_per_epoch])
+```
+
+Training cost: ~31 T4-hours (in addition to deresute costs)
+
+
+Data distribution:
+```
+             character  cards
+1        Minami Tomoka     94
+2          Nobara Hime     86
+3          Kannagi Rei     82
+4       Shinonome Aira     80
+5          Fuyuki Noel     80
+6      Kishida Natsumi     79
+7           Fuyuki Eve     78
+8        Minazuki Fuko     78
+9       Rurikawa Akiho     77
+10        Tachibana Ui     76
+11         Momose momo     75
+12      Moriya Tsukuyo     74
+13       Tateno Nozomi     74
+14       Hattori Azusa     74
+15      Minase Kaoruko     72
+16      Otonashi ritsu     72
+17         Yusa Naruko     72
+18    Shirafuji Konoha     72
+19      Takanashi Miyu     72
+20          Narumi Jun     70
+21       Shiina Yukari     70
+22     Wagatsuma Asari     70
+23       Nakazuki Sara     70
+24      Sumeragi Ayaka     68
+25      Tsukimiya Sana     68
+26      Kusunogi Arisu     68
+27       Kurusu Homura     68
+28        Mamiya Chika     68
+29          Reisen Aoi     68
+30   Emilia Bloomfield     68
+31      Kirizuka Moegi     68
+32    Koshimizu Sophia     68
+33       Mary Williams     68
+34         Hikawa Saki     66
+35           Nanjo Ren     66
+36         Ebina Ayase     66
+37           Yuki Sena     66
+38      Charlotte Dior     66
+39       Saibara Yueko     66
+40       Kazeyari Mina     66
+41    Takeda Torachiyo     66
+42    Nabatame Tsukasa     66
+43     Rurikawa Haruno     64
+44    Tokisaka Chitose     64
+45       Futami Kokoro     64
+46       Yonamise Rina     64
+47       Ellen Amedick     62
+48       SHIshido yuki     62
+49     Asahina Tatsuki     62
+50   Yukishiro Mashiro     62
+51           Soma Rena     62
+52        Lee Xiaolian     62
+53       Qiao Mingling     62
+54       Kisaragi Sora     60
+55       Hasekura Toko     60
+56     Jinguji Hatsune     60
+57          Yayoi Ruqa     60
+58      Satonaka Karin     58
+59     Madokano Marika     54
+60        Urajiro Nana     50
+61  Matsushima Michiru     50
+62        Inukawa Nene     48
+63   Nanayoshi Chihiro     44
+64             Lizette      8
+65                 Uno      4
+```
+
+
+Training config:
+```
+[model_arguments]
+v2 = false
+v_parameterization = false
+pretrained_model_name_or_path = "animefull-final-pruned.ckpt"
+
+[additional_network_arguments]
+no_metadata = false
+network_module = "lycoris.kohya"
+network_dim = 16
+network_alpha = 1
+network_args = [ "conv_dim=0", "algo=loha",]
+network_train_unet_only = false
+network_train_text_encoder_only = false
+cache_latents = false
+
+[optimizer_arguments]
+optimizer_type = "AdamW8bit"
+learning_rate = 2e-4
+max_grad_norm = 1.0
+lr_scheduler = "custom"
+
+
+[dataset_arguments]
+debug_dataset = false
+
+[training_arguments]
+save_precision = "fp16"
+save_every_n_epochs = 1
+max_token_length = 225
+mem_eff_attn = false
+xformers = true
+max_train_epochs = 31
+max_data_loader_n_workers = 0
+persistent_data_loader_workers = false
+gradient_checkpointing = false
+gradient_accumulation_steps = 1
+mixed_precision = "fp16"
+clip_skip = 2
+lowram = true
+train_batch_size = 4
+
+[sample_prompt_arguments]
+
+[saving_arguments]
+save_model_as = "safetensors"
+```