adds new Haruka LoRA

haruka/README.md

@@ -0,0 +1,52 @@
+# Igusa Haruka (Blue Archive)
+伊草ハルカ (ブルーアーカイブ) / 이구사 하루카 (블루 아카이브) / 伊草春香 (碧蓝档案)
+
+[**Download here.**](https://huggingface.co/khanon/lora-training/blob/main/haruka/chara-haruka-v1c.safetensors)
+
+## Table of Contents
+- [Preview](#preview)
+- [Usage](#usage)
+- [Training](#training)
+- [Revisions](#revisions)
+
+## Preview
+![Haruka portrait](chara-haruka-v1c.png)
+![Haruka preview 1](example-001b-v1c.png)
+![Haruka preview 2](example-002b-v1c.png)
+![Haruka preview 3](example-003b-v1c.png)
+
+## Usage
+Use any or all of the following tags to summon Haruka: `haruka, 1girl, halo, short hair with long locks, purple eyes`
+
+For her normal outfit: `school uniform, hairclip, purple jacket, collared shirt, juliet sleeves, belt, miniskirt, black skirt, boots`
+- The AI is slightly stubborn with Haruka's normal outfit, likely because she doesn't have much art that does not feature long sleeves of some sort.
+- To remove her jacket sleeves, prompt `sleeveless` and use `long sleeves` in the negative prompt.
+- To remove the purple bowtie/tie clip, prompt `collarbone` and use `bowtie` in the negative prompt.
+
+For her nervous/creepy smile: `nervous smile, nervous, sweat, wavy mouth, [closed mouth:parted lips:0.75]`
+- Use `sanpaku, constricted pupils, crazy eyes, yandere` or similar for more intense expressions.
+
+[Here is a list of all tags including in the training dataset, sorted by frequency.](all_tags.txt)
+
+## Training
+*Exact parameters are provided in the accompanying JSON files.*
+- Trained on a set of 143 images.
+  - 138 without "multiple views" (9 repeats)
+  - 5 "multiple views" (4 repeats)
+  - 3 batch size, 4 epochs
+  - `(138 * 9 + 5 * 4) / 3 * 4` = 1683 steps
+- 0.083 loss
+- Initially tagged with WD1.4 swin-v2 model. Tags pruned/edited for consistency.
+  - [Tagging methodology detailed here.](tagging%20methodology.md)
+- `constant_with_warmup` scheduler
+- 1.5e-5 text encoder LR
+- 1.5e-4 unet LR
+- 1e-5 optimizer LR
+- Used network_dimension 128 (same as usual) / network alpha 128 (default)
+  - Resized to 32 after training
+- Training resolution 832x832.
+- Trained without VAE.
+
+## Revisions
+- v1b (2023-02-18)
+  - Initial release.

haruka/chara-haruka-v1b.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b751170286f8135cd25ae375a961fb9c9df68df130d4bf3120cebfade8151168
+size 37882638

haruka/lora_chara_haruka_v1b_138i9r-5i4r.json

@@ -0,0 +1,59 @@
+{
+  "pretrained_model_name_or_path": "G:/sd/repo/models/Stable-diffusion/nai-animefull-final-pruned.safetensors",
+  "v2": false,
+  "v_parameterization": false,
+  "logging_dir": "",
+  "train_data_dir": "G:/sd/training/datasets/haruka/dataset",
+  "reg_data_dir": "",
+  "output_dir": "G:/sd/lora/trained/chara/haruka",
+  "max_resolution": "832,832",
+  "learning_rate": "1e-5",
+  "lr_scheduler": "constant_with_warmup",
+  "lr_warmup": "5",
+  "train_batch_size": 3,
+  "epoch": "4",
+  "save_every_n_epochs": "",
+  "mixed_precision": "fp16",
+  "save_precision": "fp16",
+  "seed": "31337",
+  "num_cpu_threads_per_process": 32,
+  "cache_latents": true,
+  "caption_extension": ".txt",
+  "enable_bucket": true,
+  "gradient_checkpointing": false,
+  "full_fp16": false,
+  "no_token_padding": false,
+  "stop_text_encoder_training": 0,
+  "use_8bit_adam": true,
+  "xformers": true,
+  "save_model_as": "safetensors",
+  "shuffle_caption": true,
+  "save_state": false,
+  "resume": "",
+  "prior_loss_weight": 1.0,
+  "text_encoder_lr": "1.5e-5",
+  "unet_lr": "1.5e-4",
+  "network_dim": 128,
+  "lora_network_weights": "",
+  "color_aug": false,
+  "flip_aug": false,
+  "clip_skip": 2,
+  "gradient_accumulation_steps": 1.0,
+  "mem_eff_attn": false,
+  "output_name": "chara-haruka-v1b-128",
+  "model_list": "",
+  "max_token_length": "150",
+  "max_train_epochs": "",
+  "max_data_loader_n_workers": "",
+  "network_alpha": 128,
+  "training_comment": "Character: `haruka, 1girl, halo, short hair with long locks, purple eyes`\nStandard outfit: `hairclip, garrison cap, purple jacket, collared shirt, black skirt, miniskirt, black belt, boots`\nNew Year alt: `kimono skirt, purple kimono, floral print, purple hairband, hair flower, wide sleeves, obi, geta, tabi`\nNervous expression: `nervous, sweat, wavy mouth`\nVaguely yandere-like smile: `naughty face, nervous smile`\nFlustered: `sanpaku, wide-eyed, shaded face, flustered`\n\nNot all tags are necessary.\n\n(138 normal * 9 repeats + 5 multipleviews * 4 repeats) / 3 batch size * 4 epochs = 1683 steps",
+  "keep_tokens": 2,
+  "lr_scheduler_num_cycles": "",
+  "lr_scheduler_power": "",
+  "persistent_data_loader_workers": true,
+  "bucket_no_upscale": true,
+  "random_crop": false,
+  "bucket_reso_steps": 64.0,
+  "caption_dropout_every_n_epochs": 0.0,
+  "caption_dropout_rate": 0
+}

haruka/tagging methodology.md

@@ -0,0 +1,93 @@
+Tagging methodology for Haruka (blue archive)
+
+Start with WD1.4 Swinv2 at ~0.25-0.3 confidence.
+
+- Tag unique features
+  - ie. `halo` / `demon horns` / `low wings`
+  - Remove when not present or out of view.  WD1.4 likes putting `halo` even on images where no halo is visible.
+- Tag outfit variants with a single master tag
+  - ie. for Haruka:
+    - Uniform: `school uniform`
+    - NY Alt: `japanese clothes`
+      - Add `official alternate costume`
+    - Non-canon costumes
+      - Add `alternate costume`
+  - Nudity (WD1.4 usually does this accurately)
+    - `nude` / `completely nude`
+- Prune eye colors
+  - Keep tags which describe unusual eye features (`multicolored eyes`, `heterochromia`, `slit pupils`) as they can otherwise be too subtle and inconsistently drawn for the AI to learn
+- Prune hair colors
+  - This includes `two-toned hair`, `gradiant hair`, etc.  The AI learns all of these very consistently without the tags, likely because artists tend to draw them consistently
+- Partially prune hair styles
+  - Leave key style tags like `twintails`, `ponytail`, `short hair with long locks`.
+  - Prune exceedingly common tags like `bangs`, `eyebrows visible through hair`, `hair between eyes`, etc.
+  - Prune length, except for images which differ from the character's usual length
+    - Add `alternate hairstyle` and/or `alternate hair length` for these
+- Fixup hair ornaments
+  - Prune generic `hair ornament` in favor of more specificity
+    - `hairclip` / `black headband` / `hair flower` / `hair ribbon`, etc.
+  - Consolidate tags that have color variants (`headband` >> `black headband`)
+- Consolidate outfits
+  - Only tag an item when it is actually visible. If it is only barely visible along the edge of an image, keep in mind it may be cropped during bucketing.
+  - Danbooru's wiki entry for a character often provides a good list of tags commonly used to describe a character's outfits.
+  - Haruka outfits:
+    - Uniform
+      - `garrison cap`
+      - `miniskirt`
+      - `pleated skirt`
+      - `black skirt`
+      - `boots`
+      - `hairclip`
+      - `juliet sleeves`
+      - `black belt`
+      - `red shirt`
+      - `collared shirt`
+      - `purple jacket`
+    - NY Alt
+      - `frilled kimono`
+      - `purple kimono`
+      - `kimono skirt`
+      - `floral print`
+      - `purple hairband`
+      - `hair flower`
+      - `wide sleeves`
+      - `obi`
+      - `geta`
+      - `tabi`
+- Fixup sleeves
+  - ie. `long sleeves` / `puffy long sleeves` / `detached sleeves` / `off shoulder`
+  - You only need one, but pick one and be consistent. If sleeves aren't tagged properly the tends to add them inappropriately (such as when prompting for sleeveless outfits or nudity)
+- Fixup collars
+  - ie. `detached dollar` / `collared shirt` / `choker` / etc.
+  - Same deal as sleeves, they tend to appear when unwanted if not consistently tagged according to visibility in the training data.
+- Tag expressions
+  - WD1.4 rarely tags these, but doing them manually can help the AI reproduce a character's iconic expressions well.
+  - Start by searching for images without one of these, and add them.
+    - `open mouth`
+    - `closed mouth`
+    - `parted lips`
+  - Add less common expressions
+  - `wavy mouth`
+  - `surprised` / `o_o`
+  - `embarrassed` / `flustered` / `panicked` / `swirly eyes` / `@_@`
+  - `smile` / `light smile` / `:d`
+  - `upset` / `annoyed` / `frustrated`
+  - `naughty face` / `seductive smile`
+  - `smug` / `:3`
+  - `eyes closed` / `one eye closed`
+    - WD1.4 usually gets these for you.
+- Tag camera angles/composition
+  - `cowboy shot`
+  - `upper body`
+  - `full body`
+  - `portrait`
+  - `cropped torso` / `cropped legs`
+  - `feet out of frame`
+  - `from side` / `from above` / `from below` / `from behind`
+- Tag iconic poses/actions
+  - ie. `v` / `standing on one leg` 
+  - Haruka
+    - `object hug`
+    - `holding weapon`
+- Flip through each image and use Hydrus's "related tags" feature to quickly identify important tags that might be missing.
+  - This feature looks at other images with similar tags to provide suggestions.