v2.0

A LoRA of the Hisakawa twins from THE iDOLM@STER Cinderella Girls based on 967 fanarts. More style neutral than v1.1.

Two versions are available: rank 1 and rank 4.

This shows that, perhaps unsurprisingly, 1 rank can carry info of more than 1 character.

Despite the abundance of two-subject data in the training set, the model still struggles to seperate the attributes of the twins in the image. Increasing the model rank may or may not help. This will be investigated some time in the future.

Usage

The character tags are Hisakawa Nagi and Hisakawa Hayate.

To generate both characters in the same image, use either Hisakawa Nagi / Hisakawa Hayate or Hisakawa Hayate / Hisakawa Nagi

Top related tags:

Hiwakawa Nagi,
1girl, solo, twintails, long hair, grey hair, braid, low twintails, brown eyes, looking at viewer, ribbon, hair ribbon, braided bangs, blush, shirt, simple background, long sleeves, white background, black ribbon, very long hair, bow, white shirt, jacket, parted lips, skirt, collared shirt, closed mouth, thighhighs, hair over shoulder, :o, upper body, puffy sleeves, shorts, open mouth, breasts, red bow, black jacket, collarbone, short sleeves, holding, small breasts

Hisakawa Hayate,
1girl, solo, long hair, grey hair, braid, looking at viewer, blush, smile, jewelry, breasts, braided bangs, earrings, blue eyes, very long hair, white background, shirt, open mouth, simple background, skirt, collarbone, medium breasts, white shirt, long sleeves, cleavage, pleated skirt, jacket, collared shirt, :d, bow, bare shoulders, navel, swimsuit, closed mouth, hair between eyes, holding, upper body, puffy sleeves, dress, bikini, short sleeves

Hisakawa Nagi / Hisakawa Hayate,
2girls, multiple girls, siblings, twins, braid, long hair, sisters, grey hair, twintails, brown eyes, smile, braided bangs, open mouth, blue eyes, low twintails, ribbon, blush, looking at viewer, hair ribbon, jewelry, earrings, shirt, dress, skirt, long sleeves, black ribbon, very long hair, white shirt, bow, parted lips, breasts, :d, simple background, one eye closed, v, collarbone, white background, :o, puffy sleeves, collared shirt

For specific outfits, refer to the preview images or the dataset.

Training info

Dataset: 967 fanarts from Danbooru

338 Hisakawa Nagi solo
341 Hisakawa Hayate solo
238 both twins

No balancing and tag cleaning.

For images with both characters, the order of the character tags were shuffled dynamically.

Training cost: ~2 T4-hour each for r1 and r4

Training config:

[model_arguments]
v2 = false
v_parameterization = false
pretrained_model_name_or_path = "animefull-final-pruned.ckpt"

[additional_network_arguments]
no_metadata = false
unet_lr = 0.0001
text_encoder_lr = 0.0001
network_module = "networks.lora"
network_alpha = 1
network_train_unet_only = false
network_train_text_encoder_only = false

[optimizer_arguments]
min_snr_gamma = 5
optimizer_type = "AdamW8bit"
learning_rate = 0.0001
max_grad_norm = 1.0
lr_scheduler = "constant"
lr_warmup_steps = 0

[dataset_arguments]
debug_dataset = false
dataset_repeats = 1
shuffle_caption = true
keep_tokens = 1
resolution = "512,512"
caption_dropout_rate = 0
caption_tag_dropout_rate = 0
caption_dropout_every_n_epochs = 0
color_aug = false
token_warmup_min = 1
token_warmup_step = 0

[training_arguments]
save_precision = "fp16"
save_every_n_epochs = 1
train_batch_size = 4
max_token_length = 225
mem_eff_attn = false
xformers = true
max_train_epochs = 9999
max_data_loader_n_workers = 8
persistent_data_loader_workers = true
gradient_checkpointing = false
gradient_accumulation_steps = 1
mixed_precision = "fp16"
clip_skip = 2
lowram = true