v1.1-hisakawa

A LoRA (Hadamard Product) resumed from deresute-v1.1 and tuned on 72 Starlight Stage cards and character sprites, improving consistency and likeness of the twins while retaining the ability to recognize other characters.

This is not a style-neutral model. Currently, generating both characters in the same image isn't supported. This serves as a preview model while these issues are being worked on.

This shows that official material is sufficient and selective finetuning is possible.

If there are other characters whose likeness you find that the base model doesn't capture well, please leave a comment below. The feedback would help improve the base model.

Usage

This Hadamard Product LoRA requires installing an extra extension to be used in the Web UI.

The style itself doesn't have a trigger word. For the characters, below are the character tags and respective top 20 related tags.

Hisakawa Nagi,
braid, 1girl, grey hair, brown eyes, long hair, twintails, low twintails, bangs, solo, looking at viewer, shirt, braided bangs, white background, simple background, long sleeves, skirt, shorts, thighhighs, holding, hair ribbon

Hisakawa Hayate,
long hair, grey hair, bangs, braid, 1girl, jewelry, solo, earrings, blue eyes, looking at viewer, braided bangs, smile, blush, very long hair, shirt, white background, simple background, skirt, breasts, long sleeves

As this is a finetuned model, prompts that work on the base model should also work.

Training info

Finetuned (without optimizer state) from deresute-v1.1 with resolution 768*768.

Training cost: ~1 T4-hour

Training config:

[model_arguments]
v2 = false
v_parameterization = false
pretrained_model_name_or_path = "Animefull-final-pruned.ckpt"

[additional_network_arguments]
no_metadata = false
network_module = "lycoris.kohya"
network_dim = 16
network_alpha = 1
network_args = [ "conv_dim=0", "conv_alpha=16", "algo=loha",]
network_train_unet_only = false
network_train_text_encoder_only = false

[optimizer_arguments]
optimizer_type = "AdamW8bit"
learning_rate = 1e-4
lr_scheduler = "cosine"
lr_warmup_steps = 80

[dataset_arguments]
debug_dataset = false

[training_arguments]
save_precision = "fp16"
save_every_n_epochs = 1
train_batch_size = 2
max_token_length = 225
mem_eff_attn = false
xformers = true
max_train_epochs = 40
max_data_loader_n_workers = 8
persistent_data_loader_workers = true
gradient_checkpointing = false
gradient_accumulation_steps = 1
mixed_precision = "fp16"
clip_skip = 2
lowram = true

[saving_arguments]
save_model_as = "safetensors"