README.md

metadata

license: mit
language:
  - en

[Open Grid] | [Open CivitAI] | [Show models on HuggingFace]

Here I publish some results of my experiments and my subjective opinion about hyperparams for PonyXL while I tried to make Senko-lora for that.

Batch Size == 1
TE LR == UNet LR
I didn't use gradient checkpointing.

<lora:senko_ds6_ponyxl_lr1_linear_prodigy_dim16_alpha8:1> overfit
<lora:senko_ds6_ponyxl_lr1_linear_prodigy_dim32_alpha16:1> overfit

<lora:senko_ds6_ponyxl_lr1e-4_constant_adamw8_dim32_alpha16:1> overfit
<lora:senko_ds6_ponyxl_lr1e-5_constant_adamw8_dim16_alpha1:1> doesn't work
<lora:senko_ds6_ponyxl_lr1e-5_constant_adamw8_dim16_alpha8:1> ✅ OK
<lora:senko_ds6_ponyxl_lr1e-5_constant_adamw8_dim32_alpha16:1> ✅ OK (published as senko-ponyxl-v2)

<lora:senko_ds6_ponyxl_lr3e-4_constant_adafactor_dim16_alpha1:1> ✅ OK
<lora:senko_ds6_ponyxl_lr3e-4_constant_adafactor_dim16_alpha8:1> ✅ OK
<lora:senko_ds6_ponyxl_lr3e-4_constant_adafactor_dim32_alpha16:1> ✅ OK (published as senko-ponyxl-v1)

<lora:senko_ds6_ponyxl_locon_lr1_linear_prodigy_dim16_alpha8_conv16_convalpha_8:1> breaks anatomy on complex contepts
<lora:senko_ds6_ponyxl_locon_lr1_linear_prodigy_dim16_alpha8_conv32_convalpha_16:1> TE overfit
<lora:senko_ds6_ponyxl_locon_lr1_linear_prodigy_dim32_alpha16_conv16_convalpha_8:1> TE overfit
<lora:senko_ds6_ponyxl_locon_lr1_linear_prodigy_dim32_alpha16_conv32_convalpha_16:1> TE overfit

<lora:senko_ds6_sdxl_lr1e-5_constant_adamw8_dim32_alpha16:1> doesn't work
<lora:senko_ds6_sdxl_lr3e-4_constant_adafactor_dim32_alpha16:1> doesn't work
<lora:senko_ds6_counterfeitxl_lr1e-5_constant_adamw8_dim32_alpha16:1> doesn't work