Sen-sou/Anima-LLLite-Regional-Controlnet

Experimental Anima LLLite Regional Controlnet

Apply Anima ControlNet-LLLite parameters:

Control image example: (Use basic colors as region)

Prompt: (kinda works)
base prompt
region color: region prompt

Training Parameters:

caption_extension: .txt
shuffle_caption: false
resolution: 1024
batch_size: 1
enable_bucket: true
bucket_no_upscale: true
bucket_reso_steps: 16
min_bucket_reso: 64
max_bucket_reso: 1536
num_repeats: 2

580 image/control/caption pairs
580 * 2 repeats = 1160 steps per epoch
8 epochs = 9280 total steps

learning_rate: 3e-4
max_train_epochs: 8
seed: 42
optimizer: AdamW8bit
lr_scheduler: constant
mixed_precision: bf16
save_precision: bf16
save_model_as: safetensors
save_every_n_epochs: 1
gradient_checkpointing: enabled
cache_latents_to_disk: enabled
cache_text_encoder_outputs_to_disk: enabled
vae_chunk_size: 64
vae_disable_cache: enabled

timestep_sampling: shift
discrete_flow_shift: 3.0
attn_mode: torch

cond_emb_dim: 32
lllite_cond_dim: 64
lllite_mlp_dim: 64
lllite_target_layers: self_attn_qkv
lllite_cond_resblocks: 4
lllite_use_aspp: enabled
caption_dropout_rate: 0.15