Workflow adjustment tips while using character Lora for i2v?
So I went ahead and decided to try Lora training for the first time, 40ish GB LTX 2.3 Dev model that downloaded itself. And make a character Lora. I've followed Gemini advice while using a cloud rtx pro 6000 we, adjustment on aintoolkit such as 3000 steps, disabling all quantization ( probably a bad ideas, using the Lora I was getting OOM on my rtx 5070 to and 32 GB Ram. I now have to use --lowvram with Eros fp8mixed learned model, but it's quite a bit slower especially on the tiles sampler.) and a few others things. Nevertheless the Lora seems to work but it seems quite inconsistent, and motion seems slower. So far I had the time to try gemma 3 12b it fp8 e4m3fn text encoder instead of scaled, I think it's already better, but even slower, a full generation is now 1000 seconds.
Do you guys have any tips on workflow adjustment or others to make it work better? I'm using the likeness guide helper workflow. Thanks!
Motion would depend on how you trained the lora. You want to manually compose any clips in the dataset to the exact frame rate and frame counts needed. If you have a 10s 30fps clip and let the trainer auto format it it will make it a 24fps 11 second clip, slow the motion, and then trim it, which will make motion slow down. For characters it's best to do a short very simple face/body/outfit training on images with no audio abd if you're just using eros train it on Eros or Sulphur base model.
Motion would depend on how you trained the lora. You want to manually compose any clips in the dataset to the exact frame rate and frame counts needed. If you have a 10s 30fps clip and let the trainer auto format it it will make it a 24fps 11 second clip, slow the motion, and then trim it, which will make motion slow down. For characters it's best to do a short very simple face/body/outfit training on images with no audio abd if you're just using eros train it on Eros or Sulphur base model.
Got it, so next time I should use base bf16 Eros for training, then exclusively use images for the trainings. Although are there any specific settings to change for bf16 Eros model in AI toolkit I should change before starting a i2v character training? Either for efficiency or results, and perhaps the steps count? Last time I used 3000 steps, I'm not sure if it's a lot or not.