Can you explain to me how the "Repeats" are made?
I mean this part:
Trained on a set of 250 images.
" 5 repeats "
3 batch size, 4 epochs
(250 * 5) / 3 * 4 = 1667 steps
His math is fuzzy. 1667 steps is per BS for a grad total of 5001 steps.
num train images * repeats: 1250
Total optimization steps/Max training steps: 1667
Actual final total steps: 5001
His math is fuzzy. 1667 steps is per BS for a grad total of 5001 steps.
num train images * repeats: 1250
Total optimization steps/Max training steps: 1667
Actual final total steps: 5001
What is BS?
Excuse my ignorance, I'm new to this.
I use Kohya. The point is that I only have the options "batch size", "Epochs", among others, but I don't have "repeats".
If for example I have 250 images, I put in the default folder "100_NameCharacter" that 100 is multiplied by the number of images.
250 * 100 = 25,000 Steps * 1 epoch / 6 Batch size. = 4,166 Steps. 😮
I don't know where the "repeats" comes from.
The reality is that I have never used so many images for character training, so I am confused, I always use between 15 and 45 images.
The thing is that in the tutorials I've seen they always say to put 100 by default in the image training folder, if I put it below 100 it won't work. If I lower that 100 to 5, if they give me the numbers that Khanon mentions. So I don't know if that is what is called "repeats".
BS = batchsize
edit: that 100 in the folder name means it will repeat those images 100 times.
OHHHHHH!!! Honestly, as I was writing to you, I was realizing that it could be that... Brother, thank you very much, do you recommend leaving 100 or lowering it whenever I want? Or is there some standard in the number of repetitions?
Depends as I go from 100 down to 40 depending on what I am training and judging the first train. I try to stick to 1500-2200 total steps.
Repeats refer to the number before each of the folders in your dataset, so yeah 100_Cammy means that during a single epoch, each image in that folder will be "seen" by the AI 100 times. There is a caveat, however -- batch size. When batch size is greater than 1, that means the model "looks at" more images during a single step of training (by taking the "average" of the images in the batch). This increases VRAM usage but may smooth out a lower quality dataset by reducing the influence of any one image on the model's weight updates.
In (250 * 5) / 3 * 4 = 1667 steps:
250 = number of images for the character
5 = number of repeats for those images. (ie. folder name 5_koharu is 5 repeats) You can think of this as just copying and pasting the images in that folder x number of times.
3 = my usual batch size. I recommend 3 unless you have a specific reason to change it. If you increase the batch size without increasing the number of repeats or epochs, you will train for fewer steps.
4 = number of epochs. One "epoch" is a cycle through the entire dataset, including repeats.
1667 steps = the final number of training steps. This is the value you see in Kohya once the progress bar appears during training.
To reiterate: (images * repeats) / batch_size * epochs = training_steps
Similar to @GeneralAwareness , I generally stick to ~1400-1800 training steps for character LoRAs. Adjust the repeats/epochs according to the size of your dataset to try to hit that ~1600 number.
Note that all of the training hyperparameters are interdependent and tend to influence each other. 1600 steps is my recommendation, but if you aren't using the same learning rates, batch size, or optimizer settings, you may not see the same results. That's why I provide the JSON files that you can load into the Kohya GUI (https://github.com/bmaltais/kohya_ss) to match my settings as closely as possible.
MUCHAS GRACIAS!