Issue 1:
Hey there,

How does one actually use class_weight on model.fit?

I had originally written the following method to do this, but I'm not entirely sure whether it works or not.

def calculate_class_weights(train_label):
list = train_label.tolist()

num_neg = list.count(0)
num_pos = list.count(1)

duplicate = num_pos / num_neg

class_weights={0 : (num_neg * (duplicate)) , 1: num_pos }
return class_weights

This returns a dictionary of...

{0: 34, 1: 34}

Does anyone have a working example of how to balance 2 classes using the class_weights method?

Thanks,

Keiron.

Issue 2:
I'm working in a new generator and I realized that keras was calling it more times that it was supposed.

Information about dataset:
Dataset size: 30
Training size: 21
Test size: 9
Classes: 3
Batch Size: 6

Below my fit_generator:

kbg = KerasBatchGenerator('/home/kleysonr/Downloads/keras-generator/dataset', batch_size=6, imagesize=(100, 100), test_ratio=0.3)

model.fit_generator(kbg.generate(set='train'),
steps_per_epoch=training_steps,
epochs=1,
verbose=1,
callbacks=[tbCallback],
validation_data=kbg.generate(set='test'),
validation_steps=validation_steps)

As I set only 1 epoch, I would expect to see the kbg.generate() delivering only 21 images to keras, but instead, was consumed much more, see the logs:

Epoch 1/1
Batch: 0-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 0-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 0-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 0-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 0-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 0-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 1-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 1-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 1-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 1-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 1-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 1-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 2-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 2-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 2-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 2-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 2-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 2-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 3-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 3-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 3-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
Batch: 4-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 4-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 4-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 4-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 4-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 4-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 5-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 5-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 5-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 5-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 5-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 5-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 6-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 6-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 6-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 6-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 6-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 6-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 7-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 7-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 7-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
Batch: 8-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 8-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 8-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 8-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 8-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 8-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 9-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 9-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 9-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 9-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 9-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 9-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 10-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 10-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 10-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 10-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 10-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 10-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 11-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-2.jpg
Batch: 11-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
Batch: 11-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-2.jpg
1544967955 --train-- New epoch
1/4 [======>.......................] - ETA: 3s - loss: 1.1924 - acc: 0.1667Batch: 12-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 12-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 12-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 12-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 12-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 12-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 13-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
Batch: 13-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
3/4 [=====================>........] - ETA: 0s - loss: 1.2550 - acc: 0.2222Batch: 13-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 13-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-4.jpg
Batch: 13-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-1.jpg
Batch: 13-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 14-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-5.jpg
Batch: 14-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-4.jpg
Batch: 14-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 14-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-7.jpg
Batch: 14-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
Batch: 14-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
4/4 [==============================] - 1s 351ms/step - loss: 1.2378 - acc: 0.1714 - val_loss: 1.0822 - val_acc: 0.3333

Even being necessary data for just a single epoch, keras consumed data for almost 4 epochs.

I've include the configuration workers=0 and use_multiprocessing=False to the fit_generator()

model.fit_generator(kbg.generate(set='train'),
steps_per_epoch=training_steps,
epochs=1,
verbose=1,
callbacks=[tbCallback],
validation_data=kbg.generate(set='test'),
validation_steps=validation_steps,
use_multiprocessing=False,
workers=0)

and I got was it was supposed, enough data for a single epoch

Epoch 1/1
Batch: 0-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-6.jpg
Batch: 0-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-1.jpg
Batch: 0-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-2.jpg
Batch: 0-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-5.jpg
Batch: 0-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-5.jpg
Batch: 0-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-9.jpg
1/4 [======>.......................] - ETA: 3s - loss: 1.1336 - acc: 0.1667
Batch: 1-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-9.jpg
Batch: 1-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-9.jpg
Batch: 1-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-6.jpg
Batch: 1-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-4.jpg
Batch: 1-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-10.jpg
Batch: 1-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-7.jpg
2/4 [==============>...............] - ETA: 1s - loss: 1.1540 - acc: 0.1667
Batch: 2-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-3.jpg
Batch: 2-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-3.jpg
Batch: 2-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-3.jpg
Batch: 2-3 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-10.jpg
Batch: 2-4 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-6.jpg
Batch: 2-5 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-8.jpg
3/4 [=====================>........] - ETA: 0s - loss: 1.3066 - acc: 0.2222
Batch: 3-0 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class3/c3-1.jpg
Batch: 3-1 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class1/c1-7.jpg
Batch: 3-2 <<train>> /home/kleysonr/Downloads/keras-generator/dataset/class2/c2-10.jpg
1544964579 --train-- New epoch
4/4 [==============================] - 2s 391ms/step - loss: 1.2351 - acc: 0.3238 - val_loss: 1.1089 - val_acc: 0.2222

I believe this behavior might be related with the worker=1 working in a async thread and pushing data to keras without any control.

Shouldn't this async thread take in account the number of epoch to calculate and limit how much data should be consumed and pushed to keras ? More data that what is needed is a waste of processing and memory consumption.

Best Regards.

Kleyson Rios.