Usually, we divide by the size of the minibatch, not the size of the whole dataset.