Repository: pykeen/pykeen
Issue: Possible issue in loss scaling 
Root cause: Algorithmic Error

Hi everyone,

while testing the training loop I realised that this `if-else` condition might be incorrect:
https://github.com/pykeen/pykeen/blob/b0a9abd949a9751d7afa2d8e1fe0f2c006c94519/src/pykeen/training/training_loop.py#L688-L691

If the loss has has a `mean` reduction, then the loss of each minibatch is already scaled by the number of samples in the minibatch because the loss function calls `self._reduction_method`. So if seems like at the end of the epoch we should just run the code ```epoch_loss = current_epoch_loss / len(train_data_loader)``` regardless of the reduction strategy.

In fact, when testing the training loop with this simple setup:

- A model that always predicts `1.`
- The default `MarginRankingLoss`, with the default `margin=1.` and `reduction="mean"`
- A dataset with 3 triples

I end up with an epoch loss equal to `0.3333333333333333`, while I would expect an epoch loss of `1.`, given that the loss for each sample is `score(pos) - score(neg) + margin = 1. - 1. + 1. = 1.` and hence the average over the whole dataset is still `1.`.

Am I missing something? Is this the desired behaviour? I'm happy to submit a PR with a unit-test to fix this if this is indeed a bug.

Thanks in advance.

The batch losses get rescaled in the `_forward_pass` by the individual `batch_size`, cf.

https://github.com/pykeen/pykeen/blob/b0a9abd949a9751d7afa2d8e1fe0f2c006c94519/src/pykeen/training/training_loop.py#L830-L833

Thus, we need to divide by the sum of batch sizes, i.e., the number of training instances.

Thank you @mberr. Isn't that code needed to handle the case where `sub_batch_size` is not `None`?
In my test case `this_sub_batch_size` ends up being equal to `current_batch_size`, so there is no scaling. 



> Thus, we need to divide by the sum of batch sizes, i.e., the number of training instances.

Wouldn't this line in normal circumstances always return 1?
```
this_sub_batch_size / current_batch_size
```

So loss would indeed become way too small - I generally observe that losses in pykeen are 1-2 orders of magnitude smaller than in other libraries (while the final performance is comparable)

You are right, the current version is indeed incorrect.

For calculating the true epoch loss, we would need to rescale the batch-losses by the batch size, and then finally divide by the number of instances, to account for the possiblity of the last batch being smaller than the rest.

This should be a marginal difference though, since typically `batch_size << num_instances`.

In any way, this error should luckily only affect the calculation of the final epoch loss - the individual batch losses (and thus the resulting gradients used for optimization) should be fine.

To be honest, I suspect that even with incomplete minibatches everything should work fine. The reduction function uses the true batch size (not the config parameters), so it seems like the batch loss is always averaged correctly.

<del>It seems like there would be an issue only if `sub_batch_size` is not a multiple of `batch_size` for one (or more) of the minibatches.

EDIT: actually even incomplete sub-batches seem to be handled correctly, given that the code uses the true `stop-start`.

I meant a problem like this:

| i | sample loss |
| - | ----: |
| 0 | 2 |
| 1 | 1 |
| 2 | 1 |

with batch_size = 2

**Average of batch averages**
```
batch_loss_1 = (2+1)/2 = 3/2
batch_loss_2 = 1/1
epoch_loss = (batch_loss_1 + batch_loss_1)/2 = 5/4
```

**Average of sample losses**
```
epoch_loss = (2+1+1)/2 = 4/4
```

@mberr thanks for clarifying, that makes sense. 

Anyway, I guess that users can also avoid this kind of corner case by using the `drop_last` option if they need to.