Repository: pykeen/pykeen
Issue: Why do I get an OOM error during evaluation?
Root cause: Documentation Deficiency

I have recently starting experimenting with PyKEEN and I have noticed some issues with GPU implementation details.

**Describe the bug**
GPU memory requirement blows up tenfold during evaluation in comparison to training. 
This leads to OOM errors after having waited patiently for the training to finish.

**To Reproduce**
Steps to reproduce the behavior:

```
import pykeen
from pykeen.pipeline import pipeline

result = pipeline(
    dataset='OpenBioLink',
    model='RotatE',
    regularizer="PowerSumRegularizer",
    negative_sampler='basic',
    evaluator="rankbased",
    negative_sampler_kwargs=dict(num_negs_per_pos=1),
    evaluator_kwargs=dict(filtered=True),
    evaluation_kwargs=dict(batch_size=64),
)

print(result)
```

Monitor the GPU memory usage during training using nvidia-smi. For me it sits pretty constantly at 2747MiB / 11019MiB.
However, during evaluation i get 

```
RuntimeError: CUDA out of memory. Tried to allocate 17.26 GiB (GPU 0; 10.76 GiB total capacity; 933.36 MiB already allocated; 8.90 GiB free; 1.09 GiB reserved in total by PyTorch)
```

**Expected behavior**
It evaluates batch-wise and doesn't need ten times more memory than during training.

**Environment (please complete the following information):**
 - OS: Ubuntu 18.04.5 LTS
 - Python version: 3.8.5
 - Version of this software: 1.0.5
 - Versions of required Python packages:
   - click                 7.1.2
   - click-default-group   1.2.2
   - numpy                 1.19.2
   - torch                 1.6.0
   - torch-cluster         1.5.7
   - torch-geometric       1.6.1
   - torch-scatter         2.0.5
   - torch-sparse          0.6.7
   - torch-spline-conv     1.2.0
   - tqdm                  4.51.0

**Additional information**
The magnitude of the problem obviously varies with the size of the dataset that has been chosen as well as with the method and with the evaluation batch size. However, the increase in memory consumption in relation to the training remains a lot. This basically rules out quick prototyping even on high-end consumer grade graphics cards like the RTX 2080 Ti in my case even for modestly sized graphs.

I would be very delighted for every hint on how to circumvent/alleviate this problem.


Hi @fratajcz 

This is the case because in the pipeline configuration you set
`evaluation_kwargs=dict(batch_size=64)`
This means that the evaluation is forced to run with a batch size of 64, which in this case is too big for the memory at hand because the dataset is not small and you activated evaluation filtering, which increases the memory imprint.

I suggest that you omit the line `evaluation_kwargs=dict(batch_size=64)`.
PyKEEN has memory-optimization built in, i.e. it will find the biggest possible batch size itself.

Hi @lvermue 

Thanks for the response and clarification. I didn't know that it would find the biggest possible batch size itself, that's very handy.

@fratajcz FYI we've got a list of the clever performance tricks that @mberr @lvermue and @mali-git came up with on https://pykeen.readthedocs.io/en/latest/tutorial/performance.html - many of them are related to memory usage and speed