Repository: pykeen/pykeen
Issue: MacroRankBasedEvaluator erroring in early stopper
Root cause: Resource Mishandling

### Describe the bug

When using MacroRankBasedEvaluator with early stopping in the pipeline, the weights dictionary is cleared after each post epoch evaluation when MacroRankBasedEvaluator.finalize() method is called. This results in a `KeyError` when the evaluation is run again after the next epoch as the `head` key has been removed.

### How to reproduce

```python
from pykeen.pipeline import pipeline
from pykeen.datasets import Nations

dataset = Nations()

pipeline_result = pipeline(
    dataset=dataset,
    model="TransE",
    evaluator="macrorankbased",
    evaluator_kwargs=dict(evaluation_factory=dataset.validaton),
    training_kwargs=dict(num_epochs=10),
    stopper="early",
    stopper_kwargs=dict(frequency=1, patience=100, relative_delta=0.0005),
)
```

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Darwin                   |
| Release         | 21.5.0                   |
| Time            | Wed Aug 10 12:34:40 2022 |
| Python          | 3.8.9                    |
| PyKEEN          | 1.9.0                    |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 1.11.0                   |
| CUDA Available? | false                    |
| CUDA Version    | N/A                      |
| cuDNN Version   | N/A                      |


### Additional information

One potential solution might be to instantiate weights as a `defaultdict(list)` in the MacroRankBasedEvaluator __init__ -> https://github.com/pykeen/pykeen/blob/eb556f200e0f8773803faa6e38686aef953f4241/src/pykeen/evaluation/rank_based_evaluator.py#L512

Another issue found in MacroRankBasedEvaluator process_scores -> https://github.com/pykeen/pykeen/blob/eb556f200e0f8773803faa6e38686aef953f4241/src/pykeen/evaluation/rank_based_evaluator.py#L541-L543 where the batch is detached but not sent to cpu. e.g:
```python
key_list = (
    hrt_batch[:, [TARGET_TO_INDEX[key] for key in self._get_key(target=target)]].detach().cpu().numpy().tolist()
)
```

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

@Allaway11 would you please attach the full stack trace?

Sure: 

```python-traceback
No random seed is specified. Setting to 3936677467.
No cuda devices were available. The model runs on CPU
Training epochs on cpu:   0%|                                                                                                                                   | 0/10 [00:00<?, ?epoch/s, loss=1.48, prev_loss=nan]INFO:pykeen.evaluation.evaluator:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value.                                    
INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.
INFO:pykeen.evaluation.evaluator:Evaluation took 0.03s seconds
INFO:pykeen.stoppers.early_stopping:New best result at epoch 1: 0.9475. Saved model weights to /Users/georgeallaway/.data/pykeen/checkpoints/best-model-weights-b12349ac-a6bd-4b0c-acdd-9b6855c34f88.pt
INFO:pykeen.training.training_loop:=> Saved checkpoint after having finished epoch 1.
Training epochs on cpu:   0%|                                                                                                                                   | 0/10 [00:00<?, ?epoch/s, loss=1.4, prev_loss=1.48]INFO:pykeen.evaluation.evaluator:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value.                                    
INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.
Training epochs on cpu:  10%|████████████▎                                                                                                              | 1/10 [00:00<00:01,  7.73epoch/s, loss=1.4, prev_loss=1.48]
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/georgeallaway/Documents/Code/healnet/KBC-KE/check_macro_ranks.py", line 6, in <module>
    pipeline_result = pipeline(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/pipeline/api.py", line 1291, in pipeline
    losses = training_loop_instance.train(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/training/training_loop.py", line 378, in train
    result = self._train(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/training/training_loop.py", line 734, in _train
    callback.post_epoch(epoch=epoch, epoch_loss=epoch_loss)
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/training/callbacks.py", line 438, in post_epoch
    callback.post_epoch(epoch=epoch, epoch_loss=epoch_loss, **kwargs)
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/training/callbacks.py", line 362, in post_epoch
    if self.stopper.should_stop(epoch):
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/stoppers/early_stopping.py", line 230, in should_stop
    metric_results = self.evaluator.evaluate(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/evaluation/evaluator.py", line 208, in evaluate
    rv = evaluate(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/evaluation/evaluator.py", line 682, in evaluate
    relation_filter = _evaluate_batch(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/evaluation/evaluator.py", line 802, in _evaluate_batch
    evaluator.process_scores_(
  File "/Users/georgeallaway/Library/Python/3.8/lib/python/site-packages/pykeen/evaluation/rank_based_evaluator.py", line 545, in process_scores_
    self.weights[target].append(numpy.asarray([self.precomputed_weights[target][k] for k in keys]))
KeyError: 'head'
```

@Allaway11 thanks for reporting the issue. Your suggested changes sound good to me. Would you mind creating a PR with them? :slightly_smiling_face: 

Awesome, happy to make a PR! 🙂 

Closed by #1079