Repository: pykeen/pykeen
Issue: Load models trained in older pykeen versions
Root cause: Documentation Deficiency

### What is your question

After upgrading pykeen I cannot load models trained in previous versions because some of the pickled dependencies have changed. I'm pretty sure that this is a typical pickle annoyance so I hesitate to count this as either bug / feature request. But I'm very curious how you guys deal with this issue in practice!

#### how to reproduce
- checkout pykeen v1.7.0
- produce a test model file 
```python
from pykeen.pipeline import pipeline


if __name__ == '__main__':
    result = pipeline(dataset='Nations', model='RotatE')
    result.save_to_directory("v1.7.0")
```
- checkout pykeen v1.8.2
- attempt to load the old model file
```python
import torch

if __name__ == '__main__':
    torch.load("v1.7.0/trained_model.pkl")
```

#### what happens
```shell
/home/jason/anaconda3/envs/pykeen/bin/python /home/jason/dev/pykeen/train_load.py
Traceback (most recent call last):
  File "/home/jason/dev/pykeen/train_load.py", line 4, in <module>
    model = torch.load("v1.7.0/trained_model.pkl")
  File "/home/jason/anaconda3/envs/pykeen/lib/python3.8/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/jason/anaconda3/envs/pykeen/lib/python3.8/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/home/jason/anaconda3/envs/pykeen/lib/python3.8/site-packages/torch/serialization.py", line 875, in find_class
    return super().find_class(mod_name, name)
ModuleNotFoundError: No module named 'pykeen.nn.emb'

Process finished with exit code 1
```


### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.13.0-44-generic        |
| Time            | Wed Jun  1 14:11:08 2022 |
| Python          | 3.8.12                   |
| PyKEEN          | 1.8.2                    |
| PyKEEN Hash     | daaf4520                 |
| PyKEEN Branch   |                          |
| PyTorch         | 1.10.2                   |
| CUDA Available? | true                     |
| CUDA Version    | 11.3                     |
| cuDNN Version   | 8200                     |


### Issue Template Checks

- [X] This is not a bug report (use a different issue template if it is)
- [X] This is not a feature request (use a different issue template if it is)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hi @jas-ho ,

by default we save the full model, as it allows to simply load the file and directly run the model. However, this leads to failures when the model classes' structure has changed.

The next option you can try saving only the state dict, i.e., the model's weights (cf. https://pytorch.org/docs/stable/notes/serialization.html#saving-and-loading-torch-nn-modules )
1. with old PyKEEN version, save the model's state_dict using
```python
import torch
from pykeen.pipeline import pipeline

if __name__ == '__main__':
    result = pipeline(dataset='Nations', model='RotatE')
    torch.save(result.model.state_dict(), "v1.7.0/model.state_dict.pt")
```

2. with the new PyKEEN version, instantiate the model first, then load the state-dict
```python
import torch
from pykeen.datasets import get_dataset
from pykeen.models import model_resolver

if __name__ == '__main__':
    dataset = get_dataset(dataset="Nations")
    model = model_resolver.make("RotatE", triples_factory=dataset.training)
    model.load_state_dict(torch.load("v1.7.0/model.state_dict.pt"))
```

Notice that you need to take care of instantiating the model here.

This still may break, if we change the inner structure of models such that the weight names are affected. In this case, you manually need figure out the mapping between weights. 

Hi @mberr, 

many thanks for the super quick response and providing some snippets. Unfortunately, the issue regarding renaming of model weights seems to affect already the transition 1.7 to 1.8.2 (at least for RotatE). Running your snippet 1 on 1.7.0 and snippet 2 on 1.8.2 gives (on my machine):
```shell
Traceback (most recent call last):
  File "/home/jason/dev/pykeen/train_load.py", line 8, in <module>
    model.load_state_dict(torch.load("v1.7.0/model.state_dict.pt"))
  File "/home/jason/anaconda3/envs/pykeen/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for RotatE:
	Missing key(s) in state_dict: "entity_representations.0._embeddings.weight", "relation_representations.0._embeddings.weight". 
	Unexpected key(s) in state_dict: "regularizer.weight", "regularizer.regularization_term", "entity_embeddings._embeddings.weight", "relation_embeddings._embeddings.weight". 

Process finished with exit code 1
```

Do you know of any principled approaches to this issue? 

It seems kind of similar to schema migrations in databases. Are there best practices such that model migrations becomes easy / automatic? Or do you think it's just not a big issue in practice for most people?

From our side, we do not have the resources to maintain these migrations, and only *guarantee* that you can reload model's from the same version :confused: the weight structure does not change too often though, so most of the time you might be lucky. This time, the migration from old-style RotatE to new-style happened in between (cf. https://github.com/pykeen/pykeen/pull/877).

The principled approach would be to
1. Look at the state-dict dictionaries in the old and new version, and try to match the keys.
In your specific case, I think the following should work.

```python
import torch
from pykeen.datasets import get_dataset
from pykeen.models import model_resolver

if __name__ == '__main__':
    dataset = get_dataset(dataset="Nations")
    model = model_resolver.make("RotatE", triples_factory=dataset.training)
    state_dict = torch.load("v1.7.0/model.state_dict.pt")
    # these are the changes from migrating RotatE from old-style to new-style model
    for old_name, new_name in [
        ("entity_embeddings._embeddings.weight", "entity_representations.0._embeddings.weight"),
        ("relation_embeddings._embeddings.weight", "relation_representations.0._embeddings.weight"),
    ]:
        state_dict[new_name] = state_dict.pop(old_name)
    # your new model does not have a regularizer
    for name in ["regularizer.weight", "regularizer.regularization_term"]:
        state_dict.pop(name)
    model.load_state_dict(state_dict)
```

2. Notice, that even after matching the names, it may happen that the migration fails, because the *semantic* of a weight has chanced. I remember that at some point we e.g., switched to using the PyTorch standard way of representing complex numbers in real tensors (cf. [here](https://pytorch.org/docs/stable/complex_numbers.html?highlight=complex)), from a different dimension ordering we had before. This may have affected RotatE, too (I do not remember right now). In that case, you may need to process the values of the state-dict dictionary, too.



very helpful information, thank you! do you think it's worth adding to the documentation? 

I imagine somewhere in between https://pykeen.readthedocs.io/en/stable/tutorial/checkpoints.html#word-of-caution-and-possible-errors and https://pykeen.readthedocs.io/en/stable/tutorial/checkpoints.html#word-of-caution-and-possible-errors  
I can volunteer a PR for that if you think that's the correct place for it

That would be very much welcome!

We could also consider putting that in an extra section, since it is not only valid for training checkpoint, but also for saving models e.g., via [pykeen.pipeline.PipelineResult.save_model](https://pykeen.readthedocs.io/en/stable/api/pykeen.pipeline.PipelineResult.html?#pykeen.pipeline.PipelineResult.save_model).

@mberr opened a PR https://github.com/pykeen/pykeen/pull/963