Repository: pykeen/pykeen
Issue: Shape Mismatch upon initializing pretrained ComplEx embeddings
Root cause: Documentation Deficiency

### Describe the bug

I am using [these pretrained embeddings](https://graphvite.io/docs/latest/pretrained_model) for a prediction and evaluation task on Wikidata5M. These models have an embedding dimension of 512 and are thus fairly big (~9.4 GB). I'm trying to load the embeddings for the ComplEx model using the `PretrainedInitializer` utility, but when creating the `ComplEx` instance, I'm getting this error:

```python
Traceback (most recent call last):
  File "/home/mte/KGE_Query_Completion/convert_model/create_pykeen_model.py", line 57, in <module>
    main()
  File "/home/mte/KGE_Query_Completion/convert_model/create_pykeen_model.py", line 33, in main
    model = ComplEx(
  File "/opt/conda/lib/python3.10/site-packages/pykeen/models/unimodal/complex.py", line 81, in __init__
    super().__init__(
  File "/opt/conda/lib/python3.10/site-packages/pykeen/models/nbase.py", line 349, in __init__
    self.reset_parameters_()
  File "/opt/conda/lib/python3.10/site-packages/pykeen/models/base.py", line 146, in reset_parameters_
    self._reset_parameters_()
  File "/opt/conda/lib/python3.10/site-packages/pykeen/models/nbase.py", line 100, in _reset_parameters_
    module.reset_parameters()
  File "/opt/conda/lib/python3.10/site-packages/pykeen/nn/representation.py", line 401, in reset_parameters
    self._embeddings.weight.data = self.initializer(
  File "/opt/conda/lib/python3.10/site-packages/pykeen/nn/init.py", line 240, in __call__
    raise ValueError(f"shape does not match: expected {self.tensor.shape} but got {x.shape}")
ValueError: shape does not match: expected torch.Size([4818298, 512]) but got torch.Size([4818298, 512, 2])
```

The error is raised in the `PretrainedInitializer` class here:

https://github.com/pykeen/pykeen/blob/c94213cafc5f9cebad5d23ff0a89c869ae1d5ee0/src/pykeen/nn/init.py#L237-L241

I debugged a little and tracked down the error to line 392 of `representation.py`:

https://github.com/pykeen/pykeen/blob/c94213cafc5f9cebad5d23ff0a89c869ae1d5ee0/src/pykeen/nn/representation.py#L387-L397

Here, if the model is complex, the new shape is calculated by adding two tuples together. The shape of my ComplEx embeddings is `torch.Size([4818298, 512])`, as can be seen from the error message. To my understanding, the first tuple is 1-dimensional and the second one 2-dimensional, yielding a 3-dimensional tuple with 2 as the 3rd dimension, i.e. as follows:

```python
>> shape = torch.Size([4818298, 512]
>> tuple(shape[:-1]) + (shape[-1], 2)
(4818298, 512, 2)
```

Not sure whether the ComplEx embedding is supposed to be 3-dimensional.

### How to reproduce

This is my code and I encountered the error on creating the ComplEx model. I ran the same code for other models like DistMult and TransE by replacing values and didn't run into this error. 

```python
import pickle
from pathlib import Path

import numpy as np
import torch
from pykeen.models import ComplEx
from pykeen.triples import TriplesFactory
from pykeen.nn.init import PretrainedInitializer

# Specify model to load
model_name = 'complex'

# Paths to saved embeddings and mappings
save_dir = Path(f'./embeddings/{model_name}')
triples_factory_path = save_dir / 'training_factory'
entity_embeddings_path = save_dir / 'entity_embeddings.npy'
relation_embeddings_path = save_dir / 'relation_embeddings.npy'

# Load triples factory with mappings
train_factory = TriplesFactory.from_path_binary(triples_factory_path)

# Load embeddings saved as numpy array and convert to torch.Tensor()
entity_embeddings = torch.from_numpy(np.load(entity_embeddings_path))
relation_embeddings = torch.from_numpy(np.load(relation_embeddings_path))

# Create complex model using pretrained embeddings with
# PretrainedInitializer
model = ComplEx(
    triples_factory=train_factory,
    embedding_dim=512,
    entity_initializer=PretrainedInitializer(tensor=entity_embeddings),
    relation_initializer=PretrainedInitializer(tensor=relation_embeddings)
)
```

To reduce memory consumption, I extracted the embeddings and mappings of the model and stored them in separate files. The embeddings are 2-dimensional numpy `.npy` files which are loaded and converted to `torch.Tensor()`. The train factory uses the mappings provided by the pretrained model.

I have a VM setup with 32 GB RAM, which the model almost exceeds. The memory usage steadily rises when the model is being created, to the extent that the consumed memory almost doubles (30 GB in total), as if the embeddings are somehow copied or duplicated. That is also odd.

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.10.0-27-cloud-amd64    |
| Time            | Wed Jan 17 23:39:28 2024 |
| Python          | 3.10.13                  |
| PyKEEN          | 1.10.1                   |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 2.0.0+cu118              |
| CUDA Available? | false                    |
| CUDA Version    | 11.8                     |
| cuDNN Version   | 8700                     |


### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hi @mortenterhart ,

ComplEx uses vectors of complex numbers as representation. `torch` does not yet fully support complex tensors (at least not in all places); thus, they are sometimes "flattened", i.e., a 256-dimensional complex vector is seen as a 512-dimensional real vector.

- PyKEEN internally stores complex embeddings with $d$ dimensions as a $(d, 2)$ dimensional matrix of real numbers (in line with [`torch.view_as_real`](https://pytorch.org/docs/stable/generated/torch.view_as_real.html#torch-view-as-real)). `PretrainedInitializer` currently just stores the "plain" weights, so you'd likely need to convert the complex tensor to a larger real one first and then use this.

- I do not know how the pre-trained weights from https://graphvite.io/docs/latest/pretrained_model are stored; you would need to make sure that the ordering is same as PyKEEN expects them.


Here is a small (self-contained) example how you can update your usage of `PretrainedInitializer`:
```python
import torch

from pykeen.models import ComplEx
from pykeen.nn.init import PretrainedInitializer
from pykeen.pipeline import pipeline

# we use this to get some "pre-trained" embeddings
model_kwargs = dict(embedding_dim=8)
pipeline_result = pipeline(
    model="complex", dataset="nations", model_kwargs=model_kwargs, training_kwargs=dict(num_epochs=0)
)

# we could also store them first and re-load them afterward
entity_embeddings = pipeline_result.model.entity_representations[0](indices=None)
relation_embeddings = pipeline_result.model.relation_representations[0](indices=None)


model = ComplEx(
    triples_factory=pipeline_result.training,
    **model_kwargs,
    # note the "torch.view_as_real"
    entity_initializer=PretrainedInitializer(tensor=torch.view_as_real(entity_embeddings)),
    relation_initializer=PretrainedInitializer(tensor=torch.view_as_real(relation_embeddings)),
)
```

Thank you for the detailed answer. The complex tensors in the pretrained embedding were in fact stored in 2-tuples with real and imaginary part. Flattening the tensors into real numbers with [`torch.view_as_real()`](https://pytorch.org/docs/stable/generated/torch.view_as_real.html#torch-view-as-real) did the trick for me here, so I could finally evaluate the complex embeddings.