zhiminy/EvalEng huggingface dataset:
Repository: pykeen/pykeen
Issue: Deprecated get_tail_prediction_df function in Understanding the Evaluation tutorial
Root cause: Documentation Deficiency

### Describe the bug

The function should be replaced by `predict_target`. Interestingly, running this I get Belgium located in Belgium, not sure if this is expected behavior. 

### How to reproduce

```python
from pykeen.datasets import get_dataset
from pykeen.pipeline import pipeline
dataset = get_dataset(dataset="countries")
result = pipeline(dataset=dataset, model="mure")

from pykeen.models.predict import get_tail_prediction_df

df = get_tail_prediction_df(
    model=result.model,
    head_label="belgium",
    relation_label="locatedin",
    triples_factory=result.training,
    add_novelties=False,
)
```



### Environment

```
| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 6.8.0-40-generic         |
| Time            | Fri Aug 16 10:25:39 2024 |
| Python          | 3.12.4                   |
| PyKEEN          | 1.10.2                   |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 2.3.1                    |
| CUDA Available? | false                    |
| CUDA Version    | 12.1                     |
| cuDNN Version   | 8902                     |
```

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Thanks for your report.

> Interestingly, running this I get Belgium located in Belgium, not sure if this is expected behavior.

This might be an artifact of training for a short amount of time. I increased the number of epochs in the example shown in the documentation, and now the predictions look more meaningful

|   tail_id |    score | tail_label         |
|----------:|---------:|:-------------------|
|       265 | -1.02144 | western_europe     |
|        77 | -1.7295  | europe             |
|        69 | -2.21642 | eastern_europe     |
|       216 | -2.32269 | south-eastern_asia |
|       173 | -2.39417 | northern_europe    |

Hi, yes it was just due to training for too few epochs.
Repository: pykeen/pykeen
Issue: Add a documentation section about constrainer vs. normalizer vs. regularizer
GitHub labels: enhancement

### Problem Statement

For newcomers, the distinction between *contrainer*, *normalizer* and *regularizer* might be confusing.

### Describe the solution you'd like

Include a longer description of the following in the documentation / either as a separate page, or as part of the [representations tutorial](https://pykeen.readthedocs.io/en/stable/tutorial/representations.html) (which also needs an update)

- *contrainer*: enforced; outside of gradient tracking
- *normalizer*: enforced; part of computation graph & gradient tracking
- *regularizer* encouraged by extra loss term

### Describe alternatives you've considered

It is already described to some extent in the `Embedding` documentation; however, regularizers are added elsewhere. Moreover, it would be good to have a single link to refer to from modules/models which make use of it.

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a bug report (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)

Should we make a protocol for each of these that we can more thoroughly document than the type hints we have so far?

> Should we make a protocol for each of these that we can more thoroughly document than the type hints we have so far?

Yes, sounds like a good idea!
Repository: pykeen/pykeen
Issue: Evaluation loop of pipeline crashes: torch_max_mem.api: Encountered tensors on device_types={'cpu'} while only ['cuda'] are considered safe for automatic memory utilizaiton maximization. 
Root cause: Configuration Error

### Describe the bug

Hi, 

I want to train knowledge graph embeddings with the `pipeline` function. During the evaluation I receive the warning _WARNING:torch_max_mem.api:Encountered tensors on device_types={'cpu'} while only ['cuda'] are considered safe for automatic memory utilization maximization. This may lead to undocumented crashes (but can be safe, too)._

Then the process crashes and a error _Process finished with exit code 137 (interrupted by signal 9:SIGKILL)_ is reported. 

The full output looks as follows: 

<img width="2265" alt="Screenshot 2024-10-11 at 17 27 24" src="https://github.com/user-attachments/assets/ac09ceae-7c74-4649-9c3b-12a964008689">

Does anyone know where this error is coming from and how to fix it ? 
Any help is appreciated! :-) thanks! 

Cheers, 
Luisa 

### How to reproduce

My code looks as follows : 
```python
result = pipeline(
    model=model_name,
    dataset="WN18RR",
    model_kwargs=dict(embedding_dim=embedding_dim),
    optimizer_kwargs=dict(lr=0.001),
    training_kwargs=dict(num_epochs=1, use_tqdm_batch=False),
)
```

### Environment

I am using Pykeen version 1.10.2 and Python 3.11. I am on MacOS with M1 chip but the device used here is `cpu`. 


### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hey @LuisaWerner ,

the error is likely caused by an out-of-memory issue. A short-term fix is to explicitly set a (sufficiently small) evaluation batch size, e.g.,
```python
from pykeen.pipeline import pipeline

result = pipeline(
    ...
    evaluation_kwargs=dict(batch_size=32),
)
```

Interestingly, the maximum batch size should already be set to 32 for non-`cuda` devices, cf.

https://github.com/pykeen/pykeen/blob/497756f44e8a1a93243c872d3f1c9b7c739cc315/src/pykeen/utils.py#L1560-L1586

but maybe this is not yet included in the 1.10.2 release 🤔 

EDIT: this change seems to be quite fresh, cf https://github.com/pykeen/pykeen/pull/1428 😅 

New release should be coming up soon ;)

Thanks for your fast reply. The quick-fix worked in my case! I still get the warning but the system doesn't crash. Thank you :-)
Repository: pykeen/pykeen
Issue: TriplesFactory.new_with_restriction() throws an exception when create_inverse_triples=True
Root cause: Algorithmic Error

The following code throws the following exception when `create_inverse_triples=True`, but not `create_inverse_triples=False`.

```python
from pykeen.datasets.nations import NATIONS_TEST_PATH, NATIONS_TRAIN_PATH
from pykeen.triples import TriplesFactory


def main():
    create_inverse_triples = True

    training = TriplesFactory(
        path=NATIONS_TRAIN_PATH,
        create_inverse_triples=create_inverse_triples,
    )
    testing = TriplesFactory(
        path=NATIONS_TEST_PATH,
        entity_to_id=training.entity_to_id,
        relation_to_id=training.relation_to_id,
        create_inverse_triples=create_inverse_triples,
    )

    evaluation_relation_whitelist = sorted(training.relation_to_id)[:4]
    # print(evaluation_relation_whitelist)

    testing.new_with_restriction(relations=evaluation_relation_whitelist)

if __name__ == '__main__':
    main()
```

Gives 

```python-traceback
Traceback (most recent call last):
  File "/Users/cthoyt/dev/pykeen/scratch/tst.py", line 49, in <module>
    main()
  File "/Users/cthoyt/dev/pykeen/scratch/tst.py", line 36, in main
    testing.new_with_restriction(relations=evaluation_relation_whitelist)
  File "/Users/cthoyt/dev/pykeen/src/pykeen/triples/triples_factory.py", line 641, in new_with_restriction
    relations = list(relations) + list(map(self.relation_to_inverse.__getitem__, relations))
KeyError: 'accusation_inverse'
```


@cthoyt What would be the restriction to use with inverse relations? Should the user explicitly select `inverse_relations`? Maybe not, since it is a model hyperparameter, rather than something in the data itself.

🤷‍♀️ 

i'm fried for today. will figure this out on thursday (tomorrow is Karneval... so yeah no coding will be done)



the problem is that `evaluation_relation_whitelist` contains relations with inverse suffix

Has been solved by switching to ID-based triples and deferring creation of inverse triples in #193 

I have verified that on the current `master` the snippet from above does not throw any exception.

Gotta love having an actual minimal example in an issue, right?
Repository: pykeen/pykeen
Issue: PyKEEN 1.11.3 Release

@mberr is there anything we should try getting into the next release?

- [ ] Update PyKEEN to use cookiecutter-snekpack
- [ ] #1220
- [ ] #1306

jk, released 1.11 because it's been too long

still didn't get around to these things, but made another release
Repository: pykeen/pykeen
Issue: `relation_to_inverse` does not contain inverses for all relations.

The test currently fails, since `relation_to_inverse` does not contain inverses for all relations. :thinking: 

```python
len(testing.relation_to_id) == 110
len(testing.relation_to_inverse) == 41
```

_Originally posted by @mberr in https://github.com/pykeen/pykeen/pull/147#issuecomment-725307872_

I think this is caused by re-creating the inverse relations in each TriplesFactory separately. It can be solved with #57 

Has been made obsolete by #193
Repository: pykeen/pykeen
Issue: Resolve FutureWarning with `torch>=2.5`
Root cause: External Dependency Breakage

### Problem Statement

Running code with `torch~=2.5` gives some future warnings:

> FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.

### Describe the solution you'd like

Passing `weights_only=True` in all places we use `torch.load`, and use `torch.serialization.add_safe_globals` is necessary.

### Describe alternatives you've considered

Keeping the status quo will break at some point in the future.

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a bug report (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)

Good to have this one logged - is this resolved (or partially resolved) with the PR https://github.com/pykeen/pykeen/pull/1494? 

I think yes

Sweet! 

The repo might benefit from an Issues audit to close stale or resolved issues. Is that something the maintainers can do? 

@chartsNgraphs we do this periodically, but we have a lot less steam at the moment than we have in the past

closed by #1494
Repository: pykeen/pykeen
Issue: Bugs when using lr_scheduler
Root cause: Interface Contract Mismatch

When using lr_scheduler, pykeen attempts to restore the parameter list from the instance of lr_scheduler:

https://github.com/pykeen/pykeen/blob/8b4d4811bb70a9765f4c5dd45011cdd87749502a/src/pykeen/training/training_loop.py#L111-L118

This is a clever implementation, but it may fail in cases where the parameter names do not match the attribute names. For example, in the case of pytorch.optim.lr_scheduler.LambdaLR:

https://github.com/pytorch/pytorch/blob/9990b47ea3a86427a9f1c774f6df69d60fd9a3b8/torch/optim/lr_scheduler.py#L320-L338

Here, lr_lambda is mapped to self.lr_lambdas.
Repository: pykeen/pykeen
Issue: Why can't I set a device in a pipeline?
Root cause: Documentation Deficiency

### Describe the bug

TypeError: __init__() got an unexpected keyword argument 'device'
![image](https://github.com/user-attachments/assets/825ffc9b-fb25-44d1-b4d9-cee8bebd9cf9)


### How to reproduce

```
from pykeen.pipeline import pipeline
dataset = 'Hetionet'
model = 'RGCN'
proj_name = f"{dataset}_{model}"

pipeline_result = pipeline(
    dataset=dataset,
    model=model,
    training_loop='sLCWA',
    negative_sampler='bernoulli',
    evaluator='RankBasedEvaluator',
    stopper='early',
    lr_scheduler='ExponentialLR',
    lr_scheduler_kwargs=dict(gamma=0.99,),
    result_tracker='tensorboard',
    result_tracker_kwargs=dict(
        experiment_path=f'tb-logs/{proj_name}',
    device='cpu',
    ),
)
pipeline_result.save_to_directory(proj_name)
```

### Environment

![image](https://github.com/user-attachments/assets/3f9d9637-89f4-4af3-8141-baba1c5cd04a)


### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

This problem has been solved. The reason is that the device parameter is in the wrong position and should be placed before the result parameter.
Repository: pykeen/pykeen
Issue: What's the best way to re-associate entities/relationships to their trained embeddings for Backfill?
Root cause: Documentation Deficiency

I am attempting to leverage the warm start training feature demo'd in https://github.com/pykeen/pykeen/blob/master/docs/source/examples/nn/representation/backfill.py. 

In this example, the representations for entity backfill are simulated tensors, and the relation representations are "skipped for simplicity". 

This seems to gloss over a pretty fundamental part of a warm start training pipeline - that is, the association of existing learned embeddings to the nodes they represent. How can this best be achieved? 

My use case, generally, will follow: 
1. train embeddings
2. graph data may update or new nodes & relationships be added
3. I will retrain the representation model, but starting with the original learned embeddings where I have them for that data. 

At this point, I'm stuck on how to re-associate the entities to ids to ensure that the backfill representation base_ids and base_repr features are correctly associated. 

Am I overthinking this? The support is much appreciated! 

Hi @chartsNgraphs ,

I tried to adjust the example to be more realistic and better show what needs to be done in https://github.com/pykeen/pykeen/pull/1493

Does this answer your questions already? If not, please let me know what you would additionally want to know, so we can further improve the example.

It absolutely does! Thank you very much, @mberr!
Repository: pykeen/pykeen
Issue: Add ConceptNet
Root cause: Unimplemented Feature Gap

Issue #2 brought attention to the [ConceptNet](http://conceptnet.io/) as a possible dataset to include with PyKEEN. They provide a tab-separated dump of the database [here](https://s3.amazonaws.com/conceptnet/downloads/2019/edges/conceptnet-assertions-5.7.0.csv.gz). It is not pre-stratified into training/testing/evaluation sets.

Because this file has additional columns besides head, relation, and tail, its inclusion will also require an updated to the `SingleTabbedDataset` such that the `usecols` keyword argument can be specified in the dataset's `__init__()`

Blocked by #196 because splitting algorithm is currently too slow for big datasets (with more than ~5 million triples)
Repository: pykeen/pykeen
Issue: GPU Memory fills up during hpo_pipeline trials and between subsequent runs
Root cause: Resource Mishandling

**Describe the bug**
While training my hpo_pipeline I noticed that the GPU memory gets more and more filled with every trial. This leads to some memory intensive models like ConvE bailing out on OOM errors after trial 3 or 4. Is there any way to reduce GPU memory burden between the trials?

Also I couldn't find a way to free my memory between the different runs. I do HPO for several models sequentially in a for loop, deleting the HpoPipelineResult object after i have written the results to disk and then calling torch.cuda.empty_cache() before starting the next model's HPO but it still retains GB of data in GPU memory even though all objects are deleted. The only way I found to free the memory is to exit the whole script and start a new one. Is there a way to free all the memory after a model is finished with training? 

**To Reproduce**
Steps to reproduce the behavior:
At the moment all my GPUs are busy so I couldn't come up with an MWE to reproduce the error.

**Expected behavior**
I expected the GPU memory consumption not to increase monotonically with every trial. I also expected to free my GPU memory by deleting the HpoPipelineResult object and freeing the cache.

**Environment**
OS: Ubuntu 18.04.5 LTS
Python version: 3.8.5
Version of this software: 1.0.6-dev
Versions of required Python packages:
click 7.1.2
click-default-group 1.2.2
numpy 1.19.2
torch 1.6.0
torch-cluster 1.5.7
torch-geometric 1.6.1
torch-scatter 2.0.5
torch-sparse 0.6.7
torch-spline-conv 1.2.0
tqdm 4.51.0


@fratajcz 
Thanks for reporting this. We have encountered the same issue. However, it's tricky to reproduce and also seems to differ between hardware and environment settings.
We're currently investigating this together with #129 and are working on it to provide a solution soon.

I also get this issue, particularly when using TuckER. Every single trial will die with messages like the following:

`WARNING:pykeen.training.training_loop:The training loop just failed during epoch 1 due to error CUDA out of memory. Tried to allocate 576.74 GiB (GPU 0; 31.72 GiB total capacity; 22.62 GiB already allocated; 7.87 GiB free; 22.75 GiB reserved in total by PyTorch).
`

Somehow it seems to be asking for a rather insane amount of GPU memory! It seems to be mostly be related to TuckER, although other models do occasionally request a lot of memory as well.
Repository: pykeen/pykeen
Issue: Improve "Using learned embeddings" with concrete example for subset of indices
GitHub labels: documentation

In the [Using learned embeddings](https://pykeen.readthedocs.io/en/stable/tutorial/first_steps.html#using-learned-embeddings) section of the docs it'd be nice to have some insight into how the embeddings can be accessed with the entities or relation ids, maybe add an example too. 

I'm happy to make a PR if you find this useful.

I think this tutorial is pretty good and explicit about where the indices go to get the representations, but it is currently missing a concrete example that uses a specific subset of indices instead of just passing `indices=None`, which gets everything. This can be improved

Thanks for the quick reply, would you like me to make a PR or will you make the changes?

Contribution welcome! Thanks!
Repository: pykeen/pykeen
Issue: Getting embeddings from Node literals models
Root cause: Documentation Deficiency

### Describe the bug

I have done HPO and training of ComplExLiteral model and wanted to get the embeddings of nodes for further downstream processing like ML. However when I use the normal method to get embeddings as follows, I get all the embeddings for all nodes nearly the same at -1. What should I do to get the right embeddings?
```python
import torch
model = torch.load("trained_model.pkl", map_location=torch.device('cpu'))
    
entity_embeddings_array = (
    model.entity_representations[0](indices=None).detach().cpu().numpy().real
)
```

### How to reproduce

```python
import torch
model = torch.load("trained_model.pkl", map_location=torch.device('cpu'))
    
entity_embeddings_array = (
    model.entity_representations[0](indices=None).detach().cpu().numpy().real
)
```

### Environment

| Key             | Value                       |
|-----------------|-----------------------------|
| OS              | posix                       |
| Platform        | Linux                       |
| Release         | 3.10.0-1160.80.1.el7.x86_64 |
| Time            | Tue Mar  4 14:32:53 2025    |
| Python          | 3.11.4                      |
| PyKEEN          | 1.10.2-dev                  |
| PyKEEN Hash     | UNHASHED                    |
| PyKEEN Branch   |                             |
| PyTorch         | 2.0.1+cu117                 |
| CUDA Available? | false                       |
| CUDA Version    | 11.7                        |
| cuDNN Version   | 8500                        |

### Additional information

_No response_

### Issue Template Checks

- [x] This is not a feature request (use a different issue template if it is)
- [x] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [x] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed
Repository: pykeen/pykeen
Issue: Epic: Add weights and/or features to triples
Root cause: Unimplemented Feature Gap

There are many scenarios in which triples themselves have weights, such as chemical-protein interactions, which have an activity value associated with them.

This has been requested in several places, so this issue can serve as a hub for connecting discussions

## See Also

- https://github.com/pykeen/pykeen/pull/1142
- https://github.com/pykeen/pykeen/discussions/973
- #1529
- https://github.com/pykeen/pykeen/pull/1532
- https://github.com/pykeen/pykeen/issues/1533 
  - https://github.com/pykeen/pykeen/pull/1534

@cthoyt  I'll second this! Is the edge weight feature still potentially scheduled?