zhiminy/EvalEng huggingface dataset:
Repository: pykeen/pykeen
Issue: Protobuf requirement contriant 
Root cause: Configuration Error

Will it be possible to change the constraint on the protobuf requirement which currently < 4.0.0. 


The [blame](https://github.com/pykeen/pykeen/blame/master/setup.cfg) says that it was introduced by https://github.com/pykeen/pykeen/commit/424811ad8a8e1f34ca9053eedc9f773557b18815 for https://github.com/pykeen/pykeen/pull/946; @cthoyt do you remember what the issue was?

I remember the new version of protobuf wreaking some kind of havoc, I don't remember exactly what. Shame I changed this without putting a note somewhere. @ewianda we can make a PR that removes this constraint and we can see what happens

No promises that we will be able to relax this in the end, since it is not a direct PyKEEN dependency

#1332 seems to show that relaxing this constraint doesn't have an issue, but it might also be the case that there's another thing that's limiting it, and if you have 4.0+, it will just not work

Since the tests all passed, I merged it, and will leave any hiccups to downstream to deal with 😈 

Thanks
Repository: pykeen/pykeen
Issue: Want to train a model without any evaluate or test dataset
Root cause: Architectural Constraint

Hello, 

I want to train a model on my knowledge graph via pipeline class without providing a test set and want to later use predict_target to predict potential tails for a head-relationship pair. 

The package throws an error when I define None as argument to testing in pipeline. I also tried providing a fake file unrelated to training file, again I obtain an error. How do I go around this problem without testing file affecting the training procedure? 

And may I suggest that, you do not force your users to specify a testing file if they do not need that procedure within pipeline. 

Thanks.

Hi @nerner94 ,

if you want a more customized training that what the `pipeline` function offers as a default, please take a look at what is described under "Beyond the Pipeline" [in this tutorial](https://pykeen.readthedocs.io/en/stable/tutorial/first_steps.html#beyond-the-pipeline).
Repository: pykeen/pykeen
Issue: Bug in wandb result tracker 
Root cause: Documentation Deficiency

### Describe the bug

Hi, I want to use the experiment tracking tool wandb in my pykeen pipeline and followed the instructions [here](https://pykeen.readthedocs.io/en/stable/tutorial/trackers/using_wandb.html). I  set a label to my run as described in the example below ('experiment-1')
In my code (see how to reproduce) I get the error:  "TypeError: init() got an unexpected keyword argument 'experiment' ". 

**Example from documentation**
```python
from pykeen.pipeline import pipeline

pipeline_result = pipeline(
    model='RotatE',
    dataset='Kinships',
    result_tracker='wandb',
    result_tracker_kwargs=dict(
        project='pykeen_project',
        experiment='experiment-1',
    ),
)
```

### How to reproduce

My code looks as follows. 
```python
result = pipeline(
            training=train,
            validation=valid,
            testing=test,
            model='ConvE',
            model_kwargs=dict(predict_with_sigmoid=True),
            loss=BCEWithLogitsLoss,
            training_loop='sLCWA',
            negative_sampler='bernoulli',
            epochs=100
            stopper='early',
            random_seed=123,
            device=device,
            result_tracker='wandb',
            result_tracker_kwargs=dict(project='my_wandb_project', experiment='test'),
        )
```

I get the error:  "TypeError: init() got an unexpected keyword argument 'experiment' ". 

The keyword argument seems not so match the keyword argument in `self.wandb.init`. I solved the problem by checking the [documentation](https://docs.wandb.ai/ref/python/init) of `wandb.init()` and set the keyword argument to `tags`: 

```python
result = pipeline(
            [...]
            result_tracker='wandb',
            result_tracker_kwargs=dict(project='KGE_Family', tags='test'),
        )
```

So I think this should maybe be updated in the documentation :)

### Environment

```
wandb==0.16.0
pykeen==1.10.1
```

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

@LuisaWerner PR welcome!
Repository: pykeen/pykeen
Issue: Importing model_resolver
Root cause: External Dependency Breakage

### Describe the bug

Hello all,

I would like to inform you about the RegistrationNameConflict error.
importing model_resolver throws RegistrationNameConflict.

### How to reproduce

`from pykeen.models import model_resolver`

```python
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/upb/users/c/cdemir/profiles/unix/cs/anaconda3/envs/dice/lib/python3.10/site-packages/pykeen/models/__init__.py", line 52, in <module>
    from .base import Model
  File "/upb/users/c/cdemir/profiles/unix/cs/anaconda3/envs/dice/lib/python3.10/site-packages/pykeen/models/base.py", line 20, in <module>
    from ..losses import Loss, MarginRankingLoss, loss_resolver
  File "/upb/users/c/cdemir/profiles/unix/cs/anaconda3/envs/dice/lib/python3.10/site-packages/pykeen/losses.py", line 172, in <module>
    from class_resolver.contrib.torch import margin_activation_resolver
  File "/upb/users/c/cdemir/profiles/unix/cs/anaconda3/envs/dice/lib/python3.10/site-packages/class_resolver/contrib/torch.py", line 213, in <module>
    lr_scheduler_resolver.register(ReduceLROnPlateau)
  File "/upb/users/c/cdemir/profiles/unix/cs/anaconda3/envs/dice/lib/python3.10/site-packages/class_resolver/base.py", line 170, in register
    raise RegistrationNameConflict(self, key, element, label="name")
class_resolver.base.RegistrationNameConflict: Conflict on registration of name reducelronplateau:
Existing: <class 'torch.optim.lr_scheduler.ReduceLROnPlateau'>
Proposed: <class 'torch.optim.lr_scheduler.ReduceLROnPlateau'>
``` 


### Environment

python 3.10.13
pykeen 1.10.1
torch 2.2.0+cu121
lightning '2.1.3

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Same here.
Environment
python 3.9.*
pykeen 1.10.1
torch 1.13.0+cu116

Hi, this should be solved now. Until our next release, please install from source: https://github.com/pykeen/pykeen?tab=readme-ov-file#installation--
Repository: pykeen/pykeen
Issue: MatKG
Root cause: Unimplemented Feature Gap

**Describe the benchmark KG dataset**:

This is a dataset of materials, their applications, properties, characterization methods, synthesis methods, descriptors, and symmetry labels extracted from over 5 Million scientific publications in material science. It has over 6 million triples covering 70,000 entities. 

E.g., what domain does this benchmark KG cover? Are there any special parts besides triples?

The domain is Material Science. Each triple has another field called 'count', which is a numerical parameter that is a weight. It is the number of documents in which the triples appear.

**Dataset is pre-stratified into train/test/valid** [ ]

NO

**Publication(s) or website describing benchmark KG dataset**:
https://www.nature.com/articles/s41597-024-03039-z.pdf

**URL for downloading the benchmark KG dataset**:
https://zenodo.org/records/10144972

Hi @vinven7 ,

There is currently no support for triple weights, although there has been interest in adding support for this, see #1142.

As a temporary workaround, it may be possible to add a user-defined count threshold to the KG dataset that would discard triples below a certain count. This way, the resulting KG would be in a format compatible with all existing structures.

We would be happy to accept a PR, especially since you seem to be involved in the creation of this KG. If you would like to participate in adding support for triple weights, we would also appreciate your contribution, but this seems to be a larger effort than the baseline I suggested above.
Repository: pykeen/pykeen
Issue: EarlyStopper: show progress bar
Root cause: Unimplemented Feature Gap

### Describe the bug

When an evaluation process is triggered after `frequency` epochs (a parameter of `EarlyStopper`), the `tqdm` progress bar is intentionally not displayed. I believe this could be misleading, as it may give the user the impression that the execution is stuck, whereas the process is actually ongoing in the background.

### How to reproduce

```py
from pykeen.pipeline import pipeline

pipeline_result = pipeline(
    dataset='nations',
    model='transe',
    model_kwargs=dict(embedding_dim=20, scoring_fct_norm=1),
    optimizer='SGD',
    optimizer_kwargs=dict(lr=0.01),
    loss='marginranking',
    loss_kwargs=dict(margin=1),
    training_loop='slcwa',
    training_kwargs=dict(num_epochs=20, batch_size=128),
    negative_sampler='basic',
    negative_sampler_kwargs=dict(num_negs_per_pos=1),
    evaluator_kwargs=dict(filtered=True),
    evaluation_kwargs=dict(batch_size=128),
    stopper='early',
    stopper_kwargs=dict(frequency=5, patience=2, relative_delta=0.002),
)
```

In this case, the evaluation will happen after 5 epochs but the progress bar will be hidden.

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Darwin                   |
| Release         | 23.0.0                   |
| Time            | Tue Mar  5 14:18:11 2024 |
| Python          | 3.11.5                   |
| PyKEEN          | 1.10.1                   |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 2.1.0                    |
| CUDA Available? | false                    |
| CUDA Version    | N/A                      |
| cuDNN Version   | N/A                      |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed
Repository: pykeen/pykeen
Issue: Cosine Annealing with Warm Restart LR Scheduler recieving an unexpected kwarg `T_i`
Root cause: Configuration Error

### Describe the bug

 Calling `CosineAnnealingWarmRestart` as an option in `pykeen.pipeline.pipeline` results in the following error:
  ```
CosineAnnealingWarmRestarts.__init__() got an unexpected keyword argument 'T_i
```

I think this error is because [lr_scheduler_kwargs](https://github.com/pykeen/pykeen/blob/4eb226196c2ec8edaa1775aef918e9cad91d897b/src/pykeen/training/training_loop.py#L648) has initialized a `T_i` state when `T_i` is not required to run the scheduler (`T_i` equal to `T_0` at initialization by [Torch ](https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.CosineAnnealingWarmRestarts.html#torch.optim.lr_scheduler.CosineAnnealingWarmRestarts )so no need to pass `T_i` as part of the state)

### How to reproduce

```python
res = pipeline(
    dataset=wn.WN18,
    model="RotatE",
    model_kwargs=dict(embedding_dim=200),
    training_kwargs=dict(num_epochs=5, batch_size=128),
    optimizer="Adam",
    optimizer_kwargs={"lr": 0.01},
    lr_scheduler="CosineAnnealingWarmRestarts",
    lr_scheduler_kwargs={
        "T_0": 10,
    },
)
```

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.15.0-100-generic       |
| Time            | Tue Apr  2 21:03:50 2024 |
| Python          | 3.11.8                   |
| PyKEEN          | 1.10.2                   |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 2.2.1+cu118              |
| CUDA Available? | true                     |
| CUDA Version    | 11.8                     |
| cuDNN Version   | 8700                     |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hi @turoger ,

the issue should be fixed with https://github.com/pykeen/pykeen/pull/1386 being merged to `master`. It will be included in our next release, but you can already get access by [installing from source](https://github.com/pykeen/pykeen?tab=readme-ov-file#installation--).
Repository: pykeen/pykeen
Issue: Reason for omitting validation inference triples from filtering when doing test evaluation in inductive lp example
Root cause: Documentation Deficiency

Is there a reason you omit the validation triples of the inference (test-ind) graph from filtering in your example [here](https://pykeen.readthedocs.io/en/stable/tutorial/inductive_lp.html#full-inductive-lp-example) when doing evaluation on the test set?

I would expect that the test evaluator would also include the validation triples for filtering, like

```python
test_evaluator = SampledRankBasedEvaluator(
    mode="testing",   # necessary to specify for the inductive mode - this will use inference nodes
    evaluation_factory=dataset.inductive_testing,  # test triples to predict
    additional_filter_triples=[dataset.inductive_inference.mapped_triples,dataset.inductive_validation.mapped_triples],   # filter out true inference triples
)
```

In this particular dataset, `InductiveFB15k237`, the validation and test graphs are guaranteed to be disjoint - so we don't need to filter for them; however, it would not hurt much and might be a good adjustment to make sure you do not forget to do this filtering in other datasets that do not have this guarantee.

I don't think that is true. The training and inference graph are disjoint. But not the test and validation set of the inference graph. E.g. test.txt of fb237_v1_ind contains the triple
**/m/0p9xd	/music/genre/artists	/m/01304j**
While valid.txt of fb237_v1_ind contains the triple 
**/m/02x8m	/music/genre/artists	/m/01304j**
Now for a query genre_artists(?, /m/01304j) the validation triple **/m/02x8m** would not get filtered
Repository: pykeen/pykeen
Issue: tqdm progressbar is still shown although setting `use_tqdm=False`
Root cause: Configuration Error

### Describe the bug

The tqdm progressbar is still shown, although setting `use_tqdm=False`

### How to reproduce

Run any evaluation using `use_tqdm=False`

### Environment

Unable to handle parameter in CooccurrenceFilteredModel: base
| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.15.0-100-generic       |
| Time            | Fri May 24 10:24:58 2024 |
| Python          | 3.8.19                   |
| PyKEEN          | 1.10.2                   |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 2.3.0+cu121              |
| CUDA Available? | true                     |
| CUDA Version    | 12.1                     |
| cuDNN Version   | 8902                     |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed
Repository: pykeen/pykeen
Issue: create_inverse_triples=True fails for the ILPC datasets
Root cause: Interface Contract Mismatch

### Describe the bug

When I try to load one of the ILPC datasets, the program crashes with stacktrace

```python-traceback
Traceback (most recent call last):
  File "baseline.py", line 203, in <module>
    dataset = ILPC2022Large(create_inverse_triples=False)
  File "/home/otts/storage/envs/pykeen10/lib/python3.8/site-packages/pykeen/datasets/inductive/ilpc2022.py", line 73, in __init__
    super().__init__(
TypeError: __init__() got multiple values for keyword argument 'create_inverse_triples'
```

### How to reproduce

```python
from pykeen.datasets.inductive import ILPC2022Large, ILPC2022Small
dataset = ILPC2022Large(create_inverse_triples=False)
```

### Environment

Unable to handle parameter in CooccurrenceFilteredModel: base
| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.15.0-100-generic       |
| Time            | Fri May 24 10:24:58 2024 |
| Python          | 3.8.19                   |
| PyKEEN          | 1.10.2                   |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 2.3.0+cu121              |
| CUDA Available? | true                     |
| CUDA Version    | 12.1                     |
| cuDNN Version   | 8902                     |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed
Repository: pykeen/pykeen
Issue: Improve interpretability of interaction function
GitHub labels: research

Since each interaction model has a different range (e.g., the values of some interaction functions could be between [-inf,inf], some could be [0, inf], some could be [0, 1]), it makes the interpretation difficult. Even worse, this also has to do with interaction models' constrains on embeddings, etc.

It would be good for all interaction functions to define an "interpretation function" that remaps the results onto the range of [0,1]

From @mberr:

> Obtaining such is an active research area and neither straight-forward, nor exists a "unique" solution. In case you are interested, you may want to look for keywords "model calibration" (e.g. Platt scaling), and for KG "triple classification".
Repository: pykeen/pykeen
Issue: issue in "Full Inductive LP Example"
Root cause: Documentation Deficiency

### Describe the bug

In the documentation of "[inductive link prediction](https://github.com/pykeen/pykeen/blob/master/docs/source/tutorial/inductive_lp.rst)", the last example titled "Full Inductive LP Example" (the last example) has two issues. 
Firstly, "`NSSALoss`" used in the model definition has not been imported. Second, after defining "`negative_sampler_kwargs`" in the definition of "`training_loop`", a comma ("`,`") has been missed. 

### How to reproduce

run the example

### Environment

Python 3.8.19
pykeen 1.10.2

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Please grant me access to create a PR to fix this issue, if possible. 

Hi @mohmehmo ,

thanks for looking into this!

Here is a description of how you can make a PR: https://github.com/pykeen/pykeen/blob/master/CONTRIBUTING.md#code-contribution (tl;dr: you create a fork in your own account and open the PR from there).

Hi @mberr 
It's done in PR #1405 

Thanks!
Repository: pykeen/pykeen
Issue: "Class Inheritance Diagram" of documentation about prediction in pyKeen does not show correctly 
GitHub labels: bug

### Describe the bug

The documentation page of pyKeen [prediction ](https://pykeen.readthedocs.io/en/stable/reference/predict.html#class-inheritance-diagram)has no image or description regarding the "Class Inheritance Diagram". Instead, it shows some scripts that may face an error. 

### How to reproduce

Documentation of pyKeen

### Environment

Documentation of pyKeen [here](https://pykeen.readthedocs.io/en/stable/reference/predict.html#class-inheritance-diagram).

### Additional information

Let me know how I can resolve this issue to take a look. 

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hi @mohmehmo ,

thanks for reporting the issue. Right now, I do not see where this figure is actually generated - @cthoyt do you know more?

I am aware of this but not how to fix it at the moment. I'm still on burnout leave so I don't think I'll be able to pick this up in the short term
Repository: pykeen/pykeen
Issue: List of mode's best performance config for each dataset
Root cause: Documentation Deficiency

I am trying to reproduce the results from the paper using RotatE on FB15K-237, but I am having trouble achieving the MRR reported in their paper. I've been experimenting with the hyperparameters and am wondering if there are any other ways to explore and improve the results.

`result = hpo_pipeline(
    model='RotatE',
    training=training,
    validation=val,
    testing=testing,
    epochs = 140,  
    optimizer='adam',
    model_kwargs=dict(
        embedding_dim=1000,
    ),
    negative_sampler='basic',
    negative_sampler_kwargs=dict(num_negs_per_pos=256),
    lr_scheduler_kwargs=dict(
        gamma=9,
        alpha=1
    ),
    optimizer_kwargs=dict(lr=0.00005),
    )
`
Repository: pykeen/pykeen
Issue: Hit@10 does not match with the evaluator result
Root cause: Algorithmic Error

I am testing my model using the 'predict_triples' on the same testing dataset from the pipeline, in the pipeline result, I am able to obtain a **hit@10: 0.37156747776800547** for both head and tail, for tail itself, I have **0.5217922407896023**. However, after running the following code, I am only able to have **0.2227**

<img width="661" alt="Screen Shot 2024-08-03 at 8 14 59 PM" src="https://github.com/user-attachments/assets/7c849c39-9ef7-4a31-9dac-8fe6b9dc6409">