zhiminy/EvalEng huggingface dataset:
Repository: pykeen/pykeen
Issue: AttributeError: 'TransE' object has no attribute 'entity_embeddings' - Results Plotting Demo
Root cause: Validation Gap

### Describe the bug

Dear Pykeen staff, thanks for the great library. 
I'm really starting with it and while running one of the notebooks I got the following error:
`AttributeError: 'TransE' object has no attribute 'entity_embeddings'`

This is the environment I have. 


The model runs but the second chart cant be generated. 
I would appreciate your support. 


### How to reproduce

Just by running this [notebook ](https://github.com/pykeen/pykeen/blob/master/notebooks/results_plots/results_plots.ipynb)

After executing this code

`results = pipeline(
    training=tf,
    testing=tf,
    model = 'TransE',
    model_kwargs=dict(embedding_dim=2),
    training_kwargs=dict(use_tqdm_batch=False),
    evaluation_kwargs=dict(use_tqdm=False),
    random_seed=1,
    device='cpu',
)
results.plot(er_kwargs=dict(plot_relations=True))
plt.savefig('results/toy_1.png', dpi=300)`

Actually, if I comment this line the code runs with no errors:
`results.plot(er_kwargs=dict(plot_relations=True))`


### Environment

<html><body>
<!--StartFragment-->

Key | Value
-- | --
OS | nt
Platform | Windows
Release | 10
Time | Wed Aug 17 17:46:55 2022
Python | 3.9.12
PyKEEN | 1.9.0
PyKEEN Hash | UNHASHED
PyKEEN Branch |  
PyTorch | 1.11.0
CUDA Available? | false
CUDA Version | 11.3
cuDNN Version | 8200

<!--EndFragment-->
</body>
</html>

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

duplicate of https://github.com/pykeen/pykeen/issues/1071

Hello, this issue was already reported and solved few days ago.
Please check the following closed issue (https://github.com/pykeen/pykeen/issues/1071) for the details.
Repository: pykeen/pykeen
Issue: RuntimeError when creating entity word clouds
Root cause: Validation Gap

### Describe the bug

```python-traceback
Traceback (most recent call last):
  File "examples/hello_world/hello_world.py", line 107, in <module>
    tf.entity_word_cloud()
  File "...pykeen/src/pykeen/triples/triples_factory.py", line 1215, in entity_word_cloud
    return self._word_cloud(
  File "...pykeen/src/pykeen/triples/triples_factory.py", line 1250, in _word_cloud
    uniq, counts = ids.view(-1).unique(return_counts=True)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
```

### How to reproduce

Run the [hello-world example notebook](https://github.com/pykeen/pykeen/blob/2d74d06f9137a6d0f48879408bc19c3c0807e337/notebooks/hello_world/hello_world.ipynb).

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.4.0-72-generic         |
| Time            | Thu Aug 18 18:33:16 2022 |
| Python          | 3.8.10                   |
| PyKEEN          | 1.9.1-dev                |
| PyKEEN Hash     | dfa632d3                 |
| PyKEEN Branch   | master                   |
| PyTorch         | 1.12.0+cpu               |
| CUDA Available? | false                    |
| CUDA Version    | N/A                      |
| cuDNN Version   | N/A                      |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed
Repository: pykeen/pykeen
Issue: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
Root cause: Environment Incompatibility

Hi,

I'm new to PyKEEN. I've installed it on my Mac via `pip`, opened up Python, and, following the starter's guide, tried to import the pipeline like this:
```
>>> from pykeen.pipeline import pipeline
```
and I end up with the following error:
```
OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
Abort trap: 6
```
I'm running this on Mac OS Monterey 12.5.1. The version of PyKEEN that I have is 1.9.0. I tried to file this issue using the bug template, but am unable to get an environment table because when I try running `python -m pykeen version`, I get "Segmentation fault: 11" instead.

I showed this to my supervisor, and he said that PyKEEN works fine for him on Linux.

Any help on this would be appreciated. Thank you!

Hey @yichuanma95 ,

the error seems to be not directly related to PyKEEN code but rather be an installation issue. I do not have macOS system at hand, so I cannot look into it.

However, [here](https://stackoverflow.com/questions/53014306/error-15-initializing-libiomp5-dylib-but-found-libiomp5-dylib-already-initial) is a stackoverflow discussion of the same error message you encountered, and the solutions suggested there might be applicable to your case, too.


I've tried following the solutions in that discussion, and my environment got even more messed up. Can another Mac user help me out please?
Repository: pykeen/pykeen
Issue: Symmetric LCWA
Root cause: Unimplemented Feature Gap

### Problem Statement

[Lacroix et al., 2018](http://imagine.enpc.fr/~obozinsg/papers/lacroix2018canonical.pdf) introduce a new training objective

$$
l_{i,j,k}(X) = - X_{i,j,k} + \log \left(
\sum_{k'} \exp(X_{i,j,k′})
\right) - X_{k,j+P,i} + \log \left( \sum_{i'} \exp (X_{k, j+P, i'})\right)
$$

which can be seen as a "symmetric LCWA", where for one batch of triples, we score both, heads *and* tails, given the remainder of the triple. (see [[0]](https://github.com/facebookresearch/kbc/blob/248e93136448661cac54020054b929997394f3d2/kbc/optimizers.py) [[1]](https://github.com/facebookresearch/kbc/blob/248e93136448661cac54020054b929997394f3d2/kbc/models.py#L102-L106))

The same training objective is also used by [AutoSF](https://github.com/AutoML-Research/AutoSF/blob/f640218446e74ab41f7c30a94f8a1b7188f655d3/models.py#L21-L45)

### Describe the solution you'd like

Implement this as a loop, roughly:
```python
loss = lcwa_loss(
  model.score_heads(hrt_batch[:, 1:]), hrt_batch[:, 0]
) + lcwa_loss(
  model.score_tails(hrt_batch[:, :-1]), hrt_batch[:, -1]
)
```

### Describe alternatives you've considered

<!-- -->

### Additional information

code: https://github.com/AutoML-Research/AutoSF/blob/f640218446e74ab41f7c30a94f8a1b7188f655d3/models.py#L21-L45

### Issue Template Checks

- [X] This is not a bug report (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)

Symmetric LCWA should probably be called LCWAWCL
Repository: pykeen/pykeen
Issue: Run pykeen on own KGs
Root cause: Documentation Deficiency

Can we run pykeen with our own KGs?

Hi @SRL94 ,

please use the `Discussion` section to ask questions.

For your question specifically, there is already [a section](https://pykeen.readthedocs.io/en/stable/byo/data.html) in the documentation.
Repository: pykeen/pykeen
Issue: Ablation pipeline is not recognizing evaluation_kwargs
Root cause: Configuration Error

### Describe the bug

Ablation pipeline is not recognizing evaluation_kwargs. 
Even after setting it to some other value, it's still throwing this warning :
`INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.`

### How to reproduce

```python
evaluation_kwargs={
    "batch_size":512
}
ablation_pipeline(
    models=["TransE"],
    datasets=['Nations'],
    losses=['Softplus'],
    training_loops=['lcwa','slcwa'],
    optimizers=['Adam'],
    create_inverse_triples=[True, False],
    evaluation_kwargs=evaluation_kwargs,
    stopper="early",
    stopper_kwargs=stopper_kwargs,
    directory="test/",
    n_trials=1,
    timeout=60*5, # 5min
    metric="hits@10",
    direction="maximize",
    sampler="random",
    pruner="nop",
)
```

### Environment

<html>
<body>
<!--StartFragment-->

Key | Value
-- | --
OS | posix
Platform | Linux
Release | 5.4.0-1080-gcp
Time | Sat Sep 3 16:35:14 2022
Python | 3.8.5
PyKEEN | 1.9.0
PyKEEN Hash | UNHASHED
PyKEEN Branch |  
PyTorch | 1.11.0+cpu
CUDA Available? | false
CUDA Version | N/A
cuDNN Version | N/A

<!--EndFragment-->
</body>
</html>

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

It appears there are no examples containing `evaluation_kwargs` on https://pykeen.readthedocs.io/en/stable/tutorial/running_ablation.html. I think you're looking for `evaluator_kwargs`. Unfortunately the ablation pipeline isn't a huge focus right now so your best bet is to try and do some code digging, then maybe you can write something here that's helpful for others in the future
Repository: pykeen/pykeen
Issue: Random Questions

Hi guys,

i have some random questions.

1. How can i create a custom inductive dataset? I assume the `from_labeled_triples()` function does not work. What i did was to create a TripleFactory for all four required datasets (Transductive Training, Inductive Inference, Inductive Testing and Inductive Validation) and than set the attributes of the class instance of `InductiveDataset` to each respectively. Would that be the way to go or could one use a `__init__` function? I guess an extension to load a inductive dataset from files would also be nice. 

```python
dataset = InductiveDataset()
dataset.transductive_training = transductive_training 
dataset.inductive_inference = inductive_inference
dataset.inductive_testing =  inductive_test
dataset.inductive_validation = inductive_validation
```

2. Does the InductiveDataset also work if i have a mixed szenario where i basically have an Inductive Inference graph that is just an extension of the transdutive training graph?

3. NodePiece requires  the parameter `create_inverse_triples`. However, i did not find get what that implies. Will it create inverse triples and if so, how does it know what the inverses are? Where do i have to specify or pass that information? 

4. In [example](https://pykeen.readthedocs.io/en/stable/tutorial/inductive_lp.html) you show very vividly how to evaluate the model with the test_evaluator. I wonder how I could use the trained model to generate only the embeddings for the Inductive Inference? Could you give me a tiny code snippet for it? 

6. What is your recommendation to deal with literals e.g. order values or age as integer values? 

Best regards
Chris
Repository: pykeen/pykeen
Issue: Add KG-BERT
Root cause: Unimplemented Feature Gap

### Publication Link

https://arxiv.org/pdf/1909.03193.pdf

### Reference Implementation

https://github.com/yao8839836/kg-bert

### Additional Implementations

_No response_

### Relevance

I'm working on a research project, and as part of it, I'd like to run a few LM-based KGC models (such as this one) on the PrimeKg dataset. It would be interesting to compare the performance of such models with KGE baselines you've already integrated into the library (such as TransE).

### Additional Context

_No response_

+1 on this point. 
As a more high level point, I haven't found any integrated examples of using pretrained LLMs to extract text embeddings on data as node initializations. I know pyKeen supports it, but there doesn't seem to be a worked code example in the docs or notebooks?

Hi @noy-sternlicht ,

we would be happy to accept PRs to add this model. We already have some support for using transformer based text encoders, cf. [`pykeen.nn.representation.TextRepresentation`](https://pykeen.readthedocs.io/en/stable/api/pykeen.nn.representation.TextRepresentation.html) and [this tutorial page](https://pykeen.readthedocs.io/en/stable/tutorial/representations.html#text-based), but a full reproduction of a model configuration / a research paper is often quite a bit of extra effort to figure out all the small details.


@ddofer , please also refer to the linked `TextRepresentation`. We are always happy about feedback what is missing from a user perspective, and also where and how to better link information.

What I directly saw from skimming the tutorial page again, is that it does not talk about [`pykeen.nn.init.LabelBasedInitializer`](https://pykeen.readthedocs.io/en/stable/api/pykeen.nn.init.LabelBasedInitializer.html) - this *initializer* uses a pre-trained text encoding model, but only to obtain *initial* representation which are afterwards disconnected from the text encoder. This allows them to deviate during training if this allows them to improve link prediction and makes the model faster since we do not need to run the (heavy) text encoding model each time, but we may lose some generalization capability and regularization from tying representations to their text form.
Repository: pykeen/pykeen
Issue: Add StAR
Root cause: Unimplemented Feature Gap

### Publication Link

https://dl.acm.org/doi/pdf/10.1145/3442381.3450043

### Reference Implementation

https://github.com/wangbo9719/StAR_KGC

### Additional Implementations

_No response_

### Relevance

I'm working on a research project, and as part of it, I'd like to run a few LM-based KGC models on the PrimeKg dataset. StAR uses a dual-encoder architecture (in contrast to the additional LM-based model I've requested - [KG-BERT](https://github.com/pykeen/pykeen/issues/1113), which is a cross-encoder) and it would be interesting to compare them both.

### Additional Context

_No response_
Repository: pykeen/pykeen
Issue: `negative_sampler_kwargs` are ignored unless a sampler is explicitly set in the pipeline
Root cause: Configuration Error

### Describe the bug

It seems that any `negative_sampler_kwargs` are ignored unless a sampler is explicitly set in the pipeline. For example, setting a value for the number of negative samples does not appear to make it through to the default basic sampler choice and thus the value is always 1. However if the basic sampler is explicitly chosen, setting a values for the number of negative samples works as expected.  

### How to reproduce

```python
from pykeen.pipeline import pipeline

pipeline_result = pipeline(
    dataset='Nations',
    model='TransE',
    negative_sampler_kwargs=dict(num_negs_per_pos=100),
)
print(pipeline_result.train_seconds)
print(pipeline_result.configuration["training_loop_kwargs"])
```
Running this code results in the following output on my machine:
```
0.4482271671295166
{}
```
Whereas if a sampler is set in the pipeline - 

``` python
from pykeen.pipeline import pipeline

pipeline_result = pipeline(
    dataset='Nations',
    model='TransE',
    negative_sampler="basic",
    negative_sampler_kwargs=dict(num_negs_per_pos=100),
)
print(pipeline_result.train_seconds)
print(pipeline_result.configuration["training_loop_kwargs"])
```
I get the following output. 

```
2.9099767208099365
{'negative_sampler': <class 'pykeen.sampling.basic_negative_sampler.BasicNegativeSampler'>, 'negative_sampler_kwargs': {'num_negs_per_pos': 100}}
```

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Darwin                   |
| Release         | 21.6.0                   |
| Time            | Tue Sep 20 16:44:30 2022 |
| Python          | 3.9.9                    |
| PyKEEN          | 1.9.1-dev                |
| PyKEEN Hash     | d905feb3                 |
| PyKEEN Branch   | master                   |
| PyTorch         | 1.12.1                   |
| CUDA Available? | false                    |
| CUDA Version    | N/A                      |
| cuDNN Version   | N/A                      |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed
Repository: pykeen/pykeen
Issue: Scikit-learn dependency is wrongly declared in setup.cfg
Root cause: Configuration Error

### Describe the bug

The scikit-learn dependency is wrongfully declared as "sklearn" in setup.cfg (which [actually exists](https://pypi.org/project/sklearn/)). This means unnecessarily installing "sklearn" when installing PyKEEN.

### How to reproduce

```bash
python -m venv test_venv
pip install pykeen
pip list | grep "sklearn"
```
will show
```
sklearn             0.0
```

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.17.0-1-amd64           |
| Time            | Fri Sep 23 18:24:03 2022 |
| Python          | 3.10.5                   |
| PyKEEN          | 1.9.0                    |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 1.12.1+cu102             |
| CUDA Available? | false                    |
| CUDA Version    | 10.2                     |
| cuDNN Version   | 7605                     |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Closed by https://github.com/pykeen/pykeen/pull/1122
Repository: pykeen/pykeen
Issue: Understanding the broadcasting of positive and negative scores in `MarginPairwiseLoss`

### Discussed in https://github.com/pykeen/pykeen/discussions/1120

<div type='discussions-op-text'>

<sup>Originally posted by **sbonner0** September 21, 2022</sup>
I've been trying to understand how the broadcasting works for the positive and negative scores in the `MarginPairwiseLoss` class. Essentially I am wondering why there is the need to add an extra singleton dimension in the positive scores using the following line `positive_scores = positive_scores.unsqueeze(dim=1)`?

This results in the positive scores having the shape `[batch_size, 1, 1]`, whilst the negative scores have the shape `[batch_size, num_neg_per_pos]`. Thus the result of the calculation `neg_scores - pos_scores + margin` is of shape `[batch_size, batch_size, num_neg_per_pos]`. Does this not mean that each positive score is being compared against all negatives, even those generated as corruptions of different positive triples? Is this the intended behavior or am I missing something? To me, it would make more sense if the positive scores simply had the shape `[batch_size, 1]` as this would limit computation to the correct set of negatives for a given positive triple. </div>
Repository: pykeen/pykeen
Issue: How to set device beyond the pipeline?
Root cause: Documentation Deficiency

Hi,

I know that we can set device='gpu' in pipeline(), but I did not find how to set device beyond the pipeline.
I use the following code in jupyter, but it does not work.
```
if torch.cuda.is_available():
  dev = "cuda:0"
else:
  dev = "cpu"
device = torch.device(dev)
```

here is my code:

```
# model
model = TransE(
    triples_factory=training,
    random_seed=42,
)

# optimizer
optimizer = Adam(params=model.get_grad_params())
# Pick an evaluator
evaluator = RankBasedEvaluator(
    filtered=True,  # Note: this is True by default; we're just being explicit
)
# stopper
stopper = EarlyStopper(
    model=model,
    evaluator=evaluator,
    training_triples_factory=training,
    evaluation_triples_factory=validation,
    patience=2,
    relative_delta=0.1,
    metric='mean_rank',
)
# training approach (sLCWA or LCWA)
training_loop = SLCWATrainingLoop(
    model=model,
    triples_factory=training,
    optimizer=optimizer,
)

# Train 
_ = training_loop.train(
    triples_factory=training,
    num_epochs=100,
    batch_size=256,
    stopper=stopper,
)

# Get triples to test
mapped_triples = testing.mapped_triples
# Evaluate
results = evaluator.evaluate(
    model=model,
    mapped_triples=mapped_triples,
    batch_size=1024,
    additional_filter_triples=[
        training.mapped_triples,
        validation.mapped_triples,
    ],
)
```
Repository: pykeen/pykeen
Issue: How to get "Accuracy" and "F1 Score" values?
Root cause: Documentation Deficiency

Hello, 
in the following simple example, I used pykeen to get the loss value and hits@k Values, but when I tried to get the Accuracy and F1 Score values, I don't know how to write my code. I have also read the pykeen document and the following link(https://github.com/pykeen/pykeen#metrics). could you take a look at it for me?

my code:

from pykeen.pipeline import pipeline
from matplotlib import pyplot as plt

result = pipeline(
    dataset="fb15k237",
    model="transE",
    negative_sampler="basic",
    model_kwargs=dict(embedding_dim=100),
    training_kwargs=dict(
        num_epochs=5,
        batch_size=256,
    ),
    random_seed=1234,
    negative_sampler_kwargs=dict(num_negs_per_pos=1),
)
plt.plot(result.losses)
plt.show()
print(result.get_metric("hits@10"))



Environment:

platform.system()  Windows
platform.release()  11
python 3.8.5
pykeen 1.8.2
torch 1.12.0
cuda cuda11.6
Repository: pykeen/pykeen
Issue: Load datasets via python entrypoints
Root cause: Unimplemented Feature Gap

While users can mostly implement their own datasets either by:

1. make your own triples factories
2. subclass `pykeen.datasets.DataSet`

Sometimes, they might want to integrate even further. Why not use python entrypoints to load datasets, so users can implement their own datasets that doesn't require any new pykeen or python code functionality for coding?