Repository: pykeen/pykeen
Issue: NodePiece Model takes more time to process the data when we use a subset of training data
Root cause: Algorithmic Error

### Describe the bug

I want to create embeddings for the english entities/nodes of the conceptnet graph. When I create a pipeline such that only english entities, it takes more time to process the data than calling the nodepiece model with the the complete conceptnet dataset - In Code 1, the dataset processing takes more time than Code 2. 

## Code 1
```python
from typing import List

import pykeen
import torch.nn
from pykeen.models import NodePiece
from pykeen.pipeline import pipeline
from pykeen.datasets import FB15k237, ConceptNet, Wikidata5M

# create conceptnet dataset instance and train, test and val triples for english entities
# english entities are from id - 2232753, 4588023
dataset = ConceptNet(create_inverse_triples=True)
train = dataset.training.new_with_restriction(entities=list(range(2232753, 4588023)))
test = dataset.testing.new_with_restriction(entities=list(range(2232753, 4588023)))
val = dataset.validation.new_with_restriction(entities=list(range(2232753, 4588023)))

class DeepSet(torch.nn.Module):
    def __init__(self, hidden_dim=64):
        super().__init__()
        self.encoder = torch.nn.Sequential(
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
        )
        self.decoder = torch.nn.Sequential(
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
        )

    def forward(self, x, dim=-2):
        x = self.encoder(x).mean(dim)
        x = self.decoder(x)
        return x

result = pipeline(
    training=train, testing=test, validation=val,
    model=NodePiece,
    model_kwargs=dict(
        tokenizers=["MetisAnchorTokenizer", "RelationTokenizer"],
        num_tokens=[20, 12],
        tokenizers_kwargs=[
            dict(
                num_partitions=20,  # each partition will be of about 5M / 20 = 250K nodes
                device="cpu",  # METIS on cpu tends to be faster
                selection="MixtureAnchorSelection",
                selection_kwargs=dict(
                    selections=["degree", "random"],
                    ratios=[0.5, 0.5],
                    num_anchors=2000,
                ),
                searcher="SparseBFSSearcher",
                searcher_kwargs=dict(
                    max_iter=5  # each node will be tokenized with anchors in the 5-hop neighborhood
                )
            ),
            dict(),  # empty dict for the RelationTokenizer - it doesn't need any kwargs
        ],
        embedding_dim=128,
        interaction="TransE",
        interaction_kwargs=dict(p=2),
        aggregation=DeepSet(hidden_dim=128),
    )
)

# See docs: https://pykeen.readthedocs.io/en/stable/api/pykeen.pipeline.PipelineResult.html
print("Training time (s)", result.train_seconds)
print("Evaluation time (s)", result.evaluation_seconds)
```
## Code 2 
```python
from typing import List

import pykeen
import torch.nn
from pykeen.models import NodePiece
from pykeen.pipeline import pipeline

class DeepSet(torch.nn.Module):
    def __init__(self, hidden_dim=64):
        super().__init__()
        self.encoder = torch.nn.Sequential(
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
        )
        self.decoder = torch.nn.Sequential(
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
            torch.nn.ReLU(),
            torch.nn.Linear(hidden_dim, hidden_dim),
        )

    def forward(self, x, dim=-2):
        x = self.encoder(x).mean(dim)
        x = self.decoder(x)
        return x

result = pipeline(
    dataset = 'ConceptNet',
    model=NodePiece,
    model_kwargs=dict(
        tokenizers=["MetisAnchorTokenizer", "RelationTokenizer"],
        num_tokens=[20, 12],
        tokenizers_kwargs=[
            dict(
                num_partitions=20,  # each partition will be of about 5M / 20 = 250K nodes
                device="cpu",  # METIS on cpu tends to be faster
                selection="MixtureAnchorSelection",
                selection_kwargs=dict(
                    selections=["degree", "random"],
                    ratios=[0.5, 0.5],
                    num_anchors=2000,
                ),
                searcher="SparseBFSSearcher",
                searcher_kwargs=dict(
                    max_iter=5  # each node will be tokenized with anchors in the 5-hop neighborhood
                )
            ),
            dict(),  # empty dict for the RelationTokenizer - it doesn't need any kwargs
        ],
        embedding_dim=128,
        interaction="TransE",
        interaction_kwargs=dict(p=2),
        aggregation=DeepSet(hidden_dim=128),
    )
)

# See docs: https://pykeen.readthedocs.io/en/stable/api/pykeen.pipeline.PipelineResult.html
print("Training time (s)", result.train_seconds)
print("Evaluation time (s)", result.evaluation_seconds)
```

### How to reproduce

Please refer the section

> Describe the bug

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | nt                       |
| Platform        | Windows                  |
| Release         | 10                       |
| Time            | Sun Sep 25 02:22:24 2022 |
| Python          | 3.10.4                   |
| PyKEEN          | 1.9.0                    |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 1.12.1                   |
| CUDA Available? | false                    |
| CUDA Version    | N/A                      |
| cuDNN Version   | N/A                      |

### Additional information

_No response_

### Issue Template Checks

- [X] This is not a feature request (use a different issue template if it is)
- [X] This is not a question (use the [discussions](https://github.com/pykeen/pykeen/discussions/new) forum instead)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hi @trilokpadhi ,

which time are you measuring? the overall time of both scripts?

could you also try to isolate the problem better, by removing as much code as possible while still being able to reproduce the time difference?
e.g., using `DeepSet` as aggregation seems to be the same in both snippets, so this should not make up the difference, and removing this part of the code and using the default makes it easier to investigate differences.

Since ConceptNet is quite large, it would also be good to see whether you can reproduce the difference (of course on a smaller scale) for smaller datasets, e.g., FB15k-237.

Hi @mberr 
Thanks for your reply. Just to highlight the difference, The pipeline definition in the **code block 1** creates anchor nodes  faster than the **code block 2**, The **code block 2** also gets stuck a lot of times. This happens when I exclusively create the training and testing triples with mentioning the english enttiy ids.  Could you please tell me why this could be happening ? All the other parameters of the model are same. 

## code block 1
```
result = pipeline(dataset = 'ConceptNet',
                  model=NodePiece,
                  model_kwargs=dict( ...............
                  .................................)
```
## code block 2 
```
dataset = ConceptNet(create_inverse_triples=True)
train = dataset.training.new_with_restriction(entities=list(range(2232753, 4588023)))
test = dataset.testing.new_with_restriction(entities=list(range(2232753, 4588023)))
result = pipeline(training=train, testing=test,
                  model=NodePiece,
                  model_kwargs=dictmodel_kwargs=dict( ...............
                  .................................)
```




just to be sure: are you measuring the time for 
```
train = dataset.training.new_with_restriction(entities=list(range(2232753, 4588023)))
test = dataset.testing.new_with_restriction(entities=list(range(2232753, 4588023)))
```
too? because they might be what causes the difference.

Hi @mberr,
Not really, the time taken to create the anchor nodes from the dataset in the pipeline function basically 

I added to the example code the following

```python
# See docs: https://pykeen.readthedocs.io/en/stable/api/pykeen.pipeline.PipelineResult.html
print("Training time (s)", result.train_seconds)
print("Evaluation time (s)", result.evaluation_seconds)
```

Since timing these functionalities is already standardized inside PyKEEN

Hi @cthoyt 
Thanks for your reply. Before I test the time, the issue is the code is getting stuck ( I waited for 6 hours ) when I create the model for only english entities. Is it obvious to take such huge time to just filter the triples or to process the data and the nodepiece model with these triples?? Is it that it gets OOM while filtering out the triples for only these entities. Please let me know how could I get over this issue or filter the triples faster.


> triples_factory_english = dataset.training.new_with_restriction(entities=list(range(2232753, 2235023)))

<img width="1043" alt="image" src="https://user-images.githubusercontent.com/20143249/192596034-881fb6d7-21c7-4aba-94df-e48fd5c7ae9a.png">

Just to let you know the system info 
<img width="819" alt="Screen Shot 2022-09-27 at 1 42 53 PM" src="https://user-images.githubusercontent.com/20143249/192598797-1aa1b1c0-49d8-4f05-a2d9-de9c60d9b4de.png">

Thanks


@cthoyt @mberr could you please look at this issue ? 
Thanks 

@trilokpadhi I currently do not have a large working machine available to test anything on big datasets. Please try to find a minimal working example which is as small as possible (e.g., on "nations"), but still reproduces the issue.

You also may want to use [`timeit`](https://docs.python.org/3/library/timeit.html) or (even better) torch's wrapper around it from https://pytorch.org/docs/master/benchmark_utils.html to get more reliable time measurements.