Repository: pykeen/pykeen
Issue: How to optimize and use the model?
Root cause: Documentation Deficiency

My goal is to train my DBpedia dataset with Pykeen to get the vectors of the entities (the trained vectors should carry the structural information of the original DBpedia dataset) and then get the similarity of the structures between the entities.

So far, I've run all the models in Pykeen with my DBpedia dataset, with MemoryErrors occurring the vast majority of the time.
Currently, I can only run the TransE model successfully. 

The training results of my idea in the OpenKE framework is OK, but the results obtained in the Pykeen framework are very poor. 
## I want to know how to adjust the parameters to optimize the memory and the entity vector obtained by training will be more accurate. 

My configuration:
GeForce RTX 2060 : 12GB  x8
RAM: 192GB

My DBpedia dataset：
The total of entities: 1295967.
The total of relations: 2.
The total of train triples: 3118841.

Here is my code:

```
import logging
from pykeen.pipeline import pipeline
from pykeen.triples import TriplesFactory
logging.basicConfig(level=logging.DEBUG)

# DBpedia
TRAIN_PATH = "./DBpedia/train.tsv"
TEST_PATH = "./DBpedia/test.tsv"
VALIDATE_PATH = "./DBpedia/valid.tsv"

# TriplesFactory: training, testing, validation
training = TriplesFactory.from_path(
    path=TRAIN_PATH,
    create_inverse_triples=False)

testing = TriplesFactory.from_path(
    path=TEST_PATH,
    create_inverse_triples=False,
    entity_to_id=training.entity_to_id,     ## training
    relation_to_id=training.relation_to_id)

validation = TriplesFactory.from_path(
    path=VALIDATE_PATH,
    create_inverse_triples=False,
    entity_to_id=training.entity_to_id,     ## training
    relation_to_id=training.relation_to_id)

def KnowledgeGraphEmbeddings():
        pipeline_result = pipeline(
        training=training,         # TRAIN_PATH
        testing=testing,           # TEST_PATH
        validation=validation,  # VALIDATE_PATH
        model="TransE",         # TransE PairRE ERMLPE ConvE CompGCN QuatE TuckER MuRE SimplE RGCN ComplExLiteral
        model_kwargs=dict(embedding_dim=200),   # Set 'embedding_dim=200' to work
        optimizer_kwargs=dict(lr=0.002),
        training_kwargs=dict(use_tqdm_batch=False,  # batch_size=256,
                                           num_epochs=1000),
        evaluation_kwargs=dict(use_tqdm=False),     # batch_size=256,
        stopper='early',    # Is it necessary to set?
        stopper_kwargs=dict(frequency=5, patience=2, relative_delta=0.002),  
        random_seed=1688,
        # device='gpu'   # cpu
    )
    pipeline_result.save_to_directory(f"./trained_model/{model}")
```

I hope to get your help. 
Thank you very much!


Hi @fiego ,

the default parameters of most models are those corresponding to one of the settings used in the original publication. Most of these models where trained on relatively small datasets (FB15k-237, WN18-RR) with ~15k-40k entities. Since your dataset has more entities, you may need to reduce the embedding dimension to still be able to fit the model on your GPU. Also the (training) batch size and the number of negative samples per positive one have an effect on the required GPU memory.

Some models have additional parameters affecting the memory requirement such as a separate relation embedding dimension (e.g. `TuckER`), or a hidden dimension (e.g. `ConvE`).

Finally, GNN-based models such as `RGCN` and `CompGCN` require more memory since they do not use plain embedding matrices, but have an additional message passing phase.

A small comment regarding the model choice: `ComplExLiteral` is a model specific for knowledge graphs with additional literals. Since the other models do not work with literals, and you did not describe your dataset to have them, I guess you want to use the "normal" `ComplEx` instead.

We currently do not have support for training a single model sharded over multiple GPUs, cf. https://github.com/pykeen/pykeen/issues/96. If you want to also run a hyperparameter optimization (HPO), i.e., investigate many different hyperparameter configurations to find the best one for each model on your given dataset, you can take a look at [this tutorial](https://pykeen.readthedocs.io/en/stable/tutorial/running_hpo.html). As before, you may need to adjust the ranges of hyperparameters to better suit your comparably large dataset.

> Hi @fiego ,
> 
> the default parameters of most models are those corresponding to one of the settings used in the original publication. Most of these models where trained on relatively small datasets (FB15k-237, WN18-RR) with ~15k-40k entities. Since your dataset has more entities, you may need to reduce the embedding dimension to still be able to fit the model on your GPU. Also the (training) batch size and the number of negative samples per positive one have an effect on the required GPU memory.
> 
> Some models have additional parameters affecting the memory requirement such as a separate relation embedding dimension (e.g. `TuckER`), or a hidden dimension (e.g. `ConvE`).
> 
> Finally, GNN-based models such as `RGCN` and `CompGCN` require more memory since they do not use plain embedding matrices, but have an additional message passing phase.
> 
> A small comment regarding the model choice: `ComplExLiteral` is a model specific for knowledge graphs with additional literals. Since the other models do not work with literals, and you did not describe your dataset to have them, I guess you want to use the "normal" `ComplEx` instead.

Thanks.
After I didn't set stopper='early', I got the result I wanted.