Package: dicee
DICE Embeddings: Hardware-agnostic Framework for Large-scale Knowledge Graph EmbeddingsKnowledge graph embedding research has mainly focused on learning continuous representations of knowledge graphs towards the link prediction problem.
Recently developed frameworks can be effectively applied in a wide range of research-related applications.
Yet, using these frameworks in real-world applications becomes more challenging as the size of the knowledge graph grows.We developed the DICE Embeddings framework (dicee) to compute embeddings for large-scale knowledge graphs in a hardware-agnostic manner.
To achieve this goal, we rely onPandas& Co.to use parallelism at preprocessing a large knowledge graph,PyTorch& Co.to learn knowledge graph embeddings via multi-CPUs, GPUs, TPUs or computing cluster, andHuggingfaceto ease the deployment of pre-trained models.WhyPandas& Co. ?A large knowledge graph can be read and preprocessed (e.g. removing literals) by pandas, modin, or polars in parallel.
Through polars, a knowledge graph having more than 1 billion triples can be read in parallel fashion.
Importantly, using these frameworks allow us to perform all necessary computations on a single CPU as well as a cluster of computers.WhyPyTorch& Co. ?PyTorch is one of the most popular machine learning frameworks available at the time of writing.
PytorchLightning facilitates scaling the training procedure of PyTorch without boilerplate.
In our framework, we combinePyTorch&PytorchLightning.
Users can choose the trainer class (e.g., DDP by Pytorch) to train large knowledge graph embedding models with billions of parameters.
PytorchLightning allows us to use state-of-the-art model parallelism techniques (e.g. Fully Sharded Training, FairScale, or DeepSpeed)
without extra effort.
With our framework, practitioners can directly use PytorchLightning for model parallelism to train gigantic embedding models.WhyHugging-face Gradio?Deploy a pre-trained embedding model without writing a single line of code.For more please visitdice-embeddings!InstallationClick me!Installation from Sourcegitclonehttps://github.com/dice-group/dice-embeddings.git
condacreate-ndicepython=3.10.13--no-default-packages&&condaactivatedice&&cddice-embeddings&&pip3install-e.orpipinstalldicee==0.1.3Download Knowledge Graphswgethttps://files.dice-research.org/datasets/dice-embeddings/KGs.zip--no-check-certificate&&unzipKGs.zipTo test the Installationpython-mpytest-pno:warnings-x# Runs >114 tests leading to > 15 minspython-mpytest-pno:warnings--lf# run only the last failed testpython-mpytest-pno:warnings--ff# to run the failures first and then the rest of the tests.Knowledge Graph Embedding ModelsTo see available ModelsTransE, DistMult, ComplEx, ConEx, QMult, OMult, ConvO, ConvQ, KeciAll 44 models available inhttps://github.com/pykeen/pykeen#modelsFor more, please refer toexamples.How to TrainTo see a code snippetTo Train a KGE model (KECI) and evaluate it on the train, validation, and test sets of the UMLS benchmark dataset.fromdicee.executerimportExecutefromdicee.configimportNamespaceargs=Namespace()args.model='Keci'args.scoring_technique="KvsAll"# 1vsAll, or AllvsAll, or NegSampleargs.dataset_dir="KGs/UMLS"args.path_to_store_single_run="Keci_UMLS"args.num_epochs=100args.embedding_dim=32args.batch_size=1024reports=Execute(args).start()print(reports["Train"]["MRR"])# => 0.9912print(reports["Test"]["MRR"])# => 0.8155# See the Keci_UMLS folder embeddings and all other fileswhere the data is in the following form$head-3KGs/UMLS/train.txtacquired_abnormalitylocation_ofexperimental_model_of_disease
anatomical_abnormalitymanifestation_ofphysiologic_function
algaisaentityA KGE model can also be trained from the command linedicee--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"dicee automaticaly detects available GPUs and trains a model with distributed data parallels technique. Under the hood, dicee uses lighning as a default trainer.# Train a model by only using the GPU-0CUDA_VISIBLE_DEVICES=0dicee--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"# Train a model by only using GPU-1CUDA_VISIBLE_DEVICES=1dicee--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"NCCL_P2P_DISABLE=1CUDA_VISIBLE_DEVICES=0,1pythondicee/scripts/run.py--trainerPL--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"Under the hood, dicee executes run.py script and uses lighning as a default trainer# Two equivalent executions# (1)dicee--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"# Evaluate Keci on Train set: Evaluate Keci on Train set# {'H@1': 0.9518788343558282, 'H@3': 0.9988496932515337, 'H@10': 1.0, 'MRR': 0.9753123402351737}# Evaluate Keci on Validation set: Evaluate Keci on Validation set# {'H@1': 0.6932515337423313, 'H@3': 0.9041411042944786, 'H@10': 0.9754601226993865, 'MRR': 0.8072362996241839}# Evaluate Keci on Test set: Evaluate Keci on Test set# {'H@1': 0.6951588502269289, 'H@3': 0.9039334341906202, 'H@10': 0.9750378214826021, 'MRR': 0.8064032293278861}# (2)CUDA_VISIBLE_DEVICES=0,1pythondicee/scripts/run.py--trainerPL--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"# Evaluate Keci on Train set: Evaluate Keci on Train set# {'H@1': 0.9518788343558282, 'H@3': 0.9988496932515337, 'H@10': 1.0, 'MRR': 0.9753123402351737}# Evaluate Keci on Train set: Evaluate Keci on Train set# Evaluate Keci on Validation set: Evaluate Keci on Validation set# {'H@1': 0.6932515337423313, 'H@3': 0.9041411042944786, 'H@10': 0.9754601226993865, 'MRR': 0.8072362996241839}# Evaluate Keci on Test set: Evaluate Keci on Test set# {'H@1': 0.6951588502269289, 'H@3': 0.9039334341906202, 'H@10': 0.9750378214826021, 'MRR': 0.8064032293278861}Similarly, models can be easily trained with torchruntorchrun--standalone--nnodes=1--nproc_per_node=gpudicee/scripts/run.py--trainertorchDDP--dataset_dir"KGs/UMLS"--modelKeci--eval_model"train_val_test"# Evaluate Keci on Train set: Evaluate Keci on Train set: Evaluate Keci on Train set# {'H@1': 0.9518788343558282, 'H@3': 0.9988496932515337, 'H@10': 1.0, 'MRR': 0.9753123402351737}# Evaluate Keci on Validation set: Evaluate Keci on Validation set# {'H@1': 0.6932515337423313, 'H@3': 0.9041411042944786, 'H@10': 0.9754601226993865, 'MRR': 0.8072499937521418}# Evaluate Keci on Test set: Evaluate Keci on Test set{'H@1':0.6951588502269289,'H@3':0.9039334341906202,'H@10':0.9750378214826021,'MRR':0.8064032293278861}You can also train a model in multi-node multi-gpu setting.torchrun--nnodes2--nproc_per_node=gpu--node_rank0--rdzv_id455--rdzv_backendc10d--rdzv_endpoint=nebuladicee/scripts/run.py--trainertorchDDP--dataset_dirKGs/UMLS
torchrun--nnodes2--nproc_per_node=gpu--node_rank1--rdzv_id455--rdzv_backendc10d--rdzv_endpoint=nebuladicee/scripts/run.py--trainertorchDDP--dataset_dirKGs/UMLSTrain a KGE model by providing the path of a single file and store all parameters under newly created directory
calledKeciFamilyRun.dicee--path_single_kg"KGs/Family/family-benchmark_rich_background.owl"--modelKeci--path_to_store_single_runKeciFamilyRun--backendrdflibwhere the data is in the following form$head-3KGs/Family/train.txt_:1<http://www.w3.org/1999/02/22-rdf-syntax-ns#type><http://www.w3.org/2002/07/owl#Ontology>.
<http://www.benchmark.org/family#hasChild><http://www.w3.org/1999/02/22-rdf-syntax-ns#type><http://www.w3.org/2002/07/owl#ObjectProperty>.
<http://www.benchmark.org/family#hasParent><http://www.w3.org/1999/02/22-rdf-syntax-ns#type><http://www.w3.org/2002/07/owl#ObjectProperty>.Apart from n-triples or standard link prediction dataset formats, we support ["owl", "nt", "turtle", "rdf/xml", "n3"]*.
Moreover, a KGE model can be also trained  by providingan endpoint of a triple store.dicee--sparql_endpoint"http://localhost:3030/mutagenesis/"--modelKeciFor more, please refer toexamples.Creating an Embedding Vector DatabaseTo see a code snippetLearning Embeddings# Train an embedding modeldicee--dataset_dirKGs/Countries-S1--path_to_store_single_runCountryEmbeddings--modelKeci--p0--q1--embedding_dim32--adaptive_swaLoading Embeddings into Qdrant Vector Database# Ensure that Qdrant available# docker pull qdrant/qdrant && docker run -p 6333:6333 -p 6334:6334      -v $(pwd)/qdrant_storage:/qdrant/storage:z      qdrant/qdrantdiceeindex--path_model"CountryEmbeddings"--collection_name"dummy"--location"localhost"Launching Webservicediceeserve--path_model"CountryEmbeddings"--collection_name"dummy"--collection_location"localhost"Retrieve and SearchGet embedding of germanycurl-X'GET''http://0.0.0.0:8000/api/get?q=germany'-H'accept: application/json'Get most similar things to europecurl-X'GET''http://0.0.0.0:8000/api/search?q=europe'-H'accept: application/json'{"result":[{"hit":"europe","score":1.0},{"hit":"northern_europe","score":0.67126536},{"hit":"western_europe","score":0.6010134},{"hit":"puerto_rico","score":0.5051694},{"hit":"southern_europe","score":0.4829831}]}Answering Complex QueriesTo see a code snippet# pip install dicee# wget https://files.dice-research.org/datasets/dice-embeddings/KGs.zip --no-check-certificate & unzip KGs.zipfromdicee.executerimportExecutefromdicee.configimportNamespacefromdicee.knowledge_graph_embeddingsimportKGE# (1) Train a KGE modelargs=Namespace()args.model='Keci'args.p=0args.q=1args.optim='Adam'args.scoring_technique="AllvsAll"args.path_single_kg="KGs/Family/family-benchmark_rich_background.owl"args.backend="rdflib"args.num_epochs=200args.batch_size=1024args.lr=0.1args.embedding_dim=512result=Execute(args).start()# (2) Load the pre-trained modelpre_trained_kge=KGE(path=result['path_experiment_folder'])# (3) Single-hop query answering# Query: ?E : \exist E.hasSibling(E, F9M167)# Question: Who are the siblings of F9M167?# Answer: [F9M157, F9F141], as (F9M167, hasSibling, F9M157) and (F9M167, hasSibling, F9F141)predictions=pre_trained_kge.answer_multi_hop_query(query_type="1p",query=('http://www.benchmark.org/family#F9M167',('http://www.benchmark.org/family#hasSibling',)),tnorm="min",k=3)top_entities=[topk_entityfortopk_entity,query_scoreinpredictions]assert"http://www.benchmark.org/family#F9F141"intop_entitiesassert"http://www.benchmark.org/family#F9M157"intop_entities# (2) Two-hop query answering# Query: ?D : \exist E.Married(D, E) \land hasSibling(E, F9M167)# Question: To whom a sibling of F9M167 is married to?# Answer: [F9F158, F9M142] as (F9M157 #married F9F158) and (F9F141 #married F9M142)predictions=pre_trained_kge.answer_multi_hop_query(query_type="2p",query=("http://www.benchmark.org/family#F9M167",("http://www.benchmark.org/family#hasSibling","http://www.benchmark.org/family#married")),tnorm="min",k=3)top_entities=[topk_entityfortopk_entity,query_scoreinpredictions]assert"http://www.benchmark.org/family#F9M142"intop_entitiesassert"http://www.benchmark.org/family#F9F158"intop_entities# (3) Three-hop query answering# Query: ?T : \exist D.type(D,T) \land Married(D,E) \land hasSibling(E, F9M167)# Question: What are the type of people who are married to a sibling of F9M167?# (3) Answer: [Person, Male, Father] since  F9M157 is [Brother Father Grandfather Male] and F9M142 is [Male Grandfather Father]predictions=pre_trained_kge.answer_multi_hop_query(query_type="3p",query=("http://www.benchmark.org/family#F9M167",("http://www.benchmark.org/family#hasSibling","http://www.benchmark.org/family#married","http://www.w3.org/1999/02/22-rdf-syntax-ns#type")),tnorm="min",k=5)top_entities=[topk_entityfortopk_entity,query_scoreinpredictions]print(top_entities)assert"http://www.benchmark.org/family#Person"intop_entitiesassert"http://www.benchmark.org/family#Father"intop_entitiesassert"http://www.benchmark.org/family#Male"intop_entitiesFor more, please refer toexamples/multi_hop_query_answering.Predicting Missing LinksTo see a code snippetfromdiceeimportKGE# (1) Train a knowledge graph embedding model..# (2) Load a pretrained modelpre_trained_kge=KGE(path='..')# (3) Predict missing links through head entity rankingspre_trained_kge.predict_topk(h=[".."],r=[".."],topk=10)# (4) Predict missing links through relation rankingspre_trained_kge.predict_topk(h=[".."],t=[".."],topk=10)# (5) Predict missing links through tail entity rankingspre_trained_kge.predict_topk(r=[".."],t=[".."],topk=10)Downloading Pretrained ModelsTo see a code snippetfromdiceeimportKGE# (1) Load a pretrained ConEx on DBpediamodel=KGE(url="https://files.dice-research.org/projects/DiceEmbeddings/KINSHIP-Keci-dim128-epoch256-KvsAll")For more please look atdice-research.org/projects/DiceEmbeddings/How to DeployTo see a single line of codefromdiceeimportKGEKGE(path='...').deploy(share=True,top_k=10)To see the interface of the webserviceDockerDetailsTo build the Docker image:
```
docker build -t dice-embeddings .
```To test the Docker image:docker run --rm -v ~/.local/share/dicee/KGs:/dicee/KGs dice-embeddings ./main.py --model AConEx --embedding_dim 16How to citeCurrently, we are working on our manuscript describing our framework.
If you really like our work and want to cite it now, feel free to chose one :)# Keci
@inproceedings{demir2023clifford,
  title={Clifford Embeddings--A Generalized Approach for Embedding in Normed Algebras},
  author={Demir, Caglar and Ngonga Ngomo, Axel-Cyrille},
  booktitle={Joint European Conference on Machine Learning and Knowledge Discovery in Databases},
  pages={567--582},
  year={2023},
  organization={Springer}
}
# LitCQD
@inproceedings{demir2023litcqd,
  title={LitCQD: Multi-Hop Reasoning in Incomplete Knowledge Graphs with Numeric Literals},
  author={Demir, Caglar and Wiebesiek, Michel and Lu, Renzhong and Ngonga Ngomo, Axel-Cyrille and Heindorf, Stefan},
  booktitle={Joint European Conference on Machine Learning and Knowledge Discovery in Databases},
  pages={617--633},
  year={2023},
  organization={Springer}
}
# DICE Embedding Framework
@article{demir2022hardware,
  title={Hardware-agnostic computation for large-scale knowledge graph embeddings},
  author={Demir, Caglar and Ngomo, Axel-Cyrille Ngonga},
  journal={Software Impacts},
  year={2022},
  publisher={Elsevier}
}
# KronE
@inproceedings{demir2022kronecker,
  title={Kronecker decomposition for knowledge graph embeddings},
  author={Demir, Caglar and Lienen, Julian and Ngonga Ngomo, Axel-Cyrille},
  booktitle={Proceedings of the 33rd ACM Conference on Hypertext and Social Media},
  pages={1--10},
  year={2022}
}
# QMult, OMult, ConvQ, ConvO
@InProceedings{pmlr-v157-demir21a,
  title = 	 {Convolutional Hypercomplex Embeddings for Link Prediction},
  author =       {Demir, Caglar and Moussallem, Diego and Heindorf, Stefan and Ngonga Ngomo, Axel-Cyrille},
  booktitle = 	 {Proceedings of The 13th Asian Conference on Machine Learning},
  pages = 	 {656--671},
  year = 	 {2021},
  editor = 	 {Balasubramanian, Vineeth N. and Tsang, Ivor},
  volume = 	 {157},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--19 Nov},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v157/demir21a/demir21a.pdf},
  url = 	 {https://proceedings.mlr.press/v157/demir21a.html},
}
# ConEx
@inproceedings{demir2021convolutional,
title={Convolutional Complex Knowledge Graph Embeddings},
author={Caglar Demir and Axel-Cyrille Ngonga Ngomo},
booktitle={Eighteenth Extended Semantic Web Conference - Research Track},
year={2021},
url={https://openreview.net/forum?id=6T45-4TFqaX}}
# Shallom
@inproceedings{demir2021shallow,
  title={A shallow neural model for relation prediction},
  author={Demir, Caglar and Moussallem, Diego and Ngomo, Axel-Cyrille Ngonga},
  booktitle={2021 IEEE 15th International Conference on Semantic Computing (ICSC)},
  pages={179--182},
  year={2021},
  organization={IEEE}For any questions or wishes, please contact:caglar.demir@upb.de