Package: unKR
unKR: A Python Library for Uncertain Knowledge Graph Reasoning by Representation LearningunKR is an python library foruncertainKnowledge graph (UKG)Reasoning based on thePyTorch Lightning. It provides a unifying workflow to implement a variety of uncertain knowledge graph representation learning models to complete UKG reasoning. unKR consists of five modules: 1) Data Processor handles low-level dataset parsing and negative sampling, then generates mini-batches of data; 2) Model Hub implements the model algorithms, containing the scoring function and loss function; 3) Trainer conducts iterative training and validation; 4)Evaluatorprovides confidence prediction and link prediction tasks to evaluate models' performance; 5) Controller controls the training worklow, allowing for early stopping and model saving. These modules are decoupled and independent, making unKR highly modularized and extensible. Detailed documentation of the unKR is available athere.unKR core development team will provide long-term technical support, and developers are welcome to discuss the work and initiate questions usingissue.DemoThis is a demo shows the training and testing process ofPASSLEAFmodel with unKR.DatasetsunKR provides three public UKG datasets including CN15K, NL27K, and PPI5K. The following table shows the source, the number of entities, relations, and facts of each dataset.DatasetSource#Entity#Relation#FactCN15KConceptNet1500036241158NL27KNELL27221404175412PPI5KSTRING49997271666ModelsNow, nine uncertain knowledge graph representation learning models are available and they can be divided to two types: normal and few-shot models.TypeModelNormalBEURrE,FocusE,GTransE,PASSLEAF,UKGE,UKGsE,UPGATFew-shotGMUC,GMUC+Reproduce ResultsunKR determines two tasks, confidence prediction and link prediction, to evaluate models' ability of UKG reasoning. For confidence prediction task, MSE (Mean Squared Error) and MAE (Mean Absolute Error) are reported. For link prediction task, Hits@k(k=1,3,10), MRR (Mean Reciprocal Rank), MR (Mean Rank) under both raw and filterd settings are reported. In addition, we choose high-confidence (>0.7) triples as the test data for link prediction.Here are the reproduce results of nine models on NL27K dataset with unKR. See more results athere.Confidence predictionTypeModelMSEMAENormalBEUrRE0.089200.22194PASSLEAF(ComplEx)0.024340.05176PASSLEAF(DistMult)0.023090.05107PASSLEAF(RotatE)0.019490.06253UKGE_logi0.028680.05966UKGE_rect0.033260.07015UKGsE0.122020.27065UPGAT0.029220.10107Few-shotGMUC0.013000.08200GMUC+0.013000.08600Link predictionTypeModelHits@1Hits@3Hits@10MRRMRNormalBEUrRE0.1560.3850.5430.299488.051FocusE0.8140.9180.9570.870384.471GTransE0.2220.3660.4930.3161377.564PASSLEAF(ComplEx)0.6690.7860.8760.741138.808PASSLEAF(DistMult)0.6270.7540.8560.707138.781PASSLEAF(RotatE)0.6870.8160.8840.76250.776UKGE_logi0.5250.6730.8120.623168.029UKGE_rect0.5000.6470.8000.599125.233UKGsE0.0380.0730.1300.0692329.501UPGAT0.6180.7510.8620.70169.120Few-shotGMUC0.3350.4650.5920.42558.312GMUC+0.3380.4860.6360.43845.774UsageInstallationStep1Create a virtual environment usingAnacondaand enter it.condacreate-nunKRpython=3.8
condaactivateunKRStep2Install package.Install from sourcegitclonehttps://github.com/seucoin/unKR.gitcdunKR
pipinstall-rrequirements.txt
pythonsetup.pyinstallInstall by pypipipinstallunKRData FormatFor normal models,train.tsv,val.tsv, andtest.tsvare required.train.tsv: All facts used for training in the format(h, r, t, s), one fact per line.val.tsv: All facts used for validation in the format(h, r, t, s), one fact per line.test.tsv: All facts used for testing in the format(h, r, t, s), one fact per line.For few-shot models,train_tasks.json,dev_tasks.json,test_tasks.jsonandpath_graphare required.train/dev/test_tasks.json: Few-shot dataset with one task per relation in the format{r:[[h, r, t, s], ...]}. The key of the dictionary is the task name and the values are all the facts under the task.path_graph: background knowledge, i.e., all data except training, validation and testing tasks, in the format(h, r, t, s). One fact per line.ForUKGE, thesoftlogic.tsvfile is also required.softlogic.tsv: All facts inferred by PSL in the format(h, r, t, s), one fact per line.Parameter SettingYou can set up parameters byconfigfile. The desciption of each paramter is athere.Model Trainingpythonmain.py--load_config--config_path<your-config>Model Testingpythonmain.py--test_only--checkpoint_dir<your-model-path>Model CustomizationIf you want to customize your own model using unKR, you need to create the following classes/functions.data: Implement data processing functions, includingDataPreprocess,SamplerandKGDataModule.DataPreprocess.py: 
    class unKR.data.DataPreprocess.<your-model-name>BaseSampler
    class unKR.data.DataPreprocess.<your-model-name>Data
Sampler:
    class unKR.data.Sampler.<your-model-name>Sampler
    class unKR.data.Sampler.<your-model-name>TestSampler
KGDataModule.py: 
    class unKR.data.KGDataModule.<your-model-name>DataModulelit_model: Implement model training, validation, and testing functions.<your-model-name>LitModel.py:
    class unKR.lit_model.<your-model-name>LitModel.<your-model-name>LitModelloss: Implement loss functions.<your-model-name>_Loss.py:
    class unKR.loss.<your-model-name>_Loss.<your-model-name>_Lossmodel: Implement model framework functions, classified asUKGModelandFSUKGModelbased on whether it is a few-shot model.<your-model-name>.py:
    class unKR.model.UKGModel/FSUKGModel.<your-model-name>.<your-model-name>config: Implement parameter settings.<your-model-name>_<dataset-name>.yaml:
    data_class, litmodel_name, loss_name, model_name, test_sampler_class, train_sampler_classdemo: Implement the model run file.<your-model-name>demo.pyunKR Core TeamSoutheast University: Jingting Wang, Tianxing Wu, Shilin Chen, Yunchang Liu, Shutong Zhu, Wei Li, Jingyi Xu, Guilin Qi.