Package: pkt-kg
What is PheKnowLator?PheKnowLator (Phenotype Knowledge Translator) orpkt_kgis the first fully customizable knowledge graph (KG) construction framework enabling users to build complex KGs that are Semantic Web compliant and amenable to automatic Web Ontology Language (OWL) reasoning, generate contemporary property graphs, and are importable by today’s popular graph toolkits. Please see the projectWikifor additional information.📢 Please see our preprint 👉https://arxiv.org/abs/2307.05727What Does This Repository Provide?A Knowledge Graph Sharing Hub:Prebuilt KGs and associated metadata. Each KG is provided as triple edge lists, OWL API-formattedRDF/XMLand NetworkX graph-pickled MultiDiGraphs. We also make text files available containing node and relation metadata.A Knowledge Graph Building Framework:An automatedPython 3library designed for optimized construction of semantically-rich, large-scale biomedical KGs from complex heterogeneous data. The framework also includes Jupyter Notebooks to greatly simplify the generation of required input dependencies.NOTE.A table listing and describing all output files generated for each build along with example output from each
file can be foundhere.How do I Learn More?Join and/or start aDiscussionThe ProjectWikifor available knowledge graphs,pkt_kgdata sources, and the knowledge graph construction processAZenodo Communityhas been established to provide access to software releases, presentations, and preprints related to this projectReleasesData AccessBuild DocumentationGetting StartedInstall LibraryThis program requires Python version 3.6. To install the library fromPyPI, run:pipinstallpkt_kgYou can also clone the repository directly from GitHub by running:gitclonehttps://github.com/callahantiff/PheKnowLator.gitNote.SometimesOWLTools, which comes with the cloned/forked repository (./pkt_kg/libs/owltools) loses “executable” permission. To avoid any potential issues, I recommend running the following in the terminal from the PheKnowLator directory:chmod+xpkt_kg/libs/owltoolsSet-Up EnvironmentThepkt_kglibrary requires a specific project directory structure.If you plan to run the code from a cloned version of this repository, then no additional steps are needed.If you are planning to utilize the library without cloning the library, please make sure that your project directory matches the following:PheKnowLator/||----resources/|||construction_approach/|||edge_data/|||knowledge_graphs/|||node_data/|||ontologies/|||owl_decoding/|||relations_data/DependenciesSeveral input documents must be created before thepkt_kglibrary can be utilized. Each of the input documents are listed below by knowledge graph build step:DOWNLOAD DATAThis code requires three documents within theresourcesdirectory to run successfully. For more information on these documents, seeDocument Dependencies:resources/resource_info.txtresources/ontology_source_list.txtresources/edge_source_list.txtFor assistance in creating these documents, please run the following from the root directory:python3generates_dependency_documents.pyPrior to running this step, make sure that all mapping and filtering data referenced inresources/resource_info.txthave been created. To generate these data yourself, please see theData_Preparation.ipynbJupyter Notebook for detailed examples of the steps used to build thev2.0.0 knowledge graph.Note.To ensure reproducibility, after downloading data, a metadata file is output for the ontologies (ontology_source_metadata.txt) and edge data sources (edge_source_metadata.txt).CONSTRUCT KNOWLEDGE GRAPHTheKG ConstructionWiki page provides a detailed description of the knowledge construction process (please see the knowledge graphREADMEfor more information). Please make sure the documents listed below are presented in the specified location prior to constructing a knowledge graph. Click on each document for additional information. Note, that cloning this library will include a version of these documents that points to the current build. If you use this version then there is no need to download anything prior to running the program.resources/construction_approach/subclass_construction_map.pklresources/Master_Edge_List_Dict.json➞automatically created after edge list constructionresources/node_data/node_metadata_dict.pkl➞if adding metadata for new edges to the knowledge graphresources/knowledge_graphs/PheKnowLator_MergedOntologies*.owl➞seeontology READMEfor informationresources/relations_data/RELATIONS_LABELS.txtresources/relations_data/INVERSE_RELATIONS.txt➞if including inverse relationsRunning the pkt Librarypkt_kgcan be run via the providedmain.pyscript or using themain.ipynbJupyter Notebook or using a Docker container.Main Script or Jupyter NotebookThe program can be run locally using themain.pyscript or using themain.ipynbJupyter Notebook. An example of the workflow used in both of these approaches is shown below.importpsutilimportrayfrompktimportdownloads,edge_list,knowledge_graph# initialize rayray.init()# determine number of cpus availableavailable_cpus=psutil.cpu_count(logical=False)# DOWNLOAD DATA# ontology dataont=pkt.OntData('resources/ontology_source_list.txt')ont.downloads_data_from_url()ont.writes_source_metadata_locally()# edge data sourcesedges=pkt.LinkedData('resources/edge_source_list.txt')edges.downloads_data_from_url()edges.writes_source_metadata_locally()# CREATE MASTER EDGE LISTcombined_edges=dict(edges.data_files,**ont.data_files)# initialize edge dictionary classmaster_edges=pkt.CreatesEdgeList(data_files=combined_edges,source_file='./resources/resource_info.txt')master_edges.runs_creates_knowledge_graph_edges(source_file'./resources/resource_info.txt',data_files=combined_edges,cpus=available_cpus)# BUILD KNOWLEDGE GRAPH# full build, subclass construction approach, with inverse relations and node metadata, and decode owlkg=PartialBuild(kg_version='v2.0.0',write_location='./resources/knowledge_graphs',construction='subclass,node_data='yes,inverse_relations='yes',cpus=available_cpus,decode_owl='yes')kg.construct_knowledge_graph()ray.shutdown()main.pyThe example below provides the details needed to runpkt_kgusing./main.py.python3main.py-husage:main.py[-h][-pCPUS]-gONTS-eEDG-aAPP-tRES-bKG-oOUT-nNDE-rREL-sOWL-mKGMPheKnowLator:ThisprogrambuildsabiomedicalknowledgegraphusingOpenBiomedicalOntologiesandlinkedopendata.Theprogramtakesthefollowingarguments:optionalarguments:-h,--helpshowthishelpmessageandexit-pCPUS,--cpusCPUS# workers to use; defaults to use all available cores-gONTS,--ontsONTSname/pathtotextfilecontainingontologies-eEDG,--edgEDGname/pathtotextfilecontainingedgesources-aAPP,--appAPPconstructionapproachtouse(i.e.instanceorsubclass-tRES,--resRESname/pathtotextfilecontainingresource_info-bKG,--kgKGthebuild,canbe"partial","full",or"post-closure"-oOUT,--outOUTname/pathtodirectorywheretowriteknowledgegraph-rREL,--relRELyes/no-addinginverserelationstoknowledgegraph-sOWL,--owlOWLyes/no-removingOWLSemanticsfromknowledgegraphmain.ipynbThe./main.ipynbJupyter notebook provides detailed instructions for how to run thepkt_kgalgorithm and build a knowledge graph from scratch.Docker Containerpkt_kgcan be run using a Docker instance. In order to utilize the Dockerized version of the code, please make sure that you have downloaded the newest version ofDocker. There are two ways to utilize Docker with this repository:Obtain Pre-Built Container fromDockerHubBuild the Container (see details below)Obtaining a ContainerObtain Pre-Built Containiner:A pre-built containers can be obtained directly fromDockerHub.Build Container:To build thepkt_kgdownload a stable release of this repository (or fork/clone it repository). Once downloaded, you will have everything needed to build the container, including the./Dockerfileand./dockerignore. The code shown below builds the container. Make sure to replace[VERSION]with the currentpkt_kgversion before running the code.cd/path/to/PheKnowLator(Note,thisisthedirectorycontainingtheDockerfilefile)dockerbuild-tpkt:[VERSION].Notes:UpdatePheKnowLator/resources/resource_info.txt,PheKnowLator/resources/edge_source_list.txt, andPheKnowLator/resources/ontology_source_list.txtBuilding the container “as-is” off of DockerHub will include a download of the data used in the latest releases. No need to update any scripts or pre-download any data.Running a ContainerThe following code can be used to runpkt_kgfrom outside of the container (after obtaining a prebuilt container or after building the container locally). In:dockerrun--name[DOCKERCONTAINERNAME]-itpkt:[VERSION]--appsubclass--kgfull--ndeyes--relyes--owlno--kgmyesNotes:The example shown above builds a full version of the knowledge graph using the subclass construction approach with node metadata, inverse relations, and decoding of OWL classes. See theRunning the pkt Librarysection for more information on the parameters that can be passed topkt_kgThe Docker container cannot write to an encrypted filesystem, however, so please make sure/local/path/to/PheKnowLator/resources/knowledge_graphsreferences a directory that is not encryptedFinding Data Inside a ContainerIn order to enable persistent data, a volume is mounted within theDockerfile. By default, Docker names volumes using a hash. In order to find the correctly mounted volume, you can run the following:Command 1:Obtains the volume hash:dockerinspect--format='{{json .Mounts}}'[DOCKERCONTAINERNAME]|python-mjson.toolCommand 2:View data written to the volume:sudols/var/lib/docker/volumes/[VOLUMEHASH]/_dataGet In Touch or Get InvolvedContributionPlease readCONTRIBUTING.mdfor details on our code of conduct, and the process for submitting pull requests to us.Contact UsWe’d love to hear from you! To get in touch with us, please join or start a newDiscussion,create an issueorsend us an email💌AttributionLicensingThis project is licensed under Apache License 2.0 - see theLICENSE.mdfile for details.Citing this WorkPlease see our preprint:https://arxiv.org/abs/2307.05727