Package: metakb
metakbThe intent of the project is to leverage the collective knowledge of the disparate existing resources of the VICC to improve the comprehensiveness of clinical interpretation of genomic variation. An ongoing goal will be to provide and improve upon standards and guidelines by which other groups with clinical interpretation data may make it accessible and visible to the public. We have released a preprint discussing our initial harmonization effort and observed disparities in the structure and content of variant interpretations.Getting StartedThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.PrerequisitesA newer version of Python 3, preferably 3.8 or greater. To confirm on your system, run:python3 --versionPipenv, for package management.pip3 install --user pipenvInstallingOnce Pipenv is installed, clone the repo and install the package requirements into a Pipenv environment:gitclonehttps://github.com/cancervariants/metakbcdmetakb
pipenvlock&&pipenvsyncIf you intend to provide development support, install the development dependencies:pipenvlock--dev&&pipenvsyncSetting up Neo4jThe MetaKB usesNeo4jfor its database backend. To run a local MetaKB instance, you'll need to run a Neo4j database instance as well. The easiest way to do this is from Neo4j Desktop.First, follow thedesktop setup instructionsto download, install, and open Neo4j Desktop for the first time.Once you have opened Neo4j desktop, use the "New" button in the upper-left region of the window to create a new project. Within that project, click the "Add" button in the upper-right region of the window and select "Local DBMS". The name of the DBMS doesn't matter, but the password will be used later to connect the database to MetaKB (we have been using "admin" by default). Click "Create". Then, click the row within the project screen corresponding to your newly-created DBMS, and click the green "Start" button to start the database service.The graph will initially be empty, but once you have successfully loaded data, Neo4j Desktop provides an interface for exploring and visualizing relationships within the graph. To access it, click the blue "Open" button. The prompt at the top of this window processesCypher queries; to start, tryMATCH (n:Statement {id:"civic.eid:1409"}) RETURN n. Buttons on the left-hand edge of the results pane let you select graph, tabular, or textual output.Setting up normalizersThe MetaKB calls a number of normalizer libraries to transform resource data and resolve incoming search queries. These will be installed as part of the package requirements, but require additional setup.First,follow these instructions for deploying DynamoDB locally on your computer. Once setup, in a separate terminal instance, navigate to its source directory and run the following to start the database instance:java-Djava.library.path=./DynamoDBLocal_lib-jarDynamoDBLocal.jar-sharedDbNext, navigate to thesite-packagesdirectory of your virtual environment. Assuming Pipenv is installed to your user directory, this should be something like:cd~/.local/share/virtualenvs/metakb-<variouscharacters>/lib/python<python-version>/site-packages/# replace <various characters> and <python-version>Next, initialize theVariation Normalizerby following the instructions in theREADME. When setting up the UTA database,thesedocs may be helpful.The MetaKB can acquire all other needed normalizer data, except for that ofOMIM, which must be manually placed:cddisease/# starting from the site-packages dir of your virtual environment's Python instancemkdir-pdata/omim
cp~/YOUR/PATH/TO/mimTitles.txtdata/omim/omim_<date>.tsv# replace <date> with date of data acquisition formatted as YYYYMMDDEnvironment VariablesMetaKB relies on environment variables to set in order to work.Always Required:UTA_DB_URLUsed in Variation Normalizer which relies on UTA ToolsFormat:driver://user:pass@host/database/schemaMore info can be foundhereExample:exportUTA_DB_URL=postgresql://uta_admin:password@localhost:5432/uta/uta_20210129Required when using the--load_normalizers_dbor--force_load_normalizers_dbarguments in CLI commandsRXNORM_API_KEYUsed in Therapy Normalizer to retrieve RxNorm dataRxNorm requires a UMLS license, which you can register for onehere. You must set theRxNORM_API_KEYenvironment variable to your API key. This can be found in theUTS 'My Profile' areaafter singing in.Example:exportRXNORM_API_KEY={rxnorm_api_key}DATAVERSE_API_KEYUsed in Therapy Normalizer to retrieve HemOnc dataHemOnc.org data requires a Harvard Dataverse API key. After creating a user account on the Harvard Dataverse website, you can followthese instructionsto generate a key. You will create or login to your account atthissite. You must set theDATAVERSE_API_KEYenvironment variable to your API key.Example:exportDATAVERSE_API_KEY={dataverse_api_key}Loading dataOnce Neo4j and DynamoDB instances are both running, and necessary normalizer data has been placed, run the MetaKB CLI with the--initialize_normalizersflag to acquire all other necessary normalizer source data, and execute harvest, transform, and load operations into the graph datastore.In the MetaKB project root, run the following:pipenvshell
python3-mmetakb.cli--db_url=bolt://localhost:7687--db_username=neo4j--db_password=<neo4j-password-here>--load_normalizers_dbFor more information on the different CLI arguments, see theCLI README.Starting the serverOnce data has been loaded successfully, use the following to start service on localhost port 8000:uvicornmetakb.main:app--reloadEnsure that both the MetaKB Neo4j and Normalizers databases are running.Navigate tohttp://localhost:8000/api/v2in your browser to enter queries.Running testsUnit testsExplain how to run the automated tests for this systempython3-mpytestAnd coding style testsCode style is managed byflake8and checked prior to commit.see .flake8ContributingPlease readCONTRIBUTING.mdfor details on our code of conduct, and the process for submitting pull requests to us.CommittingWe usepre-committo run conformance tests.This ensures:Check code styleCheck for added large filesDetect AWS CredentialsDetect Private KeyBefore first commit run:pre-commitinstallVersioningWe useSemVerfor versioning. For the versions available, see thetags on this repository.LicenseThis project is licensed under the MIT License - see theLICENSEfile for details