Package: omop2obo
What is OMOP2OBO?omop2obois a collection of health system-wide, disease-agnostic mappings between standardized clinical terminologies in the Observational Medical Outcomes Partnership (OMOP) common data model and several Open Biomedical Ontologies (OBOs) foundry ontologies.MotivationCommon data models have solved many challenges of utilizing electronic health records, but have not yet meaningfully integrated clinical and molecular data. Aligning clinical data to open biological ontologies (OBOs), which provide semantically computable representations of biological knowledge, requires extensive manual curation and expertise.ObjectiveTo address these limitations, we have developedOMOP2OBO, the first health system-wide integration and alignment between the Observational Health Data Sciences and Informatics’ Observational Medical Outcomes Partnership (OMOP) standardized clinical terminologies and eight OBO biomedical ontologies spanning diseases, phenotypes, anatomical entities, cell types, organisms, chemicals, metabolites, hormones, vaccines, and proteins. To verify that the mappings are both clinically and biologically meaningful, we have performed extensive experiments to verify theaccuracy,generalizability, andlogical consistencyof each released mapping set.What Does This Repository Provide?Through this repository we provide the following:Mappings:A free set ofomop2obomappings that can be used out of the box (requires no coding) covering OMOP Conditions, Drug Exposures, and Measurements. These mappings are available in several formats including:.txt,.xlsx, and.dump. We also provide a semantic representation of the mappings, integrated with the OBO biomedical ontologies, available as an edge list (.txt) and as an.owlfile. See current release for more details.A Mapping Framework:An algorithm and mapping pipeline that enables one to construct their set ofomop2obomappings. The figure below provides a high-level overview of the algorithm workflow. The code provided in this repository facilitates all of the automatic steps shown in this figure except for the manual mapping (for now, although we arecurrentlyworking on a deep learning model to address this).How do I Learn More?Join an existing or start a newDiscussionThe ProjectWikifor more details on theomop2obomappings, algorithm, and information on the experiments we ran to ensure each mapping set released is accurate, generalizable, and consistent!ReleasesAll code and mappings for each release are free to download, seeWikiWe are working on a publishing this work, but if you’d like to use the validated mappings immediately, pleasesend us an email.Current Release:Release currently in preparation and planned for*December 2020*!
-v1.0.0➞ data and code can be directly downloadedhere.Getting StartedInstall LibraryNOT YET AVAILABLE - COMING SOON!!This program requires Python version 3.6. To install the library from PyPI, run:pipinstallomop2oboYou can also clone the repository directly from GitHub by running:gitclonehttps://github.com/callahantiff/OMOP2OBO.gitSet-Up EnvironmentTheomop2obolibrary requires a specific project directory structure. Please make sure that your project directory includes the following sub-directories:OMOP2OBO/||----resources/|||clinical_data/|||mappings/|||ontologies/Results will be output to themappingsdirectory.DependenciesAPPLICATIONSThis software also relies onOWLTools. If cloning the repository, theowltoolslibrary file will automatically be included and placed in the correct repository.The National of Library Medicine’s Unified Medical Language System (UMLS)MRCONSOandMRSTY. Using these data requires a license agreement. Note that in order to get theMRSTYfile you will need to download the UMLS Metathesaurus and run MetamorphoSys. Once both data sources are obtained, please place the files in theresources/mappingsdirectory.DATAClinical Data:This repository assumes that the clinical data that needs mapping has been placed in theresources/clinical_datarepository. Each data source provided in this repository is assumed to have been extracted from the OMOP CDM. An example of what is expected for this input can be foundhere.Ontology Data:Ontology data is automatically downloaded from the user provided input fileontology_source_list.txt(here).Vocabulary Source Code Mapping:To increase the likelihood of capturing existing database cross-references,omop2oboprovides a file that maps different clinical vocabulary source code prefixes between the UMLS, ontologies, and clinical EHR data (i.e. “SNOMED”, “SNOMEDCT”, “SNOMEDCT_US”)source_code_vocab_map.csv(here). Please note this file builds off oftheseUMLS provided abbreviation mappings. Currently, this file is updated for ontologies releasedjuly 2020, clinical data normlaized toOMOP_v5.0, andUMLS 2020AA.Semantic Mapping Representation:In order to create a semantic representation of theomop2obomappings, an ontological specification for creating classes that span multiple ontologies (reosurces/mapping_semantics/omop2obo). This document only needs to be altered if you plan to utilize the semantic mapping transformation algorithm and want to use a different knowledge representation. Please the followingREADMEfor additional details on these resources.Running the omop2obo LibraryThere are a few ways to runomop2obo. An example workflow is provided below.importglobimportpandasaspdimportpicklefromdatetimeimportdate,datetimefromomop2oboimportConceptAnnotator,OntologyDownloader,OntologyInfoExtractor,SimilarStringFinder# set some global variablesoutfile='resources/mappings/OMOP2OBO_MAPPED_'date_today='_'+datetime.strftime(datetime.strptime(str(date.today()),'%Y-%m-%d'),'%d%b%Y').upper()# download ontologiesont=OntologyDownloader('resources/ontology_source_list.txt')ont.downloads_data_from_url()# process ontologiesont_explorer=OntologyInfoExtractor('resources/ontologies',ont.data_files)ont_explorer.ontology_processor()# create master dictionary of processed ontologiesont_explorer.ontology_loader()# read in ontology datawithopen('resources/ontologies/master_ontology_dictionary.pickle','rb')ashandle:ont_data=pickle.load(handle)handle.close()# process clinical datamapper=ConceptAnnotator(clinical_file='resources/clinical_data/omop2obo_conditions_june2020.csv',ontology_dictionary={k:vfork,vinont_data.items()ifkin['hp','mondo']},merge=True,primary_key='CONCEPT_ID',concept_codes=tuple(['CONCEPT_SOURCE_CODE']),concept_strings=tuple(['CONCEPT_LABEL','CONCEPT_SYNONYM']),ancestor_codes=tuple(['ANCESTOR_SOURCE_CODE']),ancestor_strings=tuple(['ANCESTOR_LABEL']),umls_mrconso_file=glob.glob('resources/mappings/*MRCONSO*')[0]iflen(glob.glob('resources/mappings/*MRCONSO*'))>0elseNone,umls_mrsty_file=glob.glob('resources/mappings/*MRCONSO*')[0]iflen(glob.glob('resources/mappings/*MRCONSO*'))>0elseNone)exact_mappings=mapper.clinical_concept_mapper()exact_mappings.to_csv(outfile+'CONDITIONS'+date_today+'.csv',sep=',',index=False,header=True)# get column names -- used later to organize outputstart_cols=[iforiinexact_mappings.columnsifnotany(jforjin['STR','DBXREF','EVIDENCE']ifjini)]exact_cols=[iforiinexact_mappings.columnsifinotinstart_cols]# perform similarity mappingiftfidf_mappingisnotNone:sim=SimilarStringFinder(clinical_file=outfile+'CONDITIONS'+date_today+'.csv',ontology_dictionary={k:vfork,vinont_data.items()ifkin['hp','mondo']},primary_key='CONCEPT_ID',concept_strings=tuple(['CONCEPT_LABEL','CONCEPT_SYNONYM']))sim_mappings=sim.performs_similarity_search()sim_mappings=sim_mappings[['CONCEPT_ID']+[xforxinsim_mappings.columnsif'SIM'inx]].drop_duplicates()# get column names -- used later to organize outputsim_cols=[iforiinsim_mappings.columnsifnotany(jforjinstart_colsifjini)]# merge dbXref, exact string, and TF-IDF similarity resultsmerged_scores=pd.merge(exact_mappings,sim_mappings,how='left',on='CONCEPT_ID')# re-order columns and write out datamerged_scores=merged_scores[start_cols+exact_cols+sim_cols]merged_scores.to_csv(outfile+clinical_domain.upper()+date_today+'.csv',sep=',',index=False,header=True)COMMAND LINE➞main.pypythonmain.py--helpUsage:main.py[OPTIONS]TheOMOP2OBOpackageprovidesfunctionalitytoassistwithmappingOMOPstandardclinicalterminologyconceptstoOBOterms.Successfullyrunningthisprogramrequiresseveralinputparameters,whicharespecifiedbelow:PARAMETERS:ont_file:'resources/oontology_source_list.txt'tfidf_mapping:"yes"ifwanttoperformcosinesimilaritymappingusingaTF-IDFmatrix.clinical_domain:clinicaldomainofinputdata(i.e."conditions","drugs",or"measurements").merge:AboolspecifyingwhethertomergeUMLSSABcodeswithOMOPsourcecodesonceortwice.onts:Acomma-separatedlistofontologyprefixesthatmatches'resources/oontology_source_list.txt'.clinical_data:Thefilepathtotheclinicaldataneedingmapping.primary_key:Thenameofthefiletouseastheprimarykey.concept_codes:Acomma-separatedlistofconcept-levelcodestouseforDbXRefmapping.concept_strings:Acomma-separatedlistofconcept-levelstringstomaptouseforexactstringmapping.ancestor_codes:Acomma-separatedlistofancestor-levelcodestouseforDbXRefmapping.ancestor_strings:Acomma-separatedlistofancestor-levelstringstomaptouseforexactstringmapping.outfile:Thefilepathforwheretowriteoutputdatato.Severaldependenciesmustbeaddressedbeforerunningthisfile.PleaseseetheREADMEforinstructions.Options:--ont_filePATH[required]--tfidf_mappingTEXT[required]--clinical_domainTEXT[required]--merge[required]--ontTEXT[required]--clinical_dataPATH[required]--primary_keyTEXT[required]--concept_codesTEXT[required]--concept_stringsTEXT--ancestor_codesTEXT--ancestor_stringsTEXT--outfileTEXT[required]--helpShowthismessageandexit.If you follow the instructions for how to format clinical data (here) and/or if taking the data that results from running our querieshere),omop2obocan be run with the following call on the command line (with minor updates to the csv filename):pythonmain.py--clinical_domaincondition--ontshp--ontsmondo--clinical_dataresources/clinical_data/omop2obo_conditions_june2020.csvJUPYTER NOTEBOOK➞omop2obo_notebook.ipynbContributingPlease readCONTRIBUTING.mdfor details on our code of conduct, and the process for submitting pull requests to us.LicenseThis project is licensed under MIT - see theLICENSE.mdfile for details.Citing this Work@software{callahan_tiffany_j_2020_3902767,author={Callahan,TiffanyJ},title={OMOP2OBO},month=jun,year=2020,publisher={Zenodo},version={v1.0.0},doi={10.5281/zenodo.3902767},url={https://doi.org/10.5281/zenodo.3902767}.}ContactWe’d love to hear from you! To get in touch with us, please join or start a newDiscussion,create an issueorsend us an email💌