Package: gnomad-db
gnomAD_DBChangelogNEW version (November 2023)release gnomAD WGS v4.0 and WES v4.0gnomad_version=["v2"|"v3"|"v4"] argument has to be specified when initializing the databaseminor fixesversion (July 2022)release gnomAD WGS v3.1.2minor bug fixesversion (December 2021)more available variant features present, checkhereget_maf_from_dfrenamed toget_info_from_dfget_maf_from_strrenamed toget_info_from_str[DEPRECATED 11.2023]genome=["Grch37"|"Grch38"] argument has to be specified when initializing the databaseWhy and WhatThe Genome Aggregation Database (gnomAD)is a resource developed by an international coalition of investigators, with the goal of aggregating and harmonizing both exome and genome sequencing data from a wide variety of large-scale sequencing projects, and making summary data available for the wider scientific community.This package scales the huge gnomAD files (on average ~120G/chrom) to a SQLite database with a size of 34G for WGS v2.1.1 (261.942.336 variants) and 98G for WGS v3.1.2 (about 759.302.267 variants), and allows scientists to look for various variant annotations present in gnomAD (i.e. Allele Count, Depth, Minor Allele Frequency, etc. -hereyou can find all selected features given the genome version). (A query containing 300.000 variants takes ~40s.)It extracts from a gnomAD vcf about 23 variant annotations. You can find further information about the exact fieldshere.The package works for all currently available gnomAD releases.(July 2022)1. Download SQLite preprocessed filesI have preprocessed and created sqlite3 files for gnomAD for you, which can be easily downloaded from here. They contain all variants on the 24 standard chromosomes.WGS gnomAD v4.0 (hg38,618'765'636variants) 36.1G zipped, 74G in total -https://zenodo.org/records/10066323/files/gnomad_db_wgs_v4.0.sqlite3.gz?download=1WES gnomAD v4.0 (hg38,161'417'006variants) 7.3G zipped, 17G in total -https://zenodo.org/records/10066310/files/gnomad_db_wes_v4.0.sqlite3.gz?download=1WGS gnomAD v3.1.2 (hg38,759'302'267variants) 46.2G zipped, 98G in total -https://zenodo.org/record/6818606/files/gnomad_db_v3.1.2.sqlite3.gz?download=1WGS gnomAD v2.1.1 (hg19,261'942'336variants) 16.1G zipped, 48G in total -https://zenodo.org/record/5770384/files/gnomad_db_v2.1.1.sqlite3.gz?download=1You can download it as:fromgnomad_db.databaseimportgnomAD_DBdownload_link="https://zenodo.org/record/6818606/files/gnomad_db_v3.1.2.sqlite3.gz?download=1"output_dir="test_dir"# database_locationgnomAD_DB.download_and_unzip(download_link,output_dir)NB this would take ~30min (network speed 10mb/s)or you can create the database by yourself.However, I recommend using the preprocessed files to save resources and time. If you do so, you can go to2. API usageand explore the package and its great features!2. API usageCongratulations, your database is set up! Now it is time to learn how to use it.First, you can install the package in the gnomad_db env or in the one which you are going to use for your downstream analysispipinstallgnomad_dbYou can use the package likeimport modulesimportpandasaspdfromgnomad_db.databaseimportgnomAD_DBInitialize database connectionMake sure to have the correct gnomad version!# pass dirdatabase_location="test_dir"db=gnomAD_DB(database_location,gnomad_version="v3")Insert some test variants to run the examples belowIf you have downloaded the preprocessed sqlite3 files, you can skip this step as you already have variants, make sure to have the correct genome version!# get some variantsvar_df=pd.read_csv("data/test_vcf_gnomad_chr21_10000.tsv.gz",sep="\t",names=db.columns,index_col=False)# IMPORTANT: The database removes internally chr prefix (chr1->1)# insert these variantsdb.insert_variants(var_df)Query variant minor allele frequencyThese example variants are assembled to hg38!# query some MAF scoresdummy_var_df=pd.DataFrame({"chrom":["1","21"],"pos":[21,9825790],"ref":["T","C"],"alt":["G","T"]})# query from dataframe AF columndb.get_info_from_df(dummy_var_df,"AF")# query from dataframe AF and AF_popmax columnsdb.get_info_from_df(dummy_var_df,"AF, AF_popmax")# query from dataframe all columnsdb.get_info_from_df(dummy_var_df,"*")# query from stringdb.get_info_from_str("21:9825790:C>T","AF")You can query also intervals of minor allele frequenciesdb.get_info_for_interval(chrom=21,interval_start=9825780,interval_end=9825799,query="AF")For more information on how to use the package, look into GettingStartedwithGnomAD_DB.ipynb notebook!NB: The package is under development and any use cases suggestions/extensions and feedback are welcome.