Package: genomic-benchmarks
Genomic Benchmarks 🧬🏋️✔️In this repository, we collect benchmarks for classification of genomic sequences. It is shipped as a Python package, together with functions helping to download & manipulate datasets and train NN models.InstallGenomic Benchmarks can be installed as follows:pipinstallgenomic-benchmarksTo use it with papermill, TF or pytorch, install the corresponding dependencies:# if you want to use jupyter and papermillpipinstalljupyter>=1.0.0
pipinstallpapermill>=2.3.0# if you want to train NN with TFpipinstalltensorflow>=2.6.0
pipinstalltensorflow-addons
pipinstalltyping-extensions--upgrade# fixing TF installation issue# if you want to train NN with torchpipinstalltorch>=1.10.0
pipinstalltorchtextFor the package development, use Python 3.8 (ideally 3.8.9) and the installation describedhere.UsageGet the list of all datasets with thelist_datasetsfunction>>>fromgenomic_benchmarks.data_checkimportlist_datasets>>>>>>list_datasets()['demo_coding_vs_intergenomic_seqs','demo_human_or_worm','dummy_mouse_enhancers_ensembl','human_enhancers_cohn','human_enhancers_ensembl','human_ensembl_regulatory','human_nontata_promoters','human_ocr_ensembl']You can get basic information about the benchmark withinfofunction:>>>fromgenomic_benchmarks.data_checkimportinfo>>>>>>info("human_nontata_promoters",version=0)Dataset`human_nontata_promoters`has2classes:negative,positive.Alllenghtsofgenomicintervalsequals251.Totally36131sequenceshavebeenfound,27097fortrainingand9034fortesting.traintestnegative123554119positive147424915The functiondownload_datasetdownloads the full-sequence form of the required benchmark (splitted into train and test sets, one folder for each class). If not specified otherwise, the data will be stored in.genomic_benchmarkssubfolder of your home directory. By default, the dataset is obtained from our cloud cache (use_cloud_cache=True).>>>fromgenomic_benchmarks.loc2seqimportdownload_dataset>>>>>>download_dataset("human_nontata_promoters",version=0)Downloading1VdUg0Zu8yfLS6QesBXwGz1PIQrTW3Ze4into/home/petr/.genomic_benchmarks/human_nontata_promoters.zip...Done.Unzipping...Done.PosixPath('/home/petr/.genomic_benchmarks/human_nontata_promoters')Getting TensorFlow Dataset for the benchmark and displaying samples is straightforward:>>>frompathlibimportPath>>>importtensorflowastf>>>>>>BATCH_SIZE=64>>>SEQ_TRAIN_PATH=Path.home()/'.genomic_benchmarks'/'human_nontata_promoters'/'train'>>>CLASSES=['negative','positive']>>>>>>train_dset=tf.keras.preprocessing.text_dataset_from_directory(...directory=SEQ_TRAIN_PATH,...batch_size=BATCH_SIZE,...class_names=CLASSES)Found27097filesbelongingto2classes.>>>>>>list(train_dset)[0][0][0]<tf.Tensor:shape=(),dtype=string,numpy=b'TCCTGCCTTTCCACTTGCACCAGTTTTCCCACCCCAGCCTCAGGGCGGGGCTGCCTCGTCACTTGTCTCGGGGCAGATCTGCCCTACACACGTTAGCGCCGCGCGCAAAGCAGCCCCGCAGCACCCAGGCGCCTCCTGGCGGCGCCGCGAAGGGGCGGGGCTGTCGGCTGCGCGTTGTGCGCTGTCCCAGGTTGGAAACCAGTGCCCCAGGCGGCGAGGAGAGCGGTGCCTTGCAGGGATGCTGCGGGCGG'>SeeHow_To_Train_CNN_Classifier_With_TF.ipynbfor more detailed description how to train CNN classifier with TensorFlow.Getting Pytorch Dataset and displaying samples is also easy:>>>fromgenomic_benchmarks.dataset_getters.pytorch_datasetsimportHumanNontataPromoters>>>>>>dset=HumanNontataPromoters(split='train',version=0)>>>dset[0]('CAATCTCACAGGCTCCTGGTTGTCTACCCATGGACCCAGAGGTTCTTTGACAGCTTTGGCAACCTGTCCTCTGCCTCTGCCATCATGGGCAACCCCAAAGTCAAGGCACATGGCAAGAAGGTGCTGACTTCCTTGGGAGATGCCATAAAGCACCTGGATGATCTCAAGGGCACCTTTGCCCAGCTGAGTGAACTGCACTGTGACAAGCTGCATGTGGATCCTGAGAACTTCAAGGTGAGTCCAGGAGATGT',0)SeeHow_To_Train_CNN_Classifier_With_Pytorch.ipynbfor more detailed description how to train CNN classifier with Pytorch.Structure of packagedatasets: Each folder is one benchmark dataset (or a set of bechmarks in subfolders), seeREADME.mdfor the format specificationdocs: Each folder contains a Python notebook that has been used for the dataset creationexperiments: Training a simple neural network model(s) for each benchmark dataset, can be used as a baselinenotebooks: Main use-cases demonstrated in a form of Jupyter notebookssrc/genomic_benchmarks: Python module for datasets manipulation (downlading, checking, etc.)tests: Unit tests forpytestandpytest-covHow to contributeHow to contribute a modelIf you beat our current best model on any dataset or just came with an interesting new idea, let us know about it: Make you code publicly available (GitHub repo, Colab...) and fill in the form athttps://forms.gle/pvkkrgHNCNmAAC1TAHow to contribute a datasetIf you have an interesting genomic dataset, send usan issuewith the description and possibly link to the data (e.g. BED file and FASTQ reference). In the future, we will provide functions to make the import easy.If you are a hero, readthe specification of our dataset formatand send us a pull request with newdatasets/[YOUR_DATASET_NAME]anddocs/[YOUR_DATASET_NAME]folders.How to improve code in this packageWe welcome new code contributors. If you see a bug, send usan issuewith aminimal reproducible example. Or even better, fix the bug and send us a pull request.Citing Genomic BenchmarksIf you use Genomic Benchmarks in your research, please cite it as follows.TextGRESOVA, Katarina, et al. Genomic Benchmarks: A Collection of Datasets for Genomic Sequence Classification. bioRxiv, 2022.BibTeX@article{gresova2022genomic,title={Genomic Benchmarks: A Collection of Datasets for Genomic Sequence Classification},author={Gresova, Katarina and Martinek, Vlastimil and Cechak, David and Simecek, Petr and Alexiou, Panagiotis},journal={bioRxiv},year={2022},publisher={Cold Spring Harbor Laboratory},url={https://www.biorxiv.org/content/10.1101/2022.06.08.495248}}