Package: get-assemblies
Table of ContentsWho might want this software?InstallationDependenciesJust tell me how to run itOverviewMore ExamplesUsageOrganismAssembly IDsNucleotide IDsNCBI JSON Docsum inputBugsContributingAuthor ContactAcknowledgmentsLicenseCopyrightsWho might want this software?This software is intended to facilitate the use of theNCBI assembly database.The intended audience is scientific researchers, computational biologists, and
bioinformaticians who need sequence data for comparative genomics projects.Anyone using NCBI sequence data (for reference genomes/transcriptomes) can
benefit as well.PhD/Masters students and undergraduates are especially encouraged to submit
issues if they are having trouble using this software.This software is written in python. I’m happy to help
those who are having trouble with software installs.Windows users, please install WSL to make use of this software. Using a Linux
distribution will make your life as a computational researcher significantly
easier.Installationget_assemblies is distributed onPyPIas a universal wheel and is
available on Linux/macOS and supports Python 3.7+. This software will work on
Windows usingWSL.get_assemblies depends onNCBI Entrez DirectwhichNO LONGERrequires Perl. Perl is
installed by default on most *nix systems. If edirect is not currently
installed, please runget_assemblies--dledirectto install.$python3-mpipinstall-Uget-assembliesDependenciesPython modules:richSeerequirements.txtfor more info.External programs:NCBI Entrez DirectYou can install external programs using theget_assemblies--dledirectcommand. These will be installed to${HOME}/edirectunless otherwise
specified.Just tell me how to run it$get_assembliesorganism'Pseudomonas fluorescens'This will find all genomes tagged as ‘Pseudomonas fluorescens’ in NCBI’s
database. By default, this command will only check to see how many genomes
fall into this category.To download metadata for the genomes:$get_assembliesorganism'Pseudomonas fluorescens'--functionmetadataCheck out themetadata.tabfile that is created after running this command.
Generally you will want to select a subset from your search. One way to do this
is to select the lines that include the genomes of interest, and then saving
the assembly accessions to a file. You can either delete the lines that you
don’t want, or usegrepto pull out the lines that you want to keep. Then
you can usecut-f15 > accs.txtto get the assembly accesions in a file.$cataccs.txt|get_assembliesassembly_ids---functiongenomes-ofnaThis will download the nucleotide fasta files for your genomes of interest.OverviewThis tool was written to make accessing genomic data from NCBI easier. The
output files are renamed such that each assembly has a Genus species strain in
the filename to make it easy to find the genomes that you’re interested in. You
won’t have to spend time renaming the files by hand.This software is effectively a wrapper for the NCBI edirect tools that makes
getting genome files easier. If you are interested in starting a comparative
genomics project, this is the tool for you.The software supports four types of input:organism input, either taxonomy rank names (e.g. Genus species, Family) or
taxidsassembly ids, either accessions or uidsnuccore ids (e.g. individual contig/chromosome names)json input (e.g. the intermediate files - docsums - produced by this script)Five file type outputs are supported:Nucleotide genome sequence (fna)Nucleotide coding sequence (ffn)Amino acid coding sequence (faa)General feature format (i.e. tab-delimited features) (gff)GenBank format (gbk)The program will attempt to find a unique prefix per genome assembly. This
prefix will be in the resulting filename. A metadata file that contains much
of the relevant information per genome will also be included. This file can
be included as a supplementary table for a manuscript in a comparative genomics
project.If you need to make phylogenetic trees with these data, check out my other
python package,automlsa2.More Examples$get_assembliesorganism'Mycobacterium'2020-10-1522:49:53,257-INFO-Found7522genomestodownload.2020-10-1522:49:53,257-INFO-Expect37610MBto52654MBofdata.$get_assembliesorganism--typeID167539--functiongenomes-ogbk2020-10-1523:10:13,822-INFO-Found1genomestodownload.2020-10-1523:10:13,822-INFO-Expect5MBto7MBofdatapendingthechosenfiletypesfordownload.chunk:1it[00:01,1.21s/it]docsums:100%|██████████████████████████████|1/1[00:00<00:00,5146.39it/s]2020-10-1523:10:16,262-INFO-Downloading1files.100%[##################################################]           1M / 1M]2020-10-1523:10:18,044-INFO-P_marinus_CCMP1375_SS120.gbksuccessfullydownloaded.download:100%|███████████████████████████████|1/1[00:01<00:00,1.78s/it]$lsdocsums0.jsonmetadata.tabget_assemblies.logP_marinus_CCMP1375_SS120.gbk$echoGCA_000269645.2|get_assembliesassembly_ids-2020-10-1523:18:04,107-INFO-Found1genomestodownload.2020-10-1523:18:04,107-INFO-Expect5MBto7MBofdatapendingthechosenfiletypesfordownload.Usage$get_assemblies-husage:get_assemblies[-h][--debug][--version][--dledirect[DLEDIRECT]]{organism,assembly_ids,nuccore_ids,json_input}...Downloadsassemblies&annotationsfromNCBI.positionalarguments:{organism,assembly_ids,nuccore_ids,json_input}Choosefromthislistofinputtypes.organismValidNCBIorganismortaxids.assembly_idsValidNCBIassemblyIDs.nuccore_idsValidNCBInucleotideaccessions.json_inputValidNCBIJSONdocsums.optional arguments:-h,--helpshow this help message and exit--debugTurn on debugging messages.--versionshow program’s version number and exit--dledirect<[DLEDIRECT]>Download edirect to given location. [~/edirect]Organism$get_assembliesorganism-husage:get_assembliesorganism[-h][--type{text,ID}][--function{check,metadata,genomes}[{check,metadata,genomes}...]][--annotation][--metadata_append][--typestrain][--keepmulti][--force][-f{abbr,full,strain}][-o{fna,ffn,gff,gbk,faa,all}[{fna,ffn,gff,gbk,faa,all}...]][--edirectEDIRECT][--debug]querypositionalarguments:queryValidNCBIorganismtexttermorIDoptional arguments:-h,--helpshow this help message and exit--type<{text,ID}>Input is text term (default) or ID--function<{check,metadata,genomes} [{check,metadata,genomes} …]>check counts, download metadata, or genomes. [check]--annotationRequire annotation? False by default, True if gbk/faa/ffn requested--metadata_appendAppend to metadata, not overwrite.--typestrainOnly download type strains.--keepmultiBy default, genomes from large multi-isolatestudies are removed.--forceForce download attempt of low-quality genomes.-f<{abbr,full,strain},--outformat{abbr,full,strain}>Output file prefix. [full]-o<{fna,ffn,gff,gbk,faa,all} [{fna,ffn,gff,gbk,faa,all} …]>Output file types.--edirectEDIRECTPath to edirect directory.--debugTurn on debugging messages.Assembly IDs$get_assembliesassembly_ids-husage:get_assembliesassembly_ids[-h][--type{acc,uid}][--function{check,metadata,genomes}[{check,metadata,genomes}...]][--annotation][--metadata_append][--typestrain][--keepmulti][--force][-f{abbr,full,strain}][-o{fna,ffn,gff,gbk,faa,all}[{fna,ffn,gff,gbk,faa,all}...]][--edirectEDIRECT][--debug]infilepositionalarguments:infileInputfilewithNCBIassemblyIDs;"-"forstdinoptional arguments:-h,--helpshow this help message and exit--type<{acc,uid}>Input is Accession (default) or ID--function<{check,metadata,genomes} [{check,metadata,genomes} …]>check counts, download metadata, or genomes. [check]--annotationRequire annotation? False by default, True if gbk/faa/ffn requested--metadata_appendAppend to metadata, not overwrite.--typestrainOnly download type strains.--keepmultiBy default, genomes from large multi-isolatestudies are removed.--forceForce download attempt of low-quality genomes.-f<{abbr,full,strain},--outformat{abbr,full,strain}>Output file prefix. [full]-o<{fna,ffn,gff,gbk,faa,all} [{fna,ffn,gff,gbk,faa,all} …]>Output file types.--edirectEDIRECTPath to edirect directory.--debugTurn on debugging messages.Nucleotide IDs$get_assembliesnuccore_ids-husage:get_assembliesnuccore_ids[-h][--function{check,metadata,genomes}[{check,metadata,genomes}...]][--annotation][--metadata_append][--typestrain][--keepmulti][--force][-f{abbr,full,strain}][-o{fna,ffn,gff,gbk,faa,all}[{fna,ffn,gff,gbk,faa,all}...]][--edirectEDIRECT][--debug]infilepositionalarguments:infileInputfilewithNCBInuccoreIDs;"-"forstdinoptional arguments:-h,--helpshow this help message and exit--function<{check,metadata,genomes} [{check,metadata,genomes} …]>check counts, download metadata, or genomes. [check]--annotationRequire annotation? False by default, True if gbk/faa/ffn requested--metadata_appendAppend to metadata, not overwrite.--typestrainOnly download type strains.--keepmultiBy default, genomes from large multi-isolatestudies are removed.--forceForce download attempt of low-quality genomes.-f<{abbr,full,strain},--outformat{abbr,full,strain}>Output file prefix. [full]-o<{fna,ffn,gff,gbk,faa,all} [{fna,ffn,gff,gbk,faa,all} …]>Output file types.--edirectEDIRECTPath to edirect directory.--debugTurn on debugging messages.NCBI JSON Docsum input$get_assembliesjson_input-husage:get_assembliesjson_input[-h][--function{metadata,genomes}[{metadata,genomes}...]][--annotation][--metadata_append][--typestrain][--keepmulti][--force][-f{abbr,full,strain}][-o{fna,ffn,gff,gbk,faa,all}[{fna,ffn,gff,gbk,faa,all}...]][--edirectEDIRECT][--debug]jsonfile[jsonfile...]positionalarguments:jsonfileInputJSONfilewithdocsums;"-"forstdinoptional arguments:-h,--helpshow this help message and exit--function<{metadata,genomes} [{metadata,genomes} …]>Download metadata and/or genomes. [metadata]--annotationRequire annotation? False by default, True if gbk/faa/ffn requested--metadata_appendAppend to metadata, not overwrite.--typestrainOnly download type strains.--keepmultiBy default, genomes from large multi-isolatestudies are removed.--forceForce download attempt of low-quality genomes.-f<{abbr,full,strain},--outformat{abbr,full,strain}>Output file prefix. [full]-o<{fna,ffn,gff,gbk,faa,all} [{fna,ffn,gff,gbk,faa,all} …]>Output file types.--edirectEDIRECTPath to edirect directory.--debugTurn on debugging messages.BugsViruses are currently not handled well, if at all. Look elsewhere to download
those.ContributingFeel free to submit bug reports or pull requests so we can improve this
software. Undoubtedly there will be some erroneous prefixes generated out
there, and I’d like to fix them.Author ContactEd DavisAcknowledgmentsSpecial thanks for helping me test the software and get the python code packaged:Alex WeisbergShawn O’NeilAlso, thanks to these groups for supporting me through my scientific career:OSU Chang LabCenter for Genome Research and Biocomputing @ OSULicenseget_assemblies is distributed under the terms listed in theLICENSEfile.
The software is free for non-commercial use.CopyrightsCopyright (c) 2020 Oregon State UniversityAll Rights Reserved.