Package: pyani
pyaniWhole-genome classification using Average Nucleotide Identitypyaniis maintained by:Leighton PritchardBailey Harringtonand we are grateful to all who have contributed to this software:Balázs BrankovicsPeter CockRobert DaveyÖzcan EsenNick Waters@ytanzawTable of ContentspyaniTable of ContentsOverviewInstallationcondapipDocker imagesNOTEInstalling from the repository/source codeIMPORTANT NOTICEObtain source codeDirect downloadClone the repository usinggitInstallation From SourceThird-party toolsHow To RunpyaniScript:average_nucleotide_identity.pyScript:genbank_get_genomes_by_taxon.pyTestingpyaniMethod and Output DescriptionAverage Nucleotide Identity (ANI)Developer notesCode Style and Pre-Commit HooksLicensingOverviewpyaniis a program that calculates average nucleotide identity (ANI) and related measures for whole genome comparisons, and renders graphical summary output. Where available, it takes advantage of multicore systems, and can integrate withSGE/OGE-type job schedulers for the sequence comparisons.ThisREADME.mdapplies only to v0.2 ofpyani.The following scripts should be visible in your$PATHafter installation:average_nucleotide_identity.pythat enables command-line ANI analysis.genbank_get_genomes_by_taxon.pythat downloads publicly-available genomes from NCBI.delta_filter_wrapper.pyis a helper script required to run delta-filter on SGE/OGE systems.InstallationThe easiest way to installpyaniv0.2 is to usecondaorpip.condais recommended for the simplest installation of third-party tool dependencies (mummerandBLAST/BLAST+).condaYou will need to install thebiocondachannel, following instructions athttps://bioconda.github.io/user/install.html. Then, to create a new environment forpyaniand install the program, issue the following command:condacreate--namepyani_envpython=3.8-y
condaactivatepyani_env
condainstallpyanipippipwill installpyaniand its Python dependencies, but not the third-party tools.pip3installpyaniDocker imagespyaniv0.2 scripts are also provided asDockerimages that can be run locally as containers on any operating system which supports Docker. To use these images, first installDocker. Then, to run the corresponding scripts, issue either:dockerrun-v${PWD}:/host_dirleightonpritchard/average_nucleotide_identityordockerrun-v${PWD}:/host_dirleightonpritchard/genbank_get_genomes_by_taxonNOTEThe-v ${PWD}:/host_dirargument enables the Docker container to see the current directory. Without this argument, the container will not be able to see your input files, or write output data.Installing from the repository/source codeIf you wish to installpyaniv0.2 from source code, you will need to download this code from GitHub either directly, or by cloning the repository.Both methods will create a directory calledpyaniwhich contains the source code for v0.2.IMPORTANT NOTICEPlease note that v0.2 is aMAINTENANCE-ONLY RELEASEand is found on the branch calledversion_0_2. Bugs and security issues will be fixed as we become aware of them, but no new features will be added. All new feature development now takes place under v0.3 (themasterbranch on the repository).Obtain source codeYou can obtain the source forpyanieithe by downloading the latest release as a compressed archive, or cloning theversion_0_2branch.Direct downloadDownload the source as.zipor.tar.gzfrom:https://github.com/widdowquinn/pyani/releases/tag/v0.2.10Extract the archived file using the appropriate commands (or by double-clicking in your file explorer/finder):unzipv0.2.10.ziportar-zxvfv0.2.10.tar.gzClone the repository usinggitDownload the currentpyanirepository withgit clone, and change branch toversion_0_2:gitclonegit@github.com:widdowquinn/pyani.git
gitcheckoutversion_0_2Installation From SourceChange directory topyani, and use thepython setup.pysetuptools command, to install the package and scripts. This willnotinstall the third-party toolsBLAST/BLAST+andmummer.cdpyani
pythonsetup.pybuild
pythonsetup.pyinstallThird-party toolsThree alignment packages are required if you wish to use all ofpyani's methods:mummer,BLAST+, and legacyBLAST.NOTE: the legacy BLAST executables provided by NCBI will not run on macOS Big Sur; we do not provide executables for this tool.*The simplest route to obtaining these tools is to useconda/bioconda:condainstallmummerblastlegacy-blast-yBut they can also be installed by following instructions from the tools' own websites.BLAST+(foranib)ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/legacy BLAST(foraniblastall)ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST/MUMmer(foranim)http://mummer.sourceforge.net/How To RunpyaniScript:average_nucleotide_identity.pyTheaverage_nucleotide_identity.pyscript - installed as part of this package - enables straightforward ANI analysis at the command-line, and uses thepyanimodule behind the scenes.You can get a summary of available command-line options withaverage_nucleotide_identity.py -h$average_nucleotide_identity.py-h
usage:average_nucleotide_identity.py[-h][-oOUTDIRNAME][-iINDIRNAME][-v][-f][-sFRAGSIZE][-lLOGFILE][--skip_nucmer][--skip_blastn][--noclobber][--nocompress][-g][--gformatGFORMAT][--gmethodGMETHOD][--labelsLABELS][--classesCLASSES][-mMETHOD][--schedulerSCHEDULER][--workersWORKERS][--SGEgroupsizeSGEGROUPSIZE][--maxmatch][--nucmer_exeNUCMER_EXE][--blastn_exeBLASTN_EXE][--makeblastdb_exeMAKEBLASTDB_EXE][--blastall_exeBLASTALL_EXE][--formatdb_exeFORMATDB_EXE][--write_excel][--subsampleSUBSAMPLE][--seedSEED][--jobprefixJOBPREFIX][…]Example data and output can be found in the directorytest_ani_data. The data are chromosomes of four isolates ofCaulobacter. Basic analyses can be performed with the command lines:average_nucleotide_identity.py-itests/test_ani_data/-otests/test_ANIm_output-mANIm-g

average_nucleotide_identity.py-itests/test_ani_data/-otests/test_ANIb_output-mANIb-g

average_nucleotide_identity.py-itests/test_ani_data/-otests/test_ANIblastall_output-mANIblastall-g

average_nucleotide_identity.py-itests/test_ani_data/-otests/test_TETRA_output-mTETRA-gThe graphical output below, supporting assignment ofNC_002696andNC_011916to the same species (C.crescentus), and the other two isolates to distinct species (NC_014100:C.segnis;NC_010338:C.sp K31), was generated with the command-line:average_nucleotide_identity.py-v-itests/test_ani_data/\-otests/test_ANIm_output/-g--gformatpng,pdf,eps\--classestests/test_ani_data/classes.tab\--labelstests/test_ani_data/labels.tabScript:genbank_get_genomes_by_taxon.pyThe scriptgenbank_get_genomes_by_taxon.py, installed by this package, enables download of genomes from NCBI, specified by taxon ID. The script will download all available assemblies for taxa at or below the specified node in the NCBI taxonomy tree.Command-line options can be viewed using:$genbank_get_genomes_by_taxon.py-h
usage:genbacnk_get_genomes_by_taxon.py[-h][-oOUTDIRNAME][-tTAXON][-v][-f][--noclobber][-lLOGFILE][--formatFORMAT][--emailEMAIL][--retriesRETRIES][--batchsizeBATCHSIZE][…]For example, the NCBI taxonomy ID forCaulobacteris 75, so all publicly-availableCaulobactersequences can be obtained using the command-line:$genbank_get_genomes_by_taxon.py-oCaulobacter_downloads-v-t75-lCaulobacter_downloads.log--emailme@my.email.domain
INFO:genbank_get_genomes_by_taxon.py:MonApr1817:22:542016INFO:command-line:/Users/lpritc/Virtualenvs/pyani3/bin/genbank_get_genomes_by_taxon.py-oCaulobacter_downloads-v-t75-lCaulobacter_downloads.log--emailme@my.email.domain
INFO:Namespace(batchsize=10000,email='me@my.email.domain',force=False,format='gbk,fasta',logfile='Caulobacter_downloads.log',noclobber=False,outdirname='Caulobacter_downloads',retries=20,taxon='75',verbose=True)INFO:SetNCBIcontactemailtome@my.email.domain
INFO:CreatingdirectoryCaulobacter_downloads
INFO:Outputdirectory:Caulobacter_downloads
INFO:PassedtaxonIDs:75INFO:EntrezESearchwithquery:txid75[Organism:exp]INFO:EntrezESearchreturns29assemblyIDs
INFO:Identified29uniqueassemblies
INFO:Taxon75:29assemblies[…]INFO:Assembly639581:271contigs
INFO:Assembly233261:17contigs
INFO:Assembly575291:48contigs
INFO:MonApr1817:25:462016INFO:Done.NOTE:You must provide a valid email to identify yourself to NCBI for troubleshooting.The number of attempted retries for each download, and the size of a batch download can be modified. By default, the script will attempt 20 download retries, and obtain sequences in batches of 10000.TestingpyaniThepyanirepository includes tests that can be run withpytest(including coverage measurement usingpytest-cov) with the following command, executed from the repository root directory:pytest-vMethod and Output DescriptionAverage Nucleotide Identity (ANI)This module calculates Average Nucleotide Identity (ANI) according to one of a number of alternative methods described in, e.g.Richter M, Rossello-Mora R (2009) Shifting the genomic gold standard for the prokaryotic species definition. Proc Natl Acad Sci USA 106: 19126-19131. doi:10.1073/pnas.0906412106. (ANI1020, ANIm, ANIb)Goris J, Konstantinidis KT, Klappenbach JA, Coenye T, Vandamme P, et al. (2007) DNA-DNA hybridization values and their relationship to whole-genome sequence similarities. Int J Syst Evol Micr 57: 81-91. doi:10.1099/ijs.0.64483-0.ANI is proposed to be the appropriatein silicosubstitute for DNA-DNA
hybridisation (DDH), and so useful for delineating species boundaries. A
typical percentage threshold for species boundary in the literature is 95%
ANI (e.g. Richter et al. 2009).All ANI methods follow the basic algorithm:Align the genome of organism 1 against that of organism 2, and identify the matching regionsCalculate the percentage nucleotide identity of the matching regions, as an average for all matching regionsMethods differ on: (1) what alignment algorithm is used, and the choice of parameters (this affects the aligned region boundaries); (2) what the input is for alignment (typically either fragments of fixed size, or the most complete assembly available).ANIm: uses MUMmer (NUCmer) to align the input sequences.ANIb: uses BLASTN+ to align 1020nt fragments of the input sequencesANIblastall: uses legacy BLASTN to align 1020nt fragments of the input sequencesTETRA: calculates tetranucleotide frequencies of each input sequenceThe algorithms takes as input correctly-formatted FASTA multiple sequence files. All sequences for a single organism should be contained in only one sequence file. Although it is possible to provide new labels for each input genome, for rendering graphical output, the names of these files are used for identification so it is best to name
them sensibly.Output is written to a named directory. The output files differ depending on the chosen ANI method.ANIm: MUMmer/NUCmer .delta files, describing each pairwise sequence alignment. Output as tab-separated plain text format tables describing: alignment coverage; total alignment lengths; similarity errors; and percentage identity (ANIm).ANIbandANIblastall: FASTA sequences describing 1020nt fragments of each input sequence; BLAST nucleotide databases - one for each set of fragments; and BLASTN output files (tab-separated tabular format plain text) - one for each pairwise comparison of input sequences. Output as tab-separated plain text tables describing: alignment coverage; total alignment lengths; similarity errors; and percentage identity (ANIb or ANIblastall).TETRA: Tab-separated plain text files describing the Pearson correlations between Z-score distributions for each tetranucleotide in each input sequence (TETRA).If graphical output is chosen, the output directory will also contain PDF, PNG and EPS files representing the various output measures as a heatmap with row and column dendrograms. Other output formats (e.g. SVG) can be specified with the--gformatargument.Developer notesThepyanipackage is presented atGitHubunder two main branches:masteris the source code underpinning the most recent/current release ofpyani. It will (almost) always be in sync with the latest release found athttps://github.com/widdowquinn/pyani/releases. The only time this code should not be in sync with the release is when there are modifications to documentation, or immediately preceding a release.developmentis the current bleeding-edge version ofpyani. It should (almost) always be in a working and usable condition, but may not be complete and/or some features may be missing or still under development.Code Style and Pre-Commit HooksThe source code forpyaniis expected to conform toflake8linting, andblackcode styling. These are enforced as pre-commit hooks using thepre-commitpackage (included inrequirements.txt).Theblackandflake8hooks are defined in.pre-commit-config.yaml. Custom settings forflake8are held in.flake8.To enable pre-commit checks in the codebase on your local machine, execute the following command in the root directory of this repository:pre-commitinstallLicensingUnless otherwise indicated, all code is subject to the following agreement:(c) The James Hutton Institute 2014-2019
(c) The University of Strathclyde 2019-2021
Author: Leighton PritchardContact:leighton.pritchard@strath.ac.ukAddress:
Leighton Pritchard
Strathclyde Institute for Pharmacy and Biomedical Sciences
University of Strathclyde
161 Cathedral Street
Glasgow
Scotland
G4 0RE
UKThe MIT LicenseCopyright (c) 2014-2019 The James Hutton Institute
Copyright (c) 2019-2021 The University of StrathclydePermission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.