Package: catastrophy
CATAStrophyCATAStrophy is a classification method for describing the lifestyles/trophic characteristics
of filamentous plant pathogens using carbohydrate-active enzymes (CAZymes).
The name CATAStrophy is a backronym portmanteau hybrid where "CATAS" means
CAZyme Assisted Training And Sorting.CATAStrophy takes HMMER3 files from searches against thedbCANCAZyme database
as input, and returns pseudo-probabilities called the relative centroid distance (RCD) of trophic class memberships for each file.To train these models, we performed principal component analysis (PCA) on the frequencies of CAZymes for a set of curated proteomes with literature support for their trophic lifestyles.
For each class in our classification system, we find the centre/geometric mean of the class in the first 16 principal components.New proteomes are classified by transforming the CAZyme frequencies using the same PCA loadings as in the training set.
We then find the closest class center in PCA space, and set that RCD score to 1.
Then for each of the other classes we find the distance between the new proteome and the class center divided (i.e. relative to) by the distance to the closest class center.
If a new proteome is equidistant between two class centroids and they are closer than any other class, then both RCD scores will be one, so the RCD method is akindof multi-label classifier.
This is useful when evaluating your organism, because it might have characteristics of multiple classes (or be so dissimilar to any class that the distance is meaningless).NEW: Users with a google account can now run the whole pipeline easily using ourGoogle colab notebook.
This offers a relatively user friendly click-interface to the pipeline (i.e. no coding experience or software installation required).Citation and further informationThe CATAStrophy method and trophic classification systems is describedhere:James K. Hane, Jonathan Paxman, Darcy A. B. Jones, Richard P. Oliver and Pierre de Wit (2020). "CATAStrophy", a Genome-Informed Trophic Classification of Filamentous Plant Pathogens – How Many Different Types of Filamentous Plant Pathogens Are There?Frontiers in Microbiology. doi:10.3389/fmicb.2019.03088InstallingCATAStrophy is a python program which can be used as a python module or via a command-line interface.
It can be installed from PyPIhttps://pypi.org/project/catastrophy/usingpip, or from anacondahttps://anaconda.org/darcyabjones/catastrophyusingconda.Users that are less familiar with python and pip might like to read ourINSTALL.mddocument which explains things in more detail, including where things will be installed and how to use virtual environments.
For details on installing and using conda see theirgetting-started guide.To install CATAStrophy and dependencies with pip:# Windows users may need to use "Python" instead of "python3"python3-mpipinstall--usercatastrophyTo install CATAStrophy and dependencies using conda:condainstall-cdarcyabjonescatastrophyUsing CATAStrophyCATAStrophy uses results ofHMMER3hmmscansearches against the dbCAN database.
The package includes apipeline which will download dbCAN and run HMMER3 for you,
or you can run HMMER yourself either locally or via the dbCAN website and provide the files to CATAStrophy manually.Running CATAStrophyTo run CATAStrophy you need to supply the input files and where to put the output.
Either the plain text (stdout) or "domain table" (--domtblout) outputs can be used.The easiest way to get a HMMER3 plain text file is to annotate your proteome using
the dbCAN online tool athttp://bcb.unl.edu/dbCAN2/blast.php(make sure the HMMER tool is selected to run), and
save the HMMER3 raw text (Select the HMMER tab, then "Download Raw HMMER output". Note that as of 2022-07-25 this is actually the domain-table output from HMMER3) results locally.
Alternatively you canrun HMMER locally, or use thepipeline scriptwhich will do this for you.WARNING: Before you run any dbCAN searches, pleaseread the section below on database versions.Assuming that you have this file locally you can run CATAStrophy like so:catastrophy-fhmmer_domtab-omy_catastrophy_results.csvmy_dbcan_results.txtThe input files are provided as positional arguments after all of the optional parameters.
The output will be a tab-delimited file (which you can open in excel) containing RCD results for each nomenclature and trophic class for each of the input files.
The-f/--formatflag specified the format of the input file and defaults tohmmer_domtab.Multiple input files can be provided using spaces to separate them:catastrophy-omy_catastrophy_results.csvdbcan_1.txtdbcan_2.txtNote that standard bash "globbing" patterns expand into a space delimited array,
so you can easily use*or subshells if you like (eg.$(find . -type f -name *.txt)etc).Database versionsCATAStrophy models are specific to the different versions ofdbCAN.
CAZyme family frequencies are at the core of the CATAStrophy method, so adding, removing, or changing the database HMMs will necessarily affect the results.
CATAStrophy will attempt to check for mismatched model versions and alert you, but could potentially give inaccurate results if a mismatch isn't detected.It is very important that you match the database version with the CATAStrophy model.To specify the version of the model to use, include the-m/--modelflag with one of the valid options (v4,v5,v6,v7,v8,v9, orv10; seecatastrophy -hfor the available model versions in your installation).catastrophy--modelv7-omy_catastrophy_results.csvmy_dbcan_results.txtThe model versions just reflect the version of dbCAN that the model was trained against (i.e. dbCAN 7 would use CATAStrophy model v7).NOTE:The dbCAN2 web-server will always search against the latest version of dbCAN.
To find the latest version number, go tohttp://bcb.unl.edu/dbCAN2/download/Databases/and find the file with the highest number with the patterndbCAN-HMMdb-V8.txt.
If we haven't yet trained a model for the latest version of dbCAN please contact us.
Otherwise you may need torun HMMER yourself.The CATAStrophy paper used version 6 of dbCAN, you may get slightly different results with different database versions.OutputOutput will be as a tab-separated values file, with the columns for the filename, nomenclature, nomenclature class, and the RCD value.
Each possible combination of label, nomenclature and class is listed in along format.For example, part of the table might look like this:catastrophyinfile1.txtinfile2.txtlabelnomenclatureclassvalueinfile1.txtnomenclature1saprotroph0.9infile2.txtnomenclature1monomertroph10.2You can also output the principal components of your proteomes along with the training data samples to reproduce the scatterplots presented in the CATAStrophy paper.
This table is not written by default, but can be specified using the--pca my_pcs.tsv.The output PCA TSV file will have the following columns looks like this:column namecolumn typedescriptionlabelstringthe label of your sample or the name of the proteome from the training data.nomenclature1stringThe literature annotation of proteomes in the training data. According to nomenclature1 from the paper. New samples will have missing values '.'nomenclature2string"nomenclature3string"nomenclature1_predstringThe main predicted nomenclature based on the RCD scores.nomenclature2_predstring"nomenclature3_predstring"nomenclature1_ancillaryComma ceparated stringsOther predicted nomenclature1 classes with an RCD score > 0.8nomenclature2_ancillaryComma ceparated strings"nomenclature3_ancillaryComma ceparated strings"PC1floatThe value of the principal component for this proteome...float"PC16float"LabelsBy default the filenames (including directories and extensions) are used as the label in the output, but you can explicitly specify
a label using the-l/--labelflag. The output from the command above will
have two lines, one containing the column headers and the other containing
results for the filemy_dbcan_results.txtwhich will have the label
"my_dbcan_results.txt".
If you don't specify a label for stdin input the label will be "<stdin>".To give it a nicer label you can run this.catastrophy-lprettier_label-omy_catastrophy_results.csvmy_dbcan_results.txtWhich would give the output line formy_dbcan_results.txtthe label "prettier_label".
Labels cannot contain spaces unless you explicitly escape them (quotes will not work).To label multiple input files you can again supply the label flag with the space separated labels.catastrophy-llabel1label2-omy_catastrophy_results.csvdbcan_1.txtdbcan_2.txtNote that if you do use the label flag, the number of labelsmustbe the same as the number of input files.If you provide the--labelas the last argument before the input positional arguments (infiles) you may need to use--to tell when you're done and that infiles should start.
This is because both--labeland infiles can take multiple arguments.catastrophy-lmylabel1mylabel2-oresults.csvinfile1.txtinfile2.txt# Finecatastrophy-oresults.csv-lmylabel1mylabel2infile1.txtinfile2.txt# Dangerous# Do this instead to tell where labels stops and infiles starts.catastrophy-oresults.csv-lmylabel1mylabel2--infile1.txtinfile2.txtUsing the CATAStrophy pipelineThe CATAStrophy pipeline script captures all of the steps described in later sections.
This is the easiest way to run the pipeline locally, and can take many Fasta files.The pipeline:Verifies that input Fasta files won't cause errors when running HMMER.Downloads the specified version of dbCAN.Prepares the HMMER formatted database.Runs hmmscan in parallel (single node only, sorry) on all fasta files.Runs the CATAStrophy pipeline on the results, captuting all outputs and making labels pretty.catastrophy-pipeline --model v10 --outdir results --ncpu 4 proteome1.fasta proteome2.fastaThe pipeline expectshmmscanandhmmpressfromHMMERto be installed in your$PATH.
You can also specify the full paths to the executables with the arguments--hmmscan_path /path/hmmscan --hmmpress_path /path/hmmpress.
The easiest way to install these programs is using conda:conda install -c bioconda hmmer, or on some Ubuntu or Debian linux distros you can install with apt:sudo apt install hmmer.The program will raise an error early if there are weird characters in your fasta files.
Basic errors such as non-standard amino acids (OBJZ etc) or gaps can be "fixed" with the--correctflag.
Essentially, gaps, spaces, and terminating stops are removed, and non-standard characters and internal stops are replaced with 'X'.If you wish to provide the dbCAN database yourself, you can provide it to the--hmmsparameter. Please make sure that it matches the version specified by--model.The output directory (default:results) will contain all of the outputs and intermediate results.
It has the basic structure.results/
  downloads/
    dbcan-database.hmm
  fastas/
    corrected.fasta  # Only if the --correct flag used
  search/
    proteome1_domtab.tsv
    proteome1_hmmer.txt
  counts.tsv  # The CAZyme counts for each proteome.
  pca.tsv  # The 16 principle components for the proteomes.
  classifications.tsv  # The trophy classifications and RCD values.Running HMMER locallyIf you have lots of proteomes to run or CATAStrophy hasn't been trained on the latest version of dbCAN yet, then you probably don't want to use the web interface.
In that case you can run the dbCAN pipeline locally usingHMMER.The following steps assume that you've installedHMMERand are using a unix-like OS.Step 1. Download dbCAN.We first need to download the dbCAN database (HMMs) to search against.
You will need to make sure that you download a version of dbCAN that it compatible with CATAStrophy.
To get a list of databases versions that is supported, you can use the--helpflag and look for the--nomenclaturehelp section.catastrophy--helpOnce you've identified the version you want to use, download the database fromhttp://bcb.unl.edu/dbCAN2/download/Databases/.
Alternatively you can use the bash commands below to download it, setting the value ofDBCAN_VERSIONto the desired version (NB. the full urlmustmatch one of the file names inhttp://bcb.unl.edu/dbCAN2/download/Databases/).DBCAN_VERSION="V10"mkdir-p./data
wget-qc-P./data"http://bcb.unl.edu/dbCAN2/download/Databases/dbCAN-HMMdb-${DBCAN_VERSION}.txt"Step 2. Prepare the dbCAN HMM databaseNow we can convert the file containing HMM definitions into a HMMER database.hmmpress./data/dbCAN-HMMdb-V10.txtThis will create several files at the same location as the.txtfile, so it's good to do this
inside a separate folder (as we've done here).Step 3. Search your proteomes against the dbCAN HMMs.Now we can run HMMERhmmscanto find matches to the dbCAN HMMs.
For demonstration, we'll save both the domain table and plain text outputs.hmmscan--domtbloutmy_fasta_hmmer.csv./data/dbCAN-HMMdb-V10.txtmy_fasta.fasta>my_fasta_hmmer.txtThe domain table is now in the filemy_fasta_hmmer.csvand the plain hmmer
text output is inmy_fasta_hmmer.txt.
Either one of these files is appropriate for use with CATAStrophy (just
remember to specify the--formatflag).Step 4. Classify your proteomes using CATAStrophy.Now we can finally find out what CATAStrophy thinks our organism is!To use the files created in step 3, you can run either of the following commands.
Remember to match the version of dbCAN with the model version in catastrophy.catastrophy--modelv10--formathmmer_domtab-omy_catastrophy_results.csvmy_fasta_hmmer.csv# orcatastrophy--modelv10--formathmmer_text-omy_catastrophy_results.csvmy_fasta_hmmer.txtCommand line argumentsOnly the input files are required and are provided as positional arguments.
Other optional parameters are below:Parameterdefaultdescription-h/--helpflagShow help text and exit.--versionflagShow program version information and exit.-f/--format"hmmer_domtab"The format that the input is provided in. All input files must be in the same format. HMMER raw (hmmer_domtab) and domain table (hmmer_domtab, default) formatted files are accepted. Files processed by the dbCAN formatterhmmscan-parser.share also accepted using thedbcanoption.-l/--labelfilenamesLabel to give the prediction for the input file(s). Specify more than one label by separating them with a space. The number of labels should be the same as the number of input files.  By default, the filenames are used as labels.-o/--outfilestdoutFile path to write tab delimited output to.-c/--countsNot writtenWrite the CAZyme counts to this tab delimited file.-p/--pcaNot writtenWrite the PCA results and best scoring RCD classes to this tab separated file. This will include the training data results in the table for comparison. Useful for plotting your data.-m/--modellatestThe version of the model (matching the dbCAN database version) to use. The latest version is used by default. Seecatastrophy -hfor list of valid options.Basic usage:# To stdout aka "-o -"catastrophyinfile1.txtinfile2.txt>results.csv# To specify an output filenamecatastrophy-oresults.csvinfile1.txtinfile2.txt# To take input from stdin use a "-"catinfile1.txt|catastrophy->results.csvChangesV0.1.0:hmmer_domtabis now the default input format, as dbCAN now outputs that for the HMMER results. Added a warning about proteomes with zero counts.