Package: maginator
MAGinatorCombining the strengths of contig and gene based methods to provide:Accurate abundances of species using de novo signature genesMAGinator uses a statistical model to find the best genes for calculating accurate abundancesSNV-level resolution phylogenetic trees based on signature genesMAGinator creates a phylogenetic tree for each species so you can associate your metadata with subspecies/strain level differencesConnect accessory genome to the species annotation by getting a taxonomic scope for gene clustersMAGinator clusters all ORFs into gene clusters and for each gene cluster you will know which taxonomic level it is specific toImprove your functional annotation by grouping your genes in synteny clusters based on genomic adjacencyMAGinator clusters gene clusters into synteny clusters - Syntenic genes are usually part of the same pathway or have similar functionsInstallationAll you need for running MAGinator is snakemake and mamba. Other dependencies will be installed by snakemake automatically.condacreate-nmaginator-cbioconda-cconda-forgesnakemakemamba
condaactivatemaginator
pipinstallmaginatorFurthermore, MAGinator also needs the GTDB-tk database downloaded. Here we download release 214. If you don't already have it, you can run the following:wgethttps://data.ace.uq.edu.au/public/gtdb/data/releases/release214/214.1/auxillary_files/gtdbtk_r214_data.tar.gz
tarxvzf*.tar.gzUsageMAGinator needs 3 input files:The clusters.tsv files fromVAMBA fasta file with sequences of all contigs, with unique namesA comma-separated file giving the position of the fastq files with your sequencing reads formatted as: SampleName,PathToForwardReads,PathToReverseReadsRun MAGinator:maginator-vvamb_clusters.tsv-rreads.csv-ccontigs.fasta-omy_output-g"/path/to/GTDB-Tk/database/release214/"A testset can be found in the test_data directory.Download the 3 samples used for the test at SRA:https://www.ncbi.nlm.nih.gov/sra?LinkName=bioproject_sra_all&from_uid=715601with the ID's dfc99c_A, f9d84e_A and 221641_AChange the paths to the read-files in reads.csvUnzip the contigs.fasta.gzRun MAGinatorRun on a compute clusterMAGinator can run on compute clusters using qsub (torque), sbatch (slurm), or drmaa structures. The --cluster argument toggles the type of compute cluster infrastructure. The --cluster_info argument toggles the information given to the submission command, and it has to contain the following keywords {cores}, {memory}, {runtime}, which are used to forward resource information to the cluster.A qsub MAGinator can for example be run with the following command (... indicates required arguments, see above):maginator...--clusterqsub--cluster_info"-l nodes=1:ppn={cores}:thinnode,mem={memory}gb,walltime={runtime}"Test dataA test set can be found in the maginator/test_data directory.Download the 3 samples used for the test at SRA:https://www.ncbi.nlm.nih.gov/sra?LinkName=bioproject_sra_all&from_uid=715601with the ID's dfc99c_A, f9d84e_A and 221641_AClone repo: git clonehttps://github.com/Russel88/MAGinator.gitChange the paths to the read-files in reads.csvUnzip the contigs.fasta.gzRun MAGinatorMAGinator can been run on the test data on a slurm server with the following command:maginator--vamb_clustersclusters.tsv--readsreads.csv--contigscontigs.fasta--gtdb_dbdata/release214/--outputtest_out--clusterslurm--cluster_info"-n {cores} --mem {mem_gb}gb -t {runtime}"--max_mem180The expected output can be found as a zipped file on Zenodo:https://doi.org/10.5281/zenodo.8279036. MAGinator has been run on the test data (using GTDB-tk db release207_v2) on a slurm server.On the compute cluster each job have had access to 180gb RAM, with the following time consumption:
real	72m27.379s
user	0m18.830s
sys	1m0.454sIf you run on a smaller server you can set the parameters --max_cores and --max_mem.Recommended workflowTo generate the input files to run MAGinator we have created a recommended workflow, with preprocessing, assembly and binning* of your metagenomics reads (the rules for binning have been copied from VAMB (https://github.com/RasmussenLab/vamb/blob/master/workflow/)).
It has been setup as a snakefile in recommended_workflow/reads_to_bins.Snakefile.The input to the workflow is the reads.csv file. The workflow can be run using snakemake:snakemake --use-conda -s reads_to_bins.Snakefile --resources mem_gb=180 --config reads=reads.csv --cores 10 --printshellcmdsPreparing data for MAGinator runsed 's/@/_/g' assembly/all_assemblies.fasta > all_assemblies.fasta
sed 's/@/_/g' vamb/clusters.tsv > clusters.tsvNow you are ready to run MAGinator.Functional AnnotationTo generate the functional annotation of the genes we recommend using EggNOG mapper (https://github.com/eggnogdb/eggnog-mapper).You can download it and try to run it on the test datamkdirtest_out/functional_annotation
emapper.py-itest/genes/all_genes_rep_seq.fasta--outputtest_out/functional_annotation-mdiamond--cpu38The eggNOG output can be merged with clusters.tsv and further processed to obtain functional annotations of the MAG, cluster or sample levels with the following command:(echo-e'#sample\tMAG_cluster\tMAG\tfunction';join-11-21<(awk'{print $2 "\t" $1}'clusters.tsv|sort)<(tail-n+6annotations.tsv|head-n-3|cut-f1,15|grep-v'\-$'|sed's/_[[:digit:]]\+\t/\t/'|sed's/,/\n/g'|perl-lane'{$q = $F[0] if $#F > 0; unshift(@F, $q) if $#F == 0}; print "$F[0]\t$F[1]"'|sed's/\tko:/\t/'|sort)|awk'{print $2 "\t" $2 "\t" $3}'|sed's/_/\t/'|sort-k1,1-k2,2n)>MAGfunctions.tsvIn this case the KEGG ortholog column 15 was picked from the eggNOG-mapper output. But by cutting e.g. column number 13, one would obtain GO terms instead. Refer to the header of the eggNOG-mapper output for other available functional annotations e.g. KEGG pathways, Pfam, CAZy, COGs, etc.MAGinator workflowThis is what MAGinator does with your input (if you want to see all parameters run maginator --help):Filter bins by sizeUse --binsize to control the cutoffRun GTDB-tk to taxonomically annotate bins and call open reading frames (ORFs)Group your VAMB clusters into metagenomic species (MGS) based on the taxonomic annotation. (Unannotated VAMB clusters are kept in the pipeline, but left unchanged)Use --no_mgs to disable thisUse --annotation_prevalence to change how prevalent an annotation has to be in a VAMB cluster to call taxonomic consensusCluster your ORFs into gene clusters to get a non-redundant gene catalogueUse --clustering_min_seq_id to toggle the clustering identityUse --clustering_coverage to toggle the clustering coverageUse --clustering_type to toggle whether to cluster on amino acid or nucleotide levelMap reads to the non-redundant gene catalogue and create a matrix with gene counts for each samplePick non-redundant genes that are only found in one MAG cluster eachFit signature gene model and use the resulting signature genes to get the abundance of each MAG clusterUse --min_mapped_signature_genes to change minimum number of signature genes to be detected in the sample to be included in the analysisUse --min_samples to alter the number of samples with the MAG cluster present in order to perform signature gene refinementPrepare for generation of phylogenies for each MAG cluster by finding outgroups and marker genes which will be used for rooting the phylogeniesUse the read mappings to collect SNV information for each signature gene and marker gene for each sampleAlign signature and marker genes, concatenate alignments and infer phylogenetic trees for each MAG clusterUse --phylo to toggle whether use fasttree (fast, approximate) or iqtree (slow, precise) to infer phylogeniesInfer the taxonomic scope of each gene cluster. That is, at what taxonomic level are genes from a given gene cluster found inUse --tax_scope_threshold to toggle the threshold for how to find the taxonomic scope consensusCluster gene clusters into synteny clusters based on how often they are found adjacent on contigsOutputabundance/abundance_phyloseq.RData - Phyloseq object for R, with abundance and taxonomic dataclusters//.fa - Fasta files with nucleotide sequence of binsgenes/all_genes.faa - Amino acid sequences of all ORFsall_genes.fna - Nucletotide sequences of all ORFsall_genes_nonredundant.fasta - Nucleotide sequences of gene cluster representativesall_genes_cluster.tsv - Gene clustersmatrix/gene_count_matrix.tsv - Read count for each gene cluster for each samplesmall_gene_count_matrix.tsv - Read count matrix only containing the genes, that does not cluster across MAG clustersynteny/ - Intermediate files for synteny clustering of gene clustersgtdbtk// - GTDB-tk taxonomic annotation for each VAMB clusterlogs/ - Log filesmapped_reads/bams/ - Bam files for mapping reads to gene clustersphylo/alignments/ - Alignments for each signature genecluster_alignments/ - Concatenated alignments for each MAG clusterpileup/ - SNV information for each MAG cluster and each sampletrees/ - Phylogenetic trees for each MAG clusterstats.tab - Mapping information such as non-N fraction, number of signature genes and marker genes, read depth, and number of bases not reaching allele frequency cutoffstats_genes.tab - Same as above but the information is split per genesignature_genes/- R data files with signature gene optimizationread-count_detected-genes.pdf - Figure for each MAG cluster displaying number of identified SG's in each sample along with the number of reads mapped.tabs/gene_cluster_bins.tab - Table listing which bins each gene cluster was found ingene_cluster_tax_scope.tab - Table listing the taxonomic scope of each gene clustermetagenomicspecies.tab - Table listing which, if any, clusters where merged in MAG cluster and the taxonomy of thosesignature_genes_cluster.tsv - Table with the signature genes for each MAG clustersynteny_clusters.tab - Table listing the synteny cluster association for the gene clusters. Gene clusters from the same synteny cluster are genomically adjacent.tax_matrix.tsv - Table with taxonomy information for MAG cluster