Package: marker-alignments
marker alignmentsThis a Python package to process and summarise alignments of metagenomic sequencing reads to a reference database of marker genes. You can use it in conjunction with an aligner likebowtie2to produce an estimate of taxa present in a metagenomic sample.Features:filtering reads on alignment propertiessummarizing by marker and taxonclustering based on multiple alignments per queryquantificationUsageInstallationInstall via pip:pip install marker_alignmentsGetting startedDownload a small example alignment file, and runmarker_alignmentswith most basic options:wget "https://raw.githubusercontent.com/wbazant/marker_alignments/main/tests/data/example.sam"

marker_alignments --input example.sam --output /dev/stdoutIf the package installed correctly, you should see a coverage report for each reference in the alignments file. Try it with your input file next, and take it from there:marker_alignments --helpwill show you all options about filtering and output types.Is this suitable for my data?This package is for analysing metagenomic reads from whole genome sequencing, aligned to a reference of your choice.An allowed input is an alignment file in.samor.bamformat, coming frombowtie2or similar enough: @SQ lines on top, reads and MD tags in the output. Seeexample .samhere.The package was developed in the context of looking for eukaryotes using marker genes - seeCORRALfor a packaged workflow. It can also be useful for analysing aligned bacterial reads, e.g. to get quantification outputs.Example - detecting eukaryotesFirst download the EukDetect reference database followingEukDetect installation instructions.Then follow this example to download an example metagenomic file, run alignments to a reference database bundled with EukDetect, and obtain a profile using suitable filtering options:REFDB_LOCATION="eukdb"

wget "ftp.sra.ebi.ac.uk/vol1/fastq/ERR274/009/ERR2749179/ERR2749179_1.fastq.gz"
wget "ftp.sra.ebi.ac.uk/vol1/fastq/ERR274/009/ERR2749179/ERR2749179_2.fastq.gz"
gunzip *gz
FASTQ_1="ERR2749179_1.fastq"
FASTQ_2="ERR2749179_2.fastq"

bowtie2 --omit-sec-seq --no-discordant --no-unal \
  -x $REFDB_LOCATION/ncbi_eukprot_met_arch_markers.fna \
  -k10,10
  -1 ERR2749179_1.fastq.gz \
  -2 ERR2749179_2.fastq.gz \
  -S ERR2749179.sam 

FILTERING_OPTS="--min-read-query-length 60 --min-taxon-num-markers 2 --min-taxon-num-reads 2 --min-taxon-better-marker-cluster-averages-ratio 1.01 --threshold-avg-match-identity-to-call-known-taxon 0.97  --threshold-num-taxa-to-call-unknown-taxon 1 --threshold-num-markers-to-call-unknown-taxon 4     --threshold-num-reads-to-call-unknown-taxon 8"

marker_alignments --input ERR2749179.sam --output ERR2749179.taxa.tsv \
  --refdb-format eukprot \
  --refdb-marker-to-taxon-path $REFDB_LOCATION/busco_taxid_link.txt \
  --output-type taxon_all \
  --num-reads $(grep -c '^@' $FASTQ_1) \
  $FILTERING_OPTSFiltering optionsBacteriaRecommended setting for bacteria is:--min-read-query-length 60 --min-taxon-num-reads 100- only use alignments of length at least 60, report organisms with at least a hundred aligned reads.EukaryotesRecommended presets for eukaryotes are:" --min-read-mapq 30 --min-read-query-length 60 --min-read-match-identity 0.9 --min-taxon-num-markers 2"if using single best alignment per query.If using multiple alignments, the following preset recommended if you're okay with relying on MCL clusters:--min-read-query-length 60 --min-taxon-num-markers 2 --min-taxon-num-reads 2 --min-taxon-better-marker-cluster-averages-ratio 1.01 --threshold-avg-match-identity-to-call-known-taxon 0.97  --threshold-num-taxa-to-call-unknown-taxon 1 --threshold-num-markers-to-call-unknown-taxon 4     --threshold-num-reads-to-call-unknown-taxon 8A simpler alternative is--min-read-query-length 60 --min-taxon-num-markers 2 --min-taxon-num-reads 2 --min-taxon-fraction-primary-matches 0.5but it does not deal with unknown taxa quite as well.All filtering options are as follows:columndescription--min-read-mapqwhen reading the input, skip alignments with MAPQ < min-read-mapq--min-read-query-lengthwhen reading the input, skip alignments shorter than min-read-query-length--min-read-match-identitywhen reading the input, skip alignments where the proportion of matching bases in the alignment is less than min-read-match-identity--min-taxon-num-markersOnly keep taxa with at least min-taxon-num-markers markers--min-taxon-num-readsOnly keep taxa with at least min-taxon-num-reads reads--min-taxon-num-alignmentsOnly keep taxa with at least min-taxon-num-alignments alignments--min-taxon-fraction-primary-matchesOnly keep taxa where no more than min-taxon-fraction-primary-matches fraction of alignments is inferior / secondary--min-taxon-better-marker-cluster-averages-ratioOnly keep taxa where the ratio between markers which have at least average match identity relative to their clusters and markers with identity below average is at least min-taxon-better-cluster-averages-ratio--threshold-avg-match-identity-to-call-known-taxonThreshold on average match identity to return taxon in reference--threshold-num-reads-to-call-unknown-taxonTo positively identify an unknown taxon (fits all criteria except match identity) expect this many reads from a taxon cluster--threshold-num-markers-to-call-unknown-taxonTo positively identify an unknown taxon (fits all criteria except match identity) expect this many markers from a taxon cluster--threshold-num-taxa-to-call-unknown-taxonTo positively identify an unknown taxon (fits all criteria except match identity) expect this many taxa from a taxon clusterReasons to apply filtersVery short alignments do not convey useful information
Our ancestors had to make do with 35-40bp shotgun reads, but we have longer ones - game changer for metagenomics! Still, a 100bp read can match on the last twenty bases at the end of a reference sequence (clipped alignments) or you could have configured the aligner to do local alignments instead of end-to-end. Either way,--min-read-query-lengthbeing something high enough (60 from EukDetect seems to work fine) addresses this problem.Low identity matches are not taxon specific
An unknown species will match as a mixture of results. The clustering option--min-taxon-better-marker-cluster-averages-ratiotries to take care of removing the overall inferior evidence, and the--threshold-avg-match-identity-to-call-known-taxononly passesThe suggested value of 0.97 has been chosen empirically. Is a bit lower than CCMetagen's 0.9841 quoted fromVu et al (2019), as this number was calculated from ribosomal subunits, we're not aware of a study that calculates average identity for BUSCOs. Most unknown taxa seem to match at around 0.9 identity, and a value 0.95 still permitted an unknownPenicillinumspecies to appear as a mixture.Threshold of evidence for making claims
Claiming a eukaryote is present based on one read would be preposterous! It's not clear how many reads are "enough" to make a claim, and actually, no number of reads is enough because off-target matches follow patterns. We suggest gaining evidence from at least two markers, and a higher standard for ambiguous hits coming from species not in the reference. You can also only report unknown species if the results indicate its two nearest taxa with--threshold-num-taxa-to-call-unknown-taxonoption.OutputsOutput typesThe default option is coverage To get most outputs, set--output_type taxon_allfor aggregated outputs, or--output_type marker_allfor the breakdown. Seemarker_alignments --helpfor other available options.QuantificationCoverage per marker= number of reads aligned to the marker times read length divided by length of a markerCoverage per taxon= average coverage between all markers with nonzero coverageCPM (copies per million)= taxon coverage divided by total number of reads and multiplied by one millionCPMs are the most useful number for comparing between different taxa and across samples, since they correct for different sequencing depth between samples, and different reference length between taxa.Please note that quantitative information might be unreliable when there is very few reads.Custom useReference databasesFor each reference database, we need to know what taxon each contig or marker sequence is for. This can be provided externally, through a lookup table with the--refdb-marker-to-taxon-pathoption, but we also try support all reference databases of interest to our users through the --refdb-format option. If--refdb-format genericproduces errors for you, please open a ticket with the details about the reference database and the raised error.Additional output optionsYou can save an intermediate database produced by providing the--sqlite-db-pathargument, and then query it with asqlite3client.How to citePlease cite our Microbiome publication,https://doi.org/10.1186/s40168-023-01505-1.