TITLE: NucBarcoder - a bioinformatic pipeline to characterise the genetic basis of plant species differences
AUTHORS: Wu Huang, Alex Twyford, Peter Hollingsworth

[DESCRIPTION]
There is now a significant number of studies that have sequenced multiple loci from the nuclear genomes of plants and these are available in public databases or in private data repositories (if the study hasn’t yet been published). Collectively these datasets have great potential for understanding the nature of genomic differences between plant species. To harness this existing information I have developed a set of workflows, with the logic behind my approach being to
●  Develop key criteria for selecting suitable datasets focusing only on datasets which have sampled multiple individuals from multiple congeneric species
●  Search the literature and public data repositories for datasets that match these criteria
●  Obtain and organise the selected datasets
●  Use these datasets to estimate the proportion of species that resolve as monophyletic units as one measure of species discrimination success
●  Establish the frequency distribution of taxonomically informative nucleotide substitutions among taxa to better understand the genomic nature of differences between plant species
●  Subsample the data to better understand the effectiveness of smaller numbers of loci in recovering maximal species discrimination, as well as evaluating the attributes of the gene regions that are most effective at telling species apart.

This protocol will focus on the last three bullet points. For the protocol for the first three bullet points please refer to another protocol "Acquiring and integrating data from multiple resources for meta-analysis". 

[GUIDELINES]
All data used in this study are stored and run at the UK Crop Diversity platform (www.cropdiversity.ac.uk).
Python (version 3+) and bash shell scripts were used to process the data.
The R Statistical Programing Language (version 4.1.0) and the Rstudio integrated development environment (Racine, 2012) were used for most of the plotting along with Excel (version 16.58) charts.
All code scripts, workflow, and an example to run the pipeline are available at https://github.com/Hazelhuangup/Species_specific_alleles_analysis.
All data used in this study are stored and run at the UK Crop Diversity platform (www.cropdiversity.ac.uk).
Python (version 3+) and bash shell scripts were used to process the data.
The R Statistical Programing Language (version 4.1.0) and the Rstudio integrated development environment (Racine, 2012) were used for most of the plotting along with Excel (version 16.58) charts.
All code scripts, workflow, and an example to run the pipeline are available at https://github.com/Hazelhuangup/NucBarcoder/tree/main/src.

[STEPS]
SECTION: Data cleaning and filtering
2. Data cleaning and filtering were performed at both the individual and locus levels. 
Different clean-up and filtering tools were used for aligned sequence formats and SNP matrices. VCFtools (0.1.17) (Danecek et al., 2011) was used to deal with the vcf format and self-written scripts were used for parsing the aligned sequences files, including format conversion, removing individuals, and removing sites. The main steps are listed in Table 2.

 

SECTION: Monophyletic Ratio
3. The Monophyletic Ratio (MR) was calculated as a simple measure of species discrimination success from different data sets. This is defined as the number of species resolved as monophyletic on a given phylogeny divided by the total number of multiple-sampled species (individuals ≥2 samples per species) from a genus. Figure 3.6. demonstrates an example where 3 species have multiple sampled individuals, 2 of them are monophyletic on the phylogeny. In this case, the monophyletic ratio is 2/3 = 0.67.

 
SECTION: The extraction of taxonomically informative Loci (Species-specific SNPs)
5. To provide a quantification of the distribution of taxonomically informative polymorphism, we
extracted information on the frequency distribution of ancestry informative loci that were either
fixed (Species-specific) or with marked nucleotide frequency differences.
Specifically, we divided SNPs into two categories: a) species-specific-SNPs which are fixed in
all individuals from one species and distinct from all other ingroup species, and b) Frequency
different SNPs where SNPs show a significant allele frequency difference in one species in
comparison to other groups. 

SECTION: Genomic region down-sampling and the impact on species resolving power
6. Performance of each locus in telling species apart
To assess the range of variation in species discrimination power among loci within datasets, we compared the discrimination success of individual loci/genes. This step is important in identifying the attributes of loci that are most informative in species discrimination which could help inform the choice of markers for future use. To do this, we built phylogenetic trees based on each single locus and looked at how many species resolved as monophyletic (compared to the equivalent figure using the total dataset). Given the scale of this task, tree-building was restricted to building quick UPGMA trees using ape_dna_dist.R and ape_dist_tree.R, and then quantifying the species resolved as monophyletic using MonoPhy.
 

To assess the relationship between nucleotide diversity of individual loci and their performance in telling species apart, we calculated the average nucleotide diversity of each single locus among all individuals in a genus using nuc.div function in R package pegas (Paradis, 2010) (script calculate_pi.R). 

 

We then plotted the number of monophyletic species resolved by this locus, against the nucleotide diversity, and the density of SSSNPs at this locus, using ggplot2 scripted in NucDiv_DensSSSNPs_No_spps_mono_by_each_gene.R. Regression curves were then fitted for both the series of nucleotide diversity and the density of SSSNPs against the number of species that are resolved as monophyletic using the CORREL function in Excel.
 
SECTION: Data cleaning and filtering
2.1. Upon acquiring each dataset, we examined how taxon re-identifications were performed and any ambiguous samples were removed based on the information given in the metadata of collaborators and in publications. Any individuals with unresolved species names (including cf., aff., and other uncategorised naming methods) and hybrids were removed from the matrix. Individuals that were outside of the focal genus of each dataset were removed except for the purpose of acting as outgroups in phylogenetic analyses. Multiple varieties or subspecies in one species were treated as a single taxon at the species level. Individuals with a high proportion of missing data were also removed, the threshold ranges from 75 ~ 80% data presence according to the data quality after manual checking. Further details on the removal of individuals and why they were removed from each genus are provided in Table 2.
SECTION: Data cleaning and filtering
2.2. After the filtering of individuals, we undertook locus filtering. For target capture or transcriptome data where genomic segment information is available, loci that were missing from 80% of all individuals were deleted. This also applies to RAD-seq/GBS data where stack or assembly information on each locus was provided. For datasets without genomic segment information, such as vcf and concatenated consensus fasta files, a nucleotide site was removed when over 40% of the individuals are missing. When the depth and quality information was accessible, usually in vcf format, nucleotide sites were retained only when the depth is within a reasonable range (lower depth limit of 2 to an upper limit of 2 to 3 times the average sequencing depth) according to the sequencing depth reported by the collaborators.
2.3. The programming scripts used in Step 1 could be found at https://github.com/Hazelhuangup/NucBarcoder/tree/main/src.
An example of using VCFtools:
 

 

 

SECTION: Genomic region down-sampling and the impact on species resolving power
7. Species discrimination success with down-sampled sequence data
To further explore how much data is needed to tell species apart, we subsampled each dataset to reduce the amount of sequence information and assess the consequent impact on the species discrimination success.
Specifically, we developed a pipeline (DS_snp.sh, available on *GitHub page, example in the code box) to downsample the data at laddered intervals. To model a reduced amount of sequence information, we tested down-sampling in terms of a) the number of SNPs, and b) the number of DNA segments (genes, exon, or assembled RAD-tags) where this information is available. The steps in both regards are the same:
1)  Select a specified number of SNPs/DNA segments randomly using python script.
2)  Build a fast UPGMA tree using R package ape and phangorn.
3)  Identify monophyletic clades using MonoPhy and count the number of monophyletic clades with python.
4)  Repeat step 1-3 for 50 times (bootstrap = 50).
5)  Change the number of SNPs/DNA segments specified and repeat steps 1-4.
6)  Visualize the result using ggplot2 (Wickham, 2011).
 


 
SECTION: Overview of the data management and the structure of NucBarcoder
1.   

SECTION: Monophyletic Ratio
3.1. If a phylogeny is in visualised format only, the identification could only be done by manual
calculation. 
SECTION: Monophyletic Ratio
4. Monophyletic clades were first counted from published phylogenies based on the data and format available. This involved either manual scoring of species resolving as monophyletic when only graphic representations of trees were available, and automated extraction of the MR for machine readable data (see section 3.4.3). Where phylogenies were not available for a given study, then we built a basic phylogeny using the aligned sequence data. Where sequence alignments were not available, we discarded these datasets due to the time constraints of producing high quality alignments from multiple different sources.
  
SECTION: Monophyletic Ratio
3.2. With phylogenies provided in newick or other text-based formats, this process could be
automated using MonoPhy (Schwery et al., 2016), which is a quick and user-friendly method
for assessing the monophyly of taxa in a given phylogeny. MonoPhy builds on the existing
packages ape 5.0, phytools (Revell, 2012), phangorn, RColorBrewer (https://CRAN.R-
project.org/package=RColorBrewer) and taxize (Chamberlain et al., 2013), and the installation
of these packages is also required. 
 

The output of MonoPhy indicates taxa are either monophyletic, non-monophyletic or
monotypic. By counting the number of species that are scored “Yes” for ‘monophyly’, divided
by the number of species with ‘tips’ > 1 (= species with multiple-samples), gives the
Monophyletic Ratio. For the example shown in Table 3.5, the monophyletic ratio is 6/7 = 0.86.
A tutorial on running MonoPhy and how to interpret the results is given at https://cran.r-
project.org/web/packages/MonoPhy/vignettes/MonoPhyVignette.html. 
Running the main analysis command ‘AssessMonophyly’ on 393 individuals using standard
settings is very rapid, and used only 0.19 seconds on a MacBook Pro with 2.2 GHz Intel 6-
Core i7 and 16 GB RAM. MonoPhy cannot be run in parallel by default.
SECTION: Monophyletic Ratio
4.1. We built phylogenies with the aligned sequences as input using IQTREE2 (Minh et al., 2020). IQTREE (Nguyen et al., 2015) is a widely used software package for phylogenetic inference using maximum likelihood. The second version incorporates Polymorphism-aware phylogenetic models (PoMo) to parse the IUPAC Ambiguity Codes (Schrempf et al., 2016), which is useful when the dataset maintains heterozygosity information. We also built quick UPGMA trees using R package ape v.5.0 (Paradis et al., 2019) and phangorn (Schliep, 2011).
The test data for evaluating the tree-building pipeline consisted of 393 individuals with 1,313,489 base pairs per individual. To build a tree using this dataset, IQTREE2 requires a minimum of 7 Gb RAM and 131 hours CPU time. With 64 CPU for parallel computing, it took a total of 2 hours and 34 minutes to finish the task. Building quick trees with ape requires less computing power because it doesn’t have a model testing and bootstrap stage. Running the whole dataset to build a UPGMA tree took 34 minutes CPU time.
 
 

SECTION: The extraction of taxonomically informative Loci (Species-specific SNPs)
5.1. To extract both types of SNPs from the aligned sequenced files, a Python script extract_ancestral_informative_SNPs*.py was developed with the following steps:
1)  For each locus, there should be at least 6 taxa whose locus coverages are greater than one (less than ~80% missing data)
2)  Calculate allele frequency for the target taxon. If the allele frequency (AF) of the major allele (M_A) is higher than 87.5%, progress to stage 3).
3)  Calculate allele frequency for the M_A across all other taxa (aggregated). If less than the 10% threshold then proceed to 4).
4)  Calculate allele frequency across other species with multi-sampled populations. If the number of individuals in this group is greater than 4 (included), and A_M is lower than 12.5% (at most only 1/8 allele belongs to the minor one), then retain the site. Otherwise, only if the minor alleles are homogeneous in a taxon would this site be retained.

The pipeline also includes the following extra calculations steps:
5)  Count the number of SSSNPs for each species that has multiple sampled individuals (e.g. the number of SNPs that are fixed and different compared to all other sampled species).
6)  Calculate the density of SSSNPs by dividing the number of SSSNPs by the average total nucleotides that do not contain missing data in that species.

Scripts for extracting ancestry informative SNPs are available in two versions, one is for the situation where the input is in vcf format (extract_ancestral_informative_SNPs_vcf.py), and one is for fasta, phylip, and nexus format (extract_ancestral_informative_SNPs_hete.py). Both of these two versions are heterozygous aware, and can accommodate heterozygosity in the designation of SSSNP calling.
 
SECTION: The extraction of taxonomically informative Loci (Species-specific SNPs)
5.3. In very large data sets consisting of many hundreds of thousands of nucleotides, one might expect to encounter shared SNPs between any group of samples purely due to random mutations. To distinguish the biological signal of SSNPs from random sampling artefacts, we developed an approach to test whether the number of SSSNPs is greater than expected due to chance alone based on a random distribution of the data. This was implemented by randomising the label of the sequences using the shuf function in bash command lines and repeating the same analysis counting the distribution of SSSNPs. We then compared the number of SSSNPs extracted from the original datasets versus the randomised label datasets using Wilcox signed-rank test (Knapp, 2017) with wilcox.test in R (parameters: paired = FALSE, alternative = "greater").
 

SECTION: The extraction of taxonomically informative Loci (Species-specific SNPs)
5.4. Computing requirements for extracting species-specific SNPs and allele-frequency-different SNPs depended largely on the dataset size, i.e. the number of sites to parse, and to some extent on the number of multiple-sampled species. Table 3.7. illustrates the run times for data sets involved in this study. 

SECTION: Genomic region down-sampling and the impact on species resolving power
7.2. For datasets in which the performance of individual genes was analysed (see section 6), we conducted a further analysis focusing on the best-performing genes. Here I started with the genes which showed the maximum amount of species discrimination, and sequentially added one gene at a time, selecting at each stage the next best-performing locus. At each stage, I recorded the number of genes used as well as the number of species that resolved as monophyletic. This analysis complements the preceding random selection of loci, by instead assessing the minimal number of best-performing loci required to achieve the maximum amount of species discrimination from a given data set.
SECTION: Genomic region down-sampling and the impact on species resolving power
7.1. An example of down-sampling.

Output file:   

Visualisation: with the Number_of_spps_mono_for_boxplot.txt as input, the data is visualised using an interactive R script   with R studio.

  

SECTION: The extraction of taxonomically informative Loci (Species-specific SNPs)
5.2. An example:
 
The essential output will be in the file *.AIL.list  

Calculate the number of SSSNPs and frequency different loci.
 
And record these basic statistics into a table.
  

To visualise and compare the distribution of SSSNPs across all species in available datasets, we produced a script plotting the density distribution of SSSNPs from all multiple sampled species annotated with whether a given species resolved as monophyletic or not, using ggplot2 embedded in the script Number_of_SSSNPs_vs_Monophyly.R.