DOI: 10.5061/dryad.k7t50
title: Data from: A protocol for species delineation of public DNA databases, applied to the Insecta

description: Public DNA databases are composed of data from many different taxa,
 although the taxonomic annotation on sequences is not always complete,
 which impedes the utilization of mined data for species-level
 applications. There is much ongoing work on species identification and
 delineation based on the molecular data itself, although applying species
 clustering to whole databases requires consolidation of results from
 numerous undefined gene regions, and introduces significant obstacles in
 data organization and computational load. In the current paper, we
 demonstrate an approach for species delineation of a sequence database.
 All DNA sequences for the insects were obtained and processed. After
 filtration of duplicated data, delineation of the database into species or
 molecular operational taxonomic units (MOTUs) followed a three-step
 process in which i) the genetic loci L are partitioned, ii) the species S
 are delineated within each locus, then iii) species units are matched
 across loci to form the matrix LxS, a set of global (multi-locus) species
 units. Partitioning the database into a set of homologous gene fragments
 was achieved by Markov clustering using edge weights calculated from the
 amount of overlap between pairs of sequences, then delineation of species
 units and assignment of species names was performed for the set of genes
 necessary to capture most of the species diversity. The complexity of
 computing pairwise similarities for species clustering was substantial at
 the COI locus in particular, but made feasible through the development of
 software that performs pairwise alignments within the taxonomic framework,
 while accounting for the different ranks at which sequences are labeled
 with taxonomic information. Over 24 different homologs, the unidentified
 sequences numbered ~194,000, containing 41,525 species ID's (98.7
 percent of all found in the insect database), and were grouped into 59,173
 single-locus MOTUs by hierarchical clustering under parameters optimized
 independently for each locus. Species units from different loci were
 matched using a multi-partite matching algorithm to form multi-locus
 species units with minimal incongruence between loci. After matching, the
 insect database as represented by these 24 loci was found to be composed
 of 78,091 species units in total. 38,574 of these units contained only
 species labeled data, 34,891 contained only unlabeled data, leaving 4,626
 units composed both of labeled and unlabeled sequences. In addition to
 giving estimates of species diversity of sequence repositories, the
 protocol developed here will facilitate species level applications of
 modern day sequence datasets. In particular, the LxS matrix represents a
 post-taxonomic framework that can be used for species level organization
 of meta-genomic data, and incorporation of these methods into phylogenetic
 pipelines will yield matrices more representative of species diversity.
description: delineation
 matrixdelineation matrix
 (LxS), 24 loci (columns), ~78,000 species units (rows). tab delimited,
 unix line breaks. spaces separate individual members of a single gene
 motu. members are representated by their originally designated taxonomy
 which is converted to species strings according to Hunt and Vogler (2008,
 MPE 47:289), followed by underscore and NCBI accession.delineation_matrixinsecta.taxkeytable giving full taxonomic information for each taxonomic string used later in the results files. taxonomic string is the first string in each row. strings are built from full heirachy up to given node, according to Hunt and Vogler (2008, MPE 47:289). 2nd column is full taxonomic name for that node, 3rd column is NCBI taxonomy number for that node.insecta.fasprocessed insect fasta database. file is compressed. entries are represented by taxonomic string followed by NCBI accession number. some duplicate data is removed from original insect db. and some particularly long DNA sequences are also ommited.Supplemental figure 1Supplementary Fig. 1: Assessment of the protocol. A: Contrasting estimates of species numbers for assigned names (x axis) and clusters based on sequence similarity (y axis). ~20 of the species dense homologs are plotted, based on species counts made only for the 26 model genera given in the methods. Upper gray point indicates COI locus, lower gray 18S. B: Contrasting the number of MOTU for each family in the dataset, where the species clustering parameters are inferred insect-wide (x-axis), or locally for each family (y-axis). Linear regression line is shown on A and B.supplement_fig1.tiff