dataesr/scientific-paragraphs-categorization huggingface dataset:
, 2019). However, m etabarcoding suffers from a series of pitfalls such as the difficulty to estimate the absolute abundance of taxa due to PCR biases, the presence of false positives and negatives and variable taxonomic resolution among taxa and genetic markers. This calls for a careful study design, the use of controls, the careful choice of a nalytical tools and a critical interpretation of the results  (Alberdi et al. , 2019). One of the difficulties of metabarcoding lies in the taxonomic assignation of sequences and the completeness of the underlying reference databases. Methods of taxonomic assignm ent can be alignment -based relying of sequence similarit ies detected by BLAST  (Altschul et al. , 1997)  or VSEARCH  (Ro gnes, Flo uri, Nichols , Quince, & Mahé, 2016)  implemented in different software  (Bokulich et al.
, 2018; Huson, Auch, Qi, & Schuster, 2007)  or based on machine learning  (Murali, Bhargava, & Wright, 2 018; Pedregosa et al. , 2011; Wang, Garrity, Tiedje, & Cole, 2007). However, for all methods, the quality of the reference database is crucial (Hlea p, Littlefair, Steinke,  Hebert, & Cristescu, 2021). Many methods are sensitive to gaps in the taxono mic coverage of the reference databas e  (Hleap et al. , 2021)  , thus the creation of a reference database with the best coverage available is highly needed. Several different markers can be used for m etabarco ding, since each of them are subject to different taxonomic bias es and provide different taxono mic resolutio n  (Ruppert, Kline, & Rahman, 2019). The most widespread markers are the ribosomal RNA markers (18S, 28 S, 16S) , the Cytochro me Oxidase C subunit I (COI) gene and internal transcribed spacer sequences (ITS)  (Creer et al.
The COI was the most sequenced gene at the beginning of the barcoding era, since it is the main maker of the Barco de of Life database (P. D. N. Hebert, Ratnasingham, & deWaard, 2003)  , and more animal taxa have been barcoded with COI than with any other markers  (Andújar et al. , 2018). This provides a solid bas is for taxonomic assignment of metabarcod ing sequences using COI as a marker. Regularly updated, curated and marker specific databases are available for ITS (UNITE  (Nilsson et al. , 2019) , PLANTiTS  (Banchi et al. , 2020)  ) and for rRNA markers  (Greengenes (DeSantis et al. , 2006)  , SILVA  (Pruesse et al. , 2007)  ). Conversely , COI sequences are deposited to two different m ajor databases, which are not COIspecific: (i) the nucleotide database of NCBI (hereafter NCB I-nt database;  Sayers et al.
, 2022) ) and their European (ENA) and Japanese equivalents (D DBJ) are generalist database s without focusing on a taxon or a gene; (ii) the Barcoding of Life Data System (BOL D ;  (Ratnasingham & Hebert, 2007)  ) contai ns barcoding sequences of several markers , but most of the sequences are from the barcoding fragment of the COI gene. Although the data overlap be tween these databases is considerable, each of them has sequences that are not found in the other database. Therefore, creating a merged database with sequences from both sources is highly desirable. A major challenge of pooling sequences from different sources into a single database is to homogenize their taxonomic lineages. This step is not trivial due to the presence of homonyms (e. g. Plecoptera is both an insect order and a moth genus), sy nonyms and misspellings.
Therefore, the only clean solution to deal with taxon names is the use of unique taxonomic identifiers (taxID) which are connected to a non -ambiguo us, hierarchical system and allow the identification of the lineage for each taxon. Both the NCBI -nt and the BOLD databas es use taxIDs, but the two systems are independent from each other, thus they cannot be simply merged. Finding the equivalent taxon names and taxIDs between the two databases call for a careful compariso n of taxon names and their lineages in order to match them. Howev er, a further complication arises from occasional incoherencies of taxo nomic lineages from different databases (e. g. Vexillata genus is a nematode belonging to the Ornithostro ngylidae family according to BOLD, but to the Trichostrongy lidae family according to NCBI taxo nomy ), which further complicates pooling of taxonomic information to a single co herent system.
Merging of COI sequences from the NCBI-nt and BOLD has been attempted in different programmes. BOLD_NCBI_Merger  (Macher, Macher, & Leese, 2017 ) uses a very simple method based on identical taxo n names, without avoiding the pitfalls of homonyms. MetaCOXI  (Balech, Sandionigi, Marzano, Pesole, & Santamaria, 2022)  obtains NCBI taxIDs and taxonomic lineages based on ENA flat files, when available. However, when this information is not offered (the sequence is present only in BOLD), NCBI taxIDs are determined by simply matching taxon names to NCBI taxonomy, witho ut checking for homonymy. Furthermore, taxon names not present in NCBI taxonomy do not receive a taxID , and therefore a taxID system is incomplete. A further difficulty of c reating custom (local) databases is sequence download ing from the original sources.
NCBI provides different means of accessing data: a whole database can be downloaded via ftp sites, and filtered subsequently, or Application Programming Interfaces (API) ar e provided for targeted downloads  (Kans, 2021). On the o ther hand, BOLD system s do not provide an easy way to download the whole public dataset , and the use of BOLD APIs need s a considerable optimization to be able to access large datasets. Although bold R package (https://docs. ropensci. org/bold/ ) is available to download data from BOLD, it is subject to failure for large taxa and takes several hours or days, according to requested data size. The mkCOInr tool was designed to create the COInr database, which includes all COI sequences from NCBI-nt and BOLD sequences, irrespective of the region of the gene covered and the taxonomic group.
All sequences have a taxID , and all taxIDs form a coherent system compatible with , but not limited to , the NCBI taxIDs , allowing to unambiguously obtain taxo nomic lineages even for taxon names with homonyms. Sequence redundancy within taxa is eliminated to reduce database size, without losing information. This database is freely available and can be easily and quickly do wnloaded from https://doi. org/10. 5281/zeno do. 6555985 , thus 6 saving the most complicated and time -consuming steps of custom database creation. Users can customi ze the downloaded database using mkCOInr scripts and format them to be able to use it with their preferred taxonomic assignment tool. It is possible to add local sequences, select or eliminate sequences o f a list of taxa , filtering sequences fo r minimum taxonomic resolutio n, and choosing a gene region.
Material and Methods mkCOInr is a series of P erl scripts that can be executed in command line, thus being easily integrated into other pipelines. They were written for Linux OS and can run on MacOS or other Unix enviro nments. The Windows Subsystem Linux (https://docs. microsoft. com/en -us/windows/wsl/ ) allows Windows user s to run mkCOInr scripts. Special care was taken to reduce dependencies to easy-to-instal l, third-party programmes without the use of special package s. BLAST  (Altschul et al. , 1997 ), vsearch (Ro gnes et al. , 2016)  , cutadapt  (Martin, 2011)  , and NSDPY (R. Hebert & Meglécz, 2022)  can all be installed either th rough the P ython Package Index (PyPI) or standard program repositories.
AND genome[Title] AND Mitochondrion[Filter])" -T -v --cds This allowed the download of all coding DNA sequences (CDS) returned with the keyword search for COI, CO1, COX I or COX1 , and CDS from complete mitochondrial genomes. The scope o f this search was intentionally very wide, and the downloaded sequences were further filtered by the format_ncbi. pl script to (i) only retain CDS with g ene and protein names corresponding to COI , and (ii) eliminate genes with introns and sequences from environmental or metagenomic samples. Sequences with more than five consecutive internal Ns, and outside of the length range of 10 0 -2000 nucleotides were also eliminated. Open nomenclature was not accepted in taxon names. If the taxID did not correspond to a correct Latin name format, the smallest taxon with a correct Latin name in the lineage was chose n for the sequence ( e.
g. Acentrella sp. AMI 1, taxID: 888165, rank: species was replaced by Acentrella, taxID: 248176, rank: genus ). Sequences were then subjected to taxonomically aware dereplication by the dereplicate. pl script. Within each taxID , all sequences that were a substring of another sequence were eliminated. This allows to reduce the size of the database without losing informatio n and keeping intraspecific variability.
BOLD A list of taxa was established from the taxo nomy page of BOLD S y stems (https://www. boldsystems. org/index. ph p/TaxBrowser_Home ), where each taxon had fewer than 500 000 specimen records. All public sequences of the above list and associated information were downlo aded from BOLD, using the download_bold. pl script that use s the BOLD APIs. For each tax on, the integrity of the downloaded files and the number of records were checked, and the download was repeated automatically in c ase of failure. From the raw downloaded files, COI sequences (COI-5P, COI-3P) were selected if they did not contain more than five consecutive intern al Ns and were in the length range of 100 -2000 nucleotides. As for NCBI sequences , the smallest taxon in th e BOLD lineage with a correct Latin name was chosen for the sequence to avoid open nomenclature.
All unique lineages were then listed with the correspo nding sequence identifiers (seque nceID) and for each lineage a taxID was determined using the add_taxids. pl script: the smallest taxon is identified in each BOLD lineage, where the name is matching a taxo n nam e in the NCBI taxonom y database (including synonyms), and at least 60% of the taxon names in the BOLD lineage match the NCBI lineage. For example, for the BOLD lineage of 'Chordata, Actino pterygii, Trachiniformes, 8 Pinguipedidae, Par apercis, Parapercis somaliensis', the Parapercis genus matches the 215380 NCBI taxID , even if the orders are different in BOLD and NCBI (Trachiniformes and Uranoscopiformes, respectively). In the next step, a taxon under the smallest taxon with NCBI taxID was attributed to an arbitrary, negative taxID, and the new taxID was integrated to the taxID system, with the NCBI taxID as a parent.
The newly created taxID was then added to the taxID system and it was characterize d by a taxon name, a taxonomic rank and the taxID of its direct parent, forming a hierarchical system. This hierarchical taxID system allows the creation of the lineage of any taxID unam biguously, even in case of homonymy and synonymy. As for NCBI sequences, the filtered BOLD dataset was dereplicated by the dereplicate. pl script. To compare the effect of using only correct Latin names (as in COInr) or accepting all taxo n names presents in the input databases , the above pipeline was run a second time using systematically the smallest taxon in each lineage, even i f it did not correspond to a correct Latin name.
The COInr database The BOLD and NCBI datasets were pooled into one single dataset by the pool_and_dereplicate. pl script, where sequences for the tax IDs shared by the two source databases were dereplicated, while sequences from taxIDs unique t o one of the sources were simply added to the combined database. T his database is a starting point to create more specific custo m databases according to the users ' needs. The core database consists of two simple-to-parse tsv files (tab separated values ). The sequence file has three columns (sequenceIDs, taxIDs and sequences ), and contains sequences of all taxo nomic groups that can cover any COI region, with variable taxonomic resolution from species to phylum level. The t axonomy file contains taxIDs, scientific names, parent tax IDs, taxonomic rank and taxonomic level index.
Customizing the COInr database The COInr database can be modifie d according to users' need s. Sequences can be selected for a list of taxa or on the contrary, removed from the database through the select_taxa. pl script. The script will also produce a lineage and a taxID for each taxon in the taxo n list, allowing users to check for potential errors due to homonyms. In case of incoherence, the taxo n list enriched by the correct taxIDs can be used to rerun the script with more precise selection. The same script also allows selecting sequences with a minimum taxonomic resolution. The select_region. pl script trim s the sequences to a specific regio n of the COI gene. Using the usearch_glo bal command of vsearc h (Ro gnes et al. , 2016) , sequences of the database are aligned to a small, taxon omically diverse pool of the sequences , which have already been trimmed to target region (target_region_fas).
The sequences of the core database are trim med according to the alignment positions. The target_region_fas file can be provided by the users or can be produced by the same script by making an E -PCR o n the co re database using cutadapt  (Martin, 2011). The COInr database can also be completed by custom sequences. Users will need a taxon name and sequenceID for each custom sequence. The format_custom. pl script will pro duce a lineage file for each input taxa, which should be checked, and eventually corrected and completed by the users. The add_taxids. pl script will add taxIDs to each lineage and complete the input taxonomy file (part of the COInr database). Sequences should then be dereplicated by the dereplicate. pl script and added to the COInr database using the pool_and_derelplicate.
Format Database The very simple format of the database (sequence file and taxono my file both in tsv format) allows users to easily o btain a database in their desired format. The format_db. pl script can pro duce databases ready to use for BLAST , RDP_classifier , and QIIME. The ' full' o ption will produce a single tsv file with sequence IDs, ranked lineages, taxIDs, and the sequences allowing user to parse, and produce basic statistics on the database content (e. g. number of sequences of each taxon).
Results Table  1  summarizes number of taxa and sequences in the initial databases before and after taxonomically aware dereplication , and after pooling and dereplicating sequences from BOLD and NCBI -nt to the COInr database. After the initial quality control, NCBI and BOL D databases contained 3. 9 M and 7. 6 M COI sequences respectively, belo nging to approximately 20 0 000 taxa with correct Latin names in both databases. Taxonomically aware dereplication within each of the source databases resulte d in 1. 7 M and 2. 8 M nonredundant sequences, correspond ing to 58% and 63% reductio n in NCBI and BOLD databases , respectively. The total number of taxa was 2 68 438 after pooling NCBI and BOLD, 69% of which was shared between the input databases, 1 4% and 17% of unique to NCBI and BOLD, respectively.
After po oling the databases and dereplication , 90% of the sequences were from taxa present in both databases, while 4% and 6% specific to NCBI and BOLD, respectively. Overall, the 11. 5 M input sequences were reduced to 3. 3 M by eliminating redundanc y between the two input databases, and within each taxon. Apart from sequences of animals, which made 99% of the database and corresponded 97% of the species , other Eukaryotes (plants, Fungi) and even some Bacteria and Archaea sequences were also present in the database (Table  2 ). Within Metazoa, 83% of the sequences were from Arthro poda that corresponds to 74% of the animal species of the database. To evaluate the effect of using non-standar d taxon names, corresponding to open nomenclature (e. g. Allograpta aff. argentipila , Alona guttata group , Macr obiotus cf.
hufelandi) or correct Latin names completed by arbitrary identifiers (e. g. sp. ACL2 485 , Abablemma BioLep730, Abacar us sp. GD 111) , two databases were created : COInr, where only correct Latin names were used and the all-names database created by the same pipeline , with the exception that all taxo n names were accepted regardless of their format (e. g. Lepidoptera sp. 096  PS -2011  was used as it is instead of the taxID of Lepidoptera order). The total number of taxa in NCBI w as more than three times higher when using all nam es (769 956 vs. 221 565). This difference was smalle r, yet considerable for the number of BOLD taxa (322 927 vs. 231 425) for the all -names and Latin names databases (Table  3 ).
Discussion The need for high-quality database can be m easured by the number of publi shed databases and method s of their construction. Several tools exist such as the CRUX database Builder integrated to Ana capa  (Curd et al. , 2019 ) , Metataxa2 Database Builder (Bengtsson-Palme et al. , 2018)  , MetaCurator  (Richardson, Sponsler, McMinn-Sauder, & Johnson, 2020)  , BCdataba ser  (Keller et al. , 2020)  , which are not marker-specific. The MIDORI database  (Leray, Ho, Lin, & Machida, 2018; Machida, Leray, Ho, & Knowlton, 2017)  contains mitochondrial sequences of 13 proteincoding genes. All the above-mentioned databases and tools are based exclusively on NCBI databases or on a dataset already containing a coherent system of lineages. Several COI -specific databases hav e also been published and are often limited to a target taxon or geographical region.
The Eukaryote CO1 Reference Set For The RDP Classifier  (Porter & Hajibabaei, 2018)  is specifically designed for the RDP classifier and focuses on Arthropoda and Chordata. It contains NCBI and BOLD sequences of at least 500 bp , but the last update is from 2019 and the script s for re-creating the database are not available. The Meta-Fish -Lib  (Collins et al. , 2021)  12 is a generali zed, dynamic reference library fishes. MitoFish  (Sato, Miya, Fukunaga, Sado, & Iwasaki, 2018)  is limited to fish mitochondrial sequences. The MARES database  (Arranz, Pearman, Aguirre, & L iggins, 2020)  is specific to marine sequences from BOLD and NCBI. The pipeline is provided to create a new database specific to the users' needs. However, a potential source of problem s for installing and using the scripts is the high need of third -party programs and packages.
METACOXI database  (Balech et al. , 2022 )  is a COI database that satisfies many criteria. It includes all Metazoan COI sequences from BOLD and NCBI (ENA) and uses NCBI taxIDs wherever possible. However, for BOLD-specific sequences without NCBI/ENA accession number, taxIDs are established by simply matching the taxon names without checking for homonymy. Furthermore, taxon names not present in NCBI taxonomy do not receive a unique taxIDs, therefor e the database lacks a coherent taxIDs system allowing to avoid all taxonomic ambiguities.
Use of accepted Latin names Both BOLD and NCBI contain a high number of taxon names at a species level , with unique taxIDs, which do not correspond to the binomial nome nclature. In most cases they correspond to taxon names of a higher level completed by an identifier or simply co mpleting the taxon name by 'sp. '. In principle, they co uld be proxies of species, but according to my findings, it is unlikely for most cases. When accepting all names as they appear in the input database, a high proportion of the COI sequences are shared between taxa, and m ost importantly a high proportio n of taxa contain only sequences that are identical to sequences of other taxa. COI is known to be variable among most species (P. D. N. Hebert, Cywinska, Ball, & deWaard, 2003)  and often shows considerable intraspecific variability  (Ratnasingham & Hebert, 2013).
The high proportion of shared sequences between taxa suggests that many of the taxa do not correspond to disti nct species, but they are the results of an unjustified over -splitting. This phenomenon is particularly pronounced in NCBI, where many abusive examples are found. Fo r example , many genus names in NCBI are completed by the sampleID of BOLD and use d as species names (e. g. Platy nothrus sp. BIOUG14078 -H10): many of the m share identical sequences, and do not even correspond to BOLD BINs (Barcode Index Numbers) which would provide some ground for species delimitation. Since the M ETACOXI database accepts all taxon names as they appear in BOLD or NCBI, it artificially inflates the number of taxa, which are in most case s uninformative to user s, hindering efficient, taxonomically aware reduction of redundancy.
Selecting the target region The COI nr database includes sequences that can cover any region of the COI gene. For taxonomic assignment methods based on sequence similarity  (Clemente, Jansson, & Valiente, 2 011; Huson et al. , 2007; Kahlke & Ralph, 2019; Wood & Salzberg, 2014)  the database can be used as it is, since sequences of the nontarget region will not be returned by BLAST o r other similarity searches. The only disadvantage would be the database size, which could be eventually reduced by selecting only the region of the sequence that cover the target region. On the other hand, for taxonomic assignment based on sequence composition or phylogeny  (Murali et al. , 2018; Nguyen, Mirarab, Liu, Pop, & Warnow, 2014; Rosen, Reichenberger, & Rosenfeld, 2011; Wang et al. , 20 07) , it is preferable to trim sequences to the target region.
Selecting the target groups Using a large database with a wide taxonomic scope is co nvenient for users analysing different dataset s with a varied taxonomic origin, since the same database can be used and can give a good first approximatio n of taxonomic assignment of sequences. It can also be helpful to detect contaminant sequences that are not expected in the study (e. g. human sequences or mo del species studied in the same lab ) or sequences outside of the target group of the study (e. g. bacteria, algae, fungi when focusing on animals). By usi ng a generalist 14 database, these sequences can be identified and eliminated. On the other hand, the presence of reference sequences from taxa not relevant to the study can also have disadvantages : the database size is higher and therefore the speed of taxonomic assignment is lower with generalist databases.
Moreover, sequences can be assigned to unexpected taxa if the taxonomic coverage of the target gro up is incomplete. This can be avoided with databases specific to the target group  (Axtner et al. , 2019; Mathon et al. , 2021; Valentini et al. , 2016). For example, many sequences from marine samples can be erroneously assigned to insect s when using a generali zed database, which is the combined result of the fact s that most marine groups are insufficiently covered in the reference databases  (Mugnai et al. , 2021) , and an overwhelming majority of the sequences are from insects (73%). Therefore, the possibility to easily create custom databases specifically tailored to the users' needs is particularly impo rtant, and the mkCOInr provides the necessary tools to make this selection.
Selecting sequences with different taxonomic resolution Another co nsideration when creating custom databases is whether to keep reference sequences with incomplete lineages. Most sequences of a reference database assigned to an insect order without further precision is likely to be useless, since most insect reference sequences are determined at least to the genus level, and the ta xonomic coverage of this group is wide. On the contrary, for less well -covered groups, especially if species or higher -level groups are difficult to identify morphologically ( e. g. Nematoda, Rotifera), reference sequences with partial lineages are still inf ormative.