dataesr/scientific-paragraphs-categorization huggingface dataset:
We identify clusters of samples in the reference database NNv1 and we will refer to those as species-groups. It is important to emphasise that the way we use the term 'species-groups' does not refer to a taxonomic classification, but to the clusters of samples defined by thresholds on the 8-mer distances between samples. Ideally, a threshold would partition the samples in the reference database NNv1 into species-groups, such that the 8-mer distance between each pair of samples within the group is smaller than the threshold and the 8-mer distance from any sample within the group to any sample outside the group is larger than the threshold. The motivating assumption is that the samples in the same species-group share more recent ancestry with each other than with samples that are members of different species-groups.
For very low thresholds, each sample will form its own species-group, which is not informative. For a threshold of 0. 1 on the sample 8-mer distance, most species-groups satisfy or nearly satisfy the partitioning condition, and each species-group contains a single species or multiple species from a known species complex or group, e. g. An. gambiae and An. coluzzii form a fine-level species-group together. We refer to the species-groups at the 0. 1 level as the fine level species-groups and they are most useful for species assignment, because they provide the highest resolution. A threshold of 0. 3 merges samples representing species from some well-known complexes, like the entire An. gambiae complex, into a single species-group. Similarly, many of the species for which we currently only have a single representative in the reference database NNv1, are merged into larger species-groups.
We refer to this level of species-groups as the intermediate level, which provides insight into the degree of similarity between different fine level species-groups. Additionally, the intermediate level species-groups can be informative when we sequence a sample whose species is not represented in the database because the species assignment at the intermediate level places the sample within its most closely related species-group in the database. Thresholds higher than 0. 3 tend to violate the partitioning condition to a greater extent, but it is desirable to include a coarse level classification to get an approximate taxonomic position of unrepresented species and of more diverged species. We selected 0. 51 as a threshold for the coarse level classification because this gives reasonably clear species-groups that roughly correspond to taxonomic series.
Nearest neighbour assignment The first step of the hierarchical assignment method performs nearest neighbour assignments to samples in the reference database at the three different levels of species-groups introduced above. The assignment is initially done independently at each target, for each sample, computing assignment proportions for the species-groups at the chosen level, normalised such that they sum up to one over all species-groups. The resulting per-target assignment proportions are then averaged over all targets, resulting in the overall sample assignment proportions at the chosen level. If the sample assignment proportion is at least 0. 8 for one species-group, the sample is classified as a member of that group. If the classification threshold of 0. 8 is not met at the chosen level, the sample remains unassigned at that level.
For example, to assign a sample s at the coarse level, we translate its target sequence corresponding to target 1 to an 8-mer count table, denoted as q s,1. Next, we compute its 8-mer distance, , to every target sequence in the reference database corresponding to 𝑑 8 𝑞 𝑠,1 , 𝑞 𝑟,1 ( ) target 1, i. e. to every. The nearest neighbours of the test sequence q s,1 are those sequences  ,1 in the database that minimise the 8-mer distance between themselves and q s,1. In other words, the nearest neighbour sequences of q s,1 are the target 1 sequences in the database with the highest percentage of matching 8-mers. The nearest neighbour sequence of q s,1 can be a sequence that occurs in a single sample in the reference database, or the nearest neighbour sequences can be the same sequence occurring in multiple samples in the database, or the nearest neighbour sequences can be distinct sequences that have the same distance to q s,1.
Targets that did not amplify in the sample are simply ignored. There are different reasons why a certain target does not get amplified in a sample. It might be that the primer binding sites for the target are too diverged or altogether absent in the test sample's genome. Or the target might not be amplified due to technical reasons like poor DNA quality. In the former case, we are implicitly using the information contained in the missingness, because we restrict our attention to the targets that did amplify in the sample, which should be the same targets that amplified in the samples of the same species contained in the reference database. In the latter case, as long as the missingness is randomly affecting the targets, ignoring missing targets does not bias the assignment proportions. So far we have assumed that the test sample was homozygous at each target.
To generalise to the heterozygous case, we compute the per-target assignment proportions separately for both target sequences and average them for the final per-target assignment proportions. It does occasionally happen that a sample has more than two different target sequences. This can be due to errors in the PCR amplification or sequencing, contamination by other samples, or a certain target region might be duplicated in the genome of some species. In the NNv1 reference database on average 1. 2% of targets per sample have more than two different sequences; in the query datasets discussed later in this article this percentage ranged from 0. 3% to 2. 3%. We deemed this small enough to simply extend the per-target assignment proportion computation to include targets with more than two different sequences, by taking the average assignment proportions over different sequences as for the heterozygous case.
This results in a high-confidence assignment to the Pyretophorus series at the coarse level as well as a high-confidence assignment to the An. gambiae complex at the intermediate level. At the fine level, the largest assignment proportion is to the An_gambiae_coluzzii species-group, but it does not meet the 0. 8 classification threshold because of the relatively high assignment proportions to other species-groups within the An. gambiae complex. So at the fine level, this sample cannot be classified with sufficient confidence to a single species. Later, we will present a method to resolve the species identities of samples within the An. gambiae complex. The heatmap shows the nearest neighbours of sample Amou-2-3, an An. moucheti specimen, at its different targets. For clarity, not all samples in the reference database have been displayed, only those in the Myzomyia and Neocellia series (except Amou-2-3).
The samples from the reference database are arranged along the x-axis and the targets along the y-axis. An entry is coloured pink if the corresponding sample from the reference database has a nearest neighbour sequence at the corresponding target. Peach entries indicate that the corresponding sample from the reference database does not carry a nearest neighbour sequence at the corresponding target. If either the test sample or the reference sample did not amplify at the corresponding target, the entry is white. The bars at the bottom show the overall assignment proportions for the displayed species-groups, only assignment proportions of at least 1% are shown. From top to bottom the assignment proportions are for the fine, intermediate and coarse level. For the three-letter code abbreviations of species-groups, see Supplementary Table  1.
The numeric abbreviations stand for 1: An. marshallii complex sp1, 2: Myzomyia sp1, 3: An. gabonensis, 4: An. culicifacies, 5: An. maculatus B; none of these species-groups represent more than 1% of the assignment. Right panel: The heatmap showing the nearest neighbours of sample Agam-35, an An. gambiae specimen. Not all samples in the reference database are displayed, only those in the Pyretophorus series (except Agam-35) as well as five samples from the Neocellia series. The numeric abbreviations stand for 6: An. gambiae complex sp1 (0. 06 assignment proportion), 7: An. christyi and MNs stands for Myzomyia_Neocellia_series. The species-group assignment has been tested on the reference database itself, by dropping out one sample at a time. The majority of samples could be assigned to the correct species-group at the fine level when using a threshold of 0.
8 assignment proportion, see Figure  3 , Supplementary Tables  2 3 4 , Supplementary Section 3. To provide context, we have included a phylogenetic tree constructed from pairwise 8-mer distances using FastME  (Lefort, Desper, and Gascuel 2015)  and displayed using TreeViewer  (Bianchini, n. d. ). If we ignore the samples that form a species-group on their own, because we do not yet have sufficient representation for those species, 61. 8% of samples are assigned correctly at the fine level, and 98. 8% and 100% at the intermediate and coarse level respectively. In most cases, the fine level species-groups consist of a single species, although in some cases they comprise multiple species. The jump in assignment success from the fine to the intermediate level is mostly caused by the An. gambiae complex, which is well-represented in the reference database.
Most samples within the complex can be assigned to the correct fine level species-group to some extent, but they only meet the assignment threshold at the intermediate level species-group assignment, where all samples in the An. gambiae complex are grouped together. This effect is seen in a few other groups as well, and motivates the VAE part of our assignment procedure (described below). For the same group of samples, the average correct assignment proportion per sample at the coarse level is 99. 4%. This shows that for samples of species that are well-represented in the reference database, there is a near perfect assignment at the coarse level. And in fact, the average correct assignment proportion of 95. 8% at the intermediate level shows that these assignments are also generally with high confidence.
At the fine level the confidence starts to break down for some samples, in particular the An. gambiae complex, which is responsible for many samples in the reference database. Then the average correct assignment per sample is 81. 3%. This shows that an additional classification method for species complexes is desirable. In NNv1, 21 species-groups at the fine level consist of a single sample. These cannot be assigned to the correct species-group in the drop-out assignment experiment. 12 of these samples are members of a larger species-group at the intermediate level and 11 of them can be assigned at the intermediate level, the other sample is not assigned at the intermediate, nor at the coarse level. The remaining nine samples only become a member of a larger species group at the coarse level.
Six of them can be classified at this level, the other three remain unclassified. The four unclassified samples are An. christyi, An. atroparvus, An. oryzalimnetes and An. cruzii. All of these are quite diverged from everything else in the reference database and as such do not exhibit a strong matching to any of the coarse level species-groups. In particular, the Kerteszia and Nyssorhynchus subgenus are underrepresented, both in number of species and number of samples, but also the basal species in for instance the Pyretophorus series are not well represented and are too diverged from the other species in this series to exhibit strong similarity to the other samples from this series. In the case of An. cruzii in the Kerteszia subgenus it is actually impossible to assign it to its coarse level species group with the current threshold of 0.
8, because of its 26 amplified targets, only 15 are also amplified in the single other sample in the reference database from the Kerteszia subgenus, which results in a theoretical maximum assignment to the correct species-group of 0. 58. Note, this is the only species-group at the coarse level for which the theoretical maximum correct assignment for a sample in the reference database is smaller than the threshold. But it again underlines the necessity to extend the reference database, both in number of species and number of samples per species. When a sample is of a species not represented in the reference database, three things can happen. If its species is much more closely related to a single species in the database than to all the others, it will likely be assigned to the species it is related to.
Alternatively, if the database contains multiple closely related species, it will be assigned at a higher level to the group that contains all these closely related species. If the sample is highly diverged from all species represented in the database, its nearest neighbours will essentially be chosen at random, and the assignment threshold will not be met. This emphasises the importance of extending the reference database, both by increasing the number of species represented and by increasing the number of samples per species, with a particular focus on capturing the within-species diversity (e. g. representing the geographic species range, representing all karyotypes when polymorphic chromosomal inversions are present). The species-groups with a single representative can be used to explore the three possible scenarios for when the test sample belongs to a species not present in the reference database: assignment to a closely related species, assignment to a group of species at the intermediate or coarse level, or no assignment.
The first scenario is represented by An. bellator. The only other representative of the Kerteszia subgenus in our reference dataset is a sample representing An. cruzii. At the fine level species-groups, the An. bellator sample is assigned for 0. 875 to An. cruzii and hence meets the threshold to be classified at the fine level. So it can happen that a sample is classified to the wrong species, if this species is the only reasonably close species in the database. This scenario is more likely for diverged groups of species with little representation in the database, for example the Nyssorhynchus and Kerteszia subgenera. An example of the second scenario is the sample Amar-3-1, in the Myzomyia series. At the fine level it has an assignment proportion of 0. 474 to the An_marshallii_complex and 0.
378 to the An. theileri species-group. All other species-groups have an assignment proportion of less than 0. 06. So this sample is more related to these two species-groups than to anything else in the reference database, but it does not belong to either of them. At the intermediate level it has an assignment proportion of 0. 850 to the An_marshallii_group species-group, which is the An_marshallii_complex and An. theileri species-group combined. Now it meets the threshold and it will be classified as a member of the An_marshallii_group species-group. An. christyi is an example of a sufficiently diverged sample that does not reach high assignment proportions for a single species-group at any level. At the coarse level, it is assigned to the Pyretophorus_series with 0. 506 and the Myzomyia_Neocellia_series with 0.
Gambiae complex classifier datasets The nearest neighbour approach is not able to confidently distinguish between closely related species that share a lot of genetic variation, but it does identify samples of those species as members of a species complex at the intermediate assignment level. To resolve the species identity inside these species complexes, we use a variational autoencoder approach specifically trained for the complex under consideration. We demonstrate this method for the An. gambiae complex, both because many of our samples fall within this complex and it is medically relevant to be able to distinguish them, and also more practically, because we have access to a large dataset of species-labelled samples. The classifier we present here contains 7 out of 8 formally named species in the complex  (Coetzee et al.
2013) , as well as two putative cryptic species  (Tennessen et al. 2021; Barrón et al. 2019). Compared to NNv1, three additional species are represented in this classifier to present as much of the diversity of the An. gambiae complex as possible. We expect that this method can be applied to other species complexes for which large species-labelled datasets are available. The An. gambiae complex classifier has been constructed using a training set (GCref v1) and a validation set (GCval v1) of species-labelled samples. We included as many described species in the An. gambiae complex as we could find or generate sequence data for. For species with wide geographic ranges and a large amount of genomic data available, we also aimed to represent the diverse geography where possible. Both GCref v1 and GCval v1 consist of amplicon sequences and in silico extracted published samples  (Fontaine et al.
2015; Neafsey et al. 2015; Tennessen et al. 2021 ; The Anopheles gambiae 1000 Genomes Consortium 2021). The samples from Nigeria and Madagascar were not species-labelled, but they were unambiguously classified by an earlier version of this classifier and we included these samples because Nigeria and Madagascar fill geographic gaps in our sampling dataset. The species represented in GCref v1 are An. gambiae (  406 ),  An. coluzzii (222), An. arabiensis (94), An. quadriannulatus (11), An. melas (3), An. merus (6), An. bwambae (3), An. tengrela (38)  and putatively An. fontenillei (  4 ). These samples are generally high coverage: 97% of samples have at least 55 of 62 targets amplified. The average number of targets tends to be lower for the samples representing species other than An. gambiae, An.
coluzzii or An. arabiensis, which are also those species represented by fewer samples, but the geographic ranges of these other species are also much more restricted so the samples we do have are likely good representatives of the species. The species represented in GCval v1 are An. gambiae (80), An. coluzzii (  15 ), An. arabiensis (30), An. melas (1), An. merus (  5 ) and An. tengrela  (12). The average number of amplicons for the species other than An. gambiae, An. coluzzii or An. arabiensis is lower than in GCref v1 set. Given that for those species, there is only a small number of samples available, we decided to use ones with at least 45 targets amplified in GCref v1 and the ones with at least 30 targets in GCval v1. Sample information for these datasets can be found in Supplementary Tables  5  and 6.
Further detail on the choice of parameter values is provided in Supplementary Information Section 4. The latent space projection is also affected by the training dataset. We observed that the presence or absence of most countries does not affect the classification, except for the Gambia and Guinea-Bissau. When these countries are removed, the accuracy to distinguish between An. gambiae and An. coluzzii considerably reduces. This is not surprising, since these samples lie on the boundary of the An. gambiae and An. coluzzii clusters and as such are crucial for assigning those species. The complete results can be found in Supplementary Information Section 5.
Within-complex species classification We use the trained and tuned VAE to assign species as follows. We input the summed 8-mer table of the test samples into the encoder of the VAE. The encoder outputs a position in latent space for each sample. Importantly, the VAE is agnostic to species labels; the species assignment happens based on the position in latent space of the test samples in relation to the latent space positions of the species-labelled reference dataset GCVAEv1. The top two panels of Figure  4  show the latent space projection of GCVAEv1. While most species form nicely isolated clusters, An. gambiae and An. coluzzii border each other closely. Interestingly, the boundary is formed by mosquitoes from The Gambia and Guinea-Bissau. These mosquitoes are labelled as An. gambiae by conventional molecular barcoding, but they cannot be confidently assigned to either An.
A nice metaphor is to imagine that you are wrapping all points corresponding to samples of a single species together in such a way that requires the minimal amount of wrapping paper. We constructed one convex hull for each species represented in the dataset, using latent space positions of the samples from GCref v1 as well as of 363 additional species-labelled samples that were not used in training the VAE, to account for possible effects caused by projecting the samples to the latent space (sample to be found information in Supplementary Table  7 ). For our classification procedure it is important that the convex hulls of different species do not overlap. When constructing convex hulls from the full sample set, only the convex hulls corresponding to An. gambiae and An. coluzzii overlap. We trimmed these hulls by iteratively removing samples from the set of points used to construct them until they didn't overlap.
We fitted the parameter value 7 on the dataset from Gabon to reflect the assumption that An. tengrela is not believed to be found in Gabon. If the latent space position of the test sample falls outside all convex hulls and outside their fuzzy boundaries, the sample is assigned 'uncertain' followed by the labels of all the species whose convex hulls are within a radius of 7 times the distance to the closest convex hull, in order of proximity. This assignment reflects the uncertainty in our classification of samples that fall in an area in latent space where no species-labelled samples fall. At the same time it gives information on our best guess or guesses for the species identity and leaves open the possibility to modify the assignments based on prior knowledge, e. g. species ranges or habitat restrictions.
For this sample we extracted only 13 targets. The remaining unassigned samples and those only assigned at the coarse level have at least 42 targets. They can be split into four groups of samples that are similar to each other in assignment proportions and mitochondrially. Because of the high amplicon recovery rate and the similarity of assignment proportions, often found in different countries, we believe that these samples represent species that are not present in NNv1. We aim to confirm the species identity for these groups by morphology and genomic comparison to publicly available data and, assuming this is successful, to include them in the next update of the reference database.
5%) we obtain at least 50 amplified targets, the minimum number required for VAE assignment. We find three species in this dataset: An. gambiae, An. coluzzii and An. arabiensis (Figure  5 ). Most samples (91. 6%) could be assigned to a single species, but we also find some samples labelled as uncertain shared between An. gambiae and one other species. The two samples that fall in the space between the An. gambiae and An. arabiensis reference samples are remarkably far away from all other samples. We cannot exclude the possibility that contamination between samples plays a role here, especially because these samples were stored with 10 individuals in ethanol in a single 1. 5 mL tube in the years before sequencing. All metadata and assignment information can be found in Supplementary Table  9.