dataesr/scientific-paragraphs-categorization huggingface dataset:
e. , 70 samples) were used as a training set. Even though each model was run 10 times on 10 distinct training sets, it remains a relatively low amount of samples to do abundance predictions and extrapolations at the global ocean scale (as a way of comparison, 181 samples allowed to predict diatoms abundance from environmental data in a Chinese river  63  ). Hence, machine learning models were not used to provide extrapolated predictions in this study, but to detect PFCs hlePFCs and the main drivers of their biogeography. However, as more and more omics datasets are collected in the global ocean  23, 26, 64, 65  , we assume that similar approaches could be conducted with much more samples in the near future, which should increase models' performances. By using less than 100 samples, we were nonetheless able to obtain 14,585 models with R 2 values over 0.
5. It highlights the potential of such quantitative approaches for predicting the abundance of key protein families in the global ocean. Moreover, our dataset was only composed of metagenomics samples, when it is hypothesized that a big part of bacterial/archaeal communities response to environmental change comes from variations in gene expression  66. This assumption was recently disputed  23  , but applying our method to metatranscriptomes in the future would allow using the environmental context to predict protein expressions instead of metagenome sequence abundances, which could help to improve the accuracy of models predictions. In the future, biogeochemical modeling should benefit from our ability to quantify and predict biological functions using environmental and omics data  10, 13, 14, 24, 67.
Methods Samples collection and MAGs. We focused our study on the 885 non-eukaryotic MAGs made publicly available  21. The whole bioinformatic workflow designed to build these MAGs, as well as all the links leading to the fasta files and Anvi'o  68  profiles for each MAG can be found at http://merenlab. org/data/tara-oceans-mags/. These MAGs were built from 93 Tara Oceans metagenomes retrieved from 61 surface samples and 32 deep chlorophyll maximum samples collected worldwide in the global ocean, using a size filter targeting free-living microorganisms (0. 2-3 μm). Original metagenomes are available under the European Bioinformatics Institute (EBI) repository with project ID ERP001736. To date, the work achieved by Delmont et al. 21. constitutes the only database of manually refined MAGs constructed using the Tara Oceans project data.
Automated binning efforts provided larger numbers of MAGs and focused on multiple sizes fractions  20  , but are subject to higher binning errors, causing sometimes obvious contigs misplacement (as discussed here: https://bjtully. github. io/posts/2018/10/re-visiting-tmed-mags/). Further information on the MAGs' genomic features, such as their completion or GC content, can be found in Supplementary Table  5  of Delmont et al. 21. Gene detection and quantification. Prodigal v2. 6. 3  69  was run to retrieve the nucleotide and protein sequences of each detected gene for each of the 885 MAGs. By concatenation, one nucleotide and one protein fasta files were created, containing each in total 1,914,171 sequences. The nucleotide sequences were then used for the mapping and quantification step (hereafter developed) whereas the protein sequences were used for building the SSN (cf.
It produced a table containing seed orthologous sequences for 677,684 of our proteins (89. 5%), the rest of them not being similar enough from any sequence in the eggNOG database. The annotation phase was then launched on these 677,684 proteins, using the seed orthologous sequences table as input to the emapper function, and the-annotate_hits_table flag. We obtain an annotation table with GO IDs, KEGG IDs, and eggNOG descriptions. KoFamScan was launched with default options and -mapper flag. The KEGG API was then used to retrieve KEGG pathways ID and descriptions for each KEGG ID identified by KoFamScan in our protein catalog. To assess for the functional homogeneity in our PFCs, we computed a homogeneity score F hom : N annot > 1 ) F hom ¼ 1 À N annot N prot ð1Þ N annot ¼ 1 ) F hom ¼ 1 ð2Þ With N annot the number of unique annotation terms found in the PFC (either KEGG IDs or eggNOG terms), and N prot the number of proteins in the PFC.
As multiple eggNOG terms can exist for similar functions (e. g. , "UBA-ThiFtype NAD FAD-binding protein" and "UBA-THIF-type NAD FAD-binding"), they can lead to artifactually low homogeneity scores. For this reason, PFCs with low homogeneity scores obtained with the EggNOG database were tagged as poorly homogeneous but were kept in the analysis. Statistics on functionally unannotated PFCs presented in Tables  1  and 2  include both (1) query sequences that did not match to any reference in public databases, and (2) query sequences that match to one or multiple references in public databases but could not yet be associated to any biological function. To assess taxonomic diversity in our PFCs, we used the taxonomic annotation of the 885 MAGs provided by Delmont et al. 21. This taxonomic annotation was inferred from 43 single-copy core genes through the combined use of CheckM  73  , RAST  74  , and manual BLAST searches (see  21  for further details).
We computed a mean abundance for each PFC in each of the 93 metagenomes, using relative protein abundances (see Gene detection and quantification). We obtained an abundance table composed of 233,756 rows, corresponding to PFCs, and 93 columns, corresponding to the 93 Tara Oceans metagenomes used in the study. Environmental dataset. For each of the 93 Tara Oceans metagenomes, we retrieved the environmental context from Faure et al. (https://figshare. com/articles/ Data_MixoBioGeo_Faure_et_al_2018/6715754)  30. To complete this environmental dataset, we added 10 climatology variables retrieved from the World Ocean Atlas:  75  temperature, salinity, density, conductivity, dissolved oxygen, percent oxygen saturation, apparent oxygen utilization, silicate, phosphate, and nitrate. Finally, we added the upper limit of the size fraction as an environmental variable, as it varied across samples.
For temperature, salinity, and conductivity we retrieved the mean and the mean seasonal anomaly at each sampling point (precision of 1°) over the 2005-2012 period. Only the mean was retrieved for density. For the 6 other variables, we retrieved the mean and the mean seasonal anomaly at each sampling point (precision of 1°) over all available years. In total, we obtained 74 environmental variables, which we reduced to 51 by getting rid of near-zero variance variables and too highly correlated ones, using options "nzv" and "corr" from the preProcess function of the caret package v6. 0 76 in R v3. 5. 3  77. A detailed description of these variables is available in Supplementary Data 2. We then scaled and centered the 51 selected environmental variables, and used a k-nearest neighbors approach to replace NA values (6.
Acknowledgements We would like to particularly thank  Loïs Maignien ,  Olivier Aumont ,  Eric Pelletier ,  Christian Tamburini ,  Sébastien Monchy ,  Thomas Mock , and  Ingrid Obernosterer  for the insightful discussions concerning this study. We also thank the  Meren Lab  (https:// merenlab. org/)and all the people involved in  Tara Oceans  for producing the data we used and making them publicly available. We would also like to thank all participants of the GOBITMAP and GREENOCEAN workshops for their useful advice. Finally, EF would like to thank  Raphaël Berthier  and  Jean-Olivier Irisson  for their advice concerning machine learning, as well as  François Duchenne ,  Elise Kerdoncuff ,  Benoît Pérez  and  Eric Bapteste  for their helpful comments. This work was funded mainly by our salary as French State agents and therefore by French taxpayers' taxes.
Data availability Instructions on how to build or download the MAGs and metagenomes used in this study are available at http://merenlab. org/data/tara-oceans-mags/. Tools and databases used for functional annotations are available at http://eggnog-mapper. embl. de/ and https://www. genome. jp/tools/kofamkoala/. All other data used in this study are available at 10. 6084/m9. figshare. 12030795, including fasta files containing nucleotide sequences of all proteins in PFCs, hlePFCs, dark PFCs, PFCs associated with Station 93, and PFCs associated with Station 93 linked with Pseudoalteromonas MAGs. In this figshare repository, we also provide summary tables including all PFC and random forest associated statistics (e. g. , all homogeneity and unknown scores, R 2 values, variables importances) for each PFCs, hlePFCs, and dark PFCs.
Human specimen HPV-ICB2 (accession number MK080568), a novel beta-2 HPV that is 7441 bp in length, was previously isolated from a human forearm skin swab sample and fully characterized in our laboratory  (Brancaccio et al. , 2019). This sample was collected from a patient enrolled in the VIRUSCAN study, a five-year (2014-2019) prospective cohort study conducted at the Moffitt Cancer Center and the University of South Florida (R01CA177586-01; "Prospective study of cutaneous viral infections and non-melanoma skin cancer"). The study was approved by the University of South Florida Institutional Review Board, and all participants provided written informed consent. All methods were carried out in accordance with relevant guidelines and regulations.
MinION library preparation Three libraries were prepared independently following the SQK-LSK109 protocol for 1D PCR barcoding amplicons (Fig. 1 ). First, longrange PCR was performed to amplify the entire HPV-ICB2 genome from a 1:100 diluted RCA product using a KAPA HIFI HotStart Ready-MixPCR kit following the manufacturer's instructions (KAPA Biosystems, Boston, MA, USA) with HPV-ICB2-specific outward-directed primers tailed with the universal sequences provided by the manufacturer. The following tailed primers were used at a final concentration of 1. 6 pM each: forward primer, 5 ′ -TTT CTG TTG GTG CTG ATA TTG CCA GAC AGA ACA CAT CTT TTG ATC C-3 ′ and reverse primer, 5 ′ -ACT TGC CTG TCG CTC TAT CTT CTC GTC CCG TGA CCC ACC CTG A-3 ′. All the above-mentioned primers were designed within a partial L1 region sequence of HPV-ICB2 (99 bp) obtained from previous next-generation sequencing data  (Brancaccio et al.
Bioinformatics data analysis Coverage was calculated using SAMtools, and the percentage of similarity was determined by the number of matches or mismatches at each base using Pysamstats v1. 1. 2 (https://github. com/alimanfoo/pysa mstats)  (Batovska et al. , 2017). Both coverage and similarity were computed on the filtered reads obtained after two steps of quality and size filtering using NanoFilt and Filtlong tools. The fully automated bioinformatics pipeline is described in Fig. 2  (https://github. com/ IARCbioinfo/MinION_pipes). A Supplementary file S1 describing all the bioinformatics tools and parameters is also available. Guppy v4. 0. 15+ (https://community. nanoporetech. com/protocols/Guppy-pro tocol) (Adrien  Leger and Tommaso Leonardi, 2019; Wick et al. , 2019)  was used to perform base-calling, de-multiplexing and barcode trimming steps.
NanoFilt v2. 7. 1 (https://github. com/wdecoster/nanofilt) (De Coster et al. , 2018)  was used to remove unspecific reads longer than 8000 bp. Filtlong v0. 2. 0 (https://github. com/rrwick/Filtlong) was used to remove reads shorter than 7000 bp and low-quality reads (the worst 90 % of the reads were removed). To assess the effectiveness of the filtering steps, defined as the similarity between the draft sequence and the reference, two BLAST analyses, using Megablast algorithm, were performed on the raw reads and on the filtered reads. Canu v2. 1 was used to reconstruct the whole genome of HPV-ICB2. Medaka v1. 0. 3 (https://github. com/nanoporetech/medaka) ("Medaka: Sequence correction provided by ONT Research (Accessed 27 Jun 2020). ," n. d. ) was used for the polishing of the draft genomes.
MinION sequencing and assembly using three independent runs (Protocol A) Three consecutive 12 -h sequencing runs, each using an independent Fig. 1. Schematic overview of the research strategy for the full-length viral genome characterization of papillomaviruses. Partial L1 sequences from putative new HPV types are available in GenBank or from next-generation sequencing data. After identification by PCR of HPV DNA-positive samples for the corresponding L1 sequence, multiply primed rolling circle amplification (RCA) was performed on the corresponding DNA sample to enrich circular DNA. To obtain the complete viral genome, first, long-range PCR was performed to amplify the entire HPV genome from a 1:100 diluted RCA product, using specific outwarddirected primers tailed with the Oxford Nanopore universal tags.
A second PCR to add the barcodes from the barcoding kit (EXP-PBC001) was performed. After DNA repair, end-prep, adapter ligation, and clean-up steps, the libraries were loaded into the MinION flow cell consecutively (Protocol A: three separate 12 -h runs) or simultaneously (Protocol B: a single 48 -h run). DNA library, were performed with the 1D PCR barcoding amplicons (SQK-LSK109) protocol on a MinION flow cell (R9. 4. 1). Before each sequencing run, the number of available nanopores was measured using the protocol "check your flow cell" in the MinKNOW software. The number of active nanopores decreased from run 1 to run 3 (1577, 1124, and 857 respectively), thus exceeding the guaranteed level (800 pores). The combined runs generated 9,354,933 raw reads (run 1: 3,186,245 reads; run 2: 2,464,705 reads; and run 3: 3,703,983 reads).
5A ), and the results showed the presence of seven nucleotide positions  (positions 882, 2184, 2612, 3001, 3301, 3491 and 3592)  with the highest error rate in all three runs. The analysis of the raw data showed that the average error rates were 2. 52 %, 1. 97 %, and 2. 17 % for runs 1, 2, and 3, respectively (Table  1 ). The reads were identified using the BLAST tool and the whole nucleotide collection (nr/nt, October 2020) database. The majority of the reads were representative of HPV-ICB2 (77. 5 %, 92. 3 %, and 63. 2 % for runs 1, 2, and 3, respectively) while the remaining reads were representative of bacteria, archaea, eukaryota, and other viral sequences (Supplementary Table  S1 ). A second BLAST alignment was performed after read-filtering (Fig. 2 ), which allowed for 99. 9 % of the sequences identified as HPV-ICB2 to be obtained in all three runs (Supplementary Table  S1 ).
98 % of the reads were identified as belonging to the HPV-ICB2 genome (Supplementary Table  S2 ). The whole HPV-ICB2 genome was obtained from each barcoded library using Canu and showed a percentage of identity to HPV-ICB2 of 99. 95 %, 99. 94 %, and 99. 90 % for BC1, BC2, and BC3, respectively, which increased to 99. 98 %, 99. 98 % and 100 % after a Medaka polishing step (Table 2, and data not shown). Finally, a consensus sequence was obtained after the alignment of the three genomes using MUSCLE. The final percentage of identity to HPV-ICB2 was 99. 98 % (1 nucleotide gap). The single nucleotide gap was located within a homopolymeric G stretch at the nucleotide position 3495 (Table  2 ). Sanger sequencing performed on the DNA libraries loaded into the flow cell demonstrated the absence of gaps before the sequencing run.
95 %. All the sequencing runs were performed applying the default starting voltage of -180 mV. Especially for run 4, adjustments of the starting voltage could have been considered to obtain better results. However, the voltage is automatically controlled by the MinKNOW software for the rest of the run time, and thus we consider these adjustments to be of minor impact. The complete protocol described in this study can be performed in less than two weeks and represents an alternative to timeconsuming methodologies for the whole-genome sequencing of HPVs; the RCA step can be performed in one day, followed by another day for the long-range PCR using HPV-ICB2-specific outward-directed primers, agarose gel purification, and library preparation. Then, the total duration of the protocol depends on the sequencing run times and on the complete bioinformatics analysis.
Conclusions Our results showed that the use of the MinION nanopore sequencer is an effective and rapid strategy for whole-genome sequencing of HPVs. Furthermore, this technology can be easily integrated into our protocol for the isolation and full characterization of novel HPV genomes  (Brancaccio et al. , 2019 (Brancaccio et al. , , 2018 (Brancaccio et al. , , 2017)). The first part of our protocol for the discovery of novel papillomaviruses relies on the use of degenerate-primer PCR to amplify part of the viral genome, combined with next-generation sequencing  (Brancaccio et al. , 2018). This strategy led to the discovery of partial L1 sequences from 105 putative new PV types in the oral cavity and skin. Also, thousands of partial genomic sequences of putative novel HPV types have been deposited in GenBank and remain to be fully characterized, some of which may display transforming activity and deserve attention (reviewed in  (Gheit, 2019) ).
Out-of-focus sections of whole slide images are a significant source of false positives and other systematic errors in clinical diagnoses. As a result, focus quality assessment (FQA) methods must be able to quickly and accurately differentiate between focus levels in a scan. Recently, deep learning methods using convolutional neural networks (CNNs) have been adopted for FQA. However, the biggest obstacles impeding their wide usage in clinical workflows are their generalizability across different test conditions and their potentially high computational cost. In this study, we focus on the transferability and scalability of CNN-based FQA approaches. We carry out an investigation on ten architecturally diverse networks using five datasets with stain and tissue diversity. We evaluate the computational complexity of each network and scale this to realistic applications involving hundreds of whole slide images.
Dynamic imaging conditions mean that FQA methods must be generalizable to different datasets. This requires that a variety of tissue types, stain types, and resolutions be used to train the model  13, 37, 38. Unfortunately, there has until recently been a shortage of large and diverse datasets for FQA purposes  39  , which increases the risk of overfitting data-driven models  40. Additionally, a principal advantage of digital pathology technology is that scanners can process images much faster than human pathologists, with some able to scan hundreds of images at a time  6, 26, 40. In clinical settings, it is ideal for scans to be completed during night-time hours, so that they are ready for diagnosis the following day  10. The high throughput of digital scans therefore requires QC pipelines that can handle this high throughput.
Computational complexity of the FQA method should not be a restricting factor, and is therefore equally important in the evaluation of the method as its performance  10, 40, 41. A drawback of deep learning models for FQA is that they are not as easily applicable as knowledge-based methods  [42] [43] [44] [45] [46] [47] [48]  , which have low computational complexity and can be easily applied without adjustment or tailoring  10. Conversely, when transferring CNNs to other computer vision applications it is usually recommended to use a process called fine-tuning  [49] [50] [51] [52] [53]  , where the network already has a majority of parameters set and is then trained to adjust the remaining parameters. This is used when the datasets and computational resources for training are limited  49. However, this increases the resources spent transferring the network to different scanners, and would ideally not be necessary for high performance.
While foregoing fine-tuning would have efficiency advantages, it could potentially produce some concerns regarding the explainability of the model and the trust in AI-based decision making  3, 27. Figure  1  shows our process for the evaluation of each deep learning model. The images are first normalized  27, 39  , then used to train a CNN. The trained model, including the fully connected layer, is then tested on the same dataset on which it was trained to validate the success of the training. Afterwards, the full model is tested on a separate dataset it has not yet seen to evaluate the transfer process  13, 54, 55  , without the use of fine-tuning. Numerous metrics are used to assess each deep learning model, including computational complexity, layer probing quality metrics, and a spatial focus quality distribution.
2. Deep CNNs: these CNNs have multiple layers and are optimized towards obtaining an accurate result using a robust framework  2, [16] [17] [18] [19] [20] [21]. These networks focus on achieving high levels of accuracy and being highly generalizable to different tissue types, stain types, and resolutions. These relatively deeper CNNs have more layers which enable the network to capture fine features that may be missed by a shallow network  56. A drawback of deep CNNs is that they are prone to overfitting when training on small datasets, which means more effort must be put into dataset creation  14. After a critical depth, CNNs can be overparametrized which worsens their ability to generalize  57. 1  also shows which studies focused on scalability and transferability, the two key pillars of a successful FQA method  3, 10.
The most popular metrics of this list, including PLCC, SRCC, ROC, and PR, are therefore evaluated in this paper to clearly compare the network performance. Heatmaps are used in four studies  2, 10, 18, 20  to spatially represent the FQA of a deep learning model. Spatial representations are important for better visualizing which features the model is able to capture well and which features it misses. They are also important for understanding the characteristics of a dataset that may make it more or less transferable to other applications  2, 10, 20. In this paper, we make the following contributions to improving FQA in digital pathology: Experiment Design: We train ten architecturally diverse CNNs on five datasets of different stain types, tissue types, resolutions, and input patch sizes. We evalute the computational complexity of each CNN, and scale this to realistic scanning applications.
We use these methods to draw conclusions about the effect of input patch size, tissue diversity, and stain diversity on the transferability of a deep learning model to other datasets. Architecture Design: We develop an automatically designed architecture using differentiable architecture search  58  on the same diverse datasets to evaluate the performance of searched architectures relative to conventional CNNs. Validation Methods: We use the knowledge gain and mappign condition metrics  63  to evaluate how well the model is learning as well as its degree of stability. We use ROC and PR metrics to evaluate the performance of each network when transferred to another dataset. We use focus quality heatmap representations to understand the spatial distribution of focus quality in an image.
Dataset Selection This section describes the datasets used for this experiment, with information about each dataset summarized in Table  2. Datasets with the suffix "64", such as FocusPath64  43  , are copies of the original dataset with a 64 x 64 input patch size. Figure  3  shows examples of patches used for each dataset. Further information about each dataset can be found in the Supplementary Materials. The tissue scans used in this work are deemed exempt from University of Toronto Research Ethics Board regulations. The datasets consist of anonymized tissue scans and metadata relevant only to the focus quality of the image. Figure 3 FocusPath 43 : The FocusPath  43  dataset contains 8640 patches of size 1024 x 1024 extracted from nine different stained slides. This dataset is useful for the development of CNNs geared towards FQA methods due to its diverse distribution of colors and stains relative to other datasets.