dataesr/scientific-paragraphs-categorization huggingface dataset:
Archaeological Samples Fit within the Genetic Cluster of Southern Levantine Cultivars. A principal component analysis (PCA) of 934 grapevine accessions from Europe, Western and Central Asia, and North Africa shows that the variation explained by the first principal component can mostly be attributed to differences between the wild and cultivated samples (SI Appendix, Fig. S4 ). This is consistent with the findings of prior studies  (30, 32). When only the cultivated accessions were analyzed, a triangular genetic structure was revealed (Fig. 2A ). One edge of the triangle outlines the axis from Western and Central Europe to the Iberian Peninsula. Another edge outlines the axis from Western Europe through Eastern Europe to Asia and the Levant. Interestingly, samples from the Southern Levant (Israel, Syria, and Lebanon in our dataset) were mostly clustered tightly together and appeared separate from other Asian locals.
We repeated our analysis of cultivated and wild Eastern accessions only (the Asian and Greek groups) and found that the distinction between the Southern Levant samples is maintained and that the archaeological samples from Israel fit within the cluster of the Southern Levantine samples (Fig. 2B ). To determine whether the ancient samples were likely cultivated variety, we executed two PCAs over the Southern Levant samples using both SNP datasets. In both analyses, the archaeological samples fell within the cultivated range (Figs. 2 C and D). Our result was further supported by a STRUCTURE analysis  (33)  in which the archaeological samples were assigned to the same clusters as the cultivated accessions (SI Appendix, Fig. S8 ). Kinship between Southern Levantine Accessions. Fig. 3A  shows the inference of kinship between all Southern Levantine accessions based on shared Identity by Decent (IBD) segments for the larger SNP datasets.
Archaeological A33 and modern Asswad Karech cultivar, which was sampled in Lebanon, appear to be very closely related; their kinship appears stronger than that of known modern parent-offspring pairs (SI Appendix, Chapter 6 and Fig. S5 ). Archaeological A31 and A32 appear to be related to a lesser extent and so do A31 and modern Be'er, a feral variety growing wild along the southern Israeli coast. Fig. 3B  features the inference of kinship between the archaeological samples and the Southern Levantine accessions based on shared IBD segments for the smaller SNP datasets. It too shows A33 and Asswad Karech to be very close and gives support to a close kinship between A31 and A32 and between A31 and Be'er. In addition, archaeological A37 appears to be related to A33 and to Asswad Karech. Further analysis of the accuracy of the imputation and phasing and how kinship is assessed are shown in SI Appendix, Chapter 6 and Figs.
Modern Asswad Karech Has Likely Parented an Ancient Grape. We estimated kinship coefficient among pairs of Southern Levantine samples by running the program KING  (34). Archaeological sample A33 and modern Asswad Karech were found to satisfy the parent-offspring criteria: Kinship coefficient (K) was estimated at 0. 3017 and IBS0 at 0. IBS0 stands for the proportions of alleles in which neither chromosome is identical by state (i. e. , the proportion of homozygote to homozygote mismatches). More details of the criteria of different relatedness categories are provided in SI Appendix, Chapter 8. Allowing for 45 sequencing errors (for diploid 6,896 sites; minimal error rate 0. 33%), all of A33 genotypes are contained in Asswad Karech, while the opposite is not true. We manually inspected the phased haplotypes and were able to match A33 haplotypes to Asswad Karech's (SI Appendix, Fig.
Adaptor contamination was removed using LeeHom software  (63)  and low-quality reads and reads shorter than 25 bp were removed using Trimmomatic version 0. 36  (64). The archaeological samples were mapped to the grapevine reference genome assembly (12X. v2) (65) using Bowtie2 version 2. 2. 5  (66). The mapped reads were evaluated for aDNA damage patterns (24) using mapDamage version 2. 0 (25) to assert that the samples are indeed ancient (SI Appendix, Fig. S2 ). We also used map-Damage to rescale the ancient sequence bases quality and improve mapping and reducing the number of genotype call errors stemming from aDNA type damage. See detailed description in SI Appendix, Chapter 4 and Tables  S6  and S7 , on measures taken to avoid misdiagnosing deamination damage as polymorphism. We also excluded reads that aligned more than once with the reference genome with their second-best alignment having less than twice the number of mismatches as the best alignment.
36, leaving sequences that were at least 75 bp long. Next, the reads were aligned to the grapevine reference genome using Bowtie2. Reads that were aligned with more than two mismatches and one gap opening were filtered out. We also excluded multiple aligned reads as described above. After this stage, five samples were eliminated due to a low number of reads remaining (<20,000), leaving 13 samples. Publicly Available Whole-Genome Sequences. Publicly available wholegenome Vitis sequences were downloaded from the NCBI database. We obtained a total of 126 samples from the short-read archive bioproject accessions PRJNA647155  (19) , PRJNA388292  (30) , and PRJNA393611  (31) , selected based on their sequencing quality (SI Appendix, Table  S3 ), and for the last bioproject, based on name compliance with the Vitis International Variety Catalogue (VIVC) grape catalog and with Lacombe et al.
(68). The samples were processed as described above for the modern native Israeli samples. Building of SNP Datasets. SNPs calling for modern and ancient samples was done using HaplotypeCaller and GenotypeGVCFs protocols of the GATK pipeline  (69). Overall, 15,414,158 SNPs were identified. About 9,988 of them were included in the 10K SNP array for which the captured samples were enriched. The remaining three shotgun ancient samples, one captured ancient sample, and nine of the native samples with coverage of less than one in this SNP array were eliminated, leaving only five ancient samples, all from Avdat, and nine native samples (SI Appendix, Tables  S1  and S2 ). Next, the genotypes of 783 cultivated and 112 wild grapevine samples from the GrapeReSeq diversity panels described by Laucou et al.
(28)  and by Le Paslier et al. (29) , respectively, were incorporated in the genotype catalog. From this catalog, we created two separate SNP datasets to be used in the analyses: One included three ancient samples and 6,928 SNPs and the other all five ancient samples and 1,032 SNPs. For each dataset, we used VCFtools  (70)  to assert that each locus had genotype call in at least 75% of the samples, and each of the samples had genotype calls in at least 60% of the loci with the minimal read depth of five (for nonpanel samples). In all of the following analyses, the abovementioned quality criteria were maintained or made stricter. PCA. We executed a PCA using the decomposition of the covariance matrix approach by running prcomp function implemented in R with the default parameters and scale=TRUE when possible.
The results were plotted using the R packages factoextra and ggplot2. The samples were geographically classified according to Bacilieri et al. (55)  with a few modifications and included only accessions sampled in Europe, Western and Central Asia, and North Africa. The full list of countries and their geographical groups can be found in SI Appendix, Chapter 8. In the PCA that included only the Southern Levantine samples, the R function stat_ellipse was used to calculate and draw ellipses with 95% confidence level around the cultivated and wild samples assuming a multivariate t distribution. Kinship Analysis. We inferred kinship between the Southern Levantine accessions in two different ways: Genotypes phasing and inference of IBD segments. Utilizing all available accessions, we only used loci with a minor allele frequency (MAF) of 3% that had genotype calls in at least 80% of the samples, leaving 6,896 and 1,030 SNPs, respective of dataset.
The genetic data were imputed and phased to infer the lengths of shared IBD segments using Beagle and refined IBD software  (71) (72) (73). Using this combined approach, IBD segments that are shared with one haplotype copy (IBD1) are built stochastically, and missing genotypes are imputed using the hidden Markov chain model. The logarithm of the odds (LOD) score of candidate segments is calculated based on the likelihood ratio in an IBD versus a non-IBD model. We were able to identify a total of 116,423 and 42,676 IBD haplotype segments with LOD > 3 among all pairs of Southern Levantine samples, respective of SNP dataset. We used the summed lengths of shared haplotypes between each sample pair to infer their relative relatedness. This analysis and the insepction of the phased haplotypes between A33 and Asswad Karech are the only analyses utilizing imputed SNP datasets.
For the larger SNP dataset, the samples were ordered and clustered using the R library pheatmap. Inference of kinship coefficient based on allele frequencies. We ran the "KING-robust" algorithm implemented KING software version 2. 2. 7 (34) to estimate kinship coefficient between each sample pair based on genotype calls. This algorithm was chosen because it accounts for biases in kinship coefficient estimation which stem from population structure. The level of relatedness was assessed based on the resulting kinship coefficient (K) and IBS0 proportion and according to the relatedness categories shown by Laucou et al. (28)  to be reliable for kinship inference in grapes (SI Appendix,  Chapter 8). In this analysis, we used the larger SNP dataset. Berry Skin Color Analysis. We used the GWAS for berry color described by Laucou et al.
Data, Materials, and Software Availability. Genomic data have been deposited in NCBI (PRJNA887737 and PRJNA887039). Previously published data were used for this work [(28) (https://urgi. versailles. inra. fr/Species/Vitis/Data-Sequences/Genotyping-data) (19, 29 (NCBI project PRJNA647155) (30) (NCBI project PRJNA388292) (31) (NCBI project PRJNA393611 and PRJNA393611)]. ACKNOWLEDGMENTS. This project was supported by the European Research Council under the European Union's Horizon 2020 Research and Innovation Program Grant 648427 and under the Marie Skłodowska-Curie grant agreement no. 842577, the Israel Science Foundation Grant 915-20, and Proof of Concept Grant 101069201. This study was conducted under the licenses of the Israel Antiquities Authority permits A7736/2016, G77/2018, G-41/2016, G-40/2017, G-26/2018, G-34/2019, and G-42/2021.
The radiocarbon research was supported by the Exilarch Foundation for the Dangoor Research Accelerator Mass Spectrometer Laboratory. Avdat excavation was funded by the S. B. 's home institution, DePaul University, and by a grant from the Fulbright US Scholar Program. We wish to thank the Kimmel Center for Archaeological Science and George Schwartzman Fund for the Laboratory and funding support for the material analysis. We thank the Israel Nature and Parks Authority for facilitating the excavations. DNA processing and other bioinformatics work were executed on the Hive computer cluster of the Faculty of Natural Science at the University of Haifa. pnas. org.
Second, expert annotations are laborious to attain, costly and prone to subjectivity. The most popular methods today rely on Multiple Instance Learning (MIL), which frames the problem as a bag classification task. WSIs are split into small workable images (tiles), which are processed separately. Features from each of the individual tiles are extracted and then aggregated to classify the WSI. The extraction of these tiles' specific representation is crucial to the downstream WSI classification task. One common approach consists of initializing the feature extractor with pre-trained weights on ImageNet, a natural image dataset. This technique allows one to extract generic features that are powerful, but that do not lie within the histopathological domain. Different strategies have been developed to extract these tile encodings taking advantage of the available data and their respective level of supervision.
A first strategy aims to learn tile features with full supervision  [Ehteshami Bejnordi et al. , 2017]. To create a supervised dataset, one or several experts manually review tiles and sort them into meaningful classes (preferably related to the downstream task of classifying the WSIs). Even though experts' annotations can bring powerful prior knowledge to the model, this technique often requires large quantities of annotations. A second strategy consists of learning tile representations through self-supervision. It leverages the unannotated data by training a convolutional neural network on a pretext task. It has proven its efficacy  [Saillard et al. , 2021 , Lu et al. , 2021]  and even its superiority to the fully supervised scheme  [Dehaene et al. , 2020]. However, this approach has a non-negligible computational cost, as training necessitates around 1000 hours of computation on a standard GPU  [Dehaene et al.
Dataset and Problem Setting The Tissue Net Challenge [DrivenData] organized in 2020, the Société Française de Pathologie (SFP) and the Health Data Hub aimed at developing methods to automatically grade lesions of the uterine cervix in four classes according to their severity. The training dataset for the challenge was made up of biopsy samples from female uterine cervix, focusing on squamous lesions (Figure  2 ). These lesions are often benign but can also be qualified as low grade or high grade depending on the 4 risk of invasion of the underlying conjunctive tissue and evolution into carcinomas. The grade of the lesions depends on the proportion of squamous epithelium affected by dysplastic criteria. Lowgrade squamous intraepithelial lesions (LSIL) are defined as having a dysplastic criteria involving less than one third of the thickness of the epithelium.
Fully Supervised Dataset 5926 annotated Regions of Interest (ROIs) of fixed size 300x300 micrometers were provided. Each ROI had roughly the same size as a tile at 10x magnification and were labeled by the severity of the lesion it contained: "Normal" (0) if tissue was normal, (1) LSIL or (2) HSIL if it presented precancerous lesions that could have malignant potential and (3) invasive squamous carcinoma (Table  1 ).
Weakly Supervised Dataset The dataset was composed of 1015 WSIs acquired from 20 different centers in France at an average resolution of 0. 234 +/-0. 0086 mpp (40X). The slide resolution varied slightly due to the multicentric provenance of the data. The class of the WSI corresponded to the class of the most severe lesions it contained (grade from 0 to 3 also). All the native WSI formats were converted to pyramidal TIFF (Tagged Image File Format). Both the WSI-level and tile-level labels have been attributed by a consortium of expert pathologists (Table  1 ).
To select the most meaningful features for each class, we trained a Lasso classifier without bias to classify the extracted feature vectors into the four classes of the dataset for the four pre-training policies. The feature vectors for each tile were first normalized and divided element-wise by the vector of features' standard deviation across all the tiles. The L1 regularization factor λ was set to 0. 01. Details about Lasso training can be found in Supplementary Materials. Contribution scores for each feature were therefore derived from the weights of the Lasso linear classifier: negative weights were removed and remaining positive weights were divided by their sum to obtain contribution scores  [0, 1]. By filtering out the negative weights, the contribution score corresponds to the proportion of attribution among the features positively correlated to a class, and allows to select feature capturing semantic information related to the class, leaving out those containing information for other classes.
Feature Extractor Pre-Training The feature extractor is initialized with pre-trained weights obtained with three distinct supervision policies: fully supervised, self-supervised or a mix of supervision. These three policies rely on the fine-tuning of a DenseNet121  [Huang et al. , 2016] , pretrained on ImageNet. The fully-supervised archicture is fine-tuned solely on the tile classification task. The SSL architecture is derived from SimCLR framework and is trained on an unlabeled dataset of 1 million tiles extracted from the slides. Finally for the mixed-supervised architecture, a supervised branch is added to the previous SSL network and trained using the mixed objective function (see Fig. 1  and Eq. 4) on the fully supervised dataset. Technical details of these three training settings are available in the supplementary material.
Self-Supervised Fine-Tuning We saved the checkpoints of the self-supervised feature extraction model at each epoch of training, allowing us to investigate the amount of time needed to reach good WSI classification performances. We computed the embeddings of the whole dataset with each of the checkpoints and trained a WSI classifier from them. Figure  3  reports the performances of WSI classification models for each of these checkpoints. SSL training led to a higher Weighted Accuracy than using ImageNet weights after 3 epochs and resulted in a gain of +4. 8% after 100 epochs. Interestingly, as little as 6 epochs 10 of training are enough to gain 4% of Weighted Accuracy: a significant boost in performance is possible with 50 GPU-hours of training. We then observe a small increase in performance until the 100th epochs.
Pre-Training Policy Comparison To compare the weights obtained with the various supervision levels, we ran a 3-fold cross-validation on the WS classification task and summarized the results in table 3. The results indicate that the SSL pre-training substantially improves the WSI classification performance. In contrast, we see that initializing the feature extractor with fully-supervised weights gives an equivalent or poorer performance than any other initialization. SSL pre-training allows us to extract rich features that are generic, yet still relevant to the dataset (unlike ImageNet). On the other hand, fully supervised features are probably too specific and seem to not represent the full diversity of the image data. The joint-optimization process manages to balance out generic and specialized features without neutralizing them: mixing the supervision levels brings significant improvements (+2%) to the performance, leading to a Weighted Accuracy of 0.
Finally, we see a diminution of the standard deviations across splits for the different pre-training policies, showing better stability for longer SSL training and more annotations. We draw different conclusions from these observations: • In this context, it is always better to pre-train the feature extractor with SSL rather than only invest in annotations. • The supervised fine-tuning needs enough annotations to bring an improvement to the WSI classification task. We can note however that even when considering the 100% annotation settings, the supervised dataset (approx. 5000 images) is still rather small in comparison to traditional image datasets. • A full SSL training is mandatory to leverage this small amount of supervised data.
Features Visualisations We generated the pseudo-images of the most important features for each class and each pre-training policy and extracted the related tiles. The Figure  5  displays the most important features along with 12 Table  4 : Relationship between self-supervision and full-supervision -Study on the performance improvement on WS classification for different proportion of labelled data versus different training time of SSL the tiles activating each feature the most for the class "Normal"  (0). Although interpretation of such pseudo images must be treated carefully, we notice that the features obtained with SSL, supervised and mixed training are indubitably more specialized to histological data than those obtained with ImageNet. Some histological patterns, such as nuclei, squamous cells or basal layers are clearly identifiable in the generated images.
Second, a small set of annotated tiles can bring benefit to the WSI classification task, up to 2% of weighted accuracy for a supervised dataset of around 5000 images. Such a set of tiles can be obtained easily by asking the pathologist to select a few ROIs that guided his decision while labeling the WSIs, which can be achieved without a strong time commitment. However this boost in performance can be reached only if the feature extractor is pre-trained with SSL, and for sufficiently long: SSL unlocks the supervised fine-tuning benefits. To further understand the differences between the range of supervision used to extract tile features, we conducted qualitative analysis on features visualizations by activation maximization and observed that features obtained from SSL, supervised or mixed trainings were more relevant for histological tasks and that class-discriminative patterns were indeed identified by the model.
Acknowledgments The authors thank  Etienne Decencière  for the thoughful discussions that helped the project. ML was supported by a  CIFRE PhD fellowship  founded by  KEEN EYE, Paris, France  and  ANRT  ( CIFRE 2019(CIFRE /1905 )). TL was supported by a  Q-Life PhD  fellowship (Q-life  ANR-17-CONV-0005 ). Furthermore, this work was supported by the  French government  under management of  Agence Nationale de la Recherche  as part of the  "Investissements d'avenir" program , reference  ANR-19-P3IA-0001  ( PRAIRIE 3IA Institute ).
94% of confirmed and reported variants are single nucleotide variants (SNVs) and span the whole genome. Missense variants account for 43% of all SNVs in MITOMAP (Figure  1A ); 58% of them are present in the Genome Aggregation Database (gnomAD)  8  and 65% in HelixMTdb  9  (Figure  1B ). As one might anticipate, the majority of gnomAD's missense variants are benign based on ClinVar, but gnomAD also includes twelve MITOMAP confirmed missense variants and an additional three that ClinVar classifies as pathogenic (Supplementary Table  1 ). In the case of new variants or without a confirmed functional effect, one should apply the full 2020 ClinGen/ACMG/AMP guidelines  10. For allele frequency (AF), they allow an evidence weight of "supporting" when a variant is found at an AF <1/50000 (<0. 002%) in mitochondrial specific databases (pathogenic criteria code "PM2"), "likely benign" when AF >0.
5% ("BS1"), or "standalone benign" when AF>1% and there is no other conflicting evidence, such as a novel occurrence in a major haplogroup branch to support pathogenicity ("BA1"). While confirmed variants are mostly rare  8  , reported, conflicting, and synergistic variants are far more common. Indeed, there are eight confirmed variants in gnomAD with AF ≥0. 002% (Supplementary Table  1 ) and 187 reported variants with AF ≥0. 002%, of which 28 with AF ≥0. 5% and 16 ≥1%. Similarly, HelixMTdb contains seven confirmed variants with frequencies ≥0. 002% (Supplementary Table  1 ) and 191 reported variants ≥0. 002%, of which 23 have AF ≥0. 5% and 13 ≥1% (Figure  1C ). Because neither of the databases is enriched for mitochondrial disorders or other clinical phenotypes, compared to the GenBank mtDNA sequence repository, these variants are unlikely to be all pathogenic.
Characteristics of the training set The APOGEE 2 training set (Dataset 1) contains 140 pathogenic and likely harmful variants and 1734 benign and likely harmless variants that have been manually curated (see Methods for a detailed description of the datasets). These were annotated with mitochondrially-tailored evolutionary, positional, and structural features and fourteen pathogenicity assessments of in silico predictors. Most of their Pearson correlation coefficients were in the range (-0. 5, 0. 5); MutationAssessor, SNAP, and PhD-SNP exhibited absolute Pearson's r values >0. 7 with more than one other feature (Supplementary Figure  1A  and Supplementary Table  2 ). Features were not always available for all variants. Missing values accounted for 1% of the evolutionary scores and in silico predictions and 0.