dataesr/scientific-paragraphs-categorization huggingface dataset:
This "tempered Riemann approximation EM" (tmp-Riemann approximation) can compute EM steps when there is no closed form thanks to the Riemann sums as well as escape the initialisation thanks to the tempering. For a bounded latent variable z ∈ [0, 1], we define the approximation as: pn,θ (z) := h( nz /n; θ) 1 Tn / z h( nz /n; θ) 1 Tn dz , for a sequence {T n } n ∈ (R * + ) N , T n -→ n∞ 1. In the following Theorem, we prove that the tempered Riemann approximation EM verifies the applicability conditions of Theorem 1 with no additional hypothesis from the regular Riemann approximation EM covered by Theorem 2. , which we call "tempered Riemann approximation EM", verifies the remain- , made by EM and tmp-EM over 1000 simulated dataset with two different initialisations. The three different parameter families, described in Fig.
Experiment 1: 6 clusters We start by demonstrating the superior performance of the tempered EM algorithm on an example mixture of K = 6 gaussians in dimension p = 2. The real parameters can be visualised on Figure  1 , where the real centroids are represented by black crosses and confidence ellipses help visualise the real covariance matrices. In addition, 500 points were simulated in order to illustrate, among other things, the weights of each class. To quantify the ability of each EM method to increase the likelihood and recover the true parameters, we generate from this model 20 different datasets with n = 500 observations. For each of these datasets, we make 200 EM runs, all of them starting from a different random initialisation. To initialise the mixture parameters, we select uniformly 6 data points to act as centroids.
Illustrative First, we observe on the left of Figure  2 , one example of the final states of the EM algorithm. The observations can be seen in green, the initial centroids are represented by blue crosses, and the parameters {μ k } K k=1 and Σ k K k=1 estimated by the EM are represented in orange. In this EM run, one of the estimated clusters became degenerated and, as counterpart, two different real clusters were fused as one by the method. On the right of Figure  2 , we observe the final state of the tmp-EM on the same dataset, from the same initialisation. This time all the clusters were properly identified. Fig. 2  EM and tmp-EM final states on the same simulation with the same initialisation. tmp-EM positioned correctly the estimated centroides, whereas the regular EM made no distinction between the two bottom classes and ended up with a degenerate class instead.
Quantitative To demonstrate the improvements made by tempering, we present aggregated quantitative results over all the simulated datasets and random initialisations. Likelihood maximisation EM and tmp-EM are optimisation methods whose target function is the likelihood of the estimated mixture parameters. We represent on Figure  3  the empirical distribution of the negative log-likelihoods reached at the end of the two methods, EM in blue, tmp-EM in orange. On those boxplots, the coloured "box" at the centre contains 50% of the distribution, hence it is delimited by the 0. 25 and 0. 75 quantiles. The median of the distribution is represented by an horizontal black line inside the box. The space between the whiskers on the other end, contain 90% of the distribution, its limits are the 0. 05 and 0.
On the other hand, the second initialisation type -which we call "2v1" -is voluntarily misguiding the algorithm by positioning two centroids on the isolated right cluster and only one centroid on the side of the two ambiguous left clusters. The blue crosses on Figure  8  represent a typical 2v1 initialisation. This initialisation is intended to assess whether the methods are able to escape the potential well in which they start and make theirs centroids traverse the empty space between the left and right clusters to reach their rightful position. For each of the three parameter families represented on Figure  6 , 1000 datasets with 500 observations each are simulated, and the two EMs are ran with both the barycenter and the 2v1 initialisation. In the case of tmp-EM, the oscillating temperature profile is used with parameters T 0 = 5, r = 2, a = 0.
Experiment on real data: Wine recognition dataset To further validate tmp-EM, we compare it once more to the unmodified EM, this time on real observations from the scikit learn  [6]  classification data base "Wine"  [4]. This dataset contains p = 13 chemical measurements of n = 178 wines each belonging to one of K = 3 families. Despite being in high dimension, this dataset is known as not very challenging (the classes are separable) and useful for testing new methods. We expect the unmodified EM to perform quite well already. For tmp-EM, we use the simple decreasing temperature profile, with no oscillations, the tempering parameters are T 0 = 100, r = 4. Table  4  shows the result of 500 runs of the EMs from different random initial points. We focus on the likelihood and the error on µ k , the other relevant metrics, not presented here, show the same tendencies.
We observe, as usual, that tmp-EM reaches in average a lower negative log-likelihood with lower variance. The class centres are also better estimated. As expected, the errors made by the EM are already fairly small, however tmp-EM manages to go further and lower the errors on each class by approximately 17%, 18% and 11% respectively. The results demonstrate that tmp-EM can improve the EM result on real data. Since this is an easy dataset, the difference is not as drastic as in the hard synthetic cases we ran the EMs by. Still, there was room to improve the EM results, and tmp-EM found those better solutions.
ZIP code We apply this IFA algorithm to the ZIP code dataset from Elements of Statistical learning. This dataset contains handwritten digits between 0 and 9. In this study, we keep only the digits 0,3, 8 (all three being ambiguously similar) and 7 (very different from the three others). We make all classes even by removing half of the 0 which are originally more numerous. When applying Independent Factor Analysis to such data, one hopes that the distinct digits will be identified as the separable sources making up the signal. We run the IFA model with a Mixture of Gaussians model with a regular and a tempered EM. In the mixing model used, each mixture is composed of two classes. The tempering was made with the oscillating profile, with hyper-parameters: T 0 = 50, b = 20, r = 3, a = 0. 02. Figure  17  displays the estimated sources by the IFA procedure with either EM or tmp-EM at their core.
Acknowledgments The research leading to these results has received funding from the  European Research Council (ERC)  under grant agreement No  678304 ,  European Union 's  Horizon 2020 research and innovation program  under grant agreement No  666992  ( EuroPOND ) and No  826421  ( TVB-Cloud ), and the  French government  under management of  Agence Nationale de la Recherche  as part of the  "Investissements d'avenir" program , reference  ANR-19-P3IA-0001  ( PRAIRIE 3IA Institute)  and reference  ANR-10-IAIHU-06  ( IHU-A-ICM ).
In addition, we generated mouse single-cell transcriptomics data to contextualize our human findings with the mouse counterpart, facilitating translational research between these species. Nature | Vol 607 | 21 July 2022 | 541 transcriptomes of corresponding mouse tissue around the time of sex determination, that is, at embryonic days (E) 10. 5, 11. 5 and 12. 5 (63,929  cells), and integrated them with a previously published dataset covering later gestational stages (E11. 5 to postnatal day (P) 5)  5  (Supplementary Table  1 ). Male and female samples were analysed separately and cell annotation was assigned on the basis of the expression of known markers and label transfer from scRNA-seq to scATAC-seq  6  (Fig. 1c , Extended Data Figs. 1a-d  and 2a -d and Supplementary Note 1). The abundant number of cells profiled in our study allowed us to resolve new somatic cell states, which were not defined in a previous human gonadal scRNA-seq study  7  (Extended Data Fig.
TFs modulating germ cell differentiation PGCs colonize the human gonads at roughly 3-5 PCW  8, 9  and, guided by the male and female supporting cells, start their differentiation into either prespermatogonia or oogonia at roughly 6 PCW. To compare the differentiation of human germ cells with that of other mammals, we integrated our human and mouse gonadal germ cells with more scRNA-seq gonadal germ cells datasets from mouse and macaque  5, 10, 11  (Extended Data Fig. 4a-d , Supplementary Table  4  and Supplementary Note 3). We used trajectory reconstruction methods to trace the differentiation of human PGCs into prespermatogonia and oocytes (Extended Data Fig. 4e ), and investigated the TF programmes that mediate these transitions. We prioritized those TFs that were differentially expressed and active in humans using both the transcriptome and open chromatin data (Extended Data Fig.
5a-d ) and compared their expression dynamics between humans, macaques and mice (Extended Data Fig. 5e  and Supplementary Table  5 ). We identify GATA4 as a primate-specific TF upregulated in PGCs. In all species analysed, we find SOX4 is active in PGCs and prespermatogonia but downregulated during oogenesis. In addition, the transition from PGCs to prespermatogonia involves the activation of EGR4, KLF6 and KLF7. Fetal oocyte differentiation is more complex than its male counterpart: it involves meiosis initiation and a spatial trajectory, with PGCs restricted to the outer cortex and cells migrating towards scRNA-seq (n = 32,889) datasets. Clusters for mesothelial, supporting and gonadal mesenchymal LHX9 + cells were defined in an independent per-lineage reanalysis and projected onto this dataset (Fig.
4c , Extended Data Fig. 8e ,f and Supplementary Table  7 ). OSE activates the primate-specific TF LHX2, which is kept active in preGC-IIa (Fig. 4c ). As they differentiate, preGC-IIb cells upregulate FOXL2 and express WNT-induced TFs (HIF1A + , FOXO1 + , FOXP1 + ), a programme shared by medullary preGC-Is, suggesting there is a higher WNT environment deeper in the ovary. Developing granulosa cells in primates upregulate the steroid hormone receptor NR1H4 and the developmental factor PBX3. To study how human pregranulosa cells in the distinct cortical and medullary microenvironments could influence germ cell differentiation, we expanded our CellPhoneDB database to (1) include non-peptide ligands and (2) link receptors with their downstream TFs (CellSign module) (Extended Data Fig. 8g , Supplementary Table  8  and Supplementary Note 6).
Two testis-specific resident macrophages Tissue-resident macrophages have a role in mouse testicular development and function  22, 23. To comprehensively characterize them in humans, we sorted cells from 11 samples using the pan-leukocyte marker CD45 and integrated them with immune cells from the main analyses (Fig. 5a , Extended Data Fig. 9a-f , Supplementary Table  1  and Supplementary Note 7). 9] [30]  ) (Fig. 5a ,b, Extended Data Fig. 9g, h , Extended Data Fig. 10a  and Supplementary Table  9 ). SIGLEC15 + and TREM2 + ftMs are rare populations in comparison to the tissue-repair macrophages characteristic of all developing tissues (2. 8% SIGLEC15 + ftMs, 5% TREM2+ ftMs, 92. 2% tissue-repair macrophages). Integration and projection using SVM of scRNA-seq datasets of myeloid cells in other developing organs  28, [31] [32] [33] [34] [35]  onto our gonadal immune manifold validated the shared transcriptomics profile between SIGLEC15 + ftMs and osteoclasts, and between TREM2 + ftMs and microglia (Fig.
Cell sorting Dissociated cells were incubated at 4 °C with 2. 5 µl of antibodies in 1% FBS in Dulbecco's PBS without calcium and magnesium (Thermo Fisher Scientific, 14190136). To isolate CD45 + and CD45 -cells, we used the antibody CD45-BUV395 BD Bioscience 563791 Clone HI30 (RUO) Flow cytometry (dilution 2. 5 µl:100 µl). 4,6-Diamidino-2-phenylindole (DAPI) was used for live versus dead discrimination. Cells were sorted using a Becton Dickinson (BD) FACS Aria Fusion with five excitation lasers (355, 405, 488, 561 and 635 nm red), and 18 fluorescent detectors, plus forward and side scatter. The sorter was controlled using BD FACS DIVA software (v. 7), and FlowJo v. 10. 3 was used for analysis.
Alignment and quantification of sc or snRNA-seq data For each sequenced scRNA-seq library, we performed read alignment to the 10X Genomics' GRCh38 v. 3. 1. 0 (human) or Mm10-2020 (mouse) reference genomes, quantification and initial quality control using the Cell Ranger Software (v. 3. 1, 10X Genomics) using default parameters. For each sequenced multimodal snRNA-seq library, we performed read alignment to the 10X Genomics' GRCh38 v. 3. 1. 0 (human) reference genome, quantification and initial quality control using the Cell Ranger ARC Software (v. 1. 0. 1, 10X Genomics) using default parameters. Cell Ranger filtered count matrices were used for downstream analysis. Downstream scRNA-seq analysis Doublet detection. We used Scrublet for cell doublet calling on a per-library basis. We used a two-step diffusion doublet identification followed by Bonferroni-false discovery rate (FDR) correction and a significance threshold of 0.
We identified highly variable genes (n = 2,000) using Seurat v3 flavour on the raw counts, which were used to correct for batch effect with single-cell variational inference (scVI) v. 0. 6. 8. In the analysis of human scRNA-seq, we corrected for sample source and donor effect in both the main and the germ and somatic reanalysis. In the analysis of mouse scRNA-seq we corrected for sample effect and origin of the dataset, this last if combined with external data (below). The resulting latent representation of each cell in the dataset was used for neighbour identification, Leiden clustering and uniform manifold approximation and projection (UMAP) visualization. General analysis was done separately on males and females in each species. Germ, gonadal somatic, endothelial and immune cells were subsequently reanalysed integrating both sexes into the same manifold, using the approach described in the previous paragraph.
Furthermore, gonadal somatic cells from samples at the time of sex specification (younger than CS23) were further reanalysed for fine-grained annotation and validation. Mouse gonad data. We combined our in-house mouse raw counts matrix with the raw count matrices from the ovarian samples profiled by Niu et al. 5  , comprising E11. 5 to P5 developmental stages (GSE136441)  5. For the Niu et al. 5  dataset, we excluded cells that expressed fewer than 1,000 genes or more than 20% of mitochondrial genes. For the analysis of mouse germ cells, we also included the mouse dataset generated by Mayère et al. 10  , which contains germ cells from mice from E10 to E18 developmental stages (GSE136220)  10. ENSEMBL gene IDs provided by the authors were converted to gene names using the appropriate genome build (GRCm38.
p5). We filtered out cells that expressed fewer than 1,000 genes or more than 20% of mitochondrial genes. Next, we concatenated male and female mouse germ cells data from our general analysis (already including data from Niu et al. 5  ) with the germ cells dataset from Mayère et al. 10  , keeping the genes shared between the three datasets. The resulting matrix was integrated by sample and origin of the dataset using scVI on the basis of the procedure described above. Macaque gonads data. In addition, we downloaded a macaque dataset profiling fetal ovaries at stages E84 and E116 (GSE149629)  11  and included it in our cross-species comparison of germ and female somatic cells. Owing to low sequencing depth, we filtered out cells expressing fewer than 300 genes and more than 20% of mitochondrial genes.
As for mice, macaque gene identifiers were converted to human genes using ENSEMBL Biomart multi-species comparison filter. Genes with several mappings were discarded. Annotation of scRNA-seq datasets cross-species. Identification, labelling and naming of the unbiased clusters was carried out on each species individually using a manual approach that we validated using a SVM classifier (see Cross-species comparison section below). For the manual approach, we first identified cluster-specific genes that we used to classify clusters into main cell types on the basis of bona fide marker genes previously reported in the literature. Next, we refined the annotation accounting for the spatiotemporal dynamics in each sex. To identify marker genes specific to a cluster, we used the TF-IDF approach from the SoupX package v.
Mouse gene identifiers were converted to human genes using ENSEMBL Biomart multi-species comparison filter. Genes with several mappings were discarded. Furthermore, genes associated with the cell cycle were removed to avoid biases. Before training the model, human cell types were downsampled to the cell type with the lowest number of cells to obtain a balanced dataset. Here, 75% of the data were used for training the model and 25% of the data were used to test the model. Raw counts were normalized and log-transformed, and the 300 most highly variable genes were selected. We then trained an SVM classifier (sklearn. svm. SVC) on human data and projected the cell type annotations onto the mouse datasets. By doing so, we obtained a predicted probability value that each cell in the mouse and macaque dataset corresponded to every given human cell type annotation.
To study the transcriptomic similarity of a given cell type across species, we compared the estimated probabilities between human-mouse matching cell types and visualized them with boxplots. A detailed description of the workflow used for cross-species comparison is reported in Supplementary Note 2. Agreement with external human gonads data. We evaluated the consistency between the main lineages identified in our study with the Smart-seq2 dataset of gonadal cells from Li et al. 7  (GSE86146). From Li et al. 7  , we downloaded the normalized transcripts per million matrix and annotated their cells using the 'FullAnnot' field provided in the S1 table of the publication. We used the scmap tool  57  to project the Li et al. annotations onto our dataset, using a similarity cut-off of 0. 5 to retrieve high confidence alignment, on each sex separately.
To speed up computational times, we downsampled our dataset to 50% size. Li et al. 's annotations were visualized onto the male and female UMAPs, respectively. To validate the new ESGCs population, we queried the 10X scRNA-seq dataset of developing testis from GSE143356 (ref. 58 ) analysed by Guo et al. 59. Here, we downloaded the raw expression count matrix, and excluded cells expressing fewer than 300 genes and more than 20% of mitochondrial genes. We carried out downstream analysis as previously described for UMAP visualization. Finally, we trained a SVM classifier (sklearn. svm. SVC) on our early human male somatic cells (<CS23) and projected cell type annotations onto the somatic cells identified by Guo et al. 59  in equivalent stages (6, 7, 8 PCW only). The label transfer workflow is analogous to that described for cross-species comparison (Supplementary Note 2), except for the initial ENSEMBL gene ID conversion, which is not necessary in this case because we are transferring labels between human datasets.
Analysis of immune cells in the gonads. Cell Ranger filtered count matrices of CD45 + enriched samples were processed using the workflow described above for the main scRNA-seq analysis (doublet detection, alignment of data across different batches with scVI and clustering). These cells were then merged with the cluster of immune cells from the non-enriched samples. The resulting clustered manifold was preliminary annotated by transferring labels from a publicly available dataset of human fetal liver haematopoiesis  31. Developing liver scRNA-seq raw counts were downloaded from ArrayExpress (E-MTAB-7407), processed with Scanpy v. 1. 7. 0 workflow described above for the main scRNA-seq analysis and filtered on the basis of the expression of CD45 (PTPRC) to exclude non-immune cells. We then trained a SVM classifier (sklearn.
svm. SVC) on the filtered liver dataset and used it to predict cell types on our gonadal immune dataset. The label transfer workflow is analogous to that described for cross-species comparison (Supplementary Note 2), except for the initial ENSEMBL gene ID conversion, which is not necessary in this case as we are transferring labels between human datasets. Predicted cell type annotations were validated or disproved by looking at the expression of known marker genes. To study the unique profile of our gonadal macrophages, we downloaded immune cells from several developing tissues: liver, skin, kidney, yolk sac, gut, thymus, placenta, bone marrow and brain  28, [31] [32] [33] [34] [35]. Raw sequencing data were downloaded from ArrayExpress (E-MTAB-7407, E-MTAB-8901, E-MTAB-8581, E-MTAB-0701, E-MTAB-9801) or Gene Expression Omnibus (GEO) (GSE141862).
For all datasets, we filtered out cells expressing fewer than 300 genes and more than 20% of mitochondrial genes. Downstream data analyses for these datasets were performed with the Scanpy v. 1. 7. 0 workflow analogously to what is described in the main scRNA-seq analysis section above. Myeloid cells from fetal liver, skin, kidney, yolk sac, gut, thymus, placenta, bone marrow and brain datasets were selected on the basis of the expression of established myeloid markers (CD14, CD68, CSF1R). We then combined the resulting myeloid dataset with our gonadal myeloid cells and used scVI with a combined batch of donor and sample to integrate across the different organs. Projection of fetal osteoclasts from Jardine et al. 35  and microglial cells from Bian et al. 29  onto our immune dataset was done using an SVM model.
Similarly, we trained an SVM model on our gonadal macrophages and projected the cell type annotations onto fetal testicular myeloid cells from Chitiashvili et al. 58. The label transfer workflow is analogous to that described for cross-species comparison (Supplementary Note 2), except for the initial ENSEMBL gene ID conversion, which is not necessary in this case as we are transferring labels between human datasets Trajectory inference in the germ and early somatic lineages. For both germ and early somatic cells, we modelled differentiation trajectories and conducted pseudotime analysis by ordering cells along the reconstructed trajectory with Palantir (v. 1. 0. 0)  60  following their tutorial. In brief, cells were subsampled to balance cell type and sex contribution (n = 500 for germ and n = 150 for somatic cells).
The top 2,000 highly variable genes were used for PCA. Next, we determined the diffusion maps from the PCA space (with five top components), and projected the diffusion components onto a t-SNE low dimensional embedding to visualize the data. Finally, we used the function run_palantir (with num_waypoints = 500) to estimate the pseudotime of each cell from the root cell. The barcode with the highest normalized expression of POU5F1 (PGC marker) or UPK3B (mesothelial marker) was used as the cell of origin in the germ and early somatic analyses, respectively. Terminal states were determined automatically by Palantir. For samples at the time of sex specification, we computed RNA velocities  61  to model early somatic development with scVelo (v. 0. 2. 4)  62  following their tutorial. Analysis was done on each sample separately in humans and mice.
To further characterize the cell fate decision process in an unbiased way, we leveraged the RNA moments with the CellRank package (v. 1. 5. 1). Specifically, CellRank uses a random walk model to learn directed, probabilistic state-change trajectories and determine initial and terminal states. We set the number of terminal states to four, letting CellRank determine the number of initial states. We extracted the fate probability of each cell ending up in one of the terminal states.