dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements. We thank  Don Kurtz  for valuable comments, which have significantly improved the quality of the manuscript. Financial support from the  National Science Centre in Poland  under project No. UMO-2017/26/E/ST9/00703  is acknowledged. SC acknowledges financial support from  Centre National d'Études Spatiales (CNES, France)  and from the  Agence Nationale de la Recherche (ANR, France)  under grant  ANR-17-CE31-0018. PN acknowledges support from the  Grant Agency of the Czech Republic  ( GA ČR 22-34467S ). The Astronomical Institute in Ondřejov is supported by the project RVO: 67985815. This paper includes data collected with the TESS mission, obtained from the MAST data archive at the  Space Telescope Science Institute (STScI). Funding for the TESS mission is provided by the  NASA Explorer   Program.
STScI  is operated by the  Association of Universities for Research in Astronomy, Inc. , under  NASA  contract  NAS 5-26555. This paper uses observations made at the  South African Astronomical Observatory (SAAO). This paper uses observations made at the  Nordic Optical Telescope (NOT). This research has made use of the SIMBAD database, operated at  CDS, Strasbourg, France. This work has also made use of data from the  European Space Agency (ESA)  mission Gaia (https://www. cosmos. esa. int/gaia), processed by the Gaia Data Processing and Analysis Consortium (DPAC, https://www. cosmos. esa. int/web/gaia/dpac/consortium). Funding for the DPAC has been provided by national institutions, in particular the institutions participating in the Gaia Multilateral Agreement. V. V. G. is a F. R. S. -FNRS Research  Associate.
To better understand the biology of pig embryos before implantation, we generated a large dataset of single-cell RNAseq at different embryonic stages (early and hatched blastocyst, spheroid and ovoid conceptus) and proteomic datasets from corresponding uterine fluids. These data were cleaned, filtered and represent a total of 34,888 cells. We first characterised the embryonic and extra-embryonic cell populations and their evolution, and identified population-specific markers of the three main populations (epiblast, trophectoderm and hypoblast). Our analysis also confirmed known functions and predicted new biological functions associated with these cell populations. We then inferred gene regulatory networks working on modules of gene regulation (regulon) and selected those that were specifically active in each embryonic population.
We confirmed the relevance of the identified regulons by a meta-analysis of two other scRNAseq datasets (porcine and human preimplantation embryos). We then linked these regulons to signalling pathways and biological processes. Our results confirm the molecular specificity and functionality of the three main cell populations and identify novel stage-specific subpopulations. In particular, we discovered two previously unknown subpopulations of the trophectoderm, one characterised by the expression of LRP2, which could represent a subpopulation of progenitor cells, and the other, expressing many proapoptotic markers, which could correspond to the cells of the Rauber's layer. We also provide new insights into the biology of these populations, their reciprocal functional interactions and the molecular dialogue established with the maternal uterine environment.
These drastic changes, which occur before implantation, are likely controlled and coordinated by key functional interactions between cells and tissues. Until recently, the study and interpretation of these interactions has been hampered by the lack of molecular tools and datasets that allow the implementation of systems biology approaches. Most of the knowledge about these interactions was inferred from observations made in other mammalian species, in particular mice, humans or cattle  [3] -  [5]. Classical genes and pathways known to control early embryonic and extra-embryonic cell specification have been tested in cultured pig embryos or explants  [6] -  [9] , but it has been difficult to gain a deeper understanding of such mechanisms due to the difficulty of accessing pig embryos at late stages of pre-implantation development, to the lack of bona fide embryonic and extra-embryonic stem cells and to the difficulty to perform functional genomics.
In the ICM, the IL6-STAT3 and PI3K-AKT pathways are mainly activated and regulate the expression of markers of naive embryonic pluripotency (KLF4, ESRRB, STAT3). When the EPI forms, these two signalling pathways decrease in favour of the TGFβ-SMAD2/3 pathway, which in turn regulates the expression of markers of From this dataset, to identify transcriptionally distinct cell populations, we performed a dimensionality reduction and clustering approach using Principal Component Analysis (PCA) and Uniform Manifold Approximation and Projection (UMAP) following the Harmony and Seurat workflows. To identify the distinct cell populations, we then visualised known population marker genes from the literature for ICM (AICDA, GGCX, HTT), EPI (POU5F1, ID1, DNMT3B), TE (DAB2, KRT8, PTGES), HYPO (COL18A1, GATA6, SERPINH1) and putative early embryonic mesendoderm (RBP4, FZD5, GPX3) (Figure  1C  and 1D ).
In parallel, we performed the same analysis on two publicly available scRNAseq datasets from pig and human pre-implantation embryos  [3] ,  [11]  and we looked for common regulons across these scRNAseq analyses (Supplementary Figure  1 ). We then generated a selection of the most specific regulons for each cell population based on the Regulon Specificity Score (RSS) (Supplementary Table  4b ).
Timely diversification of TE cell population with distinct molecular functions To further characterise the TE cell population, we selected 18,239 cells from our dataset corresponding to the TE population and performed dimensionality reduction followed by a new clustering (Figure  2A ). This led to the identification of eight TE subpopulations (Figure  2B ). All of these subpopulations show a clear TE signature by expressing GATA2, GATA3, DAB2 and PTGES (Figure  2C ), but each has specific characteristics. At the early blastocyst stage (E5), the first cell lineage decision leads to the formation of the early trophectoderm (TE Ea, light green dots in Figure  2B ). This subpopulation is characterised by the expression of early TE marker genes such as DAB2, GATA2 and PTGES  [11] ,  [14] ,  [15]  (Figure  2A  and Supplementary Table  3a ).
Hypoblast specification and differentiation To further characterise the HYPO cell population, we selected 15,335 cells from our dataset, corresponding to this population and performed dimensionality reductions followed by a new clustering (Figure  3A ). This led to the identification of four HYPO subpopulations (Figure  3B ). All of these subpopulations show a clear HYPO signature by expressing of COL18A1, SOX17, GATA4 and APOE (Figure  3C  and Supplementary Table  3a )  [11] ,  [35]  but each one presents distinct features. These four clusters can be separated into two distinct groups according to developmental time and expression of specific genes (Figure  3A ). The first group includes cells from E7 and E9 embryos and comprises early hypoblast (HYPO Ea, pink dots in Figure  3B ) and intermediate hypoblast subpopulations (Hypo In, green dots in Figure  3B ).
4. Pluripotency states follow pig epiblast development. To further characterise the ICM and EPI cell populations, we selected 1,155 cells from our dataset corresponding to these populations and performed dimensionality reductions followed by a new clustering (Figure  4A ). This revealed two distinct subpopulations (Figure  4B ), which differ in terms of cell proliferation: more cycling cells were observed in the EPI compared to the ICM (84% vs. 60% in G2/M and S phases, Supplementary Figure  3 ). The first subpopulation is mainly composed of cells from the early blastocyst stage (E5) (Figure  4A ) and corresponds to the ICM based on the expression profiles of genes associated with naive pluripotency (e. g. , ESRRB, KLF4, PDGFRA and STAT3  [11] ,  [14] ,  [15] ) (Figure  4B -4C and Supplementary Table  3a ).
UPTI (uterine plasmin/trypsin inhibitor), DAG1 and LRP2 are also known to be expressed by endometrial cells  [62] -  [64] , supporting the idea of a reciprocal loop of activation for similar signalling pathways between the endometrium and the conceptus to synchronise both for implantation. By using ligand-receptor analysis, we then associated proteins detected in the UFs (in yellow) with previously identified receptors (in blue) expressed by EPI or TE cells from E9 to E11 embryos in the scRNAseq dataset (Figure  6C ). In particular, we linked the expression of DSC2 in uterine fluid to the expression of DSG2 by TE cells. DSC2 has been found to interact with several desmoglein receptors and DSC2 is required for the formation of the blastocoel in bovine embryo  [65]. We have also highlighted the importance of protein of the extra-cellular matrix like FN1, COL28A1, which can activate integrin signalling, like ITGB1 in EPI cells (Figure  6C  and Figure  5A ).
Conclusions Our study provides new insights into the formation of the first cell lineages of the early pig embryo from single-cell gene expression datasets from E5 to E11 embryos. In particular, our data reveal unsuspected dynamic evolution and heterogeneity within extraembryonic cell populations. First, we confirmed the major difference between the early blastocyst at E5 and the later blastocysts from E7 onwards, both for ICM/EPI and early TE cells, whose transcriptional profiles are very different from the later lineages. This also suggests that the time window between E5 and E7 is crucial for the segregation of the three major lineages in the pig species and would deserve a more advanced and detailed analysis to understand the molecular mechanisms involved. For the later blastocysts, whose biology differs from that of primates and rodents, we have highlighted previously unknown subpopulations, notably from E9 onwards, at the ovoid blastocyst stage.
Production of pig embryos All the embryos were produced at the INRAE experimental unit GenESI (Rouillé, France). All the metadata associated with the biological samples used in this study have been submitted to FAANG Data Portal (https://data. faang. org/home) and BioSamples (https://www. ebi. ac. uk/biosamples/) and are summarized in the supplementary Table  1. Two distinct protocols were used for the production of pig embryos. A first batch of embryos (7 and 9 days after artificial insemination) was produced following superovulation (Supplementary information, Table  1 ). For each sow, estrous cycle was synchronized using Altrenogest (Regumate), a synthetic progestin during 18 days. The day after the end of the Altrenogest treatment, sows were superovulated using a first injection of gonadotrophin (1,200 UI PMSG) followed 72h later by an injection of 500 UI hCG.
They were stunned by electronarcosis and bled. The uterus was clamped and rapidly extracted from the abdominal cavity. Then, the embryos were collected into two tubes of 50 ml by retro-flushing of the uterine horns from the bottom of the horn (uterus) upwards (ovary) in 100 ml of physiological saline solution. The full protocol has been submitted on the FAANG Data Portal and is publicly accessible using the following link: https://api. faang. org/files/protocols/samples/INRAE_SOP_PLUS4PIGS_EMBRYOS_SAMPLI NG_PROTO2_20230131. pdf.
Preparation of single-cell suspensions Once recovered, embryos were staged, pooled and transported in Embryo holding Media (IMV Technologies) for E5 and E7 embryos or DMEM/F12 for E9 and E11 embryos to the molecular biology laboratory in a thermostatically controlled chamber at 38°C. At arrival, embryos were transferred to a 4-well dish and staged again under a stereomicroscope. When necessary, embryos of the same stage were then pooled together into a drop of DMEM/F12 or IMV Embryo holding media and processed for cell dissociation. The full protocol is publicly accessible on the FAANG Data Portal https://api. faang. org/files/protocols/samples/INRAE_SOP_PLUS4PIGS_EMBRYOS_DISSO CIATION_PROTO3_20230131. pdf. Briefly, the zona pellucida (ZP) of E5 embryos was removed by transferring embryos into drops of 0.
Production of scRNAseq libraries and sequencing Dissociated cells were washed in DMEM/F12, counted and resuspended in PBS-0. 4 % BSA according to 10X Genomics protocol: Chromium Next GEM Single Cell 3' Reagent Kits v3. 1 CG000204 Rev D (for embryos sampled in 2021) or Chromium Single Cell 3' Reagent Kits v2 CG00052 Rev F (for embryos sampled in 2017). A BioAnalyzer (or FragmentAnalyzer) profile and a Qbit quantification were performed for each sample after cDNA amplification and after library amplification. The libraries were sequenced on an Illumina HiSeq 3000 (batch1) and BGI DNBSEQ-G400 (batch 2) to obtain 144M to 328M of raw reads per library. The full protocol has been submitted on the FAANG Data Portal and is publicly accessible using the following link: https://api. faang. org/files/protocols/experiments/INRAE_SOP_PLUS4PIGS_scRNASEQ_LIB RARIES_PROTO4_20230228.
Production of an extended annotation to cover 3' UTR from porcine transcripts Raw sequencing files were mapped to the Sus scrofa genome (11. 1) using the Nextflow pipeline TAGADA (github. com/FAANG/analysis-TAGADA). Gene positions were annotated as per Ensembl build 102 and genes were filtered based on their biotype annotation to only contains genes matching one of this category: protein-coding, long intergenic non-coding RNA, antisense, immunoglobulin or T-cell receptor. We then used the quant3p script (github. com/ctlab/quant3p) to extend genes in the 3' exon for those where reads aligned past annotated genes. We used genome parameter set to 1,341,049,888 and bam file alignment. Some additional modifications to the annotation were performed including gene deletion, addition or positions change.
These changes are summarized in the project's GitHub repository (https://github. com/Goultard59/pig_embryo_scrnaseq/blob/master/1_generating_matrices/RE ADME_annotation_extension. md) and the gtf file is available there: https://github. com/Goultard59/pig_embryo_scrnaseq/blob/master/1_generating_matrices/Sus _scrofa. Sscrofa11. 1. 102_10_26. filtered. gtf. Then CellRanger (version 6. 1. 1) was executed on this annotation to produce count matrices.
Single-cell RNA sequencing analysis Raw gene expressions were converted to a Seurat object using the Seurat R package (version 4. 3. 0) [75]. Cells were removed if they had more than 10,000 or fewer than 500-300 expressed genes, or if >25-10% of their UMIs were derived from the mitochondrial genome with adjustment between samples (Supplementary Table  2 ). After filtering, the gene expression matrices for each sample from the same stages were merged, normalized and scaled using Seurat's function with the default settings. We then performed data integration using Harmony  [76]. To reduce the dimensionality of this dataset, gene expression matrices were analysed by principal component analysis (PCA). The first 10-20 principal components were further used as input with adjustments between stages (Supplementary Table  2 ) for UMAP dimensionality reduction using the RunUMAP function with the selected number of principal components and default parameters.
Secondary analysis Functional analysis (Gene Ontology, Human Phenotype Ontology, miRBase and Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment) was performed in order to obtain the most significant pathways for each cell population using the GSVA package  [77]  with parameters set to ssgea method and sz minimum of 1. A linear model was then applied to the output matrix, followed by empirical Bayes statistics for differential enrichment analysis between cell types (Supplementary Table  S3c ). Then, DEG analyses were performed in a pairwise fashion between the different stages (Supplementary Table  S3b ) and between cell populations (Supplementary Table  S3a ), using the FindMarkers function of the Seurat package, with a filter set at 0. 05 for p-value and at 25% for the minimum percentage of cells expressing DEG in the given identity.
SCENIC analysis In order to run SCENIC on the pig genome, a custom database for RcisTarget and GRNboost was built. Transcription factor lists were generated following the methods used for the AnimalTFDB 3. 0  [81]  with the pig Ensembl build 102. The motif to Transcription Factor annotation was adapted to pig by gene orthology using OrthoFinder  [82]. The motif database was built using aertslab scripts (github. com/aertslab/create_cisTarget_databases) with the best transcript score for each gene of the genomes based on the 10kb upstream regions. The library of motifs used in this manuscript is comprised of 10,560 PWMs from several sources  [83]. The SCENIC pipeline was run using the VSN pipeline with an aggregation of 10 runs  [84]  (github. com/vib-singlecell-nf/vsn-pipelines). For each regulon, RSS was computed based on the specificity results  [85]  (Supplementary Table  4b ).
Meta-analysis comparisons Processed scRNAseq expression matrix from Meistermann et al. (2021) datasets were downloaded from https://gitlab. univ-nantes. fr/E114424Z/meistermannbruneauetalprocessed using raw counts and annotations of cells. Raw sequencing reads (fastq files) from Zhi et al. (2021)  [11]  datasets were downloaded from Genome Sequence Archive (GSA) using accession number CRA003960. Reads were then split using sabre (https://github. com/najoshi/sabre) with the parameters pairedEnd mode and max mismatch of 2. We used the barcode list from the Supplementary information Table  1  of  [11]  as the input list of barcodes used for sabre. Reads were then trimmed using Trim Galore  [86]. Finally, the reads were mapped using the same script used for the human dataset, available at (https://gitlab.
univ-nantes. fr/E114424Z/SingleCell_Align) with an hisat index based on 11. 1 pig reference genome, a pig annotation reference ENSEMBL 104 and pairedEnd mode. For SCENIC analysis, the previous pipeline was applied to Zhi et al. datasets (Supplementary Figure  9 ). For Meistermann et al. , we applied the previous pipeline with human adaptation as the v9 Motif2TF annotations and the feather file-based hg38 with TSS+/-10kbp was used (Supplementary Figure  8 ). Then a comparison between the transcription factor identified in those two and ours was performed (Supplementary Figure  10 ).
Ligand Receptor analysis Ligand to transcription factor pathways was inferred using CellComm from FUSCA  [87]. Transcription factors were retrieved from our previous SCENIC analysis. Ligands and receptors were identified with LIANA packages  [88]  where gene names were converted from humans to pigs with OmniPath  [89]. The package was run to use sca, natmi, logfc, connectome, call_italk and call_connectome methods. Inference between all population at all stages was made with default parameters following CellCom tutorials. Protein-protein interaction was obtained from OmniPath with the conversion from human to pigs (Supplementary Table  5a  and 5b ). After interactions inference, the top 20 ranked pathways by possible autocrine or paracrine interactions were kept for visualization. In order to produce circos plot visualization of interactions between the different populations, we considered the ligand to receptor activity score (Supplementary Table  5a ) corresponding to the matching criteria.
MS spectra were acquired using a top-15 data-dependent acquisition method on a Q Exactive HF-X mass spectrometer. Protein identification and quantification was performed with MaxQuant (v. 1. 6. 1. 0) and the NCBI RefSeq Sus scrofa database. For the database search, the following parameters were used: enzyme: Trypsin/P; missed cleavages ≤ 2; 4. 5 ppm mass tolerance for precursor main search; 20 ppm mass tolerance MS/MS search; carbamidomethylation of cysteine as fixed modification and acetyl (Protein N-terminus) as well as oxidized methionine as variable modifications. Label-free quantification was used.
Proteomics data analysis The output table proteinGroups. txt from MaxQuant was loaded into Perseus  [90]  v. 1. 6. 7 for downstream analyses. Data were filtered to remove contaminants, reverse peptides that match a decoy database and proteins identified only by modified peptides. The matrix was normalised by log 2 , and the samples were categorized in three categories early (E5), intermediate (E7) and late (E10). Proteins were kept if there were present in 70% of at least one category. Finally, missing values were replaced by a normal distribution (Supplementary information, Table  S6 ). To assign protein names, we used the most identified protein IDs (Majority Protein IDs from MaxQuant output), then converted the RefSeq Protein Accession to gene symbol and ENSEMBL gene pig using gprofiler.
The remaining ambiguous proteins were LOC100521789, assigned to AMY2, LOC110259139, assigned to PCDHA11, and ARF3, assigned to ARF1. PCA were performed using the R function autoplot from the package ggfortify  [91]. Differential expression tests between early and late categories were produced using Perseus Volcano plot function with parameters t-test, both sides, 250 randomizations, an FDR threshold of 0. 05 and a S0 of 0. 1. Interaction between uterine fluids and receptor from TE and EPI were produced using LIANA : single cell data from EPI and TE at stages E9 and E11 were pulled together with 200 « artificial cells » produced from protein differentially enriched in the late categories. All cells were processed on LIANA using OmniPath ressources and orthologue genes converted using gprofiler and methods connectome, logfc, natmi, sca, cellphonedb, cytotalk, call_squidpy, call_cellchat, call_connectome, call_italk.
Acknowledgments We are grateful to the genotoul bioinformatics platform  Toulouse Occitanie  ( Bioinfo Genotoul , https://doi. org/10. 15454/1. 5572369328961167E12 )for providing help, computing and storage resources. We are grateful to people from the  INRAE  experimental farm (https://doi. org/10. 15454/1. 5572415481185847E12 )who took care of the animals. We are grateful to  Sylvain Bourgeais  and the people from the slaughterhouse of the  Unité Expérimentale de Physiologie Animale de l'Orfrasière INRAE  (https://www6. val-deloire. inrae. fr/uepao). We thank  Marie-Jose Mercat  from the  IFIP, Institut du porc  for her helps and funding. We also thanks  Sarah Djebali ,  Nathalie Beaujean ,  Nathalie Vialaneix ,  Camille Berthelot  and  Laurent David  for sharing data, codes and pipelines and for their helpful suggestions.
Data availability All raw sequencing data and associated metadata are available in FAANG Data portal (https://data. faang. org/home) and ENA under accession number PRJEB60517. The code used for the analysis is available at (github. com/Goultard59/pig_embryo_scrnaseq/). The mass spectrometry proteomics data have been deposited to the ProteomeXchange Consortium via the PRIDE  [93]  partner repository with the dataset identifier PXD042421.
Regarding sparse matrix hardware accelerators, they often employ specialized architectures and algorithms that leverage the Sparsity of the matrix to minimize memory usage and optimize computational operations. They can provide efficient storage and processing of sparse matrices, allowing for faster matrix-vector multiplications, matrix factorization, and other operations commonly used in scientific computing, machine learning, and graph analytics. They play a crucial role in accelerating computations involving sparse matrices, enabling faster and more efficient processing of data with a reduced memory footprint and improved performance. The objective of the FVLLMONTI project (fvllmonti. eu), of which this research work is a part, is to create 3D stacked hardware layers for neural networks.