dataesr/scientific-paragraphs-categorization huggingface dataset:
However, optimizing the average accuracy over a set of diverse relations may not be relevant  (Gladkova, Drozd, and Matsuoka 2016). For this reason, we did not try to optimize them. The hyper-parameters we used are: min_count=3, maxn=0, window size=5 , model=skipgram, epoch=20, lr=0. 05. The other parameters are set to their default values. Results. Table  3  gives the results obtained with the method by analogy. The MRR scores are high, all above 0. 6 showing that analogy is efficient for the detection of synonymy. Quadruplets included in quadruplet_RefIATE yield the best results with a MRR score of 0. 744. This good result may be due to the fact that the simple terms and the biterms in quadruplet_RefIATE come from the same database, i. e. , IATE. Another interesting outcome is the similar results obtained with the quadruplets built from specialized environment databases and from general language dictionaries.
Acquiring other types of lexical relations We are also interested in acquiring other types of lexical relations between MWTs, in particular opposition relations and hierarchical relations which are considered to be essential terminological relations (L'Homme 2020). However, because no terminological resource provides such relations for the MWTs of the environmental domain, we created a dataset of such relations by semantic projection. We noticed that more than 80% of pairs of synonymous biterms extracted from IATE share a lexical word. Semantic projection seems therefore suitable for the identification of lexical relations if we admit that the pairs of biterms that have one of their constituents in an opposition or hierarchical relation also share this characteristic. In this section, we present the creation by semantic projection of a dataset of four lexical relations between nominal biterms.
Generation of semantically related biterms by semantic projection The creation of the dataset involves several steps. We first extracted nominal term candidates that contain two lexical words from the PANACEA corpus using TermSuite  (Cram and Daille 2016). Among these, we have kept only those whose frequency in the PANACEA corpus is above 5. We then formed pairs of biterm candidates using semantic projection. We applied the inference rule in (1) to the pairs of STs from RefDicoenviro in order to form pairs of candidate biterms, without any additional restriction on the order of the constituents nor on their syntactic structure. More formally, let M W T 1 and M W T 2 be two biterm candidates such that voc(M W T 1 ) = {ST 1 , W 1 } and voc(M W T 2 ) = {ST 2 , W 2 } where voc(x) is the set of the lexical words of x.
For example, voc(protection de la flore 'flora protection') = {flore 'flora', protection 'protection'}. If ST 1 and ST 2 are two STs connected by a relation R in RefDicoenviro and if W 1 = W 2 , then we predict that M W T 1 and M W T 2 are also connected by R 14. Semantic projection can be stated as (1) where M is the set of biterms that occur in PANACEA, S the 14 Provided that the meaning of the two biterms is compositional. set of STs and L is the set of the lexical words that occur in PANACEA. ∀M W T 1 ∈ M, ∀M W T 2 ∈ M, such that ∃ST 1 ∈ S, ∃ST 2 ∈ S, ∃W ∈ L, ∃R, (1) [voc(M W T 1 ) = {ST 1 , W } ∧ voc(M W T 2 ) = {ST 2 , W } ∧ R(ST 1 , ST 2 )] ⇒ R(M W T 1 , M W T 2 ) For symmetric relations (e. g. , synonymy, antonymy), when the semantic projection yields both R(M W T 1 , M W T 2 ) and R(M W T 2 , M W T 1 ), we only kept one pair in the dataset.
We then checked that the candidates are terms by looking up in the online terminology banks TERMIUM Plus 15  , Le Grand Dictionnaire 16  and IATE 17. We consider any candidate that belong to one of these banks and to a field falling under the environmental domain as a term of the environment domain. As a result, we ended up with 80 pairs of biterms linked by ANTI, 51 pairs linked by HYP and 100 pairs linked by QSYN. The dataset is small because many of the terms extracted by TermSuite are too specific (e. g. , conservation des papillons 'butterfly conservation') to be present in one of the banks. We will call these 231 quadruplets DataProjSem 18. The biterm pairs in DataProjSem have then been manually checked by three annotators in order to assess whether the relation between the STs is preserved between the biterms.
Acquiring the other lexical relations by means of masked language models Test dataset used in the MLM experiments. The MLM experiments are evaluated by means of a test set made up of the quadruplets of the DataProj where both STs belong to the vocabulary of the model. This subset contains 61 pairs of ANTI, 16 pairs of hypernymy, 16 pairs of hyponymy and 71 pairs of QSYN. Experimentation. We performed an MLM experiment like the one presented in Section 4. 2 using 100 contexts (or less) extracted from the PANACEA corpus for each biterm that appears in one of the quadruplets of the test set. As before, we used the CamemBERT-large model. Results and discussion. In this experiment, the rank of the semantically related terms is established with respect to a vocabulary made up of the STs in DiCoEnviro that occur in the PANACEA corpus and in the model vocabulary (a total of 796 STs).
Discussion In this paper, we proposed two DSM-based methods for acquiring lexical relations between biterms. These methods have been evaluated on two datasets: synonyms extracted from IATE and biterm pairs constructed by semantic projection and linked by ANTI, HYP and QSYN. First, it is worth noting that the two datasets are complementary since the intersection of DataIATE and DataProj's QSYN pairs contains only 9 pairs of biterms. This complementarity is probably due to the nature of IATE and DiCoEnviro. IATE's synonyms are intended for translation and writing while DiCoEnviro was created to describe and structure terms in the environment domain. In addition, there is a slight difference in the relations themselves. Some relations labelled as synonymy in DataIATE are hypernymic 19  such as vie sauvage:animaux sauvages 'wildlife:wild animals'; similarly, QSYN contains co-hyponymy relations such as réseau ferroviaire:réseau routier 'railroad network:road network'.
Our two methods perform similarly for synonymy i. e. , on both dataset, analogy works better than lexical substitution. However, the two methods give better results on the QSYN pairs than on the synonymous biterms extracted from IATE, possibly because QSYN biterm pairs created by semantic projection are more constrained: they share the same syntactic structure and the two STs they contain are semantically related. This shows that the semantic compositionality of MWTs is captured by MLM and somehow modeled by analogy. The main difference between our two methods is that MLM predictions are highly contextdependent while analogy predictions are context-independent because the FastText model is not contextual. The latter is therefore more suitable for identifying lexical relations. These observations confirm those of  Peters et al.
(2018)  who show that contextual language models under-perform on semantic relation identification with respect to static embedding models when evaluated on analogy tasks. We also observe that the results of the analogy-based method are better for symmetrical relations than for asymmetrical relations. Two reasons may explain this difference. The first is that we use a symmetrical distance (i. e. , cosine) which might be less suitable for asymmetrical relations. One possible solution could be to use a directional similarity measure such as Weed-sPrec  (Weeds and Weir 2003)   (Weeds and Weir 2003). For example,  Lenci and Benotto (2012)  show that WeedsPrec perform better than cosine for hypernym identification with an Average Precision score of 0. 4 vs 0. 23. The second reason is that the difference between the embeddings of two words does not accurately represent the relation that exists between them.
The best Top 1 accuracy score obtained by  Chaudhri et al. (2022)  for analogy equations containing MWTs is 0. 51 (with the seq2vec ELMo model). It is significantly lower than ours which is 0. 697. Consequently, there is potentially a strong potential for progress that would justify carrying out the same tasks using the methods we propose. Our results can also be compared with those of  Hazem and Daille (2018)  for synonymy. They propose different methods to capture synonymy between MWTs in the domain of wind energy, a field close to that of the environment. Hazem and Daille (2018)  obtain a MAP of 0. 349 for French MWTs using a semantic projection method and a word2vec model in order to take advantage of distributional relations. This value is close to the one we obtained on the dataset extracted from IATE using the lexical substitution based method with a MRR of 0.
We observe that the two methods are quite complementary. Of the 10 quadruplets for which the solution provided by the analogy-based method is beyond the 5 th position, 6 have a solution that is in the TOP 5 for the lexical substitution method. Conversely, of the 66 quadruplets for which the solution provided by the substitution-based method is beyond the 5 th position, 62 have a solution that is in the TOP 5 for the analogy method. However, the size of our dataset is not large enough to enable us to effectively characterize the quadruplets on which the models fail. The polysemous nature of scientific vocabulary is a well-known phenomenon. The environment domain which encompasses several sub-domains results in domain specific meanings of certain terms, Terms such as gaz 'gaz' are used across scientific domains such as energy or chemistry having different meanings.
Conclusion In this paper, we explored the capacity of distributional semantic models to capture lexical semantic relations between MWTs. We concentrated on nominal biterms which are well represented in terminological databases. We used masked language models and analogy in order to predict the lexical relations under consideration. For our experiments, we used two datasets: the first is composed of synonymous biterms extracted from IATE, the second is created by semantic projection using various terminological resources and covering a wider range of lexical relations. The results show that analogy capture synonymy, antonymy and hyponymy better than masked language models. This result suggests that analogy does, to some extent, capture the semantic composition encoded in the static embeddings; conversely, compositional generalization of Transformer-based models seems weaker.
RefIATE is made up of 551 pairs of synonymous simple terms (nouns or adjectives) belonging to the environment domain and extracted form IATE, such as rejet:déversement 'discharge:spill'. IATE (Interactive Terminology for Europe) is a EU's terminology database made up of more of 8 million terms in the official languages of the European Union covering 20 domains. It is primarily intended for the translators working for the EU.
RefIATE is made up of 551 pairs of synonymous simple terms (nouns or adjectives) belonging to the environment domain and extracted form IATE, such as rejet:déversement 'discharge:spill'. IATE (Interactive Terminology for Europe) is a EU's terminology database made up of more of 8 million terms in the official languages of the European Union covering 20 domains. It is primarily intended for the translators working for the EU. • QSYN (523 pairs): synonymy, quasi-synonymy and co-hyponymy (diesel:gazole 'diesel:gas oil'; charbon:pétrole 'coal:oil');.
An integrated cell atlas of the lung in health and disease Single-cell technologies have transformed our understanding of human tissues. Yet, studies typically capture only a limited number of donors and disagree on cell type definitions. Integrating many single-cell datasets can address these limitations of individual studies and capture the variability present in the population. Here we present the integrated Human Lung Cell Atlas (HLCA), combining 49 datasets of the human respiratory system into a single atlas spanning over 2. 4 million cells from 486 individuals. The HLCA presents a consensus cell type re-annotation with matching marker genes, including annotations of rare and previously undescribed cell types. Leveraging the number and diversity of individuals in the HLCA, we identify gene modules that are associated with demographic covariates such as age, sex and body mass index, as well as gene modules changing expression along the proximal-to-distal axis of the bronchial tree.
Mapping new data to the HLCA enables rapid data annotation and interpretation. Using the HLCA as a reference for the study of disease, we identify shared cell states across multiple lung diseases, including SPP1 + profibrotic monocyte-derived macrophages in COVID-19, pulmonary fibrosis and lung carcinoma. Overall, the HLCA serves as an example for the development and use of large-scale, cross-dataset organ atlases within the Human Cell Atlas. Rapid technological improvements over the past decade have allowed single-cell datasets to grow both in size and number 1. This has led consortia, such as the Human Cell Atlas, to pursue the generation of large-scale reference atlases of human organs 2,3. To advance our understanding of health and disease, such atlases must capture variation between individuals that is expected to impact the molecular phenotypes of the cells in a tissue.
Whereas the generation of atlases at this scale by single research groups is currently not feasible, integrating datasets generated by the research community at large will enable capture of the diversity of the cellular landscape across individuals. Several foundational studies have started to map the cellular landscape of the healthy human lung 4-6. These studies each have a specific bias due to their choice of experimental protocol and technologies, and are therefore not tailored to serve as a universal reference. The studies moreover include only a limited number of samples and individuals, thus lacking the scale and diversity to capture the full cellular heterogeneity present within the lung as well as across individuals. Integrated single-cell atlases provide novel insights not obtained in individual studies.
Recent reference atlases have led to the discovery of unknown cell types 7-9 , the identification of marker genes that are reproducible across studies  7, 10, 11  , the comparison of animal and in vitro models with human healthy and diseased tissue  7, 12, 13  and patient stratification for disease endotypes  14, 15. However, many currently available integrated atlases are limited in the number of human samples  7, 8, [10] [11] [12] 16  , datasets 16 or cell types  7, 9, 12, 17, 18  per organ, as well as donor metadata  12, 13, 17, 19, 20  (for example, age, body mass index (BMI) and smoking status), or focus mainly on a specific disease  14, 15, 17. These limitations constrain the potential of atlases to serve as a reference, as they fail to represent and catalog the diversity of cellular phenotypes within the healthy organ and across individuals.
across datasets by mapping the collected cell identity labels for every dataset as provided by the data generator to the hierarchical reference framework, showing varying cell type proportions per sample (Fig. 2c ). To optimally remove dataset-specific batch effects, we evaluated 12 different data integration methods on 12 datasets  [4] [5] [6] [21] [22] [23] [24] [25]  (Fig. 2d  and Supplementary Fig. 1 ) using our previously established benchmarking pipeline  28. We use  the top-performing integration method, scANVI, to create an integrated embedding of all 584,444 cells of 107 individuals from the collected datasets: the HLCA core (Fig. 3a ).
Consensus cell type annotations based on the HLCA core A large-scale integrated atlas provides the unique opportunity to systematically investigate the consensus in cell type labeling across datasets. To identify areas of consensus and disagreement, we iteratively clustered the HLCA core and investigated donor diversity and cell type label agreement in these clusters using entropy scores (see Methods). Most clusters contained cells from many donors (Extended Data Fig. 1a ). Clusters with low donor diversity (n = 14) were largely immune cell clusters (n = 13), representing donor-or donor group-specific phenotypes. Similarly, a high diversity of (contradictory) cell type labels (high label entropy) can identify both annotation disagreements between studies and clusters of doublets (Methods).
Most clusters (61 out of 94) showed low label entropy, suggesting overall agreement of coarse cell type labels across datasets (Fig. 3b ). The remaining 33 clusters exhibited high label entropy, highlighting cellular phenotypes that were differently labeled across datasets (Fig. 3b ). For example, the immune cluster with the highest label entropy contained many cells that were originally mislabeled as monocytes and macrophages but were actually type 2 dendritic cells (Fig. 3c  and Extended Data Fig. 1b ). Thus, populations with high label entropy identify mislabeled cell types, indicating the need for consensus re-annotation of the integrated atlas. As a first step to achieve such a consensus on the diversity of cell types present in the HLCA core, we performed a full re-annotation of the integrated data on the basis of the original annotations and six datasets into a single tissue atlas is a critical step in achieving the goals of the Human Cell Atlas  2.
In this resource, we present an integrated single-cell transcriptomic atlas of the human respiratory system, including the upper and lower airways, from published and newly generated datasets (Fig. 1 ). The Human Lung Cell Atlas (HLCA) comprises data from 486 donors and 49 datasets, including 2. 4 million cells, which we re-annotated to generate a consensus cell type reference. The HLCA expands our understanding of the healthy lung and its changes in disease and can be used as a reference for analyzing future lung data. Together, we provide a roadmap for building and using comprehensive, interpretable and up-to-date organ-and population-scale cell atlases.
Data integration establishes the HLCA core To build the HLCA, we collected single-cell RNA sequencing (scRNA-seq) data and detailed, harmonized technical, biological and demographic metadata from 14 datasets (11 published and three unpublis hed)  [4] [5] [6] [21] [22] [23] [24] [25] 26, 27. These datas ts include samples from 107 individuals, with diversity in age, sex, ethnicity (harmonized as detailed in Methods), BMI and smoking status (Fig. 2a ). Cells were obtained from 166 tissue samples using a variety of tissue donors, sampling methods, experimental protocols and sequencing platforms (Supplementary Tables  1  and 2 ). Anatomical locations of the samples were projected onto a one-dimensional (1D) common coordinate framework (CCF), representing the proximal (0) to distal (1) axis of the respiratory system, to standardize the anatomical location of origin (Fig.
2a  and Supplementary Tables  2  and 3 ). Consensus definitions of cell types based on single-cell transcriptomic data across studies-particularly of transitional cell states-are lacking. To enable supervised data integration and downstream integrated analysis, we harmonized cell type nomenclature by building a five-level hierarchical cell identity reference framework (Methods, Supplementary Table  4  and Fig. 2b ). We then unified cell type labeling Resource https://doi. org/10. 1038/s41591-023-02327-2 expert opinions (consensus annotation; Methods and Fig. 3d ). Each of the 61 annotated cell types (Supplementary Table  5 ) was detected in at least four datasets out of 14, often in specific parts of the respiratory system, and different cell types showed varying fractions of proliferating (MKI67 + ) cells (Extended Data Fig.
The HLCA recovers rare cell types and identifies novel ones Rare cell types, such as ionocytes, tuft cells, neuroendocrine cells and specific immune cell subsets, are often difficult to identify in individual datasets. Yet, combining datasets in the HLCA core provides better power for identifying these rare cell types. Ionocytes, tuft and neuroendocrine cells make up only 0. 08, 0. 01 and 0. 02% of the cells in the HLCA core according to the original labels, and were originally identified in only seven, two and four datasets out of 14, respectively. Despite their low abundance, these cells formed three separate clusters of the HLCA core (Fig. 3f ). Our re-annotation increases the number of datasets in which these cells are detected up to threefold and identifies both cells falsely annotated as monocytes, tuft cells or neuroendocrine cells, as well as originally undetected rare cells (Fig.
c, Cell type composition per sample, based on level 2 labels. Samples are ordered by anatomical region CCF score. d, Summary of the dataset integration benchmarking results. Batch correction score and biological conservation score each show the mean across metrics of that type, as shown in Supplementary Fig. 1 , with metric scores scaled to range from 0 to 1. Both Scanorama and fastMNN were benchmarked on two distinct outputs: the integrated gene expression matrix and integrated embedding (see output). The methods are ordered by overall score. For each method, the results are shown only for their best-performing data preprocessing. Methods marked with an asterisk use coarse cell type labels as input. Preprocessing is specified under HVG (that is, whether or not genes were subsetted to the 2,000 (HVG) or 6,000 (FULL) most highly variable genes before integration) and scaling (whether genes were left unscaled or scaled to have a mean of 0 and a standard deviation of 1 across all cells).
HLCA-based analysis of lung data highlights new cell types The HLCA core contains an unprecedented diversity of donors, sampling protocols and cell identities, and can serve as a transcriptomic reference for lung research. New datasets can be mapped to this reference to substantially speed up data analysis by transferring consensus cell identity annotations to the new data. We tested this on a recently released multimodal lung dataset  40  (Methods, Fig. 6a  and Extended Data Fig. 6 ). Overall, the transferred labels were correct in the majority of cases, with 68% of the cells correctly labeled, 14% of labels incorrectly labeled and 18% set to unknown due to highly uncertain labeling (Fig. 5b  and Methods). Uncertain labels were observed specifically in continuous transitions from one cell type to another and among cellular identities not present in the HLCA core, including rare cell identities (erythrocytes (n = 328), chondrocytes (n = 42), myelinating Schwann cells (n = 7), nonmyelinating Schwann cells (n = 29) and nerve-associated fibroblasts (n = 66); Fig.
Clustering of the projected dataset with the HLCA reference showed that cells expressing the suggested tumor-associated marker ACKR1 were also abundant in healthy tissue from the HLCA core, specifically in venous endothelial cells (both pulmonary and systemic, Fig. 5c  and Supplementary Fig. 6a-c ). This suggests that ACKR1 is a general marker of venous endothelial cells rather than a tumor-specific endothelial cell marker. Similarly, the reported normal endothelial cell marker EDNRB characterizes aerocyte capillary endothelial cells, both in tumor and in healthy tissue (Fig. 5c  and Supplementary Fig. 6d ). As endothelial cell numbers in the original study were low, correctly identifying and distinguishing these cell types without a larger healthy reference is challenging. Thus, by serving as a comprehensive healthy control, the HLCA prevents misinterpretation of limitations in sampling and experimental design as meaningful differences between healthy and diseased tissue.