dataesr/scientific-paragraphs-categorization huggingface dataset:
Data Availability The references of the original papers and their bibliographical information, the annotation guidelines sources, and the developed code are available at the GitHub repository https://github. com/lfoppiano/SuperMat. The guidelines are freely accessible at https://supermat. readthedocs. io. The data transformation scripts were written in Python and can be run from the command line. They require Beauti-fulSoup (https://www. crummy. com/software/BeautifulSoup/), an open-source library for parsing XML and HTML formats. The data analysis scripts were developed as Jupyter notebooks (https://jupyter. org/) which can easily output results and graphs in the browser. The open source annotation tool is INCEpTION (https://inception-project. github. io/). The content was acquired using biblioglutton (https://www.
Conclusions In this paper we described the construction of an annotated linked dataset from scientific publications on superconductors development. SuperMat aims to establish a solid infrastructure where to build or improve TDM processes in superconductor ma-terials domain. We annotated 142 full-text articles where the data was automatically extracted from the PDF document and encoded through the XML TEI guidelines providing a basic structure of the original document. The dataset is validated by domain experts and provides 16052 entities of six categories, and 1398 links between materials, properties and conditions. This approach can be extended to other materials domains following similar methodology.
Acknowledgements We would like to thank  Tanifuji Mikiko  for her continuous support, as well as the enthusiasm and the openness with which she lead the Data PlatForm Data Center (DPFC, https://www. nims. go. jp/eng/research/materials-data-pf/index. html) at  NIMS. Our warmest thanks to  Patrice Lopez , the author of Grobid (https: //github. com/kermitt2/grobid)and other TDM open-source projects.
Phenotype The design of the probes was carried out with the Agilent SureSelect application (www. agilent. com/genomics/suredesign, accessed on 4 January 2017). The list of genes and the genomic positions of control SNPs were submitted to SureDesign for the choice of capture probes according to the following specifications: cover the exons of the genes of interest as well as 25 bp of intronic sequence on either side of the exons and without the UTR regions, with 3 probes per nucleotide in the target regions (3× tiling). Coding regions of all the transcripts of the genes of interest were obtained by SureDesign via several sources (Refseq, ensembl, CCDS). The genomic coordinates of the proposed design were assessed in the UCSC genome browser  [14]  by using genome assembly Human GRCh37/hg19.
Data Analysis Sequencing data was analyzed by STARK (Stellar Tools from raw sequencing data Analysis to variant RanKing), our in-house bioinformatics pipeline. STARK adopts the Genome Analysis Toolkit (GATK) recommendations  [18]. It performs reads demultiplexing, alignment to the reference human genome (GRCh37/hg19), indel (insertion or deletion) realignment, bam recalibration, variant detection (calling), and variant annotation steps. It takes as input the raw sequencing data (BCL, FASTQ, or unaligned BAM) and generates annotated results (VCF) as well as quality reports. SNV/indels were then annotated and ranked using VaRank  [19]. VaRank incorporates the annotations retrieved by the Alamut Batch software (Interactive Biosoftware, France) as well as allele frequency from our internal exome database.
Annotations include HGVS nomenclature (genomic, cDNA, and proteic), and population database frequencies from the 1000 Genomes Project (  [20] , http://www. 1000genomes. org/, accessed on 15 February 2021) and gnomAD databases (  [21] , http://gnomad. broadinstitute. org/, accessed on 15 February 2021). Data processing and analysis were performed as described previously with minor changes  [22]. Copy number variants (CNV) have been called using the CANOES program  [23]  and annotated using AnnotSV  [24]  with similar databases as SNV/indel, but also including DGV and dbVar. Variant filtering has been carried out according to allele frequency in 1000 Genomes Project data and in GnomAD (filtering out allele frequency when >1%). Variant classification was performed following guidelines for the interpretation of sequence variants according to the American College of Medical Genetics and Genomics (ACMG)/Association for Molecular Pathology (AMP)  [25].
We consider each transmission mode without prioritizing one. Pathogenicity of missense variants were predicted via available software like Provean/SIFT (http://provean. jcvi. org/, accessed on 15 February 2021  [26] ) and PolyPhen-2 (http://genetics. bwh. harvard. edu/pph2/, accessed on 15 February 2021  [27] ). Adapted ACMG/AMP guidelines for single-gene copy number variants were used for interpretation of CNV  [28].
Identity Control and Confirmation of Mutations The identity of samples was systematically checked via control SNPs by real-time PCR and then compared with the sequencing results. The predesign TaqMan PCR assay was performed independently on the extracted DNA. The amplification was executed on the patient series in simplex with a blank PCR and a control DNA with known genotype according to manufacturer's protocol (FastStart TaqMan ® Probe Master, Merck KGaA, Darmstadt, Germany). Selected candidate variations have been confirmed via Sanger sequencing. Primers for Sanger have been designed using the Primer3Plus online program for the region harboring the identified variation. Primer pairs were checked via In-SilicoPCR (https: //genome. ucsc. edu/cgi-bin/hgPcr, accessed on 8 February 2020) and primer BLAST (https://www.
Acknowledgments: We thank  Timur Gürgan  and  Senior Clinical Embryologist Halil Ruso  from  Gürgan Clinic Women's Health and IVF Center, Ankara, Turkey , and the IVF specialist from  CMCO, Strasbourg University Hospital, Strasbourg, France , for their assistance with the clinical data for patients with mutations. This study also involved samples from  Germethéque Biobank  (Toulouse, France). We would like also to thank  Robert Drillien  for his critical reading of this manuscript.
Data Availability Statement: Data generated in this study are included in the following article and corresponding Supporting Information. The raw sequencing data generated in the course of this study are not publicly available due to the protocol and the corresponding consents used that did not include such information. All variants have been submitted to ClinVar (SCV001478453, SCV001478457, SCV001478463, SCV001478464, SCV001478466, SCV001478469, SCV001478470, SCV001478471) (https://www. ncbi. nlm. nih. gov/clinvar).
The mission will offer great advantages, in terms of sensor characteristics, compared with existing ocean colour sensors: 242 bands from 420-2450 nm at 6. 5 nm in the visible and near-infrared (VNIR) and 10 nm in the shortinfrared (SWIR), SNR ≥ 400:1 in the visible (VIS), spatial resolution of 30 m, and a revisit period of 4 days with 30 • across-track viewing. We tested Polymer using HICO data acquired at seven coastal areas encompassing a wide range of optical characteristics, and where multispectral or hyperspectral in situ radiometric measurements are available. In addition, we compared its performance against that of L2gen. Finally, we implemented Polymer in the EnMAP-Box, an open source plugin for QGIS for the processing and analysis of EnMAP data, and tested the application using simulated EnMAP imagery.
Satellite Data and Validation HICO  [22, 23]  level 1B and level 2 images were obtained for the seven sites from NASA's ocean colour website  [24]. The level 1B data set was processed with Polymer version 4. 11, using the default settings. The Polymer-normalised water-leaving reflectance was converted to Rrs by dividing by π  [1]. The HICO level 2 data set corresponds to Rrs products processed with the hyperspectral-L2gen  [6] , and was used for comparison. To derive high-quality match-ups, a strict quality control was applied, including a time window of ±3 h between in situ measurements and HICO overflights and a spatial homogeneity check over a 3 × 3 pixel box centered on the location of the in situ data. All nine pixels were checked, regarding the quality control flags provided by the AC algorithms.
These match-ups were excluded from the multispectral match-up data set before computing the statistics; however, we further looked at these unusual data points and found that, in both cases, the flags 'case-2' and 'inconsistency' were raised. Normally these flags do not invalidate the results; particularly 'case-2', which is raised when the optimisation uses initial values adjusted for 'case 2' water areas/pixels  [37]. The inconsistency flag is raised, if at any band, ρ atm or t*ρ + w exceeds the ρ TOA corrected for Rayleigh scattering  [37]. As these were the only cases in the multispectral match-up data set with this flag combination, we searched in the hyperspectral match-up data set for similar cases. There were three match-ups with both flags (i. e. , 'case-2' and 'inconsistency') raised, but the values of Rrs were in the range of the other match-ups.
EnMAP-Box The Polymer algorithm is currently being integrated into the EnMAP processing tool (EnPT,  [38] ) of the EnMAP-Box  [39] , using the wrapper module ACwater. The EnMAP-Box is a software toolbox for processing hyperspectral data, used as a plugin of the QGIS [40] software, and provides the interface to the processing suites developed for imaging spectroscopy. EnPT is a pre-processing chain to process EnMAP data from Level 1B to Level 2A, which can be used as a standalone Python package or within the EnMAP-Box. It features radiometric, geometric, and atmospheric corrections, as well as orthorectification. AROSICS (An Automated and Robust Open-Source Image Co-Registration Software for Multi-Sensor Satellite Data,  [41, 42] ) is used to detect and correct spatial misregistrations. Atmospheric correction is performed with SICOR (Sensor Independent Atmospheric Correction of optical Earth observation data from multi-and hyper-spectral instruments,  [43] ) over land surfaces and with Polymer over water surfaces.
The module ACwater is embedded in the EnPT suite, which contains the specifications and technical framework required to apply the Polymer algorithm to EnMAP data. A critical part of the implementation is the application of the Polymer algorithm to the two separate EnMAP detectors (VNIR and SWIR) in sensor geometry, as they have a spatial shift of around 20 pixels along-track and 1 pixel across-track, depending on the image position. The ACwater module for running Polymer within EnPT is publicly available, along with its documentation and installation instructions, at  [44]. A first application of Polymer to simulated EnMAP data is presented in Figure  9 , which shows that Polymer can provide realistic atmospheric correction over water. The Rrs values are in the same range of those observed at the different AERONET-OC sites and Pensacola Bay (Figure  2 ).
Conclusions Exploring the fine spectral resolution of hyperspectral sensors can help to improve current water colour remote sensing retrievals, going beyond standard algorithms/retrievals (e. g. , band ratio/chlorophyll-a concentration), for such applications as the distinction of different phytoplankton functional groups and water optical constituents. However, the quality of hyperspectral radiometric retrievals depends, in part, on successful AC. In this scope, we presented the first evaluation of hyperspectral radiometric satellite retrievals from Polymer AC, based on HICO data collected between 2010 and 2014 at seven study sites. Polymer HICO retrievals were validated against in situ multi-and hyper-spectral data. The performance statistics of the Polymer AC showed good agreement with in situ measurements.
Acknowledgments: The NASA Ocean Biology Processing Group is acknowledged for HICO level 1 and 2 satellite data. We thank all the principal investigators responsible for the AERONET-OC sites: B. Holben, S. Ahmed, A. Gilerson, G. Zibordi, H. Feng, H. Sosik, B. Gibson, R. Arnone, and S. Ladner. The HyperSAS data were supplied by Darryl Keith and Blake Schaeffer from the U. S. Environmental Protection Agency Office of Research and Development. We also thank Karl Segl for the simulated EnMAP scene and the three anonymous reviewers for their constructive comments.
Scalability and sustainability are the corner stones to unleash the potential of beyond fifth-generation (B5G) ultra-dense networks that are expected to handle massive and heterogeneous services. This implies that the transport of the underlying raw monitoring data should be minimized across the network, and urges to bring the analysis functions closer to the data collection points. While federated learning (FL) is an efficient tool to implement such a decentralized strategy, real networks are generally characterized by time-and space-varying users distributions, traffic profiles, and channel conditions. This makes the data collected across different points non independent and identically distributed (non-IID), which is challenging for FL tasks. To cope with this issue, we first introduce a new a priori metric that we call dataset entropy, whose role is to capture the distribution, the quantity of information, the unbalanced structure and the "non-IIDness" of a dataset independently of the models.
This entropy is calculated using a clustering scheme based on a similarity matrix defined over both the features and the supervised output spaces, and is targeting classification as well as regression tasks. The FL aggregation server then uses the reported dataset entropies to devise i) an entropy-based federated averaging scheme, and ii) a stochastic participant selection policy to significantly stabilize the training, minimize the convergence time, and reduce the corresponding computation cost. Numerical results are provided to illustrate all these advantages.
To cope with this issue, it has been proposed to distribute the available data publicly to participants. However, such a dataset may not always exist, or the participants may refuse to download them for security reasons. Thus, an alternative solution is to construct an approximately IID dataset using inputs from a limited number of privacy insensitive participants  [6]. In the Hybrid-FL protocol, the server asks random participants if they allow their data to be uploaded. During the participant selection phase, apart from selecting participants based on computing capabilities, participants are selected such that their uploaded data can form an approximately IID dataset in the server, i. e. , the amount of collected data in each class has close values. Thereafter, the server trains a model on the collected IID dataset, and merges this model with the global model trained by the participants.
B. Contributions In this paper, our contribution is two-fold. • We first introduce the concept of the entropy of a dataset in both classification and regression tasks, where we jointly consider the features and supervised outputs to characterize the distribution of its samples and the underlying quantity of information based on a custom spectral clustering strategy. This generalized entropy captures the diversity of a dataset as well as its unbalanced structure and non-IIDness. • By leveraging the introduced entropy as an a priori information, we devise two novel FL strategies, namely, i) Entropy-weighted federated aggregation which involves all the CUs in the FL training task while prioritizing the most balanced and uncorrelated datasets (i. e. , those maximizing the entropy) and ii) Entropy-driven stochastic policy for selecting only a subset of CUs to take part in the FL task.
II. NETWORK DESCRIPTION AND DATA COLLECTION A. Edge-RAN As depicted in Fig. 1 , the considered network corresponds to a beyond 5G edge-RAN under the central unit (CU)/distributed unit (DU) functional split, where each transmission/reception point (TRP) is co-located with its DU, while all CUs are hosted in an edge cloud where they run as virtual network functions (VNFs). Each CU k (k = 1,. , K) performs RAN key performance indicators (KPIs) data collection to build its local dataset D k = {x (i) k , y (i) k } D k i=1 of size D k , where x (i) k stands for the input features vector while y (i) k represents the corresponding output. Given that this dataset is generally nonexhaustive to train accurate analytical models, the CU takes part in a federated learning task wherein an operational subsystem (OSS) server-located at the core cloud-plays the role of a model aggregator.
B. Data Collection Table  I  shows the features and the supervised output of the local datasets, which have been collected from a live LTE-Advanced (LTE-A) RAN with a granularity of 1 hour. The considered TRPs cover areas with different traffic profiles-both in space and time-that tightly depend on the heterogeneous users distribution and behavior in each context (e. g. , residential zones, business zones, entertainment events,. ). On the other hand, the radio KPIs are correlated with the time-varying channel conditions. These realistic datasets are therefore non-IID, which is more challenging for FL algorithms as studied in  [11].
III. PROPOSED ENTROPY-BASED FEDERATED LEARNING To tackle the FL convergence in practical non-IID setups, we seek an objective and compressed metric capturing both the distribution of a dataset and its quantity of information, while not depending on the local models. In this regard, we introduce the notion of dataset entropy that is a sufficient statistic to characterize the unbalanced structure of a dataset, as well as its independence from other datasets. Specifically, the entropy is maximized under a uniform distribution with low probability mass function (PMF). By relying on the a priori entropies of all CUs, the aggregation server can implement novel CUs selection and models combining schemes to accelerate and stabilize the FL convergence.
A. Dataset Entropy Since we are targeting a generalized definition of the entropy, the labels of a classification dataset are not reliable to reflect the distribution of data since it does not apply to regression tasks where the supervised output is continuous, and it omits the effect of the input features. In particular, samples with different feature values but presenting approximately similar outputs are not providing the same information and might not necessarily fit in the same group of data. Therefore, in order to accurately discern the samples, we consider a joint approach where both the features and the supervised output are used. To that end, each CU uses a clustering algorithm that operates on the socalled similarity matrix S k whose entries measure the logical correlations between the dataset samples vectors including both the features and the supervised output, i.
e. , x(i) k = x (i) k y (i) k. ( 1 ) This matrix is built using a radial basis function (RBF) kernel with parameter σ. As such, the (i, j)-th matrix element is given by s (k) i,j = exp   - d x(i) k , x(j) k σ 2   , (2) where d stands for the pairwise logical distance between samples' vectors i and j. Since basic clustering algorithms usually require the target number of clusters as an input, we resort to the well-established self-tuning spectral clustering (STSC) technique that uses the eigenvalues and eigenvectors of the similarity matrix. This enables to automatically cluster a dataset into an appropriate number of clusters that minimizes a custom cost function defined in terms of the coefficients of a rotated and normalized version of matrix S k  [12]. Let us assume that for CU k, the clustering yields n k clusters C k,1 ,.
, C k,n k with probabilities Pr (C k,1 ) ,. , Pr (C k,n k ) over dataset D k. The corresponding entropy is then defined as ε k = - n k p=1 Pr (C k,p ) log{Pr (C k,p )}. (3) By letting the CUs report their dataset entropies {ε k } K k=1 to the aggregation server before starting the training, it becomes possible to devise advanced entropy-driven FL strategies that prioritize the CUs with high entropy datasets.
In this case, the model averaging at round t is performed as W (t+1) = k∈{k1,. ,km} D k D W (t) k. ( 7 ) Where D is the total samples over all CUs datasets. This entropy-driven stochastic policy is summarized in Algorithm 1, where L(•, •) stands for the mean square error (MSE) loss function, and b is the bias, while the rest of FL setting parameters is provided in Table  II. s (k) i,j = exp - d x(i) k ,x (j) k σ 2 , 1 ≤ i, j ≤ D k # Clustering CU k clusters D k based on the eigenvetors of matrix S k return n k , Pr C k,1 ,. , Pr C k,n k # Calculate Dataset Entropy ε k = - n k p=1 Pr C k,p log Pr C k,p CU k reports ε k to the aggregation server end parallel for # Federated Learning # Server Generates Probability Distribution for k = 1,. , K do π k = exp{ε k } K l=1 exp{ε l } , k = 1,. , K end Server initializes W (0) with random Gaussian weights for t = 0,.
A. Settings and Baselines The structure of the global model weights matrix W has been defined by the server to satisfy the findings of  [13] , where the authors have estimated the required number Q of neurons per layer based on the number H of hidden layers, the dataset sizes D k , and the number of features F as Q = F + max k=1,. ,K D k H , (8) which is confirmed via Fig. 3 4 , where the best setting of the DNN model neurons turns out to be Q = 4 for H = 3. As a benckmark, the performance of our proposed approaches is compared with LossFedAvg  [11]  and FedAvg  [3]. FL settings are listed on Table. II, where FL system consists of K = 6 DUs running local DNN with a learning rate η = 0. 001 for T = 20 rounds.
B. Numerical Results Analysis Figures 5a and 5b illustrate the gains achieved by the entropy-weighted approach compared to the baseline FedAvg and LossFedAvg. The comparison is done for both balanced and unbalanced non IID datasets. As showcased in Table  III , the entropy metric varies in balanced datasets, since the clustering technique takes into account the correlation between features as well as the supervised output. In the unbalanced scenario, the entropy difference between CUs is even clearer and demonstrates also that datasets with smaller size can sometimes yield more clusters compared to larger datasets, which further corroborates the role of the introduced entropy metric in characterizing a dataset efficiently. A slightly lower losses are met with the entropy-weighted approach rather than the entropy stochastic policy, but both methods have the same convergence trend.
V. CONCLUSION In this paper, we have introduced a novel a priori metric termed dataset entropy to characterize the distribution, the quantity of information, the unbalanced structure and the "non-IIDness" of a dataset independently of the models. This entropy is calculated via a generalized clustering strategy that relies on a custom similarity matrix defined over both the features and the supervised output spaces, and supporting both classification and regression tasks. The entropy metric has been then adopted to develop i) an entropy-based federated averaging scheme, and ii) a stochastic CU selection policy to significantly stabilize the training, minimize the convergence time, and reduce the corresponding computation cost. Numerical results have been provided to corroborate these findings.
: K, m, η, T , L. # See Table II parallel for k = 1,. , K do # Calculate RBF-based Similarity Matrix entries RBF Kernel-Based Self-Tuning Spectral Similarity Matrix Clustering Probability Datasets' Distribution Entropies Fig. 2. Entropy-Driven Stochastic Federated Learning Policy. Algorithm 1: Entropy-Driven Stochastic Federated Learning Policy. Input.