dataesr/scientific-paragraphs-categorization huggingface dataset:
Then the REI is defined as:  Finally, mREI is obtained by averaging over all severities. REI per,sev net = EI per,sev net -EI per,sev V. The results are described in Table  1  for the CIFAR-10 dataset. The raw error rates under each type of perturbation can be found in the Appendix. We observe that Parseval alone is not able to help with the mREI, despite reducing the clean set error. On the other hand, the proposed regularizer and its combination with Parseval training decreases the clean set accuracy but increases the relative performance under perturbations by a significant amount. This experiment supports the fact that the proposed regularizer can significantly improve robustness to most types of perturbations introduced in  [7]. It is worth pointing out that this finding does not hold for Impulse Noise, Fog, and Contrast.
C) Experiments on challenging benchmarks In this subsection we verify the ability of the proposed regularizer to increase robustness on the CIFAR-10 dataset while being combined with recent techniques of adversarial data augmentation. This is important as those methods are seen as the state of the art for adversarial robustness. Adversarial data augmentation consists of augmenting the training set during the training stage by using the same kind of attacks as those described in the last subsection. We refer to techniques using adversarial data augmentation using the letter A.
D) Experiments with other datasets In this final subsection, we test the generality of the method using the CIFAR-100 and ImageNet32x32 datasets, with a subset of the perturbations used for CIFAR-10. Gaussian noise is applied 10 times to each of the networks for a total of 30 different runs. An SNR of 33 is used for FGSM and 15 for Gaussian noise. Images are normalized in the same way as the experiments with CIFAR-10. Standard data augmentation is used for CIFAR-100. Results on CIFAR-100 are shown in Table  5  as the mean over three different initializations. We observe that as it was laplacian networks: bounding indicator function smoothness for neural networks robustness      the case on CIFAR-10, the proposed method and the combination of the methods is the most robust on these test cases.
The former takes into account both small variations around the training set examples and the margin. The latter enforces small variations of the smoothness of label signals on similarity graphs obtained at intermediate layers of a DL network architecture. We have empirically shown with our tests that the proposed regularizer can lead to improved robustness in various conditions compared to existing counterparts. We also demonstrated that combining the proposed regularizer with existing methods can result in even better robustness for some conditions. Future study includes a more systematic study of the effectiveness of the method with regards to other datasets, models, and perturbations. Recent studies have shown that adversarial noise is partially transferable between models and dataset  [31, 32]  and therefore we are confident about the generality of the method in terms of models and datasets.
CIFAR-10 For the CIFAR-10 dataset we do a 100 epoch training. Our learning rate starts at 0. 1. After half of the training (50 epochs) the learning rate decreases to 0. 001. We do not add data augmentation, as it can interfere with some of the perturbations (elastic transformations for example). We tested multiple parameters of β, the Parseval tightness parameter, γ the weight for the smoothness difference cost, and m the power of the Laplacian. We found that the best values for this specific architecture, dataset, and training scheme were: β = 0. 01, γ = 0. 0001, m = 2. The PreActRes-Net18 network depicted in Fig. A1  is used in all experiments for CIFAR-10.
CIFAR-100 We perform experiments using the WideResNet28-10  [34]  architecture, and we added standard data augmentation (random crops and random horizontal flipping) and dropout with probability of 30 after the first convolution of each residual block. We train for 200 epochs, starting with a learning rate of 0. 1 and divide the learning rate by 5 in epochs 60, 120, and 160. Momentum of 0. 9 is used and weight decay of 5 × 10 -4. We tested multiple parameters of β, the Parseval tightness parameter, γ the weight for the smoothness difference cost, and m the power of the Laplacian. We found that the best values for this specific architecture, dataset and training scheme were: β = 0. 0003, γ = 0. 01, m = 2. FGSM is applied after normalization of the input.
IMAGENET32x32 We perform experiments using the WideResNet28-5  [34]  architecture, and we added standard data augmentation (random crops and random horizontal flipping). We follow the training procedure from  [27] , where training is done for 30 epochs, starting with a learning rate of 0. 1 and divide the learning rate by 5 in epochs 10 and 20. Momentum of 0. 9 is used and weight decay of 5 × 10 -4. We tested multiple parameters of β, the Parseval tightness parameter, γ the weight for the smoothness difference cost, and m the power of the Laplacian. We found that the best values for this specific architecture, dataset and training scheme were: β = 10 -6 , γ = 0. 01, m = 2.
Marked animal data During the study period, the local Alpine ibex colony comprised ca 200 individuals (minimum number alive recorded from block counts: 161-233; adult females only, i. e. , 2 yrs. or older: 63-99; see more information on Alpine ibex conservation status and the studied colony in Appendix S2). From 2010 to 2015, we caught (July-October) 24 adult female ibex by dart-gunning with sedation, and equipped them with GPS-GSM sensors (Vectronic Aerospace GmbH), collecting and transmitting locations, and with 2-axes accelerometers, storing data which were downloadable only at collar retrieval. At capture, we recorded the reproductive status (active lactation), that we confirmed via direct observation (female feeding the kid) during the same season, and re-assessed during the following year (except two females that could not be observed in the second season; Appendix S2).
GPS collars were programmed to attempt a location every hour, and to drop-off after 54 weeks. We obtained locations for 24 individuals (fix success rate: 95%; outliers excluded as in  Urbano & Cagnacci, 2014; median location error <10 m, Párraga Aguado et al. , 2017). We derived activity status from 18 of 24 collars that could be retrieved after drop-off (days with incomplete accelerometer data <1%, excluded from subsequent analysis). We classified accelerometer data with a binomial activity index based on a threshold value  (Gervasi et al. , 2006: 'active'  or 'inactive' status, with 5-minute resolution according to sensor output; Appendix S3). We validated the automatic classification with direct observations between July and September, choosing coarse behavioural categories that would hold throughout the year (Appendix S3).
Statistical analysis We analysed data in R 4. 0. 2 (R Core Team, 2016) using the package mgcv 1. 8-33  (Wood, 2018)  to fit Generalized Additive Mixed Models (GAMM) to our empirical dataset. In all sets of models, we included individual identity as a random factor on the intercept and used Akaike Information Criterion (AIC;  Burnham et al. 2002)  for model selection. The sample size was 24 and 18 animals for analyses based on GPS and activity data, respectively, minus two animals for analyses including reproductive status. Annual behavioural responses to temperature, vegetation productivity and its rate of change (Q1) To evaluate whether activity and altitudinal movement of female ibex varied over the year in relation to thermal shelter and access to resources (Q1, Figure  1A ), we first modelled the pattern of annual variation in 'totalactive-time-per-day' (P1a) and 'mean-altitude-used-perday' (P1b) by fitting GAMM models to the week as linear effect or cyclic cubic smoother (Table  S6.
Modular electronic case report forms (eCRFs) with core, extended, and optional datasets allow data capture tailored to the participating site's variable interests and resources. The eCRFs cover all key data elements required by regulatory authorities [European Medicines Agency (EMA)] and the European Rare Disease (ERD) platform. They capture genotype, phenotype, and progression and include demographic data, biomarkers, comorbidity, medication, magnetic resonance imaging (MRI), and longitudinal clinician-or patient-reported ratings of ataxia severity, non-ataxia features, disease stage, activities of daily living, and (mental) health status. Moreover, they are aligned to major autosomal-dominant spinocerebellar ataxia (SCA) and sporadic ataxia (SPORTAX) registries in the field, thus allowing for joint and comparative analyses not only across ARCAs but also with SCAs and sporadic ataxias.
The registry is at the core of a systematic multi-component ARCA database cluster with a linked biobank and an evolving study database for digital outcome measures. Currently, the registry contains more than 800 patients with almost 1,500 visits representing all ages and disease stages; 65% of patients with established genetic diagnoses capture all the main ARCA genes, and 35% with unsolved diagnoses are targets for advanced next-generation sequencing. The ARCA Registry serves as the backbone of many major European and transatlantic consortia, such as PREPARE, PROSPAX, and the Ataxia Global Initiative, with additional data input from SPORTAX. It has thus become the largest global trial-readiness registry in the ARCA field.
The ARCA Registry was launched in 2013 in order to apply this concept to the field of ARCAs, and it remains the only multicenter registry fully dedicated to ARCAs and early-onset ataxias (EOAs), which are known to be enriched but not exclusive for ARCAs  (1, 2). The overarching goal of the ARCA Registry is to become a key facilitator enabling trial readiness by • providing an easily accessible, web-based, good clinical practice (GCP)-conforming, and general data protection regulation (GDPR)-compliant multicenter multi-trial registry infrastructure platform as a backbone for global trial-readiness efforts in ARCAs; • building cohorts of sufficient size for trial-readiness studies and upcoming treatment trials through aggregating ARCA patients in an accessible, standardized, multicenter fashion around the world; • characterizing the phenotypic spectra for ARCAs, which will inform treatment trial design and especially outcome selection for future treatment trials; • collecting real-world natural history data for ARCAs acquired during daily clinical life across a large range of centers across the world, thereby informing design, planning, and modeling of treatment trials; and • providing a continuous database backbone for trial-readiness ataxia consortia around the world, e.
Participating sites maintain access to their data entered in the ARCA Registry, with the possibility to easily export and systematically analyze locally aggregated datasets. Access to full multisite datasets is provided for specific projects upon request by a standardized project template and provided to the project submitter after evaluation of the request. The physician-reported multidomain datasets in the ARCA Registry are the core of a larger systematic multi-component ARCA database cluster (Figure  1 ). For longitudinal collection of biomaterials, the ARCA Registry is linked to an ARCA biomaterial database built on REDCap. To facilitate wholeexome and whole-genome sequencing in all patients with unsolved ARCA, the ARCA Registry is moreover linked to next-generation sequencing (NGS) data on the genomics research platform GENESIS  (13, 14).
GENESIS is a userfriendly collaborative cloud-based analysis and matchmaking platform that encompasses the largest ataxia NGS dataset collection worldwide (>2,000 ataxia NGS datasets), aggregated via the PREPARE consortium (PREPARE-GENESIS) (see below). While the ARCA Registry and the GENESIS platform are two distinct databases, subjects from the registry are linked to the GENESIS platform via an ID generated by the ARCA biomaterial database. Ongoing developments of this multicomponent ARCA database cluster will include an ARCA multistudy database as a repository for features of digital outcomes such as magnetic resonance imaging (MRI), digital-motor sensors (APDM, Q-Motor), and optical coherence tomography and for patient-driven entry of patient-reported outcome measures (PROMs).
CAPTURING PHENOTYPIC SPECTRA, PHENOTYPIC EVOLUTION, AND DISEASE PROGRESSION OF AUTOSOMAL RECESSIVE CEREBELLAR ATAXIAS: THE ELECTRONIC CASE REPORT FORMS The eCRFs of the ARCA Registry are designed to characterize the clinical heterogeneity of phenotypic spectra and natural history phenotypic evolution of ARCAs, thus helping in the selection of outcomes and planning of upcoming treatment trials (sample size calculation, trial duration, etc. ) as wellmodeling of trial endpoints and treatment effects. Different degrees of eCRFs details-characterized as "core, " extended, " and "optional" datasets-allow data capture tailored to the participating site's variable interests and resources (Table  1 ). In brief, the eCRFs include clinical scales and composite measures, clinician-reported outcome measure and PROMs, biomarker outcomes, and quantitative performance measures: • The core dataset in the ARCA Registry comprises demographic data (with ethnic background), genetic diagnosis (with types of sequencing performed), different scores to measure disease severity like the Friedreich Ataxia Rating Scale (FARS) Functional Stage  (15) , the Scale for the Assessment and Rating of Ataxia (SARA) (  16 ), systematic phenotyping using the Inventory of Non-Ataxia Signs (INAS)  (17)  with customized amendments (e.
g. , bradykinesia, ptosis, or the head impulse test), the presence and onset of typical clinical ARCA features (e. g. , ataxia, epilepsy, cognitive impairment, and diabetes), ARCA biomarkers (serum and neurophysiology), and relevant comorbidities (including alcohol intake). • The extended dataset adds questionnaires on health status and depression (EQ-5D and PHQ-9)  (18, 19) , diseaserelevant medication and treatment effects, and a summary of MRI findings. • Optional datasets include the possibility to report pediatric features (e. g. , pregnancy and birth or developmental milestones), and the ARSACS Disease Severity Index as a disease-specific outcome measure  (20). The Patient's Global Impression of Change (PGI-C; core dataset)  (21, 22)  and the FARS Activity of Daily Living (ADL; extended datasets) (  15 ) have recently been implemented as "anchor measures, " i.
CURRENT DATA IN THE ARCA REGISTRY: A DESCRIPTIVE OVERVIEW OF 800 PATIENTS AND 1,500 VISITS From its foundation in 2013 until now, more than 800 patients with almost 1,500 visits have been recruited to the ARCA Registry. In the past 5 years, there has been a considerable increase in longitudinal data, currently reaching up to eight annual followup visits in the first patients (Figure  3A ). Follow-up core datasets including SARA or INAS from at least two visits and 13 sites are available in >300 patients. Follow-up extended datasets such as MRI summary data or the self-rated assessment of health status by EQ-5D from at least two visits are available from >200 patients. In addition to its longitudinal coverage, the ARCA The ARCA Registry with its phenotypic and longitudinal data has enabled large clinico-genetic cohort studies to delineate the phenotypic spectrum and longitudinal disease progression of major and novel ARCAs.
It thus fulfills the requirements of primary datasets that can be used to describe natural history progression models and plan treatment trials in almost all of the most frequent ARCAs, including pharmacometric modeling of outcome measures and treatment effects. For example, for RFC1-ataxia, it has helped to reveal multisystemic phenotypes mimicking cerebellar type multiple system atrophy and progressive supranuclear palsy, and hyperkinetic movement disorders such as chorea and dystonia, and provided first sample size calculations based on longitudinal SARA assessments  (24). For COQ8A/ADCK3-ataxia, the ARCA Registry facilitated the delineation of clinico-genetic associations, and the longitudinal analysis of SARA scores has provided the first systematic, groupbased evidence for a possible treatment effect of coenzyme Q10  (25).
European Medicines Agency Registry Standards for Data Collection The ARCA Registry captures the key EMA data elements (3), including administrative information (e. g. , site, contact dates, registry entry and exit dates, and reason for registry exit), patient data (e. g. , age, sex, and alcohol as lifestyle factor), disease features (including diagnosis, disease duration, severity/staging, genetic information, and biochemical tests if appropriate), relevant comorbidities, and disease-related or relevant concomitant medical treatment (Table  2 ). The ARCA Registry is also enrolled in the European Directory of Registries of the European Rare Disease Registry Infrastructure (ERDRI. dor). As a constituent registry of the evolving Rare Neurological Disease Registry of the European Research Network on Rare Neurological Diseases (ERN-RND)  (28) , the ARCA Registry will provide the common data elements defined by the ERDRI  (29) , which adds more systematic coding of diagnosis (Orpha code), genetic diagnosis (HGVS) or phenotype (HPO), resources for research (e.
g. , biosampling and link to biobank), and a classification of disability (Table  3 ). As a cross-disease database, the ERN-RND registry collects general information on demographics, genetics, and phenotype, which allows the identification of centers that look after specific (and often genetically defined) patient groups. By contrast, as a disease-group specific database, the ARCA Registry collects complementary in-depth data to enable trial readiness in recessive ataxias. Both registries are well-interconnected, as the common ERN dataset can be easily extracted from the ARCA Registry and imported into the ERN-RND registry. This link with the ERN-RND registry also adopts the FAIR principles in the ARCA Registry, ensuring that its data are f indable, accessible, interoperable, and reusable between countries  (30).
European Medicines Agency Standards for Data Quality In line with the EMA standards of data quality (3), the ARCA Registry aims for consistency, completeness, accuracy, and timeliness. Consistency of data is facilitated by common standardized eCRFs, by clearly defined variables and selection of questions with binary outcomes, and by the implementation of clinical scales with high interrater reliability, especially SARA, INAS, and FARS stage or ADL  (15, 16, 31). Completeness of data in core datasets is automatically checked online as the first step of a continuous database-embedded monitoring process; this resulted in 95% (e. g. , for clinical features of ARCAs) to 99% (e. g. , for SARA or INAS) CRF completion rate. Following automated online control of data plausibility and consistency within and between different CRFs at the time of data entry, accuracy of data is afterwards controlled offline in the second monitoring step.
German Autosomal Recessive Cerebellar Ataxias/Early-Onset Ataxia Network The German network on ARCAs and EOAs, launched in 2013 by the German Center for Neurodegenerative Diseases (DZNE), comprises five major German ataxia sites (Tübingen, Bonn, Munich, Magdeburg, and Rostock). The network has established the first version of the ARCA Registry, which was co-hosted by the Ataxia Study Group. Every effort was made to ensure that the Registry is fully aligned in its data fields and database system with other major SCA and sporadic ataxia registries  (32) , likewise hosted by the Ataxia Study Group. Since then, the German ARCA/EOA network has contributed >400 subjects to the ARCA.
ARCA Global ARCA GLOBAL and its sister platform SCA GLOBAL together comprise the Ataxia Global Initiative (AGI). The AGI presents a worldwide multi-stakeholder (academia, industry, and patient organizations) platform coordinating and preparing all necessary steps for trial readiness in autosomal-dominant (SCA GLOBAL) and autosomal recessive (ARCA GLOBAL) ataxias  (7, 10). With establishing trial-ready cohorts and cross-center harmonized clinician-reported outcome measures and PROMs as one of its key tasks, the AGI uses the ARCA Registry as one of its key registries. This reflects the fact that the ARCA Registry already captures all outcome measures that were stipulated by the AGI as the common core set of clinical outcome measures to be used by ataxia centers worldwide. Moreover, the AGI builds on the ARCA Registry as one of its major trial-readiness resources, as this registry readily allows data-download and dataset preparation for further workup, e.
g. , by the Critical Path to Therapeutics for the Ataxias (CPTA) consortium of the Critical Path Institute (C-Path), which aims to prepare regulatory approval by the FDA and the EMA for clinical endpoints in genetic ataxias. In addition, the SPATAX network, which includes all types (i. e. , not only autosomal recessive) of ataxias and HSPs, has contributed subsets of data to the ARCA Registry.
LIMITATIONS AND OUTLOOK The ARCA Registry faces several limitations and open challenges that remain be to be addressed: • The sustainability of the ARCA Registry depends on strong commitment by the contributing centers as well as projectbased funding, with fluctuations in patient recruitment, participating sites, and monitoring performance. Technical improvements and new software implementations often come along with variable latencies. Even the "core dataset" may exceed the possibilities of a clinical appointment in many ataxia centers, but further minimization of the core dataset would need to be carefully weighed against the minimal required data necessary to really help in preparing trial readiness as well as against registry standards put forward by public authorities. The timeliness of monitoring is governed by batch monitoring of each site by site, which provides the opportunity for focused local revision of data and files but also leads to periodic delays for those sites that had just been monitored.
CONCLUSIONS The ARCA Registry has (i) enabled the harmonization of clinical outcomes across ataxia centers around the world; (ii) has demonstrated its capacity to act as a centralized database for genotype-phenotype and natural history studies in the >100 ARCAs, already exemplified for COQ8A-, RFC1-, and POLG-related ataxias; and (iii) aggregates the necessary largescale longitudinal progression datasets for calculating sample sizes, modeling trial designs and randomization procedures, and running pharmacometric models simulating treatment effect sizes for anticipated clinical trials. Given its adoption by many international ARCA sites and networks; its GCP, GDPR, and EMA compliance; its web-based data capture; and its connections to a constantly growing multi-component ARCA database cluster, the ARCA Registry is well-placed to become a global trial-readiness registry for ARCAs.
The ARCA Registry with its phenotypic and longitudinal data has enabled large clinico-genetic cohort studies to delineate the phenotypic spectrum and longitudinal disease progression of major and novel ARCAs. It thus fulfills the requirements of primary datasets that can be used to describe natural history progression models and plan treatment trials in almost all of the most frequent ARCAs, including pharmacometric modeling of outcome measures and treatment effects. For example, for RFC1-ataxia, it has helped to reveal multisystemic.