dataesr/scientific-paragraphs-categorization huggingface dataset:
It is also expected that research on RDs will increase our understanding of the pathophysiology of common chronic diseases, as RDs often represent a "model of dysfunction" severely affecting a limited number of biological pathways. Identifying the locks is essential. Yet, RD clinical management and research in France and Europe have been hampered by a lack of resources at several levels: few scientists work on only one given disease; few patients per disease and patients are scattered over large geographic areas, causing difficulties in gathering data on their disease; existing databases as well as biological collections, when existing, are usually local, small, incomplete, not always quality-controlled, of heterogeneous formats and contents, and are rarely accessible or standardized for allowing interoperability; phenomes are often complex and partially described along with time, with insufficient interdisciplinary cooperation.
Given the specificities of RDs-limited number of patients per country, scarcity of relevant knowledge and expertise, and fragmentation of research-they have been considered as a distinctive domain of very high national and European added value. A supporting research program was therefore required to provide essential information on disease history and characteristics, and to foster the identification of underlying molecular mechanisms, genotype/phenotype correlations. New knowledge would ultimately lead to better targeted care and treatments. The structuring boost given by the PNMR2 was considered as an opportunity to make collective efforts for building RD cohorts and therefore to propose the RaDiCo (Rare Disease Cohorts) project to the national call on cohorts of the first Investments for the Future Program (https:// anr.
The PNMR2 also fostered the development of the National RD Data Bank ("Banque Nationale de Données Maladies Rares" (BNDMR), Fig. 1 ) built as part of the CEMARA initiative  [8] , which started in 2007 at Necker Enfants-Malades Hospital (AP-HP). The BNDMR, which is dedicated to public health issues, aims at collecting general epidemiological and public health data on all patients with a RD in France, on the basis of a common RD minimum data set  [9]  and a unique identifier  [10]  for each RD patient. The RD cohorts also had to integrate, whenever appropriate, non-French RD expert centers and patients, in order to overcome the low number of patients in one single country with respect to the sample sizes needed for proper statistical power, thereby anticipating the emergence of the future RD European Research Networks.
Results Thirty-three letters of intent were received after the publication of the RaDiCo call for RD cohorts (see Material and Methods). This call led to select 16 national and/or European RD cohort projects on July 15th, 2014. Among these, 3 have been discontinued, secondary to decisions of the Scientific and Plenary committees of the RaDiCo program (see Material and Methods): after demand from the principal investigators (PIs) for two of them, since the cohorts had not started, and another in 2019 because it could not start in due time. The groups of RDs targeted by the 13 current cohorts appear in Table  1. The general framework of the RaDiCo cohorts is presented below (Table  2 ), with or without associated biocollections at each site, together with the planned inclusion and follow-up period.
Patient information in RD registries is generally collected as de-identified data from numerous sources, requiring the data to be federated. Transforming nominative data into de-identified data is thus a key issue, while minimizing the number of identity duplicates. We proposed a method enabling patient identity federation and RD data de-identification while preserving the pertinence of the provided data with a RD patient identifier. A critical issue for cohort information platforms is to enforce strict control on access privileges and particularly for patients themselves. Cerberus, a comprehensive access control Scheme was designed to provide an access control scheme that covers design, implementation, deployment and maintenance operations  [31]. It enables a targeted access to the RaDiCo GENIDA information platform by patients themselves who can capture their own data, or via an access to targeted medical data through explicit authorization by the study sponsor, when required, to practitioners and/or researchers.
Together with other European colleagues, we proposed an overview of the current situation and experiences of the national RD registries in Europe  [32]  and suggested recommendations for improving the quality of RD registries  [33, 34]. These two papers illustrated part of the design of our e-Health approach. Moreover, RaDiCo is involved in the French PNMR3, notably in its 11. 4 Action that reinforces the role of RaDiCo related to the integration of RD research data while developing operational links with the national RD databank by offering technical and ethical-regulatory support for the setting up by the RD health networks of new interoperable data warehouses and by studying the possibility of accommodating them on the RaDiCo platform. We also established operational links with the data platform of the European Joint Program on RD [35] and the ERNs currently being set up.
Discussion The RaDiCo program is intended to promote, through the support of RD e-cohorts' projects, the collection of phenotypic data for epidemiological and clinical research purposes in connection with basic and translational research. To this end, we implemented an operational team (cf. Material and Methods section) with the mission to establish a centralized platform of expert services and tools to ensure installation and follow-up of the RD e-cohorts, via consulting services, for instance legal & regulatory services, or clinical research quality tools, development and provision of innovative information technology tools (electronic case report forms (eCRF), interoperability solutions, or access to e-health tools). The preparatory period of the cohorts was slowed down by the uncontrollable delays required to obtain all the ethical and regulatory authorizations, as well as the evolving regulations both at the EU level, with the new European regulation on personal data safety and security (GDPR)  [36] , and at the national level, with modifications of the law on research involving human subjects (French «Jardé law»)  [37].
Legal contracts were set up for consortium and collaboration agreements, enabling data collection and sharing. At the cohort level, we industrialized our production processes, as well as tooling methods such as standardized operating procedures or documents shared online. Recurring difficulties are the shortage of manpower for data entry, the difficulties in identifying cases in the different hospital information systems, or the legal barriers to collect data from deceased children. We implemented solutions through identifying key data and prioritization of the fields related to the main objectives to be completed by the clinical research technicians with support from residents for filling the medical information. Clinical research technicians have been recruited by the RDHNs and specific resources have been obtained through targeted research projects supported by Public-Private Partnerships or applications to specific grants.
We explored the impact of RaDiCo for the medical community. Based on current information provided by all the cohorts' participating teams, the program should contribute to significant improvements in patients' care and outcome. Indeed, the design of the cohorts is made in such a way that it allows structured collection of disease symptoms at various stages of the pathological processes. Expected benefits include better knowledge of the natural history of all the investigated RDs, recognition of relevant comorbidities, novel proposals for healthcare management, and, ultimately, better quality of life for the patients and their families. In line with the objectives specifically defined for each cohort, other expectations are: identification of relevant disease biomarkers for diagnosis, disease severity and exacerbations; patient selection for clinical and therapeutic trials; and production of novel quality of life questionnaires.
In line with these needs, the Ministry of Health has set up the RD national data bank. All the reference centers and their network of teams in France have the obligation to implement this data bank. Standardized and detailed patient information collected through the cohort programs support the production of economic indicators. Moreover, access to the French Health Data Hub  [38]  that is dedicated to cross-referencing the health databases and facilitating their use by research teams with full respect of user privacy of the health system, will open access to patients' health expenditures from three main databases: the National Health Insurance Fund [39], hospital stays from the French hospital discharge database (PMSI)  [40]  and to the causes of death from the Inserm Centre for Epidemiology on the Medical Causes of Death, in charge of the national statistics of medical causes of death  [41].
Economic indicators aim at monitoring the development and implementation of national strategies for RDs, which imply homogeneity and standardization of patient management by the medical structures dedicated to RDs. All the teams of the RaDiCo cohorts are members of the RD Coordinating Centers in France. Among their tasks, through their organizations and their multidisciplinary teams, all the cohorts actively participate in the production of the RDs National Protocols for Diagnosis and Care  [42]. The ERNs are networks of RD centers of expertise, healthcare providers and laboratories that are organized across borders. Most of the medical teams associated with the RaDiCo program are involved in the activities of the ERN in their field. They participate in the implementation of European data registries, in the production of clinical practice recommendations, and in the setting of epidemiological and economic indicators.
Conclusion The RaDiCo program promotes the collection of RD phenotypic data of different types for epidemiological and clinical research purposes in connection with basic and translational research. The RaDiCo platform offers the cohorts an information system based on the cloud principle (Information as a Service) together with a common core of services and specific procedures for some cohorts to ensure installation and follow-up of the RD e-cohorts. The information system of RaDiCo can drive a virtually unlimited number of RD e-cohorts, a major strength compared to other IS that usually deal with only one cohort. Currently, 13 RD e-cohorts are implemented and other national, European and international cohorts are targeted.
RaDiCo's governance An Institutional Committee is composed of the founding institutions including the following: Inserm (coordinating institution), 6 Universities: Paris Descartes and Paris Diderot now merged in the University of Paris, as well as Sorbonne, Paris-Est Créteil, Aix-Marseille and Montpellier universities; and the « Entreprises du Médicament», together with the Alliance for Research and Innovation of the Industries of Health. These bodies are linked to the RaDiCo program by an agreement established with the National Research Agency. Among other missions, the Institutional Committee regularly follows the implementation of the project including the budget implementation. It validates, on proposal of the Scientific Committee, the annual program of activities. It decides the creation, composition, missions and the functioning of all ad-hoc governing bodies.
The RaDiCo Scientific Committee for the whole Program is composed of 18 members with complementary expertise associating coordinators of RD Coordinating Centers, RD Healthcare Networks, epidemiologists, biostatisticians, experts in Information Systems, molecular geneticists, directors of Inserm Research Units in the field of RDs, and representatives of Inserm Thematic Institutes. Its role is to support the Executive Committee concerning strategic issues and scientific governance of the program, including management of the call for projects and specific follow-up of cohorts; and to contribute to the national call for RD cohort projects launched by RaDiCo in 2014. An Executive Committee is shared by all the cohorts. It is composed of 4 members with complementary expertise (SG, AC, PL and SA).
Its role is to ensure the implementation, development and monitoring of the entire program. Its main missions are operational management and deployment planning of the sustainability and internationalization of the RaDiCo platform, the links with each RD cohort, and factorization of the shares. It is responsible for the scientific management of the Program which is implemented at different levels: by the Executive Committee, the Scientific Committee, the Inserm Thematic Institutes of "Public health", "Technology for Health", "Genetics, Genomics and Bioinformatics", as well as by the 3 scientific governance bodies of each RD cohort. Indeed, each cohort has its own governance, on a scheme comparable to that of the whole program, with a Plenary Committee, a Scientific Committee and an Executive Committee.
The RaDiCo platform The RaDiCo platform is managed by the IS team together with the e-Health team. It uses exchange format and data security in compliance with the European directive on the General Data Protection Regulation. The RaDiCo work plan for each cohort is structured to achieve four levels of interoperability: technological, semantic, syntactic, and institutional. This allows networking and optimising the use of existing RD patient cohorts at the EU level, while allowing integration of new types of data and technologies. The data of each RaDiCo cohort and associated services have to be compliant with the FAIR principles (Findable, Accessible, Interoperable and Reusable) (https:// www. go-fair. org/ fair-princ iples/) by both people and computers. An important step in the FAIR data approach is to publish existing and new datasets from RaDiCo cohorts in a semantically interoperable format that can be understood by computer systems.
The RaDiCo clinical research unit RaDiCo Clinical research unit helps the PIs of the cohorts to finalize their project according to the SPIRIT recommendations (https:// www. equat or-netwo rk. org/ repor ting-guide lines/ spirit-2013-state ment-defin ingstand ard-proto col-items-for-clini cal-trials/), to submit the protocol to the ethical and legal authorities and obtain the corresponding agreements; design the e-CRF (electronic Case Report Form); implement the e-CRF on the IS platform; organize the data capture and the data quality control; monitor the follow-up according to 19 Key Performance Indicators; perform the data management; data freeze the data; prepare and perform the data analysis. Patients are recruited through RD clinical sites for 12 cohorts. One cohort (GENIDA) proposes self-enrollment and self-reporting according to a pre-defined chart.
Collected data for a given cohort were defined in an e-CRF. They are collected along with time (according to each cohort design, either quarterly, semi-annual or annual) and managed via the RaDiCo platform through REDCap (Research Electronic Data Capture) which is a secure, web-based application from the Vanderbilt University, USA. It was designed to support data capture for research studies, providing an intuitive interface for validated data entry; audit trails for tracking data manipulation and export procedures; automated export procedures for seamless data downloads to common statistical packages; and procedures for importing data from external sources.
Acknowledgements Inserm is the sponsor of the RaDiCo cohorts. Study data were collected and managed using REDCap (Research Electronic Data Capture). We warmly acknowledge the RaDiCo team and the PIs of the RaDiCo cohorts:  C. Bodemer ,  P. Calvas ,  N. Chassaing ,  A. Clement ,  C. Corpechot ,  V. Cottin ,  E. Escudier ,  B. Fautrel ,  S. Georgin-Lavialle ,  L. Heidet ,  B. Héron ,  X. Jeunemaître ,  B. Knebelmann ,  A. Linglart ,  B. Maitre ,  J. L. Mandel ,  I. Netchine ,  A. Servais , and  S. Vicart , as well as their colleagues and teams, who contribute to the  RaDiCo program  (https:// www. radico. fr/ en/ accue il).
Communication Energy & Environmental Science were performed via the homemade LabView s based software using a three-point weight comparison method (for details, see Note S7, ESI †). The device measurements were performed at 200 mV s À1 scan speed. EQE measurements were performed using PV-Tools LOANA equipment. When measuring perovskite top cells, the tandem devices were light-biased by IR LEDs (930 nm); when measuring silicon bottom cells, the tandem devices were light-biased by a blue LED to saturate the subcells. To enable the near short-circuit conditions, MPP voltages were applied on the devices. All device characterizations were performed at ambient air with RH B50% and without any encapsulation.
The spectra were analyzed with Casa XPS software. The individual peak envelopes were fitted by a Gaussian (70%)-Lorentzian (30%) (GL30) function after Shirley-type background subtraction. UPS measurements were performed in the same instrument on the same samples with a (CAE) of 10 eV, using a He (I) excitation discharge lamp (hn = 21. 22 eV) (Focus). A bias of À9. 97 V was applied to the sample. The photoelectrons were collected at an angle of 901 between the sample and analyzer.
The data was acquired in photon counting mode using the Streak Camera software (HPDTA). The photogenerated carrier concentration at t = 0 was calculated using the formula  64  of n 0 = aÁf/E photon where a is the absorption coefficient at the excitation wavelength, f is the fluence of the laser and E photon is the excitation photon energy. The PL lifetimes were determined by using the general rate equation (Àdn/ dt = k 1 n + k 2 n 2 ) however, excluding Auger recombination, since the initial carrier concentration was low (n 0 = 4 Â 10 16 cm À3 ), where Auger recombination is negligible in perovskites. 65.
Absolute PL measurements Perovskite films were deposited on pre-cleaned glass substrates or on ESLs (np-SnO x and a-NbO x ) using the same recipe as used for device fabrication. The absolute PL spectra of the samples Energy & Environmental Science Communication were collected using a hyperspectral imaging system coupled to a microscope with 2 nm spectral resolution (Photon etc. IMA). Samples were excited with 532 nm laser at B1 sun illumination condition (before the measurement, the power density of the laser was adjusted with a reference perovskite solar cell). The absolute calibration procedure of the setup is reported in detail elsewhere. 66 The collected data was analyzed via home-built MatLab code using modified generalized Plank law to get quasi-Fermi level splitting using the formula: 67 where A (E,y) is the absorption probability of a photon with energy E, incident at an angle y with respect to the surface normal.
DFT analysis Calculations were performed with the DFT code Quantum Espresso,  70  the Perdew-Wang generalized-gradient approximation exchange-correlation functional (xc),  71  projectoraugmented waves,  72  [3] and the DFT-D2 method  73  to include van der Waals interactions. The energy cutoff for the planewave basis was set at 100 Rydbergs. The calculation of densities of states (DOS) employed the Monkhorst-Pack scheme with Gaussian broadening and 4 Â 4 Â 1 k-grids. 74 In total energy calculations on large supercells, only the gamma point was used for k-sampling. Structures were rendered with VESTA. 75 To model O vacancies in the bulk of SnO.
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics. Measuring progress in NLG relies on a constantly evolving ecosystem of automated metrics, datasets, and human evaluation standards. Due to this moving target, new models often still evaluate on divergent anglo-centric corpora with wellestablished, but flawed, metrics. This disconnect makes it challenging to identify the limitations of current models and opportunities for progress. Addressing this limitation, GEM provides an environment in which models can easily be applied to a wide set of tasks and in which evaluation strategies can be tested. Regular updates to the benchmark will help NLG research become more multilingual and evolve the challenge alongside models. This paper serves as the description of the data for which we are organizing a shared task at our ACL 2021 Workshop and to which we invite the entire NLG community to participate.
Introduction Natural language generation is the task to automatically generate understandable texts, typically using a non-linguistic or textual representation of information as input  (Reiter and Dale, 2000). These texts aim to fulfill an underlying communicative goal (e. g. , to produce a summary of an article) while remaining faithful to the input information, fluent, grammatical, and natural-looking. An NLG system needs to be robust to shifts in the data distribution and be able to produce text in many different languages. Finally, it is often desired that repeated interactions with the model produce diverse outputs, for example, to explain concepts in multiple ways or to become a more interesting conversational agent. These optimization objectives can often be conflicting  (Hashimoto et al.
, 2019)  and, as a result, evaluations that focus only on a single aspect may fail to recognize the drawbacks of a particular method. To demonstrate this trade-off, consider an improvement on the CNN-DM summarization dataset  (Hermann et al. , 2015; Nallapati et al. , 2016)  measured by the ROUGE-L met-ric  (Lin, 2004). Since ROUGE only tests the extent to which a generated summary has a lexical overlap with a reference summary, it can erroneously produce high scores for fluent, yet meaningless and unfaithful outputs as long as many of the same words are used  (Maynez et al. , 2020; Gabriel et al. , 2020). Moreover, ROUGE tends to favor systems that produce longer summaries  (Sun et al. , 2019). It is thus crucial to carefully assess the progress of NLG toward all of its goals at the same time in ways that evolve alongside the models.
This is currently not the case; new models are evaluated on different datasets, most of which focus only on the English language  (Bender, 2019) , and using these flawed metrics. Moreover, while human evaluations of generated texts can provide complementary insights to automatic evaluation  (Manning et al. , 2020) , it can also lead to contradicting results since studies often omit crucial replication details and assume different definitions of the measured quantities  (Howcroft et al. , 2020). We propose a living benchmark called GEM (Generation, Evaluation, and Metrics) that aims to enable research on a wide range of NLG challenges. To avoid the fallacy of encouraging hill climbing on a leaderboard  (Linzen, 2020) , GEM focuses on an in-depth evaluation of model outputs across human and automatic evaluation that aims to uncover shortcomings and opportunities for progress.
As datasets, metrics, and models improve, the benchmark environment will improve as well, replacing "solved" tasks with more challenging ones, incorporating newly developed metrics, and addressing discovered flaws in the experimental setup, as demonstrated in Figure  1. Making all model outputs available under an open-source license will support evaluation research and integrating new metrics will, in turn, help their adoption and increase the robustness of model evaluations. The initial set of eleven included datasets is presented in Table  1. They measure specific generation challenges, such as the content selection and planning (What to say?), and the surface realization (How to say it?) (Reiter and Dale, 2000; Gatt and Krahmer, 2018). Models need to be capable of paraphrasing, simplification, and others.