dataesr/scientific-paragraphs-categorization huggingface dataset:
The sustainability challenge The problem of ensuring the sustainability of software developed in academic projects has been recognized as a major element of project outcome. It is expected that research projects should develop their own sustain ability strategy for the time after the proj ect ends. Often this is addressed by a high quality of the product, a suitable license and by building a community of potential users. To support the research software commu nity in the UK, the Software Sustainability Institute  [1]  was founded to support better and more sustainable research software. An example cited by the Software Sustainability Institute is BioJS  [2] , an open source software project that developed visualization tools for biological data. Apart from fostering its technical experience, BioJS has established a user and developer community by means to reward contributors and to keep com munity members motivated.
Sustainability issues of research infrastructure tools and services have been analyzed with the result that researchers have to work with software that often is not fit for purpose. Most aca demic software seems to have a short life span before it disappears, or is not updated anymore  [3]. The EU commission expects that EU proj ects create an exit/sustainability strategy  [4]  outlining what will happen to project output after the end of the project. To address this challenge, EU projects tend to develop a sus tainability plan and create ideas how to use products at local, national and European lev els. For example, the EU projects SITEX [5], BIOCORE  [6] , METIS  [7] , GENOVATE  [8]  and EMBRC  [9]  developed strategies by creat ing plans for training, workshops, technical maintenance, for the inclusion of potential users, online portals and by addressing innova tion issues (knowledge exchange, intellectual property rights, promotion and marketing).
This especially concerns the question, if it is sustainable to provide software for personalized medicine based on an Open Source model. The development of Open Source software is popu lar among researchers and nearly all tools developed in the bioinformatics area are Open Source solutions. As example of Open Source Software that has reached a considerable distribution in the research community, Informatics for Integrating Biology and the Bedside (i2b2)  [12]  a data warehouse used by clinical researchers for discovery research must be mentioned. Its evolution over the past 10 years has been from a local research data solution to one disseminated more widely than any other software of the National Centers for Biomedical Computing (NCBC). A central point is that this Open Source Software is supported by a large user community that actively supports developers to enhance its capaci ties and functionalities.
For pmedicine, business opportunities emerge in the area of the development of novel, highly intercon nected tools for the field of personalized medicine  [15]  and in their proven user friendliness  [16]. To address the sustainability challenge, to identify opportunities for personalized medicine tools to create capabilities that can develop into competitive advantages and to find solutions to fill the financial gap created by the need to maintain software solutions, a joint pmedicine/ECRIN workshop was held. In June 2015, representatives from the personalized medicine project pmedicine [14] and ECRIN infrastructure  [17, 18]  gathered at the ECRIN center in Paris to examine opportunities and challenges facing the sustainability of software tools and services developed to support clinical trials in personalized medicine.
In contrast to the rather homogeneous structure of pmedicine and its IT infrastructure, ECRIN is a dis tributed and thus heterogenic infrastructure harboring many different clinical trial centers that use their own software and employ their own processes. For ECRIN, it is therefore important to manage its network in a way that ensures that there is no competition between ECRIN centers and national networks, but mutual sup port and cooperation. For this reason, ECRIN cannot force any new software upon its many members. Several aspects associated with the way clinical tri als are conducted determine the kind of collaboration possible for ECRIN and the constraints limiting any  [50]. This software called VISTA Trials is the clinical data management system of EORTC [51], a fully integrated and webbased solu tion applicable to trials in all therapeutic areas.
Any employment of a software solution from pmedicine in ECRIN must consider these basic conditions. In general, ECRIN is interested in collaboration with other ESFRI infrastructures and with pmedi cine. As a sustainable infrastructure ECRIN is inter ested in nextgeneration tools and wants to employ them to improve clinical trial conduct and to develop the clinical trial process further. A possibility to include pmedicine competence in an EU project is to provide a link to the CORBEL cluster project  [52]. Because of the conditions mentioned above, ECRIN cannot simply commit to the use of certain tools, but is only able to recommend their use to ECRIN mem bers. In fact, only the sponsor of a clinical trial has the authorization to choose the appropriate data manage ment tools and in this context, ECRIN may promote the use of ObTiMA as data management tool of choice to trial sponsors.
The possibilities for collaboration between pmed icine and the ECRIN community was analyzed in the personalized medicine project with the help of requirements, use cases, value propositions  [22, 23]. pmedicine tools are able to support large, interna tional GCP trials. The most widely employed clini cal data management systems (CDMS) in ECRIN are MACRO and Capture System, followed by eRe search Network, CleanWeb, GCP Base  [53]. Increas ingly the Open Source solution OpenClinica is being used in ECRIN trials. This heterogeneity in software solutions raises questions about the interoperability. Can different products be used in a single trial and can clinical trials data be exchanged between differ ent CDMS? The possibility of using different CDMS for the collection of data in a single trial by the joint exchange of trial data and metadata has been demon strated [54] and pmedicine can rely on this result to advance the integration of ObTiMA with other data management solutions.
Not all of these data standards are already supported by pmedicine. For ECRIN the ObTiMA data management tool received the highest interest, followed by the biobank ing tool. Both tools can be combined and used not only for trials in personalized medicine, but also for other ECRIN clinical trials. For ECRIN any novel tool to be introduced to ECRIN trials centers must show an added value for ECRIN; it must be able to integrate with already existing ECRIN tools and pro cesses. In general, ECRIN has to deal with two con flicting constraints when choosing new software tools: personal data protection as well as the need for trans parency and optimal usage of data. It was recognized that any integration of a patient empowerment tool into the ECRIN data management processes will raise problems concerning data protection and privacy pro tection.
Open Source Software & computer systems validation It has become standard practice for research software developed by academic researchers to be released under an opensource license. In this sense, pmedi cine tools will be made freely available under an Open Source license. This should enable researchers to read, understand, modify the code and to use software for their own research purposes. In contrast to other future science group Ensuring sustainability of software tools & services by cooperation with a research infrastructure Perspective Open Source projects that developed an operating system or an office suite, pmedicine cannot rely on a large potential user community. Therefore, based on the experiences gained by other Open Source projects, complementary forms of commercial licensing and feebased charges for provided services should not be excluded by pmedicine  [59].
It can guarantee that the software product is fit for purpose. Open Source Software is known to be difficult to validate. Thus, for the validation of its tools pmed icine cannot rely on the support of a software com pany, but must create all documents and conduct the validation process themselves. A strategy for the vali dation and certification of ObTiMA and DoctorEye exists already in pmedicine [62] employing standards for GCP compliance of computerized systems used for clinical trials, like ICHGCP  [63] , PIC/S  [64]  and GAMP  [65]. In addition, validation is an area where ECRIN can contribute experience.
pmedicine members will support STaRC by becoming equity holders, by the provision of tools, services and licenses and by gen eral support. Providing maintenance for personalized medicine tools will result in costs for STaRC and it is expected that at least in the beginning, these expen ditures will exceed the costs that can be afforded by STaRC. It was discussed, if maintenance of personal ized medicine tools could be sufficiently delegated to an Open Source community. For pmedicine, this seems to be an unrealistic scenario, because of the limited size of the user community of tools for personalized medi cine. Thus, the classic Open Source concept of provid ing the entire software as Open Source, with STaRC playing a consulting/training role will not be viable. Additional sources of funding will be necessary and in consequence, STaRC as a nonprofit organization will depend on additional public funding.
STaRC will be a nonprofit organization, because the pressure to perform financially would move STaRC's goals toward profitable changes of the software and not to the ones best for researchers and for patients. Any nonprofit business model does have its limita tions. For instance, access to startup and operating capital will be more difficult and STaRC's capability to license software to industrial partners will be lim ited. Thus, the support by a large infrastructure like ECRIN that itself is sustainable may be an important help for STaRC, both for the employment of tools and for reaching out to potential users. On the other hand, key strengths of STaRC are the possibility to provide a complete set of tools to enable any user to participate in personalized medicine. Users can also build on the strong patient focus and the inte gration of biobanking/imaging with data collection of pmedicine tools [16].
Sustainability through collaboration Of all the different ESFRI infrastructures only ECRIN conducts routinely complex clinical trials and employs for this purpose many different software solu tions  [53]. After experiencing the integrative scenarios of pmedicine tools during the workshop, ECRIN mem bers expressed their interest in pmedicine to improve clinical trials. On the other hand, for ECRIN data cen ters it can become expensive to maintain their own data management solution including the provision of regular updates and the validation and certification of software according to the GCP standard  [46]. To address this problem, pmedicine has developed a novel concept for the employment of personalized medicine tools in the research infrastructure ECRIN  [23, 67]. It consists of a combination of reciprocal services, complemented by the provision of Open Source and feebased tools.
This concept includes the combined provision of ObTiMA as a service (SaaS) and as Open Source version. The Open Source version is aimed at users that do not need the additional modules, like biobanking, imaging, adverse events reporting, data pseudonymization and data warehouse access, but are content with using only the data management functions of ObTiMA. The ser vice version provides all additional modules and sup port against a charge that will be different for academic or commercial sponsors of clinical trials. The combination of Open Source projects supported by services provided by companies that help customers with installation, configuration and troubleshooting is a wellknown concept; MySQL  [68]  and Linux  [69]  are familiar examples  [70]. In the area of research software for personalized medicine, the absence of a large user community, the complexity of the IT infrastructure and the multitude of necessary tools hamper the devel opment of such service companies  [71].
To achieve this aim, personalized medicine tools must be employed in real clinical trial situations and ECRIN could support this aim by encouraging ECRIN trial units to test and evaluate the software. In this way, with the help of ECRIN, pmedicine tools will become known to relevant members of the European clinical research community. In addition, it was sug gested to extend the collaboration to the joint search for public funding for further development and employ ment of personalized medicine tools. Collaboration must be organized and paid for and for this purpose, a durable equilibrium between support by public funding and feebased charges for services must be found. As a result of our discussions, we are convinced that only a couple of joint and interrelated activities can lead to successful support of sustainability of soft ware developed for research purposes (Box 2).
The maintenance of such infrastruc tures cannot be achieved by a single project alone, but requires collaboration and partnerships between project partners, infrastructures and expert centers. Before developing a new software solution, develop ers should check whether a reuse of existing tools and services is possible. Newly developed software should be built in a modular way for interaction with other tools and for integration in einfrastruc tures that may contribute to the maintenance and service provision of these tools. An active approach to include research infrastructures and einfrastruc tures should be considered. Especially EUDAT  [73]  and RDA  [74]  are building a generic data exchange layer and a generic data sharing framework. In the case of EUDAT, this is the Collaborative Data Infra structure (CDI) for panEuropean solutions.
Collaboration possibilities were confined to certain issues, like support of standards and participation in board memberships. BBMRI creates a panEuropean biobanking infra structure for biomedical and biological research that builds on already existing infrastructures, resources and technologies and that is embedded into European ethical and legal frameworks dealing with biosamples from patients and healthy persons[30]. To employ per sonalized medicine tools in BBMRI, for example, to use ObTiMA to manage biosamples for clinical trials, BBMRI must learn much more about this tool. It was suggested that ObTiMA should support the minimal datasets of BBMRI[31]. Nonetheless, BBMRI pro posed to examine possible overlaps between BBMRI, pmedicine and ELIXIR that could be used for collaboration purposes.
Introduction: We see increased use of existing observational data in order to achieve fast and transparent production of empirical evidence in health care research. Multiple databases are often used to increase power, to assess rare exposures or outcomes, or to study diverse populations. For privacy and sociological reasons, original data on individual subjects can't be shared, requiring a distributed network approach where data processing is performed prior to data sharing. Case Descriptions and Variation Among Sites: We created a conceptual framework distinguishing three steps in local data processing: (1) data reorganization into a data structure common across the network; (2) derivation of study variables not present in original data; and (3) application of study design to transform longitudinal data into aggregated data sets for statistical analysis.
We applied this framework to four case studies to identify similarities and differences in the United States and Europe: Exploring and Understanding Adverse Drug Reactions by Integrative Mining of Clinical Records and Biomedical Knowledge(EU-ADR),Observational Medical Outcomes Partnership(OMOP), the Food and Drug Administration's (FDA's) Mini-Sentinel, and the Italian network-the Integration of Content Management Information on the Territory of Patients with Complex Diseases or with Chronic Conditions (MATRICE). Findings: National networks (OMOP, Mini-Sentinel, MATRICE) all adopted shared procedures for local data reorganization. The multinational EU-ADR network needed locally defined procedures to reorganize its heterogeneous data into a common structure. Derivation of new data elements was centrally defined in all networks but the procedure was not shared in EU-ADR.
We applied this framework to four case studies to identify similarities and differences in the United States and Europe: Exploring and Understanding Adverse Drug Reactions by Integrative Mining of Clinical Records and Biomedical Knowledge(EU-ADR),Observational Medical Outcomes Partnership(OMOP), the Food and Drug Administration's (FDA's) Mini-Sentinel, and the Italian network-the Integration of Content Management Information on the Territory of Patients with Complex Diseases or with Chronic Conditions (MATRICE). Findings: National networks (OMOP, Mini-Sentinel, MATRICE) all adopted shared procedures for reorganize its heterogeneous data into a common structure. Derivation of new data elements was.
Introduction Observational studies based on secondary use of existing data collected in the process of health care delivery have the potential to deliver sound evidence quickly enough to support health policy making, which it is often subject to time constraints [Salmon2012],  1  thus complementing evidence generated by means of primary data collection. However, some epidemiological questions, especially those concerning rare events, rare exposures, and small groups of patients, require more data than is available in any single observational database. 2, 3, 4 herefore a growing number of studies use data from networks of databases, sometimes from different countries. Although some of these networks were formed ad hoc for a particular study, several more permanent networks have now been established, where the partners have agreed on an infrastructure and workflow to be reused for different studies.
Privacy regulations and concerns about data ownership and interpretation prevent easy central pooling of original health care data that is now stored in different databases and can be used for secondary purposes. 5 In spite of these barriers several approaches can be used to still employ this data for secondary purposes and pool the results. For example, investigators at each data source can independently create a protocol and execute the study, and estimates are only generated afterward through meta-analysis. A further step is to share the protocol across sites, but asking the local partners to adapt it to their local data and to implement it in their own usual software, to produce local estimates for meta-analysis that are compatible by design. However, most networks now go even further and adopt a distributed analysis approach: each database is locally transformed to a representation that is similar across the network, and one single computer program performing the analysis is shared and executed at each site.
Volume 4 (2016) Issue Number 1 Meanwhile, the United States Food and Drug Administration's (FDA's) Mini-Sentinel Project  10  was developed to support medical product safety monitoring and now includes 18 data partners within a distributed network. Also in the United States, from 2010 to 2014 the Observational Medical Outcomes Partnership (OMOP)  11  performed methodological research on drug safety studies and developed tools and a database network for performing risk identification. Other networks have been developed in other countries, like the Canadian Network for Observational Drug Effect Studies (CNODES) project in Canada and the Asian Pharmacoepidemiology Network (ASPEN) network in Asia. 4  Pharmacoepidemiology is not the only field where the opportunities for combining multiple databases are increasing: in the context of public health or health services research, gathering data from different regions or countries has the added value that different policies can be compared.
Mini-Sentinel and EU-ADR are also used to evaluate the impact of regulatory actions. And the Italian network-the Integration of Content Management Information on the Territory of Patients with Complex Diseases or with Chronic Conditions (Integrazione dei Contenuti Informativi per la Gestione sul Territorio di Pazienti con Patologie Complesse o con Patologie Croniche)(MATRICE) Project,  12, 13  funded by the Italian Ministry of Healthcreated a distributed network to evaluate the impact of health policies on quality and equity of health care. We developed a conceptual framework to analyze the process of data management in a network of databases adopting the distributed analysis approach to perform observational studies. We applied the framework to four case studies, and identified similarities and substantial differences.
Sampling and Data Collection Some of the paper's authors first conceived of the conceptual framework as an abstraction of the process in place in the European network EU-ADR and in the Italian network MATRICE. They reached out to the authors participating in the United States networks OMOP and Mini-Sentinel, to compare networks of different continents. Data collection was performed via document (scientific papers and websites) analysis and interviews with coauthors. The manuscript was reviewed by all the authors.
The Four Networks The EU-ADR Project was funded by the European Commission under Framework Programme 7 (FP7) and ran from 2008 to 2012 with the aim of producing a computerized integrated system for the early detection of drug safety signals. The project used data from eight databases from four European countries (Denmark, Italy, the Netherlands, and the United Kingdom) covering a population of about 20 million individuals overall with almost 60 million person-years (PYs) of follow-up. 3 Subsequently, the EU-ADR workflow has been further improved and applied in several collaborative drug-safety studies concerning NSAIDs (SOS),  14, 15  pandemic influenza vaccine (VAESCO),  16  the arrhythmogenic potential of drugs (ARITMO),  17  and hypoglycemic drugs (SAFEGUARD). 18 The subjects of the studies performed in this network include methodology  19, 20, 21, 22  drug utilization, disease incidence,  23  signal detection,  24  testing,  25, 26  filtering,  27  and substantiation.
31 At the same time, several studies focusing on methodology have been completed. 32, 33, 34, 35, 36, 37 In 2015 the Mini-Sentinel pilot transitioned to the Sentinel system that is become part of the FDA's regulatory framework. OMOP was a public-private partnership that ran from 2010 to 2014 and was part of the Innovation in Medical Evidence Development and Surveillance (IMEDS) program of the Reagan-Udall Foundation for the FDA. Its goal was to help determine best practices for use of observational health care data. OMOP currently maintained five commercial databases covering 164. 9 million persons in its own central venue, and its data partner network included six other databases covering an additional 105 million persons [Stang2010. 8, 38 The network was used to develop tools for performing observational studies in a database network, including the OMOP Common Data Model (CDM),  39  the OMOP Vocabulary,  40  and tools for assessing data quality,  41  as well as research into the development and evaluation of methods for drug-associated risk identification.