dataesr/scientific-paragraphs-categorization huggingface dataset:
The Berkeley Open Infrastructure for Network Computing system (BOINC 1  ) allows data to be processed for a range of projects, including physics, climate change, and biology. In volunteer thinking, participants typically visit a website where they are presented with data and they are trained to analyse the data according to a certain research protocol. Examples include classifying galaxies (Galaxy Zoo 2 ), folding proteins (Foldit 3 ), and transcribing weather information (Old Weather 4 ). In addition to science research, volunteer thinking projects can be found within humanities research. For example, volunteers can transcribe unstudied manuscripts written by the philosopher Jeremy Bentham (Transcribe Bentham 5  ). In participatory sensing, participants typically download a mobile phone app which allows them to collect data by utilizing sensors that are already integrated in their mobile phone, e.
g. Wifi, GPS receivers, camera, microphone. For example, volunteers can collect data about local animal species (Project Noah 6  ), and noise levels (WideNoise 7  ). Sometimes volunteers are also asked to submit behavioural information, such as rating how happy they feel (Mappiness 8  ), or tweeting errors they had experienced that day (Errordiary 9  ). Usually citizen cyberscience projects are designed by scientists and volunteers are asked to assist in some way. Projects that work in the opposite way -where citizens design their own projects and the scientists' role is to facilitate this work -are known as extreme citizen science. An example of this is the Sapelli project, where scientists worked with Congo tribes to design a mobile app to help them map parts of the rainforest that were important to them  (Vitos et al.
INTERVIEW STUDY The aim of the current interview study was to build upon our previous work  (Jennett et al. , 2016) , exploring the different kinds of creativity that volunteers experience in citizen cyberscience. Our sample consisted of volunteers and scientists from the 4 Citizen Cyberlab pilot projects (presented in sections 4 and 5): 1. GeoTag-X 2. Virtual Atom Smasher (VAS) 3. Synthetic Biology -iGEM competition 4. Extreme Citizen Science -including Do-It-Yourself (DIY) and Air Quality Monitoring (AQM) Together these pilot projects represent different kinds of citizen cyberscience. In some projects volunteers only take part in data analysis (GeoTag-X, VAS), while in other projects volunteers are involved in generating research questions (DIY) and the whole research process (AQM). Some projects involve social aspects, such as team work (iGEM).
GeoTag-X The GeoTag-X pilot, developed by UNITAR/UNOSAT, aims to teach volunteers to perform in-depth analyses of media coming out of a disaster. This involves recognizing the important information in a photo and creating relevant, structured datasets that can be used by those working on the ground in the disaster response. Over 20 projects have been launched on the GeoTag-X website 10. These projects cover different disaster-related events, such as floods, and earthquakes, along with more slow moving events, such as droughts, and pollution. GeoTag-X volunteers can be divided into two categories:  volunteer-developers, who help to develop the series of questions and multiple choice answers for one (or more) of the GeoTag-X projects; and  volunteer-analysers, anyone that visits the GeoTag-X website and analyses photos for one (or more) of the GeoTag-X projects.
As one scientist describes: "…they had a background in computer programming and bioinformatics. So they were able to analyse the data that came in and keep it in a databased for people. Something that no member of the core team had any understanding of doing, so it was pretty great that we found people that were able to do those kinds of things […] And we made sure to give them credit for it, you know it's on the website, so that they can forever on their portfolios and resumes, they have now that they made this thing and it was showcased for this project and it was really great. " One of the students involved in developing Minder (an app to share ideas) explained that she joined the iGEM team because she wanted to be more interdisciplinary: "I was interested in many different things especially the connection between biology, medicine and informatics.
CONCLUSION In our research we have shown that volunteers experience many different kinds of creativity in citizen cyberscience. We conducted 96 interviews in total: 86 with volunteers and 10 with scientists. To our knowledge, this is the largest interview dataset in citizen cyberscience research. Our results reveal several creative activities that are a result of active involvement in citizen cyberscience: discussing ideas, suggesting improvements, development of new technologies, gamification, artwork, creative writing, outreach activities, and the development of new research projects. We conclude that the majority of creative products are communityrelated. Most creative activities do not lead to scientific discoveries; however, they are still important for a project's success. Creative products come from the wish to optimize one's own activity: making things work better for themselves, and therefore, potentially for all members.
ACKNOWLEDGEMENTS We would like to thank all of our interview participants. Thank you to  Muki Haklay ,  Francois Grey , and  Ariel Lindner , for their leadership and encouragement throughout  Citizen Cyberlab. 124 Jennet et al. / Human Computation (2016) 4:1 Thank you to our reviewers for their valuable feedback. Also a special thanks to the Human Computation journal editors,  Eglė Marija Ramanauskaitė  and  Pietro Michelucci , for their positive support and advice. This research was funded by the  EU project Citizen Cyberlab  (Grant No  317705 ). Charlene Jennett is currently funded by the  Open   3D  project ( EPSRC grant   EP/M013685/1 ).
Methods From a list of 19 medical journals, three were retained for analysis, after a random selection: British Medical Journal (BMJ), Annals of Oncology (AoO) and British Journal of Cancer (BJC). All original articles published between March 2009 and March 2014 were screened. Only studies whose main outcome was accompanied by a corresponding statistical test were included in the analysis. Studies based on censored data were excluded. Primary outcome was to assess quality of reporting for description of primary outcome measure in RCTs and of variables of interest in OBS. A logistic regression was performed to identify covariates of studies potentially associated with concordance of tests between Methods and Results parts.
Study selection Observational studies or interventional studies in English-language, focusing on a particular subject related to oncology and published between March 2009 and March 2014 in BMJ, AoO or BjC journals were eligible for inclusion. The main outcome of the study was to be accompanied by a corresponding statistical test. Exclusion criteria were: Phase I, II, or IV trials; descriptive studies, meta-analyses, systematic reviews, prediction model building studies, test validation studies, recommendations, letter to the editor, medico-economic studies and fundamental research studies. Studies based on censored data and / or having the survival as primary endpoints were excluded from the analysis for two reasons: first tests used (log-rank test, cox proportional hazards survival analysis) are very specific ones.
Data collection One author (RR) extracted the following data from included studies and a second author (MO) checked the extracted data: Name of publishing journal, year of publication, number of patients included, cancer studied, study type, description of outcome measures in Methods (interventional studies), definition of variables in Methods (observational studies), description of statistical test related to main endpoint/variables in Methods, type and name of statistical test, same name of statistical test used for main endpoint/variables assessment in Results, as compared to Methods, name of this test in Results, significant results for main endpoint, concordant conclusion on the main endpoint result.
Cancer research involves numerous disciplines. The multiplicity of data sources and their heterogeneous nature render the integration and the exploration of the data more and more complex. Translational research platforms are a promising way to assist scientists in these tasks. In this article, we identify a set of scientific and technical principles needed to build a translational research platform compatible with ethical requirements, data protection and data-integration problems. We describe the solution adopted by the CARPEM cancer research program to design and deploy a platform able to integrate retrospective, prospective, and day-to-day care data. We designed a three-layer architecture composed of a data collection layer, a data integration layer and a data access layer. We leverage a set of open-source resources including i2b2 and tranSMART.
The article also noted that there was a movement away from 'home-grown' systems toward more commonly used systems (such as i2b2  [7, p. 2] ). Among the obstacles, data access was listed as an obstacle for project approval and a continued difficulty. Paradoxically, the public acceptance of such data repository is overall fairly positive. A survey from 2013 by Grande et al. [8]  investigated the public preferences toward secondary uses of electronic health information. The main concerns identified were the use of the data, before the data sensitivity and the user. The survey showed a general willingness to share health information for research use, especially in the context of research hospitals. In this article, we describe the solution adopted by the CARPEM cancer research program to design, implement and deploy a translational research platform able to face the technical and scientific challenges, and the issues underlined by the CTSA survey.
The CARPEM program The French government introduced in 2009 a 4-year plan aiming at strengthening the resources dedicated to cancer research. This plan recommended the creation of a competitive policy label to ensure the organization and management of interdisciplinary research and knowledge dissemination programs. In 2012, the French National Cancer Institute (INCa) granted eight SIRICs (Site de Recherche Intégré sur le Cancer in French, or Integrated Cancer Research Site) labels in France. SIRICs' ambitions are to provide new operational resources to oncology research, to optimize and accelerate the production of knowledge and to favor knowledge dissemination and application in patient care. The CARPEM (CAncer Research and PErsonalized Medicine) program is one of these eight SIRICs, focused on selected tumor types: digestive, endocrine, head and neck, hematological, lung, ovarian, and renal tumors.
The CARPEM program develops the following cross-disciplinary research objectives leading to the development of personalized medicine: (a) identification of genetic-and immune-based prognostic and predictive biomarkers with the help of translational project and epidemiological cohorts, (b) development of experimental therapeutics, and (c) integration of ethics and assessment of practices in the CARPEM research program. The CARPEM Translational Research Platform is a transversal project working with all the themes and tumor localization of the CARPEM research program. The platform aims at collecting and integrating all data related to cancer patients from the participating clinical and research institutions. The goal is to provide researchers with a complete picture of the expression of the disease, in terms of comorbidity, biology test results, genomic data, and so forth.
Data collected are heterogeneous in their nature and their semantics: structured, partially structured and unstructured clinical data of various data-types and various levels of granularity (from molecular biology analyses to phenotypes). CARPEM aims at collecting a large array of data, including: (i) phenotypic structured data, including routine care records and Clinical Trial observations, outcomes and adverse events; (ii) high content biomarker data, such as gene expression and proteomics data; (iii) unstructured text-data. Data are produced and stored in various places: three participating hospitals (the HEGP, the Cochin Hospital, and the Necker Hospital), research laboratories from many different specialties (several geographical localizations). Within each site, the information systems and software are heterogeneous and there is no unique patient identifier across the CARPEM data sites.
Within the institutions, the data media are heterogeneous. Retrospective data often come as text or Microsoft Excel spreadsheet, more rarely as organized relational databases. Clinical data are collected from institutional Clinical Data Warehouses (CDWs), from dedicated software (chemotherapy management software…), or from text-based clinical reports. Finally prospective data are fed to the system through Clinical Data Management Systems. The data collection process highly depends on the type of data: data collected in the context of a clinical trial have been strictly controlled and curated at different levels; whereas routine care data may exhibit missing information.
Related work Platforms enabling an integrated view of clinical and 'omics' data have received lot of attention in the recent years. Platforms of every size, capacities and goals have been developed  [9]. For example, projects such caBIG  [10]  were designed as early as 2004 for multiple purposes, including to enable data sharing at a U. S. national scale. However, this project is not suitable for the exploration of local datasets, or as a local data integration solution. Alternatively, standalone platforms appeared (e. g. iCOD  [11] , BRISK  [12]  or tranSMART  [13] ). These platforms enable an efficient exploration of the data. They are usually composed of a database, and an application server. These solutions are reasonably easy to install, and might also be shared in virtual appliances.
Datasets The CARPEM program integrates both general university hospitals with a strong activity in oncology and oncology-related care, and research centers. These institutions generate a large range of data types, going from clinical data collected during day-to-day care to data type at the forefront of mod-ern techniques. The platform integrates data from patients previously or currently treated for a cancer in one of the participating clinical institutions (namely, the Cochin Hospital, the HEGP, and the Necker Hospital). There is no uniform consent collection within the CARPEM program. For samples collection, the clinician relies on informed consent. The secondary use of care data leverages an opt-out solution. The overall data collected by the platform can be categorized as follow: • Clinical data from the clinical care systems • Retrospective data from clinical research groups (e.
S. NIH. It is used by internationally by a broad community of users in institution of various sizes. i2b2 "Informatics for Integrating Biology and the Bedside"  [7]  is an NIH-funded National Center for Biomedical Computing (NCBC) based at Partners HealthCare System in Boston. It provides an infrastructure for CDWs, which has been adopted by numerous academic hospitals around the world  [25] [26] [27]. The i2b2 warehouse uses an Entity-Attribute-Value (EAV) data model for its adaptability and dynamic nature; concepts are stored separately in a hierarchical data model. Prior to the development of the platform, we had conducted a review of the biomedical literature to identify translational research software allowing the combined exploration of clinical and omics data  [9]. In this review, we compared seven platforms on seven axes (including types of data handled, existence of a community of users).
We decided to adopt tranSMART. tranSMART  [13]  is a translational research platform allowing the integration of clinical data and omics data. It provides exploration and analysis tools. The tranSMART platform is developed by an active open-source community. We use the 1. 2 PostgreSQL version of tranSMART. Talend Open Studio  [28]  is an open-source data integration software allowing a graphical design of ETL pipelines. FreeMind  [29] is a free mind-mapping tool. It allows user to "edit a hierarchical set of ideas around a central concept"  [30]. We leverage the software strength to easily edit trees of concepts. CDISC ODM  [31] : We also leverage a community based format from the CDISC consortium. The CDISC ODM standard is designed to represent clinical studies, and to enable the storage of clinical data.
Harmonization process for prospective data One of the goals of the CARPEM program is to stimulate data sharing and collaboration among researchers. CARPEM researchers are organized into working-groups by localization of the tumor: lung cancer, ovarian cancer, colon cancer, and so forth. The task of the working-groups is to harmonize the data collection process among the participants of the group. Seek and describe available data. Each group identifies existing retrospective datasets, and describes them (in terms of variables, number of patients, time period covered by the dataset, produced publications, etc. ). Establish minimum shared datasets. In parallel, the working group constitutes a prospective cohort by determining a set of shared variables. We first establish semantic mappings between the items collected by the different groups.
We identify common items shared through the groups, and items used only by a limited number of research groups. We then select a set of item by consensus method. This step is complex and time consuming. Finally, We filter out the establish list to keep only a minimum set of items. Standardize data collection tools. This step demand a significant amount of work for clinicians as it requires an evaluation of the clinical care collection tools. To summarize, we build a data dictionary defining preferred terms for each item, units, how the information should be structured and formatted. When possible, we identify mapping to concept in standard terminology. This task is highly time consuming, but allows sharing a common understanding of collected data, and has strong clinical implications.
Security and technical implementation of the platform The platform runs on 7 Linux servers: three production servers (one dedicated to databases, the other two to applications), two development servers (one dedicated to databases, the second to applications), one data-integration and record-linkage server and one data-capture server. Servers have AMD Opteron 6320 processors (8 cores, 2. 8GHz) and RAM ranging from 16 to 64 GB. The total disk space dedicated to the infrastructure is 14TB. In addition to the use of RAIDs for system and data, backups of data and systems are made on a regular basis on a NAS. In-house development is Java-based, user interfaces are web-browser accessible. The platform is hosted within the network of the HEGP, and beneficiates from institutional firewalls from the HEGP and from our parent institution the AP-HP.
Data extraction and curation Data are collected either through a secured file exchange system, or directly extracted from the information system within the CARPEM program institutions. Hospitals and research institutes generate two types of data: (1) structured data (e. g. billing code, relational databases, Case Report Form…) and (2) unstructured data (mostly free-text reports such as discharge summary or imaging reports). As of today, the platform does not manage free-text reports. However, information contained within unstructured data is very valuable. We therefore developed specific parsers to extract structured knowledge from free text. Parsers are based on the structure of the documents and are dedicated to specific tasks. We will continue to develop such tools on a study-based basis.
One difference often highlighted by some authors between clinical research data and secondary use of care data is quality  [6]. During a clinical trial, data are controlled at virtually every step of their life cycle. Whereas, raw clinical data do not beneficiate from any data-management. To ensure the best quality possible, the translational research platform team performs data quality control to detect issues in datasets. More specifically, data are checked for different types of errors including missing information (including identification of the patient and description of the disease: localization, grade…) and consistency (e. g. chronology of dates). For retrospective datasets, the team optimizes the data representation to insure that data are leveraged to maximize their intrinsic value.
Data representation. The data-exploration interfaces of tools such as i2b2 or tranSMART rely on a hierarchical organization of the variables to express subsumption relationships (parent-child relations), e. g. to navigate, or to expand queries. When possible, we leverage standard medical taxonomies. However, for specific studies, if the clinician needs to maintain the organization of the data, we leverage FreeMind  [29]  to build a model of a local terminology. An item can be assigned to one or more concept. A study can therefore be associated with a local terminology as well as with a standard terminology.
Standardization of the exchange format Despite the heterogeneity of input formats (e. g. retrospective MS Excel datasets, data from the institutional CDW, relational databases), the actions performed on the data are always identical: recordlinkage, de-identification, data quality control, standard storage. Instead of developing ad-hoc ETL processes for each new entry format, we chose to transform every input file to a unique exchange format based on the CDISC ODM standard  [31]. The exchange ODM file is composed of 3 parts: the administrative data part holds all information regarding the provider, the date of generation of the dataset, etc. The metadata describes the structure of the variables (data definition, range, units…). Finally, the actual data are stored in the clinical data part of the document.
We developed a limited vendor-extension to the original CDISC ODM format to ensure the best interoperability between all parts of our system. Our vendor-extension of the ODM model adds a notion of "Personal Identifiable Information" status (Identifying or not). The ODM model is limited in depth (with a maximum of four hierarchical levels of depth), we also leverage the reference to hierarchies (standard or local) to enable the organization of concepts without depth limitation. Because of its single central role, the creation of the ODM exchange file is critical to the rest of the process. Both the data-managers and the data-provider (researcher or clinicians) are involved in this step. The ODM exchange file metadata provides us with a data dictionary describing at a high level the content of the datasets.
Record-linkage CARPEM patients (and their data) come from a variety of locations and institutions: one of the CARPEM program hospitals, molecular biology laboratories. No unique patient identifier is shared across these institutions. The platform provides a unique CARPEM identifier and attempt to link records from the different sources. We rely on first name, last name, sex and date of birth as our main identification key. Alternatively, institutional identifiers can also be used to identify patients within a given context. We leverage the Rochester Epidemiology Project  [32]  mapping rules, and link records through a series of decreasingly selective rules. The record-linkage is fully automated. However, all record-linkage choices are recorded, and can be later corrected if needed. Straightforwardly, the strictest rule defines that two individuals sharing exactly the same first and last name, as well as same gender and date of birth are considered to be the same person.