What incentives increase data sharing in health and medical research? A systematic review

BackgroundThe foundation of health and medical research is data. Data sharing facilitates the progress of research and strengthens science. Data sharing in research is widely discussed in the literature; however, there are seemingly no evidence-based incentives that promote data sharing.MethodsA systematic review (registration: doi.org/10.17605/OSF.IO/6PZ5E) of the health and medical research literature was used to uncover any evidence-based incentives, with pre- and post-empirical data that examined data sharing rates. We were also interested in quantifying and classifying the number of opinion pieces on the importance of incentives, the number observational studies that analysed data sharing rates and practices, and strategies aimed at increasing data sharing rates.ResultsOnly one incentive (using open data badges) has been tested in health and medical research that examined data sharing rates. The number of opinion pieces (n = 85) out-weighed the number of article-testing strategies (n = 76), and the number of observational studies exceeded them both (n = 106).ConclusionsGiven that data is the foundation of evidence-based health and medical research, it is paradoxical that there is only one evidence-based incentive to promote data sharing. More well-designed studies are needed in order to increase the currently low rates of data sharing.

Related papers:

title: The Open Knowledge Foundation: Open Data Means Better Science
abstract: Open data leads to better science, but overcoming the barriers to widespread publication and availability of open scientific data requires a community effort. The Open Knowledge Foundation Open Data in Science Working Group describes their role in this movement.

title: Meta-research: Evaluation and Improvement of Research Methods and Practices
abstract: As the scientific enterprise has grown in size and diversity, we need empirical evidence on the research process to test and apply interventions that make it more efficient and its results more reliable. Meta-research is an evolving scientific discipline that aims to evaluate and improve research practices. It includes thematic areas of methods, reporting, reproducibility, evaluation, and incentives (how to do, report, verify, correct, and reward science). Much work is already done in this growing field, but efforts to-date are fragmented. We provide a map of ongoing efforts and discuss plans for connecting the multiple meta-research efforts across science worldwide.

title: Perspectives on Open Science and Scientific Data Sharing: An Interdisciplinary Workshop
abstract: Looking at Open Science and Open Data from a broad perspective. This is the idea behind "Scientific data sharing: an interdisciplinary workshop", an initiative designed to foster dialogue between scholars from different scientific domains which was organized by the Istituto Italiano di Antropologia in Anagni, Italy, 2-4 September 2013.We here report summaries of the presentations and discussions at the meeting. They deal with four sets of issues: (i) setting a common framework, a general discussion of open data principles, values and opportunities; (ii) insights into scientific practices, a view of the way in which the open data movement is developing in a variety of scientific domains (biology, psychology, epidemiology and archaeology); (iii) a case study of human genomics, which was a trail-blazer in data sharing, and which encapsulates the tension that can occur between large-scale data sharing and one of the boundaries of openness, the protection of individual data; (iv) open science and the public, based on a round table discussion about the public communication of science and the societal implications of open science. There were three proposals for the planning of further interdisciplinary initiatives on open science. Firstly, there is a need to integrate top-down initiatives by governments, institutions and journals with bottom-up approaches from the scientific community. Secondly, more should be done to popularize the societal benefits of open science, not only in providing the evidence needed by citizens to draw their own conclusions on scientific issues that are of concern to them, but also explaining the direct benefits of data sharing in areas such as the control of infectious disease. Finally, introducing arguments from social sciences and humanities in the educational dissemination of open data may help students become more profoundly engaged with Open Science and look at science from a broader perspective.

title: Reproducible Research Practices and Transparency across the Biomedical Literature
abstract: There is a growing movement to encourage reproducibility and transparency practices in the scientific community, including public access to raw data and protocols, the conduct of replication studies, systematic integration of evidence in systematic reviews, and the documentation of funding and potential conflicts of interest. In this survey, we assessed the current status of reproducibility and transparency addressing these indicators in a random sample of 441 biomedical journal articles published in 2000–2014. Only one study provided a full protocol and none made all raw data directly available. Replication studies were rare (n = 4), and only 16 studies had their data included in a subsequent systematic review or meta-analysis. The majority of studies did not mention anything about funding or conflicts of interest. The percentage of articles with no statement of conflict decreased substantially between 2000 and 2014 (94.4% in 2000 to 34.6% in 2014); the percentage of articles reporting statements of conflicts (0% in 2000, 15.4% in 2014) or no conflicts (5.6% in 2000, 50.0% in 2014) increased. Articles published in journals in the clinical medicine category versus other fields were almost twice as likely to not include any information on funding and to have private funding. This study provides baseline data to compare future progress in improving these indicators in the scientific literature.

title: A systematic review of barriers to data sharing in public health
abstract: BackgroundIn the current information age, the use of data has become essential for decision making in public health at the local, national, and global level. Despite a global commitment to the use and sharing of public health data, this can be challenging in reality. No systematic framework or global operational guidelines have been created for data sharing in public health. Barriers at different levels have limited data sharing but have only been anecdotally discussed or in the context of specific case studies. Incomplete systematic evidence on the scope and variety of these barriers has limited opportunities to maximize the value and use of public health data for science and policy.MethodsWe conducted a systematic literature review of potential barriers to public health data sharing. Documents that described barriers to sharing of routinely collected public health data were eligible for inclusion and reviewed independently by a team of experts. We grouped identified barriers in a taxonomy for a focused international dialogue on solutions.ResultsTwenty potential barriers were identified and classified in six categories: technical, motivational, economic, political, legal and ethical. The first three categories are deeply rooted in well-known challenges of health information systems for which structural solutions have yet to be found; the last three have solutions that lie in an international dialogue aimed at generating consensus on policies and instruments for data sharing.ConclusionsThe simultaneous effect of multiple interacting barriers ranging from technical to intangible issues has greatly complicated advances in public health data sharing. A systematic framework of barriers to data sharing in public health will be essential to accelerate the use of valuable information for the global good.

title: Avoidable Waste in the Production and Reporting of Research Evidence
abstract: “Research results should be easily accessible to people who need to make decisions about their own health... Why was I forced to make my decision knowing that information was somewhere but not available? Was the delay because the results were less exciting than expected? Or because in the evolving fi eld of myeloma research there are now new exciting hypotheses (or drugs) to look at? How far can we tolerate the butterfl y behaviour of researchers, moving on to the next fl ower well before the previous one has been fully exploited?”

title: Increasing value and reducing waste in biomedical research regulation and management
abstract: After identification of an important research question and selection of an appropriate study design, waste can arise from the regulation, governance, and management of biomedical research. Obtaining regulatory and governance approval has become increasingly burdensome and disproportionate to the conceivable risks to research participants. Regulation and governance involve interventions that are assumed to be justified in the interests of patients and the public, but they can actually compromise these interests. Inefficient management of the procedural conduct of research is wasteful, especially if it results in poor recruitment and retention of participants in well designed studies addressing important questions. These sources of waste can be minimised if the following four recommendations are addressed. First, regulators should use their influence to reduce other causes of waste and inefficiency in research. Second, regulators and policy makers should work with researchers, patients, and health professionals to streamline and harmonise the laws, regulations, guidelines, and processes that govern whether and how research can be done, and ensure that they are proportionate to the plausible risks associated with the research. Third, researchers and research managers should increase the efficiency of recruitment, retention, data monitoring, and data sharing in research through use of research designs known to reduce inefficiencies, and further research should be done to learn how efficiency can be increased. Finally, everyone, particularly those responsible for health-care systems, should promote integration of research into everyday clinical practice. Regulators and researchers should monitor adherence to each of these recommendations and publish metrics.

title: Reproducing Statistical Results
abstract: The reproducibility of statistical findings has become a concern not only for statisticians, but for all researchers engaged in empirical discovery. Section 2 of this article identifies key reasons statistical findings may not replicate, including power and sampling issues; misapplication of statistical tests; the instability of findings under reasonable perturbations of data or models; lack of access to methods, data, or equipment; and cultural barriers such as researcher incentives and rewards. Section 3 discusses five proposed remedies for these replication failures: improved prepublication and postpublication validation of findings; the complete disclosure of research steps; assessment of the stability of statistical findings; providing access to digital research objects, in particular data and software; and ensuring these objects are legally reusable.

title: Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency
abstract: Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.

title: Microattribution and nanopublication as means to incentivize the placement of human genome variation data into the public domain
abstract: The advances in bioinformatics required to annotate human genomic variants and to place them in public data repositories have not kept pace with their discovery. Moreover, a law of diminishing returns has begun to operate both in terms of data publication and submission. Although the continued deposition of such data in the public domain is essential to maximize both their scientific and clinical utility, rewards for data sharing are few, representing a serious practical impediment to data submission. To date, two main strategies have been adopted as a means to encourage the submission of human genomic variant data: (1) database journal linkups involving the affiliation of a scientific journal with a publicly available database and (2) microattribution, involving the unambiguous linkage of data to their contributors via a unique identifier. The latter could in principle lead to the establishment of a microcitation‐tracking system that acknowledges individual endeavor and achievement. Both approaches could incentivize potential data contributors, thereby encouraging them to share their data with the scientific community. Here, we summarize and critically evaluate approaches that have been proposed to address current deficiencies in data attribution and discuss ways in which they could become more widely adopted as novel scientific publication modalities. Hum Mutat 33:1503–1512, 2012.

title: Speeding up research with the Semantic Web
abstract: Data for Rare Diseases are often distributed. Ideally, we can combine relevant data and biological insights from any place in the world and use it directly as input for computational analysis. However, too often data is poorly described making it hard to find, hard to assess its quality, and hard to integrate with other data. A valid question is: ‘Why can’t we analyse data as if it came from one global database?’. Here we introduce the Semantic Web as an enabling technology for making data interoperable and thereby expediting biological insight. The Semantic Web ‘language’ is RDF: the Resource Description Framework. It uses the ‘hyperlink’ mechanism known from the internet to refer to data instead of web pages. Meaningful relations are specified as triples: subject, predicate, object. For example, ‘CAPN3’, ‘interacts with’, ‘ParvB’. Written in RDF: http://www.uniprot.org/uniprot/P20807 http://www.conceptwiki.org/index.php/Concept: adf6044e-5c2b-11df-b0cb-001517ac506c http://bio2rdf.org/geneid:29780 While RDF is meant for computers, we see that: (i) RDF triples convey meaning; (ii) hyperlinks specify the location of data, which might be different databases (even within a triple); (iii) data items are also references to other RDF documents with more triples (e.g. try http://www.uniprot. org/uniprot/Q13547 in a browser). A hyperlink can be in any number of triples, effectively creating the world wide database of meaningfully linked data that is needed in the study of Rare Diseases. Ontologies can also be encoded in RDF, thereby extending the functionality to a global knowledge base. New experiments and discoveries can continually add information to this knowledge base. For example, the Semantic Web can help us to find drug targets for Rare Diseases. For this purpose, OpenPhacts  is integrating compounds from Chemspider [http://chemspider.com], proteins from UniProt [http:// uniprot.org], pathways from WikiPathways [http://wikipathways.org], and documents from PubMed [http:// www.ncbi.nlm.nih.gov/pubmed/]. We also make DNA sequence variations from the Leiden Open Variation Database (LOVD [http://www.lovd.nl]) available in RDF, and visualised via the UCSC genome browser. However, a number of barriers must be overcome. First, databases pre-dating the Semantic Web are used abundantly and must be integrated. This is usually an expensive and tedious task. Secondly, building a scientific reputation often conflicts with data sharing. Therefore, we have developed a data publishing framework called Nanopublication: an application of RDF that links authorship to individual datum (attribution). This creates a transparent and equitable incentive for data sharing. Nano publications also provide incentives for the exposure of legacy data. In conclusion, Nano publications and Semantic Web technology makes data easier to find and directly applicable to integrative analyses.

title: Open source drug discovery – A limited tutorial
abstract: Open science is a new concept for the practice of experimental laboratory-based research, such as drug discovery. The authors have recently gained experience in how to run such projects and here describe some straightforward steps others may wish to take towards more openness in their own research programmes. Existing and inexpensive online tools can solve many challenges, while some psychological barriers to the free sharing of all data and ideas are more substantial.

title: The collaboratory for MS3D: a new cyberinfrastructure for the structural elucidation of biological macromolecules and their assemblies using mass spectrometry-based approaches.
abstract: Modern biomedical research is evolving with the rapid growth of diverse data types, biophysical characterization methods, computational tools and extensive collaboration among researchers spanning various communities and having complementary backgrounds and expertise. Collaborating researchers are increasingly dependent on shared data and tools made available by other investigators with common interests, thus forming communities that transcend the traditional boundaries of the single research laboratory or institution. Barriers, however, remain to the formation of these virtual communities, usually due to the steep learning curve associated with becoming familiar with new tools, or with the difficulties associated with transferring data between tools. Recognizing the need for shared reference data and analysis tools, we are developing an integrated knowledge environment that supports productive interactions among researchers. Here we report on our current collaborative environment, which focuses on bringing together structural biologists working in the area of mass spectrometric based methods for the analysis of tertiary and quaternary macromolecular structures (MS3D) called the Collaboratory for MS3D (C-MS3D). C-MS3D is a Web-portal designed to provide collaborators with a shared work environment that integrates data storage and management with data analysis tools. Files are stored and archived along with pertinent meta data in such a way as to allow file handling to be tracked (data provenance) and data files to be searched using keywords and modification dates. While at this time the portal is designed around a specific application, the shared work environment is a general approach to building collaborative work groups. The goal of this is to not only provide a common data sharing and archiving system, but also to assist in the building of new collaborations and to spur the development of new tools and technologies.

title: THE GLOBAL ALZHEIMER’S ASSOCIATION INTERACTIVE NETWORK (GAAIN)
abstract: The Dementias Platform United Kingdom (DPUK) is an integrated dementias research environment for rapid analysis of multiple independent data sets. DPUK enables risk stratification for the conduct of early phase trials, in-silico experiments, and technology development for molecular and structural imaging, induced pluripotent stem cells (iPSCs) and bio-informatics.  The DPUK informatics environment integrates cohort datawith EHRs, genetic, imaging and wearables data. The secure data analysis environment brings the researcher to the data and is accessed through the DPUK portal http://www.dementiasplatform.uk/. Within the data portal a series of metadata tools enable an increasingly deeper dive into each cohort prior to a formal data access request. DPUK includes data on 34 cohortswith access to over 2million subjects. Studies are ongoing on synapse function, vascular disease mechanisms, innate and adaptive immunity, and genetics of neurodegenerative diseases.  DPUK acts as a communications hub between researchers and data owners to facilitate access to multiple data sets through a single access request. Although established to improve access to UK data, DPUK is able to host data from elsewhere, should that be of scientific value.

title: Data Sharing for Computational Neuroscience
abstract: Computational neuroscience is a subfield of neuroscience that develops models to integrate complex experimental data in order to understand brain function. To constrain and test computational models, researchers need access to a wide variety of experimental data. Much of those data are not readily accessible because neuroscientists fall into separate communities that study the brain at different levels and have not been motivated to provide data to researchers outside their community. To foster sharing of neuroscience data, a workshop was held in 2007, bringing together experimental and theoretical neuroscientists, computer scientists, legal experts and governmental observers. Computational neuroscience was recommended as an ideal field for focusing data sharing, and specific methods, strategies and policies were suggested for achieving it. A new funding area in the NSF/NIH Collaborative Research in Computational Neuroscience (CRCNS) program has been established to support data sharing, guided in part by the workshop recommendations. The new funding area is dedicated to the dissemination of high quality data sets with maximum scientific value for computational neuroscience. The first round of the CRCNS data sharing program supports the preparation of data sets which will be publicly available in 2008. These include electrophysiology and behavioral (eye movement) data described towards the end of this article.

title: Facilitating a culture of responsible and effective sharing of cancer genome data
abstract: Rapid and affordable tumor molecular profiling has led to an explosion of clinical and genomic data poised to enhance the diagnosis, prognostication and treatment of cancer. A critical point has now been reached at which the analysis and storage of annotated clinical and genomic information in unconnected silos will stall the advancement of precision cancer care. Information systems must be harmonized to overcome the multiple technical and logistical barriers to data sharing. Against this backdrop, the Global Alliance for Genomic Health (GA4GH) was established in 2013 to create a common framework that enables responsible, voluntary and secure sharing of clinical and genomic data. This Perspective from the GA4GH Clinical Working Group Cancer Task Team highlights the data-aggregation challenges faced by the field, suggests potential collaborative solutions and describes how GA4GH can catalyze a harmonized data-sharing culture.

title: Developing and implementing an institute-wide data sharing policy
abstract: The Wellcome Trust Sanger Institute has a strong reputation for prepublication data sharing as a result of its policy of rapid release of genome sequence data and particularly through its contribution to the Human Genome Project. The practicalities of broad data sharing remain largely uncharted, especially to cover the wide range of data types currently produced by genomic studies and to adequately address ethical issues. This paper describes the processes and challenges involved in implementing a data sharing policy on an institute-wide scale. This includes questions of governance, practical aspects of applying principles to diverse experimental contexts, building enabling systems and infrastructure, incentives and collaborative issues.

title: Data Sharing.
abstract: The aerial view of the concept of data sharing is beautiful. What could be better than having high-quality information carefully reexamined for the possibility that new nuggets of useful data are lying there, previously unseen? The potential for leveraging existing results for even more benefit pays appropriate increased tribute to the patients who put themselves at risk to generate the data. The moral imperative to honor their collective sacrifice is the trump card that takes this trick. However, many of us who have actually conducted clinical research, managed clinical studies and data collection and analysis, and curated data sets have . . .

title: Data Sharing - Is the Juice Worth the Squeeze?
abstract: Making clinical trial data widely available is ethically imperative and scientifically justified, but the experience with clinicalstudydatarequest.com suggests that we need to find ways to improve the use and output of data-sharing projects before broadening the effort.

title: Towards evidence based research
abstract: To avoid waste of research, no new studies should be done without a systematic review of existing evidence, argue Hans Lund and colleagues