DOI: 10.25934/00005853
title: Data-sharing and re-analysis for main studies assessed by the European Medicines Agency - a
cross-sectional study on European Public Assessment Reports


description: The influence of main trials (i.e. evidence used for drug marketing approval) as assessed by the European Medicine Agency (EMA) is paramount. These studies have a major impact on drug Marketing Authorizations and can change the practices of European medical practitioners and the care offered to millions of patients in the European Union. Because of the major financial conflicts of interest inherent in the evaluation of pharmaceuticals (1, 2), stakeholders are typically more confident when the results and conclusions of these studies can be verified. For a long time, however, transparency was lacking and the individual patient data (IPD) and accompanying material (code, protocol, data analysis plan, etc.) to reproduce these analyses were unavailable. An empirical analysis suggests that only a small number of re-analyses of randomized controlled trials (RCTs) have been published to date; of these, only a minority were conducted by entirely independent authors (3). Data-sharing enabling such re-analyses is being increasingly mandated in medicine. 
And indeed, the EMA aimed to pioneer transparency in this field when, in November 2010, it decided to share every piece of documentation received, in the wake of the first version of policy 0043 (4). As part of its transparency policy, the EMA publishes European Public Assessment Reports (EPAR) after the European Commission’s decision on the specific medicines. These reports include, amongst other documents, results of main trials (5). On October 2nd 2014 the EMA released its policy 0070 on “publication of clinical data for medicinal products for human use”(6). The agency describes a two-step approach. From 1st of January 2015 clinical reports on medicines submitted for Marketing Authorization have been published. A second step includes the publication of IPD. A date for the implementation of this step still needs to be fixed. However,  as a result of Brexit and the relocation of the EMA to the Netherlands, further developments and renovation have been stopped for the moment (7, 8). Efforts are therefore still needed to reach full transparency in the EMA.
On the other hand, biopharmaceutical companies (i.e. Pharmaceutical Research and Manufacturers of America [PhRMA] and the European Federation of Pharmaceutical Industries and Associations [EFPIA] endorsed a commitment ‘to enhancing public health through responsible sharing of clinical trial data’ in a manner that is consistent with 3 main principles: safeguarding the privacy of patients, respecting the integrity of national regulatory systems, and maintaining incentives for investment in biomedical research (9). Despite this commitment from 2013, an audit found that data availability was reached for only 9/61 (15%) clinical trials on medicines sponsored by the pharmaceutical industry and first published between 1 July 2015 and 31 December 2015 in the top 10 journals of general and internal medicine (10). If such low rates of data-sharing were also to be observed for main trials, it would invalidate any efforts towards reproducibility for these important studies. 
However, the environment for data-sharing is changing fast. And indeed, data-sharing Platforms like ViVli, YODA project or Clinical Study Data Request are more and more widely used. In fall of 2019 these platforms gathered a large number of trials sponsored by the pharmaceutical industry. The three together reached about 8000 RCTs in November 2019 (11). Despite this available data, re-analyses are still sparse. Among the 88 published outputs we identified resulting from data-sharing on these platforms, only 3 were re-analyses: “Restoring Study 329” by Le Noury et al. which contradicted the initial publication, a trial that was already known to be misreported (12), a reanalysis of the TORCH trial suggesting an overestimation of the treatment effect in the original study (13), and the reanalysis of the “SMART-AF” trial which came to similar conclusions to the original study (14).

As part of a global research program on reproducibility in therapeutic research (ReiTheR, funded by the French National Research Agency), we designed the present cross-sectional study to assess inferential reproducibility (i.e. when IPD is available, whether qualitatively similar conclusions can be drawn from a reanalysis of the original trials) for main studies assessed by the EMA. 


Up to now, there has been no clear picture of the inferential reproducibility of the main studies in European Assessment Reports. These main studies are used to assess the efficacy of new medical products which will influence the lives of millions of people in the European Union. If data availability were to prove low it should urge the EMA to implement an even stronger data-sharing policy. If re-analyses of available data show low reproducibility, it would argue for independent re-analyses at the time of the approval. On the other hand, if there were to be no issues in terms of reproducibility, it would reinforce the confidence one can have in EMA’s transparency concerning processes and decisions. 

In 2019, the EU Clinical Trial Regulation 536/2014 will come into force and will further extend the boundaries of data-sharing and transparency in the EMA as well as within the European Union. The EU Portal and Data Base is being developed, creating a single-entry point for submitting clinical trials in the EU. A further advantage is that not only information about clinical trials included in Marketing Authorization Applications (MAA) can be found on the portal, but the aim is to have data about every single trial conducted in the European Union, whether or not it is part of an MAA (37). Our research could help to highlight the interest of the future regulation.  

To further enhance the impact of this project, the reproducibility of our results will be checked by comparison with another reproducibility project in the ReiTheR project.