# Overview of the EU PAS register post-authorization studies performed in Europe from September 2010 to December 2018

Janet Sultana, Salvatore Crisafulli, Mariana Almas, Ippazio Cosimo Antonazzo, Esme Baan, Claudia Bartolini, Maria Paola Bertuccio, Fedele Bonifazi, Annalisa Capuano, Antonella Didio, et al.

► To cite this version:

Janet Sultana, Salvatore Crisafulli, Mariana Almas, Ippazio Cosimo Antonazzo, Esme Baan, et al. Overview of the EU PAS register post-authorization studies performed in Europe from September 2010 to December 2018. Pharmacoepidemiology and Drug Safety, 2022, 10.1002/pds.5413 . hal-03575236

HAL Id: hal-03575236

https://hal.science/hal-03575236

Submitted on 15 Feb 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
ORIGINAL ARTICLE

<figure><img src="image_2.png" /></figure>

# Overview of the European post-authorisation study register post-authorization studies performed in Europe from September 2010 to December 2018

Janet Sultana<fnref n="1" />,<fnref n="2" /> | Salvatore Crisafulli<fnref n="3" /> | Mariana Almas<fnref n="4" /> |

Ippazio Cosimo Antonazzo<fnref n="5" />,<fnref n="6" /> | Esme Baan<fnref n="7" /> | Claudia Bartolini<fnref n="6" /> |

Maria Paola Bertuccio<fnref n="3" /> | Fedele Bonifazi<fnref n="8" />,<fnref n="9" /> | Annalisa Capuano<fnref n="10" />,<fnref n="11" /> |

Antonella Didio<fnref n="8" />,<fnref n="9" /> | Vera Ehrenstein<fnref n="12" /> ID | Mariagrazia Felisi<fnref n="8" />,<fnref n="13" /> |

Carmen Ferrajolo<fnref n="10" />,<fnref n="11" /> | Andrea Fontana<fnref n="14" /> | Remy Francisca<fnref n="7" /> |

Annie Fourrier-Reglat<fnref n="15" /> | Joan Fortuny<fnref n="16" /> | Rosa Gini<fnref n="6" /> ID | Giulia Hyeraci<fnref n="6" /> |

Christel Hoeve<fnref n="7" /> | Christos Kontogiorgis¹⁷ ID | Valentina Isgrò¹⁸ |

Panagiotis-Nikolaos Lalagkas¹⁷ | Luca L'Abbate¹⁸ | Deborah Layton¹⁹ |

Annalisa Landi<fnref n="8" />,<fnref n="9" /> | Silvia Narduzzi¹⁹ | Leonardo Roque Pereira²⁰ |

Georgios Poulentzas¹⁷ | Concetta Rafaniello<fnref n="10" />,<fnref n="11" /> | Giuseppe Roberto<fnref n="6" /> ID |

Giulia Scondotto<fnref n="3" /> | Liberata Sportiello<fnref n="10" />,<fnref n="11" /> | Maddalena Toma<fnref n="8" />,<fnref n="9" /> |

Massoud Toussi¹⁹ | Katia Verhamme<fnref n="7" /> | Elisabetta Volpe<fnref n="8" />,<fnref n="9" /> | Gianluca Trifirò¹⁸ |

On behalf the Working Group 3 of ENCePP (Inventory of EU data sources and methodological approaches for multisource studies)

<footnote n="1">¹Pharmacy Department, Mater Dei Hospital, Msida, Malta</footnote>

<footnote n="2">²Exeter College of Medicine and Health, Exeter, UK</footnote>

<footnote n="3">³Department of Medicine, University of Verona, Verona, Italy</footnote>

<footnote n="4">⁴Real World Solutions Department, IQVIA, Lisbon, Portugal</footnote>

<footnote n="5">⁵Research Centre on Public Health (CESP), University of Milan-Bicocca, Milan, Italy</footnote>

<footnote n="6">⁶Agenzia Regionale di Sanità della Toscana, Florence, Italy</footnote>

<footnote n="7">⁷Department of Medical Informatics, Erasmus Medical Centre, Rotterdam, The Netherlands</footnote>

<footnote n="8">⁸TEDDY European Network of Excellence for Paediatric Clinical Research, Pavia, Italy</footnote>

<footnote n="9">⁹Fondazione per la Ricerca Farmacologica Gianni Benzi Onlus, Bari, Italy</footnote>

<footnote n="10">¹⁰Department of Experimental Medicine, University of Campania "Vanvitelli", Naples, Italy</footnote>

<footnote n="11">¹¹Campania Regional Center of Pharmacovigilance and Pharmacoepidemiology, Naples, Italy</footnote>

<footnote n="12">¹²Department of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark</footnote>

<footnote n="13">¹³CVBF Consorzio per Valutazioni Biologiche e Farmacologiche, Pavia, Italy</footnote>

<footnote n="14">¹⁴Unit of Biostatistics, Fondazione IRCCS Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy</footnote>

<footnote n="15">¹⁵Univ. Bordeaux, INSERM, Bordeaux Population Health Research Center, Team of Pharmacoepidemiology, UMR 1219, Bordeaux, France</footnote>

<footnote n="16">¹⁶RTI Health Solutions, Barcelona, Spain</footnote>
**Correspondence**

Gianluca Trifirò, Department of Diagnostics and Public Health, University of Verona, Verona, Italy.

Email: gianluca.trifiro@univr.it

**Funding information**

Università degli Studi di Verona Within the CRUI-CARE Agreement

**Abstract**

**Background:** The European post-authorisation study (EU PAS) register is a repository launched in 2010 by the European Medicines Agency (EMA). All EMA-requested PAS, commonly observational studies, must be recorded in this register. Multi-database studies (MDS) leveraging secondary data have become an important strategy to conduct PAS in recent years, as reflected by the type of studies registered in the EU PAS register.

**Objectives:** To analyse and describe PAS in the EU PAS register, with focus on MDS.

**Methods:** Studies in the EU PAS register from inception to 31st December 2018 were described concerning transparency, regulatory obligations, scope, study type (e.g., observational study, clinical trial, survey, systematic review/meta-analysis), study design, type of data collection and target population. MDS were defined as studies conducted through secondary use of >1 data source not linked at patient-level. Data extraction was carried out independently by 14 centres with expertise in pharmacopoeiemiology, using publicly available information in the EU PAS register including study protocol, whenever available, using a standardised data collection form. For validation purposes, a second revision of key fields for a 15% random sample of studies was carried out by a different centre. The inter-rater reliability (IRR) was then calculated. Finally, to identify predictors of primary data collection-based studies/versus those based on secondary use of healthcare databases) or MDS (vs. non-MDS), odds ratios (OR) and 95% confidence intervals (CI) were calculated fitting univariate logistic regression models.

**Results:** Overall, 1426 studies were identified. Clinical trials (N = 30; 2%), systematic reviews/meta-analyses (N = 16; 1%) and miscellaneous study designs (N = 46; 3%) were much less common than observational studies (N = 1227; 86%). The protocol was available for 63% (N = 360) of 572 observational studies requested by a competent authority. Overall, 36% (N = 446) of observational studies were based fully or partially on primary data collection. Of 757 observational studies based on secondary use of data alone, 282 (37%) were MDS. Drug utilisation was significantly more common as a study scope in MDS compared to non-MDS studies. The overall percentage agreement among collaborating centres that collected the data concerning study variables was highest for study type (93.5%) and lowest for type of secondary data (67.8%).

**Conclusions:** Observational studies were the most common type of studies in the EU PAS register, but 30% used primary data, which is more resource-intensive. Almost half of observational studies using secondary data were MDS. Data recording in the EU PAS register may be improved further, including more widespread availability of study protocols to improve transparency.

<footnote n="17">¹⁷Democritus University of Thrace, Alexandroupolis, Greece</footnote>

<footnote n="18">¹⁸Department of Diagnostics and Public Health, University of Verona, Verona, Italy</footnote>

<footnote n="19">¹⁹Data Science Hub, Real World Solutions, IQVIA, London, UK</footnote>

<footnote n="20">²⁰Department of Datascience & Biostatistics, University Medical Center Utrecht, Utrecht, The Netherlands</footnote>
### KEYWORDS

EU PAS register, multi-database studies, post-authorization studies

### Key Points

* The present study provides a detailed description of all studies registered in the European post-authorisation study (EU PAS) register from its inception till the end of 2018, focusing on various aspects of study design and multiple database studies specifically.

* Our results showed a steady increase in the number of observational studies registered, providing the most recent updated review of the EU PAS register.

* We leveraging a network of pharmacoepidemiologists from various centres belonging to European Network of Centres for Pharmacoepidemiology and Pharmacovigilance to collect additional and detailed information on methodological aspects of observational studies.

* The present study is the only one that evaluates the inter-rater reliability of different investigators during the data collection process, thus providing indirect measure of the completeness and accuracy of the data collected

## 1 | INTRODUCTION

The European post-authorisation study (EU PAS) register is a publicly available repository of post-authorisation (PAS) studies developed and supported by the European Medicines Agency (EMA) through the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP).¹ Current pharmacovigilance legislation in Europe requires to make the study protocols and summary of results publicly available for post-authorisation safety studies (PASS) imposed as a condition to the Marketing Authorisation (RMP category 1) or as a specific obligation in the context of a Marketing Authorisation under exceptional circumstances (RMP category 2). This register is also aimed to host non-imposed studies, such as those required as per Risk Management Plan (RMP category 3)² and all observational studies performed on authorised medicinal products, including effectiveness studies. The EU PAS register is the platform through which such studies, as well as other non-imposed studies, are made accessible online. Indeed, the aim of the EU PAS register, in line with ENCePP's mandate, is to improve the transparency of studies conducted within the EU area and beyond.

The EU PAS register has been described in detail in several previous studies. Engel & Almas et al, in a review of 189 PASS identified in the Pharmacovigilance Risk Assessment Committee (PRAC) Meeting Minutes from July 2012 to July 2015,³ searched the availability of those studies in the EU PAS and showed that only 93 of the studies were registered in the EU PAS register and among them only 43% had available protocols. Analysis of PRAC comments identified several methodological concerns, and limited reasoning behind the methodological decisions and feasibility considerations. Slightly more PASS used primary data collection (i.e., data prospectively collected for the purpose of the study) except those studies assessing drug utilisation where secondary use of already existing healthcare data was leveraged. Another review, conducted by Carroll et al and focusing on studies available on the EU PAS register as of October 2016,⁴ corroborated that primary data collection was more common in studies aimed at assessing safety and effectiveness and less common when

assessing drug utilisation. In 2018, Vora et al and Farcas et al,⁵,⁶ used the EU PAS register to explore specifically studies evaluating the implementation and impact of risk minimisation measures.

All these previously conducted EU PAS register-based studies relied primary on the data reported within the register, without validation of the collected information or additional information on methodological aspects based on expert review. Another gap in these previous studies is the lack of focus on multiple-database studies (MDS), which are observational studies conducted using more than one source of routinely collected data (e.g., claims database, electronic health records [EHRs]). MDS are of particular importance because they allow the accrual of a very large cohort of patients, which is of particular relevance to paediatric populations and rare diseases, as well as several other populations of special interest. Indeed, the number of MDS has been increasing over the years. Since EU PAS register is a platform for the mandatory recording of data on observational studies as per EU legislation, identifying and describing such studies and their impact on the landscape of observational research is therefore of great value as this has never been done to our knowledge.⁷

Given how quickly observational research is growing, the aim of the present descriptive study was to conduct an updated and detailed review of the studies that were registered in the EU PAS register from its inception till the end of 2018, providing additional information on data type (e.g., distinguishing between claims data, EHRs, etc.) and study design (e.g., distinguishing between descriptive studies, cohort studies, case-control studies, etc.). Another aim of this study was to conduct an assessment of the inter-rater reliability following the collection of data.

## 2 | METHODS

### 2.1 | Data collection

A dataset containing all studies found in the EU PAS register from its inception to 31st December 2018 was provided by the EMA. The EU
PAS register is publicly accessible online (http://www.encepp.eu/encepp/studySearch.htm). The data collection was carried out independently by 34 investigators from 13 academic centres or contract research organisations being part of ENCePP on common and detailed instructions and using the same electronic case report form (Figure 1). The resulting dataset contained data from the EU PAS register concerning different aspects of study transparency (ENCePP Seal, protocol and availability of publication), regulatory obligations, methodology, target population, scope and drug under study (chemically synthetised vs. biological drugs, orphan drugs). EU PAS register data was supplemented by retrieving data from study protocols or publications, if available, on source of funding, whether a study was an MDS (defined as a study using more than 1 source of already existing databases which could not be linked at patient level), study design, use of reference drug for formal comparison (if any). The full protocol for data collection, including the fields provided by the EMA, is provided in Appendix S1A. Once all the data was collected it was harmonised based on pre-defined criteria.

To evaluate how consistent data collection was, data retrieved by investigators from a centre was checked by investigators from another centre for a 15% random sample of studies using the same protocol used for the main data collection; each investigator validated five studies. Any disagreements were resolved through the intervention of a third centre. This was done for the following key variables: setting, study type (new classification), study design, type of secondary data used, whether the study was an MDS, use of reference drug for formal comparison, drug type, whether the study drug was an orphan drug. After data collection was completed, automated quality check of data entry was conducted for the following fields: study type, data collection method, type of secondary data if applicable, whether the study was MDS,

study type and study design. In brief, all investigators were asked to collect de novo data concerning these fields while blinded to previous assessment done by investigators belonging to a different centre.

## 2.2 | Statistical analysis

The cumulative frequency of study registration in the EU PAS register was plotted. This was stratified by type of study, data collection and by MDS or non-MDS status specifically. An overview of all studies was provided using descriptive statistics. This was done by stratifying at a high level by type of study (classified as clinical trials, observational studies, systematic review/meta-analysis, questionnaire-based surveys and others).

Finally, odds ratios (OR) and 95% confidence intervals (CIs) were calculated fitting univariate logistic regression models to investigate whether study-related variables (e.g., study type, data source, etc.) were associated with the use of primary data versus use of secondary data as a (reference) and whether they were associated with non-MDS versus MDS (reference) as a reference.

To better understand the data in the EU PAS register and add value to that data with the inclusion of further information related to methodology a large number of investigators was involved for collecting data.

Cohen's Kappa statistic was calculated in order to evaluate the inter-rater reliability (IRR). This was done by single variables and macro-categories consisting of several variables. Kappa estimates and their 95% CI were obtained by the resampling bootstrap method to account for heterogeneity between academic centres. Bootstrap replicate number was set equal to 100 000.

<figure><img src="image_3.png" /><figcaption>**FIGURE 1** The methodological approach for EU PAS register review. EU PASs, European post-authorization studies [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>
All statistical analyses were performed using R Studio (version1.3).

## 3 | RESULTS

After excluding two duplicates, a total of 1426 PASs were identified in the EU PAS register from its inception to 31st December 2018. The majority of studies were observational (N = 1227; 86.0%). Almost a quarter of all observational studies were based on primary data collection (N = 299; 24.4%; Figure 2). Two-thirds of observational studies were based only on secondary use of existing healthcare data, that is, data not collected primarily for research purposes (N = 757; 61.7%). N = 299 (24.4%) were based on primary data and N = 147 (12.0%) were based on both primary data collection and secondary use of databases.

The frequency of studies registered in the EU PAS register showed a steady increase in the number of observational studies from 2010 to 2016, while a slight decrease was observed during the last 2 years under study (Figure 3). Almost a third of observational studies (N = 718; 58.5%) had a protocol deposited in the EU PAS register (Table 1), similar to most study types except for clinical trials, where only N = 11 (36.7%) had a protocol deposited. The protocol was available for 63% (N = 360) of observational studies requested by a competent authority. Only 6.9% of all observational studies had information on ENCePP Seal, while the availability of a publication was much more common: about a third of all observational studies

had a publication available. Just under half of all studies were required by a competent authority. The most common type of risk management plan (RMP) classification for observational studies was RMP 3, that is, required by a competent authority (29.3%). Information on the RMP status was missing in 53 (4.3%) observational studies and it was classified as ‘non-EU RMP only’ for 102 (8.3%) of them.

The scope of most observational studies was risk assessment (56.7%), followed by drug utilisation (36.2%). A large proportion of observational studies was conducted included the elderly (82.2%). Only 36.0% (N = 442) of observational studies were conducted included children, and not all of these studies included children exclusively. With regards to drugs of interest, biologic drugs were studied in 24.3% of all observational studies; orphan drugs were the focus of 12.8% of observational studies.

Concerning the type of secondary data used, among observational studies the use of claims, EHRs or existing registries was similar, at 13.4, 14.9 and 11.1%, respectively (Figure 4). The most commonly used study design among observational studies was the cohort study (52.4%), followed by other types of descriptive studies (23.5%), such as drug utilisation studies, disease epidemiology and pharmacokinetics (Figure 5). Overall, N = 319 (26.0%) of observational studies were classified as MDS.

The overall percentage of agreement in data categorisation among collaborating centres that collected the data concerning study variables was highest for study type (93.5%) and lowest for type of

<figure><img src="image_4.png" /><figcaption>**FIGURE 2** Flowchart of study types in the EU PAS register. ‘Others’ refers to any study design other than clinical trials, observational studies, systematic reviews, observational studies or questionnaires. Unknown refers to any study design that was not specified. EU PASs, European post-authorization studies [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>
<figure><img src="image_5.png" /></figure>

**Abbreviations:** MDS - multiple database studies

**FIGURE 3** Cumulative frequency of studies recorded in the EU PAS register over time. The study by Carrol et al, is included as a benchmark of the most recent overview of the EU PAS register, including however only studies requested by the European Medicines Agency. MDS, multiple database studies; EU PASs, European post-authorization studies [Colour figure can be viewed at wileyonlinelibrary.com]

secondary data (67.8%; Table 2). The low level of agreement for secondary data is expected to have an impact on the overall level of agreement. These results were largely in line with total kappa coefficients. The values of Cohen's kappa and the centre variations in Cohen's kappa for key, along with their 95% CIs, are shown in Figures S1B1 and S1B2, respectively.

Compared to studies based on the secondary use of data, those based on primary data collection were less likely to have a protocol deposited, to be funded by public entities and to use a reference drug for formal comparison, while they were more likely to be funded by pharmaceutical companies (Table 3). In general, there was no substantial difference in study design between studies based on primary data collection or secondary use of data, although descriptive studies were slightly more common in the former.

Only a third of all claims- and EHR-based observational studies were requested by a regulator (Table 4). Claims data were more commonly used for risk assessment than EHRs (60.8% vs. 40.7%).

Among observational studies, as compared to non-MDS, MDS were more likely to have a protocol deposited in the EU PAS register (OR: 1.9; 95% CI: 1.4-2.5), to have an ENCePP Seal (OR: 3.0; 1.9-

4.7), to be funded by national/international drug agencies (OR: 1.9; 95% CI: 1.1-3.4) and to include children as study population (OR: 2.1; 95% CI: 1.5-2.6). Moreover, almost two-thirds of MDS were requested by a regulator. Interestingly, drug utilisation was significantly more common as a study scope in MDS compared to non-MDS studies (OR: 1.3; 95% CI: 1.1-1.7). However, MDS were less likely to consider orphan drugs as the main exposure (OR: 0.5; 95% CI: 0.3-0.8) (Table 5).

## 4 | DISCUSSION

The present study provides a detailed description of all studies registered in the EU PAS register from its inception until the end of 2018, focusing and providing detail on various aspects of study design and multiple database studies specifically. As expected, as compared to the most recent review of EU PAS register available,⁴ our results showed a steady increase in the number of observational studies registered. The present study adds to the available literature by providing the most recent updated review of the EU PAS register, including all
<table><caption>TABLE 1 Description of all studies identified in the EU PAS register</caption><thead><tr><th></th><th>Clinical trials<br>N = 30 (%)<br>N (%)</th><th>Observational studies<br>N = 1227 (%)<br>N (%)</th><th>Systematic reviews/meta-analyses N = 16 (%)<br>N (%)</th><th>Questionnaire-based<br>surveys N = 107 (%)<br>N (%)</th></tr></thead><tbody><tr><td>Protocol deposited</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>11 (36.7)</td><td>718 (58.5)</td><td>10 (62.5)</td><td>63 (58.9)</td></tr><tr><td>No</td><td>19 (63.3)</td><td>509 (41.5)</td><td>6 (37.5)</td><td>44 (41.1)</td></tr><tr><td>ENCePP Seal</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>1 (3.3)</td><td>85 (6.9)</td><td>0 (0.0)</td><td>1 (0.9)</td></tr><tr><td>No</td><td>29 (96.7)</td><td>1142 (93.1)</td><td>16 (100.0)</td><td>106 (99.1)</td></tr><tr><td>Requested by a regulator</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>10 (33.3)</td><td>571 (46.5)</td><td>5 (31.2)</td><td>68 (63.6)</td></tr><tr><td>No</td><td>18 (60.0)</td><td>637 (51.9)</td><td>11 (68.8)</td><td>39 (36.4)</td></tr><tr><td>Unknown</td><td>2 (6.7)</td><td>19 (1.5)</td><td>0 (0.0)</td><td>0 (0.0)</td></tr><tr><td>Status</td><td></td><td></td><td></td><td></td></tr><tr><td>Planned</td><td>4 (13.3)</td><td>168 (13.7)</td><td>4 (25.0)</td><td>15 (14.0)</td></tr><tr><td>Ongoing</td><td>8 (26.7)</td><td>523 (42.6)</td><td>2 (12.5)</td><td>31 (29.0)</td></tr><tr><td>Finalised</td><td>18 (60.0)</td><td>536 (43.7)</td><td>10 (62.5)</td><td>61 (57.0)</td></tr><tr><td>Source of funding</td><td></td><td></td><td></td><td></td></tr><tr><td>Pharmaceutical company</td><td>20 (66.7)</td><td>1005 (81.9)</td><td>10 (62.5)</td><td>97 (90.7)</td></tr><tr><td>National/international drug agency</td><td>0 (0)</td><td>53 (4.3)</td><td>3 (18.8)</td><td>1 (0.9)</td></tr><tr><td>Public entities excluding drug<br>agencies</td><td>8 (26.7)</td><td>65 (5.3)</td><td>2 (12.5)</td><td>5 (4.7)</td></tr><tr><td>Self-funded</td><td>1 (3.3)</td><td>23 (1.9)</td><td>1 (6.2)</td><td>0 (0)</td></tr><tr><td>More than one source</td><td>0 (0)</td><td>62 (5.1)</td><td>0 (0)</td><td>4 (3.7)</td></tr><tr><td>Unknown</td><td>1 (3.3)</td><td>19 (1.5)</td><td>0 (0)</td><td>0 (0)</td></tr><tr><td>RMP status</td><td></td><td></td><td></td><td></td></tr><tr><td>Not applicable</td><td>15 (50.1)</td><td>600 (48.9)</td><td>11 (68.8)</td><td>25 (23.4)</td></tr><tr><td>EU RMP 1</td><td>1 (3.3)</td><td>83 (6.8)</td><td>0 (0)</td><td>10 (9.3)</td></tr><tr><td>EU RMP 2</td><td>1 (3.3)</td><td>30 (2.4)</td><td>0 (0)</td><td>3 (2.8)</td></tr><tr><td>EU RMP 3</td><td>6 (20.0)</td><td>359 (29.3)</td><td>3 (18.7)</td><td>42 (39.3)</td></tr><tr><td>Non-EU RMP only</td><td>6 (20.0)</td><td>102 (8.3)</td><td>2 (12.5)</td><td>21 (19.6)</td></tr><tr><td>Missing—no info at all</td><td>1 (3.3)</td><td>53 (4.3)</td><td>0 (0)</td><td>6 (5.6)</td></tr><tr><td>Multidatabase study</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>-</td><td>319 (26.0)</td><td>-</td><td>-</td></tr><tr><td>No</td><td>-</td><td>864 (70.4)</td><td>-</td><td>-</td></tr><tr><td>Unknown</td><td>-</td><td>44 (3.6)</td><td>-</td><td>-</td></tr><tr><td>Data strategy</td><td></td><td></td><td></td><td></td></tr><tr><td>Local data extraction and analysis,<br>common protocol</td><td>-</td><td>49 (4.0)</td><td>-</td><td>-</td></tr><tr><td>Local data extraction and central<br>analysis on patient-level raw data</td><td>-</td><td>48 (3.9)</td><td>-</td><td>-</td></tr><tr><td>Study-specific local data extraction in<br>a common data model and central<br>analysis</td><td>-</td><td>40 (3.3)</td><td>-</td><td>-</td></tr><tr><td>General local data extraction in a<br>common data model and central<br>analysis</td><td>-</td><td>48 (3.9)</td><td>-</td><td>-</td></tr><tr><td>Not applicable</td><td>-</td><td>678 (55.3)</td><td>-</td><td>-</td></tr><tr><td>Unknown</td><td>-</td><td>169 (13.8)</td><td>-</td><td>-</td></tr><tr><td>Missing data</td><td>-</td><td>195 (15.8)</td><td>-</td><td>-</td></tr></tbody></table>

(Continues)
<table><caption>TABLE 1 (Continued)</caption><thead><tr><th></th><th>Clinical trials<br>N = 30 (%)<br>N (%)</th><th>Observational studies<br>N = 1227 (%)<br>N (%)</th><th>Systematic reviews/meta-<br>analyses N = 16 (%)<br>N (%)</th><th>Questionnaire-based<br>surveys N = 107 (%)<br>N (%)</th></tr></thead><tbody><tr><td>Product lifecycle</td><td></td><td></td><td></td><td></td></tr><tr><td>Pre-marketing (for any indication)</td><td>4 (13.3)</td><td>4 (0.3)</td><td>0 (0.0)</td><td>0 (0.0)</td></tr><tr><td>Post-marketing</td><td>17 (56.7)</td><td>1125 (91.7)</td><td>11 (68.8)</td><td>95 (88.8)</td></tr><tr><td>Not applicable</td><td>6 (20.0)</td><td>82 (6.7)</td><td>5 (31.2)</td><td>11 (10.3)</td></tr><tr><td>Unknown</td><td>3 (10.0)</td><td>16 (1.3)</td><td>0 (0.0)</td><td>1 (0.9)</td></tr><tr><td>Use of reference drug for formal comparison</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>7 (23.3)</td><td>336 (27.4)</td><td>4 (25.0)</td><td>4 (25.0)</td></tr><tr><td>No</td><td>22 (73.4)</td><td>844 (68.8)</td><td>10 (62.5)</td><td>10 (62.5)</td></tr><tr><td>Unknown</td><td>1 (3.3)</td><td>47 (3.8)</td><td>2 (12.5)</td><td>2 (12.5)</td></tr><tr><td>Scope of the study</td><td></td><td></td><td></td><td></td></tr><tr><td>Disease epidemiology</td><td>4 (13.3)</td><td>212 (17.3)</td><td>4 (9.5)</td><td>2 (12.5)</td></tr><tr><td>Risk assessment</td><td>13 (43.3)</td><td>696 (56.7)</td><td>15 (35.7)</td><td>11 (68.8)</td></tr><tr><td>Drug utilisation</td><td>4 (13.3)</td><td>444 (36.2)</td><td>6 (14.3)</td><td>1 (6.3)</td></tr><tr><td>Effectiveness evaluation</td><td>17 (56.7)</td><td>372 (30.3)</td><td>27 (64.3)</td><td>15 (93.8)</td></tr><tr><td>Other*</td><td>14 (46.7)</td><td>246 (20.0)</td><td>21 (50.0)</td><td>3 (18.8)</td></tr><tr><td>Population of interest—age</td><td></td><td></td><td></td><td></td></tr><tr><td>Children</td><td>5 (16.7)</td><td>442 (36.0)</td><td>7 (43.8)</td><td>25 (23.4)</td></tr><tr><td>Adults</td><td>27 (90.0)</td><td>1103 (89.9)</td><td>15 (93.8)</td><td>105 (98.1)</td></tr><tr><td>Elderly persons</td><td>19 (63.3)</td><td>1008 (82.2)</td><td>13 (81.3)</td><td>98 (91.6)</td></tr><tr><td>Unknown</td><td>0 (0.0)</td><td>11 (0.9)</td><td>1 (6.3)</td><td>1 (0.9)</td></tr><tr><td>Population of interest—special populations</td><td></td><td></td><td></td><td></td></tr><tr><td>Immunocompromised</td><td>1 (3.3)</td><td>87 (7.1)</td><td>1 (6.3)</td><td>4 (3.7)</td></tr><tr><td>Hepatic impairment</td><td>1 (3.3)</td><td>94 (7.7)</td><td>0 (0.0)</td><td>2 (1.9)</td></tr><tr><td>Renal impairment</td><td>2 (6.7)</td><td>108 (8.8)</td><td>0 (0.0)</td><td>5 (4.7)</td></tr><tr><td>Pregnant women</td><td>2 (6.7)</td><td>132 (10.8)</td><td>2 (12.5)</td><td>3 (2.8)</td></tr><tr><td>Breast-feeding women</td><td>0 (0.0)</td><td>13 (1.1)</td><td>0 (0.0)</td><td>0 (0.0)</td></tr><tr><td>Other</td><td>3 (10.0)</td><td>73 (5.9)</td><td>2 (4.8)</td><td>1 (6.3)</td></tr><tr><td>Drug of interest—general</td><td></td><td></td><td></td><td></td></tr><tr><td>Non-biologic</td><td>19 (63.3)</td><td>754 (61.5)</td><td>10 (62.5)</td><td>70 (65.4)</td></tr><tr><td>Biologic</td><td>6 (20.0)</td><td>298 (24.3)</td><td>2 (12.5)</td><td>26 (24.3)</td></tr><tr><td>Both biologic and non-biologic</td><td>0 (0.0)</td><td>25 (2.0)</td><td>0 (0.0)</td><td>1 (0.9)</td></tr><tr><td>None</td><td>5 (16.7)</td><td>109 (8.9)</td><td>4 (25.0)</td><td>8 (7.5)</td></tr><tr><td>Unknown</td><td>0 (0.0)</td><td>41 (3.3)</td><td>0 (0.0)</td><td>2 (1.9)</td></tr><tr><td>Drug of interest—orphan drugs</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>5 (16.7)</td><td>157 (12.8)</td><td>0 (0.0)</td><td>14 (13.1)</td></tr><tr><td>No</td><td>25 (83.3)</td><td>1024 (83.5)</td><td>16 (100.0)</td><td>89 (83.2)</td></tr><tr><td>Unknown</td><td>0 (0.0)</td><td>46 (3.7)</td><td>0 (0.0)</td><td>4 (3.7)</td></tr><tr><td>Publication available</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>12 (40.0)</td><td>359 (29.3)</td><td>7 (43.8)</td><td>29 (27.1)</td></tr><tr><td>No</td><td>18 (60.0)</td><td>868 (70.7)</td><td>9 (56.2)</td><td>78 (72.9)</td></tr></tbody></table>

Abbreviations: PASs, European post-authorization studies; RMP, risk management plan.

PAS rather than just PASS. To do this, we leveraging a network of pharmacoepidemiologists from various centres belonging to ENCePP to collect additional and detailed information on methodological

aspects of observational studies. Furthermore, to our knowledge, the present study is the only one that validates the inter-rater reliability of different collaborating centres during the data collection process, thus
<figure><img src="image_6.png" /><figcaption>**FIGURE 4** Distribution of different types of secondary data among all observational studies. EHR, electronic health records [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**FIGURE 5** Different types of study designs among observational studies [Colour figure can be viewed at wileyonlinelibrary.com]</figcaption></figure>

providing indirect measure of the completeness and accuracy of the data collected in the EU PAS register.

From our study, it appears that only 61.7% of observational studies were conducted through the secondary use of existing healthcare data. This finding indicates that the remaining one third of observational studies were conducted using the much more resource-intensive method of collecting data ad hoc, that is, primary data collection. It is not known whether such studies could have been easily conducted using routinely collected healthcare data, which provide detailed information on population characteristics, exposure and outcome. In general, we expected fewer studies to have been

conducted using primary data, as large use of primary data source can reflect the need to improve the access to data for research purpose. In line with previous studies, primary data were more commonly used for effectiveness and risk assessment studies,³,⁴ while secondary data was more likely to be used for drug utilisation studies and risk assessment studies. The increase in use secondary data source was sharpest in the last 2 years, while until 2016 the use of primary data source was in 56% of the 316 studies identified in the EU PAS register.⁴

Of note, some markers of high ethical standards and transparency such as protocol deposition are often missing. This result has to be
<table><caption>TABLE 2 Inter-rate reliability agreement among collaborating centres in data categorisation</caption><thead><tr><th>Variables</th><th>Categories</th><th>Kappa coefficient<sup>a</sup></th><th>Agreement N = 214 (%)</th><th>Total kappa coefficient<sup>a</sup></th><th>Interpretation of total kappa coefficient</th></tr></thead><tbody><tr><td rowspan="6">Study type</td><td>Clinical trials</td><td>0.795</td><td rowspan="6">200 (93.5)</td><td rowspan="6">0.765</td><td rowspan="6">Substantial agreement</td></tr><tr><td>Observational studies</td><td>0.758</td></tr><tr><td>Systematic reviews/ meta-analyses</td><td>1.000</td></tr><tr><td>Questionnaire-based surveys</td><td>0.769</td></tr><tr><td>Others</td><td>0.795</td></tr><tr><td>Unknown</td><td>-</td></tr><tr><td rowspan="4">Data Collection</td><td>Primary</td><td>0.717</td><td rowspan="4">171 (79.9)</td><td rowspan="4">0.652</td><td rowspan="4">Substantial agreement</td></tr><tr><td>Secondary</td><td>0.666</td></tr><tr><td>Primary and secondary (mixed)</td><td>0.562</td></tr><tr><td>Unknown</td><td>-</td></tr><tr><td rowspan="5">Drug type</td><td>Non-biologic</td><td>0.685</td><td rowspan="5">176 (82.2)</td><td rowspan="5">0.668</td><td rowspan="5">Substantial agreement</td></tr><tr><td>Biologic</td><td>0.827</td></tr><tr><td>Both biologic and non-biologic</td><td>-</td></tr><tr><td>None</td><td>0.497</td></tr><tr><td>Unknown</td><td>-</td></tr><tr><td rowspan="3">Use of reference drug for formal comparison</td><td>Yes</td><td>0.659</td><td rowspan="3">171 (79.9)</td><td rowspan="3">0.663</td><td rowspan="3">Substantial agreement</td></tr><tr><td>No</td><td>0.621</td></tr><tr><td>Unknown</td><td>0.127</td></tr><tr><td rowspan="4">Setting</td><td>Routine</td><td>0.493</td><td rowspan="4">193 (90.2)</td><td rowspan="4">0.518</td><td rowspan="4">Moderate agreement</td></tr><tr><td>Experimental</td><td>0.829</td></tr><tr><td>Unknown</td><td>-</td></tr><tr><td>Not applicable</td><td>0.509</td></tr><tr><td rowspan="7">Secondary data</td><td>Chart abstraction</td><td>0.481</td><td rowspan="7">145 (67.8)</td><td rowspan="7">0.501</td><td rowspan="7">Moderate agreement</td></tr><tr><td>Claims database</td><td>0.131</td></tr><tr><td>EHR</td><td>0.457</td></tr><tr><td>Existing registry</td><td>0.505</td></tr><tr><td>Not applicable-not secondary data</td><td>0.728</td></tr><tr><td>More than 1</td><td>0.579</td></tr><tr><td>Unknown</td><td>0.314</td></tr><tr><td rowspan="3">Multiple database study</td><td>Yes</td><td>0.503</td><td rowspan="3">176 (77.6)</td><td rowspan="3">0.485</td><td rowspan="3">Moderate agreement</td></tr><tr><td>No</td><td>0.478</td></tr><tr><td>Unknown</td><td>0.274</td></tr><tr><td rowspan="3">Orphan drug</td><td>Yes</td><td>0.478</td><td rowspan="3">179 (83.6)</td><td rowspan="3">0.453</td><td rowspan="3">Moderate agreement</td></tr><tr><td>No</td><td>0.422</td></tr><tr><td>Unknown</td><td>-</td></tr></tbody></table>

Note: -, empty cell in the dataset.

balanced with the fact that registration is legally binding and subject to financial penalties only for imposed PASS at the time of final study report. Finalised studies that are RMP 1 and 2 with protocol available were 68.5%. It was found that 37% of clinical trials did not have a

deposited protocol while 58% of observational trials did. Of 1204 observational studies, 15 (1.2%) had no information whatsoever on study design, highlighting the importance of transparency. Transparency in research along with methodological rigour are essential to

<footnote><sup>a</sup>Kappa result be interpreted as follows: ≤0 no agreement and 0.01-0.20 as none to slight, 0.21-0.40 as fair, 0.41-0.60 as moderate, 0.61-0.80 as substantial, and 0.81-1.00 as almost perfect agreement.<sup>b</sup></footnote>
<table><caption>TABLE 3 Description of the observational studies identified in the EU PAS register. Secondary data was considered the comparator</caption><thead><tr><th></th><th>Primary and secondary data (mixed) N = 147 (%)</th><th>Primary data N = 299 (%)</th><th>Secondary data N = 757 (%)</th><th>Primary data versus Secondary data OR (95% CI)</th></tr></thead><tbody><tr><td>Protocol deposited</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>95 (64.6)</td><td>146 (48.8)</td><td>476 (62.9)</td><td>0.5 (0.4-0.7)</td></tr><tr><td>No</td><td>52 (35.4)</td><td>153 (51.2)</td><td>281 (37.1)</td><td>-</td></tr><tr><td>Source of funding</td><td></td><td></td><td></td><td></td></tr><tr><td>Pharmaceutical company</td><td>125 (85.0)</td><td>272 (91.0)</td><td>587 (77.5)</td><td>2.9 (1.9-4.4)</td></tr><tr><td>National/international drug agency</td><td>4 (2.7)</td><td>5 (1.7)</td><td>43 (5.7)</td><td>0.3 (0.1-0.7)</td></tr><tr><td>Public entities excluding drug agencies</td><td>9 (6.1)</td><td>7 (2.3)</td><td>48 (6.3)</td><td>0.3 (0.1-0.8)</td></tr><tr><td>Self-funded</td><td>1 (0.7)</td><td>5 (1.7)</td><td>17 (2.2)</td><td>0.7 (0.2-2.0)</td></tr><tr><td>More than one source</td><td>8 (5.4)</td><td>8 (2.7)</td><td>45 (5.9)</td><td>0.4 (0.2-0.9)</td></tr><tr><td>Unknown</td><td>0 (0.0)</td><td>2 (0.7)</td><td>17 (2.2)</td><td>0.2 (0.0-1.2)</td></tr><tr><td>RMP status</td><td></td><td></td><td></td><td></td></tr><tr><td>Not applicable</td><td>59 (40.1)</td><td>133 (44.5)</td><td>398 (52.6)</td><td>0.7 (0.5-0.9)</td></tr><tr><td>EU RMP 1</td><td>8 (5.4)</td><td>24 (8.0)</td><td>49 (6.5)</td><td>1.2 (0.7-2.1)</td></tr><tr><td>EU RMP 2</td><td>4 (2.7)</td><td>11 (3.7)</td><td>15 (2.0)</td><td>1.8 (0.8-4.1)</td></tr><tr><td>EU RMP 3</td><td>52 (35.4)</td><td>77 (25.8)</td><td>222 (29.3)</td><td>0.8 (0.6-1.1)</td></tr><tr><td>Non-EU RMP only</td><td>17 (11.6)</td><td>42 (14.0)</td><td>41 (5.4)</td><td>2.8 (1.8-4.4)</td></tr><tr><td>Missing—no info at all</td><td>7 (4.8)</td><td>12 (4.0)</td><td>32 (4.2)</td><td>0.9 (0.4-1.8)</td></tr><tr><td>Study design</td><td></td><td></td><td></td><td></td></tr><tr><td>Cohort study</td><td>65 (44.2)</td><td>156 (52.2)</td><td>409 (54.0)</td><td>0.9 (0.7-1.2)</td></tr><tr><td>Cross-sectional study</td><td>7 (4.8)</td><td>20 (6.7)</td><td>31 (4.1)</td><td>1.6 (0.9-2.9)</td></tr><tr><td>Case-control study</td><td>7 (4.8)</td><td>7 (2.3)</td><td>28 (3.7)</td><td>0.6 (0.2-1.4)</td></tr><tr><td>Case cross-over study</td><td>0 (0.0)</td><td>3 (1.0)</td><td>1 (0.1)</td><td>-</td></tr><tr><td>Nested case-control study</td><td>0 (0.0)</td><td>1 (0.3)</td><td>20 (2.6)</td><td>0.1 (0.0-0.9)</td></tr><tr><td>Other type of descriptive studies</td><td>43 (29.3)</td><td>89 (29.8)</td><td>172 (22.7)</td><td>1.4 (1.0-1.9)</td></tr><tr><td>Other types of analytic studies</td><td>7 (4.8)</td><td>15 (5.0)</td><td>37 (4.9)</td><td>1.0 (0.5-1.9)</td></tr><tr><td>More than 1 study design</td><td>15 (10.2)</td><td>2 (0.7)</td><td>52 (6.9)</td><td>0. (0.0-0.3)</td></tr><tr><td>Unknown</td><td>3 (2.0)</td><td>6 (2.0)</td><td>7 (0.9)</td><td>-</td></tr><tr><td>Use of reference drug for formal comparison</td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>32 (21.8)</td><td>46 (15.4)</td><td>255 (33.7)</td><td>0.3 (0.2-0.5)</td></tr><tr><td>No</td><td>110 (74.8)</td><td>242 (80.9)</td><td>475 (62.7)</td><td>-</td></tr><tr><td>Unknown</td><td>5 (3.4)</td><td>11 (3.7)</td><td>27 (3.6)</td><td>-</td></tr><tr><td>Setting</td><td></td><td></td><td></td><td></td></tr><tr><td>Routine</td><td>140 (95.2)</td><td>282 (94.3)</td><td>740 (97.8)</td><td>0.4 (0.2-0.7)</td></tr><tr><td>Experimental</td><td>3 (2.0)</td><td>7 (2.3)</td><td>11 (1.5)</td><td>1.6 (0.6-4.2)</td></tr><tr><td>Unknown</td><td>2 (1.4)</td><td>9 (3.0)</td><td>1 (0.1)</td><td>-</td></tr><tr><td>Not applicable</td><td>2 (1.4)</td><td>1 (0.3)</td><td>5 (0.7)</td><td>-</td></tr><tr><td>Scope of the study</td><td></td><td></td><td></td><td></td></tr><tr><td>Disease epidemiology</td><td>38 (25.9)</td><td>40 (13.4)</td><td>130 (17.2)</td><td>0.7 (0.5-1.1)</td></tr><tr><td>Risk assessment</td><td>96 (65.3)</td><td>178 (59.5)</td><td>404 (53.4)</td><td>1.3 (0.9-1.6)</td></tr><tr><td>Drug utilisation</td><td>48 (32.7)</td><td>99 (33.1)</td><td>287 (37.9)</td><td>0.8 (0.6-1.0)</td></tr><tr><td>Effectiveness evaluation</td><td>54 (36.7)</td><td>131 (43.8)</td><td>176 (23.2)</td><td>2.5 (1.9-3.4)</td></tr><tr><td>Other*</td><td>34 (23.1)</td><td>74 (24.7)</td><td>136 (18.0)</td><td>1.5 (1.1-2.1)</td></tr></tbody></table>

(Continues)
<table><caption>TABLE 3 (Continued)</caption><thead><tr><th></th><th>Primary and secondary data (mixed) N = 147 (%)</th><th>Primary data N = 299 (%)</th><th>Secondary data N = 757 (%)</th><th>Primary data versus Secondary data OR (95% CI)</th></tr></thead><tbody><tr><td colspan="5"><strong>Population of interest—age</strong></td></tr><tr><td>Children</td><td>62 (42.2)</td><td>101 (33.8)</td><td>272 (35.9)</td><td>0.9 (0.7–1.2)</td></tr><tr><td>Adults</td><td>130 (88.4)</td><td>273 (91.3)</td><td>679 (89.7)</td><td>1.2 (0.7–1.9)</td></tr><tr><td>Elderly persons</td><td>110 (74.8)</td><td>246 (82.3)</td><td>634 (83.8)</td><td>0.9 (0.6–1.2)</td></tr><tr><td>Unknown</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td colspan="5"><strong>Population of interest—special populations</strong></td></tr><tr><td>Immunocompromised</td><td>7 (4.8)</td><td>27 (9.0)</td><td>51 (6.7)</td><td>1.3 (0.8–2.2)</td></tr><tr><td>Hepatic impairment</td><td>11 (7.5)</td><td>24 (8.0)</td><td>57 (7.5)</td><td>1.1 (0.6–1.7)</td></tr><tr><td>Renal impairment</td><td>8 (5.4)</td><td>25 (8.4)</td><td>71 (9.4)</td><td>0.9 (0.5–1.4)</td></tr><tr><td>Pregnant women</td><td>21 (14.3)</td><td>32 (10.7)</td><td>76 (10.0)</td><td>1.1 (0.7–1.6)</td></tr><tr><td>Breast-feeding women</td><td>0 (0.0)</td><td>1 (0.3)</td><td>11 (1.5)</td><td>0.2 (0.0–1.7)</td></tr><tr><td>Other</td><td>11 (7.5)</td><td>22 (7.4)</td><td>39 (5.2)</td><td>1.4 (0.8–2.5)</td></tr><tr><td colspan="5"><strong>Drug of interest—general</strong></td></tr><tr><td>Non-biologic</td><td>76 (51.7)</td><td>172 (57.5)</td><td>495 (65.4)</td><td>1.9 (1.4–2.6)</td></tr><tr><td>Biologic</td><td>50 (34.0)</td><td>94 (31.4)</td><td>145 (19.2)</td><td>-</td></tr><tr><td>Both biologic and non-biologic</td><td>5 (3.4)</td><td>6 (2.0)</td><td>14 (1.8)</td><td>-</td></tr><tr><td>None</td><td>13 (8.8)</td><td>19 (6.4)</td><td>73 (9.6)</td><td>-</td></tr><tr><td>Unknown</td><td>3 (2.0)</td><td>8 (2.7)</td><td>30 (40.0)</td><td>-</td></tr><tr><td colspan="5"><strong>Drug of interest—orphan drugs</strong></td></tr><tr><td>Yes</td><td>21 (14.3)</td><td>54 (18.1)</td><td>78 (10.3)</td><td>1.9 (1.3–2.8)</td></tr><tr><td>No</td><td>125 (85.0)</td><td>233 (77.9)</td><td>646 (85.3)</td><td>-</td></tr><tr><td>Unknown</td><td>1 (0.7)</td><td>12 (4.0)</td><td>33 (4.4)</td><td>-</td></tr><tr><td colspan="5"><strong>Publication available</strong></td></tr><tr><td>Yes</td><td>51 (34.7)</td><td>58 (19.4)</td><td>247 (32.6)</td><td>0.5 (0.4–0.7)</td></tr><tr><td>No</td><td>96 (65.3)</td><td>241 (80.6)</td><td>510 (67.4)</td><td>-</td></tr></tbody></table>

Abbreviations: 95% CI, 95% confidence interval; PASs, European post-authorization studies; OR, odds ratio; RMP, risk management plan.

increase the standards of pharmacoepidemiological research in Europe.⁹ Competent authorities should keep emphasising the importance of study registration in order to increase the level of transparency in the research landscape, particularly for publicly funded research. Another measure of high ethical standards and transparency, the ENCePP Seal, was only available in 7% of all observation studies. The ENCePP Seal identifies studies in the register that adhere to the entirety of the Code's provisions, including timelines for the submission of documents to the ENCePP Secretariat. However, obtaining this Seal is associated with an added bureaucratic burden that researchers may not be willing to undertake. The usefulness of the ENCePP Seal on a broader scale may be somewhat limited if so few studies undertake the procedures to obtain it. Other initiatives such as the revision of the ENCePP Code of Conduct have recently been undertaken but it is not known to what extent this is being adopted.¹⁰

Another interesting finding of this study with important implications concerns the level of agreement seen between pharmacoepidemiologists in the process of data collection from the EU PAS register. While there

was substantial agreement for study type, data collection, drug type and use of reference drug for formal comparison, there was only moderate agreement for the rest of the fields, including study setting, classification of secondary data, whether the study was an MDS and whether the study drug was an orphan drug. High agreement was not reached for any of the fields, due to ambiguous, missing or conflicting reporting. These findings are thought provoking and suggest that data in the EU PAS register was not always clear and complete. Indeed, the quality of data entry in the EU PAS register is not monitored at all as with other similar platforms such as clinicaltrials.gov. In the latter, National Library of Medicine (NLM) staff conducts a review of registered study records for obvious errors or inconsistencies, with important issues being communicated directly to the investigators.¹¹ However, this does not guarantee complete and accurate records for all studies.¹² In the EU PAS register, it is completely up to the investigator's discretion to conduct data entry correctly and accurately as no quality control is conducted. The very small number of duplicate studies identified, two, is potentially an indicator that in this sense, the register's data is of satisfactory quality.
<table><caption>TABLE 4 Description of specific types of mixed and secondary data</caption><thead><tr><th></th><th>Chart abstraction<br>N = 63 (%)</th><th>Claims database<br>N = 171 (%)</th><th>EHRs<br>N = 189 (%)</th><th>Existing registry<br>N = 166 (%)</th><th>More than one type of data<br>N = 290 (%)</th></tr><tr><th></th><th>N (%)</th><th>N (%)</th><th>N (%)</th><th>N (%)</th><th>N (%)</th></tr></thead><tbody><tr><td>Protocol deposited</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>36 (57.1)</td><td>84 (49.1)</td><td>126 (66.7)</td><td>103 (62.0)</td><td>209 (72.1)</td></tr><tr><td>No</td><td>27 (42.9)</td><td>87 (50.9)</td><td>63 (33.3)</td><td>63 (38.0)</td><td>81 (27.9)</td></tr><tr><td>Requested by a regulator</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>32 (51.1)</td><td>55 (32.2)</td><td>64 (33.9)</td><td>67 (46.4)</td><td>150 (51.7)</td></tr><tr><td>No</td><td>31 (49.2)</td><td>115 (67.3)</td><td>119 (63.0)</td><td>86 (51.8)</td><td>133 (45.9)</td></tr><tr><td>Unknown</td><td>0 (0.0)</td><td>1 (0.6)</td><td>2 (3.2)</td><td>3 (1.8)</td><td>7 (2.4)</td></tr><tr><td>Source of funding</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Pharmaceutical company</td><td>56 (88.9)</td><td>134 (78.4)</td><td>138 (73.0)</td><td>135 (81.3)</td><td>218 (75.2)</td></tr><tr><td>National/international drug agency</td><td>2 (3.2)</td><td>10 (5.8)</td><td>17 (9.0)</td><td>9 (5.4)</td><td>12 (4.1)</td></tr><tr><td>Public entities excluding drug<br>agencies</td><td>4 (6.3)</td><td>12 (7.0)</td><td>10 (5.3)</td><td>5 (3.0)</td><td>28 (9.7)</td></tr><tr><td>Self-funded</td><td>1 (1.6)</td><td>5 (2.9)</td><td>2 (1.1)</td><td>3 (1.8)</td><td>7 (2.4)</td></tr><tr><td>More than one source</td><td>0 (0.0)</td><td>7 (4.1)</td><td>15 (7.9)</td><td>12 (7.2)</td><td>20 (6.9)</td></tr><tr><td>Unknown</td><td>0 (0.0)</td><td>3 (1.8)</td><td>7 (3.7)</td><td>2 (1.2)</td><td>5 (1.7)</td></tr><tr><td>RMP status</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Not applicable</td><td>28 (44.4)</td><td>108 (63.2)</td><td>122 (64.6)</td><td>73 (44.0)</td><td>122 (42.1)</td></tr><tr><td>EU RMP 1</td><td>9 (14.3)</td><td>4 (2.3)</td><td>12 (6.3)</td><td>10 (6.0)</td><td>20 (6.9)</td></tr><tr><td>EU RMP 2</td><td>1 (1.6)</td><td>1 (0.6)</td><td>2 (1.1)</td><td>9 (5.4)</td><td>5 (1.7)</td></tr><tr><td>EU RMP 3</td><td>18 (28.6)</td><td>41 (24.0)</td><td>31 (16.4)</td><td>49 (29.5)</td><td>114 (39.3)</td></tr><tr><td>Non-EU RMP only</td><td>3 (4.8)</td><td>14 (8.2)</td><td>13 (6.9)</td><td>14 (8.4)</td><td>15 (5.2)</td></tr><tr><td>Missing—no info at all</td><td>4 (6.3)</td><td>3 (1.8)</td><td>9 (4.8)</td><td>11 (6.6)</td><td>14 (4.8)</td></tr><tr><td>Data model</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Local data extraction and analysis,<br>common protocol</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>Local data extraction and central<br>analysis on patient-level raw<br>data</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>Study-specific local data extraction<br>in a common data model and<br>central analysis</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>General local data extraction in a<br>common data model and central<br>analysis</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>Not applicable</td><td>36 (57.1)</td><td>109 (63.7)</td><td>122 (64.6)</td><td>93 (56.0)</td><td>54 (18.6)</td></tr><tr><td>Unknown</td><td>10 (15.9)</td><td>21 (12.3)</td><td>15 (7.9)</td><td>17 (10.2)</td><td>77 (26.6)</td></tr><tr><td>Missing</td><td>11 (17.5)</td><td>29 (17.0)</td><td>16 (8.5)</td><td>41 (24.7)</td><td>57 (19.7)</td></tr><tr><td>Study design</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Cohort study</td><td>22 (34.9)</td><td>109 (63.7)</td><td>90 (47.6)</td><td>83 (50.0)</td><td>154 (53.1)</td></tr><tr><td>Cross-sectional study</td><td>5 (7.9)</td><td>1 (0.6)</td><td>12 (6.3)</td><td>8 (4.8)</td><td>10 (3.4)</td></tr><tr><td>Case-control study</td><td>1 (1.6)</td><td>6 (3.5)</td><td>8 (4.2)</td><td>10 (6.0)</td><td>10 (3.4)</td></tr><tr><td>Case cross-over study</td><td>0 (0.0)</td><td>0 (0.0)</td><td>0 (0.0)</td><td>1 (0.6)</td><td>0 (0.0)</td></tr><tr><td>Nested case-control study</td><td>0 (0.0)</td><td>4 (2.3)</td><td>8 (4.2)</td><td>1 (0.6)</td><td>8 (2.8)</td></tr><tr><td>Other types of descriptive studies</td><td>23 (36.5)</td><td>29 (17.0)</td><td>50 (27.5)</td><td>39 (23.5)</td><td>63 (21.7)</td></tr><tr><td>Other types of analytic studies</td><td>7 (11.1)</td><td>3 (1.8)</td><td>11 (5.8)</td><td>15 (9.0)</td><td>9 (3.1)</td></tr><tr><td>More than 1 study design</td><td>3 (4.8)</td><td>16 (9.4)</td><td>8 (4.2)</td><td>8 (4.8)</td><td>31 (10.7)</td></tr><tr><td>Unknown</td><td>2 (3.2)</td><td>3 (1.8)</td><td>0 (0.0)</td><td>1 (0.6)</td><td>5 (1.7)</td></tr></tbody></table>

(Continues)
<table><caption>TABLE 4 (Continued)</caption><thead><tr><th></th><th>Chart abstraction<br>N = 63 (%)</th><th>Claims database<br>N = 171 (%)</th><th>EHRs<br>N = 189 (%)</th><th>Existing registry<br>N = 166 (%)</th><th>More than one type of data<br>N = 290 (%)</th></tr><tr><th></th><th>N (%)</th><th>N (%)</th><th>N (%)</th><th>N (%)</th><th>N (%)</th></tr></thead><tbody><tr><td colspan="6"><strong>Use of reference drug for formal comparison</strong></td></tr><tr><td>Yes</td><td>7 (11.1)</td><td>86 (50.3)</td><td>47 (24.9)</td><td>45 (27.1)</td><td>97 (33.4)</td></tr><tr><td>No</td><td>52 (82.5)</td><td>73 (42.7)</td><td>138 (73.0)</td><td>114 (68.7)</td><td>187 (64.5)</td></tr><tr><td>Unknown</td><td>4 (6.3)</td><td>12 (7.0)</td><td>4 (2.1)</td><td>7 (4.2)</td><td>6 (2.1)</td></tr><tr><td colspan="6"><strong>Scope of the study</strong></td></tr><tr><td>Disease epidemiology</td><td>6 (9.5)</td><td>26 (15.2)</td><td>36 (19.0)</td><td>44 (26.5)</td><td>49 (16.9)</td></tr><tr><td>Risk assessment</td><td>28 (44.4)</td><td>104 (60.8)</td><td>77 (40.7)</td><td>95 (57.2)</td><td>175 (60.3)</td></tr><tr><td>Drug utilisation</td><td>31 (49.2)</td><td>58 (33.9)</td><td>73 (38.6)</td><td>50 (30.1)</td><td>117 (40.3)</td></tr><tr><td>Effectiveness evaluation</td><td>25 (39.7)</td><td>37 (21.6)</td><td>54 (28.6)</td><td>43 (25.9)</td><td>54 (18.6)</td></tr><tr><td>Other*</td><td>17 (27.0)</td><td>31 (18.1)</td><td>31 (16.4)</td><td>33 (19.9)</td><td>53 (18.3)</td></tr><tr><td colspan="6"><strong>Population of interest—age</strong></td></tr><tr><td>Children</td><td>15 (23.8)</td><td>42 (24.6)</td><td>60 (31.7)</td><td>67 (40.4)</td><td>133 (45.9)</td></tr><tr><td>Adults</td><td>58 (92.1)</td><td>148 (86.5)</td><td>176 (93.1)</td><td>147 (88.26)</td><td>256 (88.3)</td></tr><tr><td>Elderly persons</td><td>54 (85.7)</td><td>144 (84.2)</td><td>160 (84.7)</td><td>127 (76.5)</td><td>240 (82.8)</td></tr><tr><td colspan="6"><strong>Population of interest—special populations</strong></td></tr><tr><td>Immunocompromised</td><td>2 (3.2)</td><td>8 (4.7)</td><td>11 (5.8)</td><td>21 (12.7)</td><td>16 (5.5)</td></tr><tr><td>Hepatic impairment</td><td>3 (4.8)</td><td>8 (4.7)</td><td>14 (7.4)</td><td>21 (12.7)</td><td>23 (7.9)</td></tr><tr><td>Renal impairment</td><td>4 (6.3)</td><td>13 (7.6)</td><td>13 (6.9)</td><td>22 (13.3)</td><td>26 (9.0)</td></tr><tr><td>Pregnant women</td><td>2 (3.2)</td><td>18 (10.5)</td><td>14 (7.4)</td><td>29 (17.5)</td><td>33 (11.4)</td></tr><tr><td>Breast-feeding women</td><td>0 (0.0)</td><td>1 (0.6)</td><td>0 (0.0)</td><td>4 (2.4)</td><td>6 (2.1)</td></tr><tr><td>Other</td><td>5 (7.9)</td><td>12 (7.0)</td><td>13 (6.9)</td><td>5 (3.0)</td><td>11 (3.8)</td></tr><tr><td colspan="6"><strong>Drug of interest—general</strong></td></tr><tr><td>Non-biologic</td><td>30 (47.6)</td><td>116 (67.8)</td><td>130 (68.8)</td><td>91 (54.8)</td><td>189 (65.2)</td></tr><tr><td>Biologic</td><td>25 (39.7)</td><td>33 (19.3)</td><td>35 (18.5)</td><td>39 (23.5)</td><td>57 (19.7)</td></tr><tr><td>Both biologic and non-biologic</td><td>2 (3.2)</td><td>3 (1.8)</td><td>1 (0.5)</td><td>7 (4.2)</td><td>5 (1.7)</td></tr><tr><td>None</td><td>3 (4.8)</td><td>12 (7.0)</td><td>18 (9.5)</td><td>22 (13.3)</td><td>29 (10.0)</td></tr><tr><td>Unknown</td><td>3 (4.8)</td><td>7 (4.1)</td><td>5 (2.6)</td><td>7 (4.2)</td><td>10 (3.4)</td></tr><tr><td colspan="6"><strong>Drug of interest—orphan drugs</strong></td></tr><tr><td>Yes</td><td>8 (12.7)</td><td>14 (8.2)</td><td>16 (8.5)</td><td>31 (18.7)</td><td>20 (6.9)</td></tr><tr><td>No</td><td>53 (84.1)</td><td>147 (86.0)</td><td>168 (88.9)</td><td>129 (77.7)</td><td>258 (89.0)</td></tr><tr><td>Unknown</td><td>2 (3.2)</td><td>10 (5.8)</td><td>5 (2.6)</td><td>6 (3.6)</td><td>12 (4.1)</td></tr><tr><td colspan="6"><strong>Publication available</strong></td></tr><tr><td>Yes</td><td>19 (30.2)</td><td>63 (36.8)</td><td>73 (38.6)</td><td>45 (27.1)</td><td>94 (32.4)</td></tr><tr><td>No</td><td>44 (69.8)</td><td>108 (63.2)</td><td>116 (61.4)</td><td>121 (72.9)</td><td>196 (67.6)</td></tr></tbody></table>

Abbreviations: EHR, electronic healthcare record; RMP, risk management plan.

In addition, there are several open text fields, missing categories in the different attributes and certain categories used in more than one attribute which makes it challenging to identify and summarise study types in the EU PAS register information. It should be noted that there is currently no glossary in the EU PAS register to promote harmonisation of terms used. We argue that the terminology to be used should be based on a single accepted definition that is presented in a glossary. The reason is that the classification of studies and other relevant aspects (e.g. data sources, methods etc.) was often found to be conducted using

ambiguous or even incorrect terms. For the EU PAS register to hold promise of a greater transparency and accuracy, by serving as a common repository to share pharmacoepidemiological research, it would be key to further standardise its data elements, and create completion guidelines to offer an unequivocal interpretation of the fields by study registrants.

With our working definition of MDS as studies which use more than one secondary data source, we found more than 20% of observational studies classified as MDS. These studies were more likely to have the ENCePP Seal and to have protocol deposited, all markers of
<table><caption>TABLE 5 Multiple database studies versus non-multiple database studies. MSD was considered the comparator</caption><thead><tr><th></th><th>MDS N = 319 (%)</th><th>Non-MDS N = 864 (%)</th><th>OR (95% CI)</th></tr></thead><tbody><tr><td>Protocol deposited</td><td>222 (69.6)</td><td>492 (56.9)</td><td>1.9 (1.4-2.5)</td></tr><tr><td>EnCePP seal</td><td>42 (13.4)</td><td>43 (4.9)</td><td>3.0 (1.9-4.7)</td></tr><tr><td>Requested by a regulator</td><td>184 (57.7)</td><td>372 (43.0)</td><td>1.6 (1.2-1.9)</td></tr><tr><td>Status</td><td></td><td></td><td></td></tr><tr><td>Planned</td><td>48 (15.3)</td><td>115 (13.4)</td><td>1.2 (0.8-1.7)</td></tr><tr><td>Ongoing</td><td>114 (36.5)</td><td>382 (44.2)</td><td>0.7 (0.5-0.8)</td></tr><tr><td>Finalised</td><td>157 (50.3)</td><td>367 (42.4)</td><td>1.3 (1.1-1.7)</td></tr><tr><td>Source of funding</td><td></td><td></td><td></td></tr><tr><td>Pharmaceutical company</td><td>250 (78.4)</td><td>715 (82.7)</td><td>0.7 (0.5-1.1)</td></tr><tr><td>National/international drug agency</td><td>21 (6.6)</td><td>31 (3.6)</td><td>1.9 (1.1-3.4)</td></tr><tr><td>Public entities excluding drug agencies</td><td>20 (6.3)</td><td>44 (5.1)</td><td>1.3 (0.7-2.2)</td></tr><tr><td>More than one source</td><td>5 (1.6)</td><td>42 (4.8)</td><td>1.3 (0.7-2.2)</td></tr><tr><td>Self-funded</td><td>19 (6.0)</td><td>17 (1.9)</td><td>0.8 (0.3-2.1)</td></tr><tr><td>Unknown</td><td>4 (1.3)</td><td>15 (1.7)</td><td>0.7 (0.2-2.3)</td></tr><tr><td>PI employed by study funder</td><td>170 (53.3)</td><td>560 (64.8)</td><td>0.6 (0.5-0.8)</td></tr><tr><td>RMP status</td><td></td><td></td><td></td></tr><tr><td>EU RMP 1</td><td>134 (42.0)</td><td>50 (5.8)</td><td>1.7 (1.1-2.7)</td></tr><tr><td>EU RMP 2</td><td>30 (9.4)</td><td>24 (2.7)</td><td>0.7 (0.3-1.7)</td></tr><tr><td>EU RMP 3</td><td>6 (1.9)</td><td>225 (26.0)</td><td>1.8 (1.4-2.3)</td></tr><tr><td>Non-EU RMP only</td><td>123 (38.6)</td><td>82 (9.5)</td><td>0.5 (0.3-0.8)</td></tr><tr><td>Not applicable</td><td>15 (4.7)</td><td>442 (51.1)</td><td>0.7 (0.5-0.9)</td></tr><tr><td>Missing—no info at all</td><td>11 (3.4)</td><td>41 (4.7)</td><td>0.7 (0.4-1.4)</td></tr><tr><td>Data collection</td><td></td><td></td><td></td></tr><tr><td>Primary</td><td>-</td><td>298 (34.5)</td><td>-</td></tr><tr><td>Secondary</td><td>282 (90.3)</td><td>452 (52.3)</td><td>6.9 (4.8-10.1)</td></tr><tr><td>Mixed</td><td>35 (11.2)</td><td>108 (12.5)</td><td>0.9 (0.6-1.3)</td></tr><tr><td>Unknown</td><td>2 (0.6)</td><td>6 (0.7)</td><td>-</td></tr><tr><td>Secondary data</td><td></td><td></td><td></td></tr><tr><td>Claims database</td><td>26 (8.3)</td><td>132 (15.2)</td><td>0.5 (0.3-0.7)</td></tr><tr><td>Product lifecycle</td><td></td><td></td><td></td></tr><tr><td>Post-marketing</td><td>2 (0.6)</td><td>795 (92.0)</td><td>0.9 (0.6-1.4)</td></tr><tr><td>Not applicable</td><td>291 (91.2)</td><td>60 (7.0)</td><td>0.8 (0.4-1.3)</td></tr><tr><td>Pre-marketing</td><td>18 (5.6)</td><td>2 (0.2)</td><td>-</td></tr><tr><td>Unknown</td><td>8 (2.5)</td><td>8 (0.9)</td><td>-</td></tr><tr><td>Study design</td><td></td><td></td><td></td></tr><tr><td>Cohort study</td><td>76 (23.8)</td><td>456 (52.8)</td><td>1.1 (0.8-1.3)</td></tr><tr><td>Case-control study</td><td>168 (52.7)</td><td>33 (3.8)</td><td>0.7 (0.3-1.6)</td></tr><tr><td>Case cross-over study</td><td>13 (4.1)</td><td>4 (0.5)</td><td>-</td></tr><tr><td>Cross-sectional study</td><td>9 (2.8)</td><td>45 (5.2)</td><td>0.7 (0.4-1.4)</td></tr><tr><td>Nested case-control study</td><td>0 (0.0)</td><td>15 (1.7)</td><td>1.1 (0.4-2.9)</td></tr><tr><td>Other types of analytic studies</td><td>6 (1.9)</td><td>44 (5.1)</td><td>0.8 (0.4-1.5)</td></tr><tr><td>Other types of descriptive studies</td><td>13 (4.1)</td><td>222 (25.7)</td><td>0.9 (0.7-1.2)</td></tr></tbody></table>

(Continues)
<table><caption>TABLE 5 (Continued)</caption><thead><tr><th></th><th>MDS N = 319 (%)</th><th>Non-MDS N = 864 (%)</th><th>OR (95% CI)</th></tr></thead><tbody><tr><td>More than 1 study design</td><td>32 (10.0)</td><td>34 (3.9)</td><td>2.6 (1.6-4.3)</td></tr><tr><td>Unknown</td><td>2 (0.6)</td><td>11 (1.3)</td><td>-</td></tr><tr><td>Use of reference drug for formal comparison</td><td>102 (32.7)</td><td>224 (25.9)</td><td>1.3 (0.9-1.7)</td></tr><tr><td>Setting</td><td></td><td></td><td></td></tr><tr><td>Routine</td><td>308 (96.6)</td><td>835 (86.9)</td><td>1.1 (0.6-2.3)</td></tr><tr><td>Experimental</td><td>7 (2.2)</td><td>13 (1.5)</td><td>1.4 (0.6-3.6)</td></tr><tr><td>Not applicable</td><td>2 (0.6)</td><td>6 (0.7)</td><td>0.9 (0.2-4.7)</td></tr><tr><td>Unknown</td><td>2 (0.6)</td><td>10 (1.2)</td><td>-</td></tr><tr><td>Scope of the study</td><td></td><td></td><td></td></tr><tr><td>Disease epidemiology</td><td>58 (18.2)</td><td>145 (16.8)</td><td>1.1 (0.8-1.5)</td></tr><tr><td>Risk assessment</td><td>187 (58.6)</td><td>488 (56.5)</td><td>1.1 (0.8-1.4)</td></tr><tr><td>Drug utilisation</td><td>132 (41.4)</td><td>295 (34.1)</td><td>1.3 (1.1-1.7)</td></tr><tr><td>Effectiveness evaluation</td><td>55 (17.2)</td><td>297 (34.4)</td><td>0.4 (0.3-0.5)</td></tr><tr><td>Other</td><td>52 (16.3)</td><td>188 (21.8)</td><td>0.7 (0.5-1.1)</td></tr><tr><td>Population of interest—age</td><td></td><td></td><td></td></tr><tr><td>Children</td><td>154 (48.3)</td><td>284 (32.9)</td><td>2.1 (1.5-2.6)</td></tr><tr><td>Adults</td><td>286 (89.7)</td><td>776 (89.8)</td><td>0.9 (0.6-1.4)</td></tr><tr><td>Elderly persons</td><td>266 (83.4)</td><td>703 (81.4)</td><td>1.1 (0.8-1.6)</td></tr><tr><td>Unknown</td><td>3 (0.9)</td><td>8 (0.9)</td><td>1.1 (0.3-4.1)</td></tr><tr><td>Population of interest—special populations</td><td></td><td></td><td></td></tr><tr><td>Immunocompromised</td><td>22 (6.9)</td><td>64 (7.4)</td><td>0.9 (0.6-1.6)</td></tr><tr><td>Hepatic impairment</td><td>26 (8.2)</td><td>67 (7.8)</td><td>1.1 (0.7-1.7)</td></tr><tr><td>Renal impairment</td><td>28 (8.8)</td><td>76 (8.8)</td><td>0.9 (0.6-1.6)</td></tr><tr><td>Pregnant women</td><td>37 (11.6)</td><td>93 (10.8)</td><td>1.1 (0.7-1.7)</td></tr><tr><td>Breast-feeding women</td><td>6 (1.9)</td><td>7 (0.8)</td><td>2.5 (0.8-7.3)</td></tr><tr><td>Other</td><td>14 (4.4)</td><td>57 (6.6)</td><td>0.6 (0.4-1.2)</td></tr><tr><td>Drug of interest (general)</td><td></td><td></td><td></td></tr><tr><td>Biologic (vs. all the others)</td><td>66</td><td>270</td><td>0.7 (0.6-1.1)</td></tr><tr><td>Orphan drug (vs. all the others)</td><td>26</td><td>149</td><td>0.5 (0.3-0.8)</td></tr><tr><td>Publication available</td><td>101 (31.7)</td><td>250 (28.9)</td><td>1.2 (0.9-1.5)</td></tr></tbody></table>

Abbreviations: 95% CI, 95% confidence intervals; MDS, multiple database study; OR, odds ratio.

high transparency. Orphan drugs were not commonly investigated using MDS, although such studies may have a lot of potential in the rare disease field given the small population sizes expected and the increasing need of merging data from different sources to speed up the development of these medicines.

Our analysis has several strengths compared with previous studies as it provides a detailed overview of the studies registered on EU PAS register, with systemic data collection being conducted by a network of established centres of excellence in pharmacoepidemiology. It is very important to underline that data collection was carried out by researchers independently based on common and very detailed instructions. These instructions were shared, agreed and made accessible and available among all reviewers, to avoid potential errors in the classification method. Again, to further check, blind quality control of

data entry was conducted for some fields, to obtain an overall agreement. In addition, this is the first time that an evaluation of the MDS landscape has been attempted.

However, this study also has some limitations. The most important limitation concerns the accuracy of the data collection, which was sometimes limited by the lack of information recorded in the EU PAS register or lack of clarity of such information. The lack of data and ambiguities in classification of some studies may limit the robustness of our analyses. In these instances, the judgement on categorising characteristics of a study was somewhat subjective. This is a limitation inherent to the way that data is collected in the EU PAS register and to the lack of quality control of such data. Furthermore, since the registration of a study is voluntary, the EU PAS register is not representative of the whole landscape of studies in Europe and some
studies may have not been captured. Another limitation is that the data was collected only until 2018. However, we hold that the present study still has the added value of providing an overview of the EU PAS register compared to the most recently published reviews, which described studies until 2016.⁴

## 5 | CONCLUSION

Observational studies were the most common type of studies in the EU PAS register, but almost one third of observational studies used primary data, which is more resource-intensive. One quarter of observational studies were MDS. MDS hold untapped potential to investigate special populations, such as patients with rare diseases and paediatric patients. A detailed analysis of IRR during data collection suggested that information recorded in the EU PAS register is often not clear or complete. Increasing the quality of study-related information would considerably increase the transparency of research.

### ACKNOWLEDGEMENT

Open Access Funding provided by Universita degli Studi di Verona within the CRUI-CARE Agreement.

### CONFLICT OF INTEREST

The authors declare no conflicts of interest.

### ETHICS STATEMENT

For this study, there was no need to set up or directly manage databases in which there might have been identifiable or identified patient data; therefore, according to local regulations, obtaining an ethics committee approval to carry out this study was not necessary.

### ORCID

Vera Ehrenstein https://orcid.org/0000-0002-3415-3254

Rosa Gini https://orcid.org/0000-0002-6250-877X

Christos Kontogiorgis https://orcid.org/0000-0001-7835-8246

Giuseppe Roberto https://orcid.org/0000-0001-6478-6442

### REFERENCES

1. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. The European Union electronic Register of Post-Authorisation Studies (EU-PAS Register). Accessed August 31, 2021. http://www.encepp.eu/encepp_studies/indexRegister.shtml

2. European Medicines Agency. GVP Module VIII; 2017. Accessed August 31, 2021. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-gvp-module-viii-post-authorisation-safety-studies-rev-3_en.pdf

3. Engel P, Almas MF, De Bruin ML, Starzyk K, Blackburn S, Dreyer NA. Lessons learned on the design and the conduct of post-authorization safety studies: review of 3 years of PRAC oversight. *Br J Clin Pharmacol*. 2017;83(4):884-893.

4. Carroll R, Ramagopalan SV, Cid-Ruzafa J, Lambrelli D, McDonald L. An analysis of characteristics of post-authorisation studies registered on the ENCePP EU-PAS register. *F1000Res*. 2017;6:1447. doi: 10.12688/f1000research.12198.2

5. Vora P, Artime E, Soriano Gabarró M, Qizilbash N, Singh V, Asiimwe A. A review of studies evaluating the effectiveness of risk minimisation measures in Europe using the European Union electronic register of post-authorization studies. *Pharmacoepidemiol Drug Saf*. 2018;27:695-706. doi:10.1002/pds.443

6. Farcas A, Huruba M, Mogosan C. Study design process and outcome indicators of post-authorization studies aimed at evaluating the effectiveness of risk minimization measures in the EU-PAS register. *Br J Clin Pharmacol*. 2019;85(3):476-491. doi:10.1111/bcp.13824

7. Gini R, Sturkenboom MCJ, Sultana J, et al. Different strategies to execute multi-database studies for medicines surveillance in real-world setting: a reflection on the European model. *Clin Pharmacol Ther*. 2020;108(2):228-235.

8. McHugh ML. Inter-rater reliability: the kappa statistic. *Biochem Med (Zagreb)*. 2012;22(3):276-2829.

9. Blake KV, Prilla S, Accadebled S, et al. European medicines agency review of post-authorisation studies with implications for the European network of Centres for Pharmacoepidemiology and Pharmacovigilance. *Pharmacoepidemiol Drug Saf*. 2011;20(10):1021-1029. doi:10.1002/pds.2209

10. Gini R, Fournie X, Dolk H, et al. The ENCePP code of conduct: a best practise for scientific independence and transparency in non-interventional postauthorisation studies. *Pharmacoepidemiol Drug Saf*. 2019;28(4):422-433.

11. Clinicaltrials.gov. Glossary of Common Site Terms; 2021. Accessed August 31, 2021. https://clinicaltrials.gov/ct2/about-studies/glossary

12. Miron L, Gonçalves RS, Musen MA. Obstacles to the reuse of study metadata in ClinicalTrials.gov. *Sci Data*. 2020;7:443.

### SUPPORTING INFORMATION

Additional supporting information may be found in the online version of the article at the publisher's website.

**How to cite this article:** Sultana J, Crisafulli S, Almas M, et al. Overview of the European post-authorisation study register post-authorization studies performed in Europe from September 2010 to December 2018. *Pharmacoepidemiol Drug Saf*. 2022;1-17. doi:10.1002/pds.5413