# The majority of observational studies in leading peer-reviewed medicine journals are not registered and do not have a publicly accessible protocol: a scoping review

S Leducq, Faaris Zaki, Loes M Hollestein, Christian Apfelbacher, Nikhil

Prasanna Ponna, Rishabh Mazmudar, Sonia Gran

► To cite this version:

S Leducq, Faaris Zaki, Loes M Hollestein, Christian Apfelbacher, Nikhil Prasanna Ponna, et al.. The majority of observational studies in leading peer-reviewed medicine journals are not registered and do not have a publicly accessible protocol: a scoping review. Journal of Clinical Epidemiology, 2024, 170, pp.111341. 10.1016/j.jclinepi.2024.111341. hal-04789292

HAL Id: hal-04789292

https://hal.science/hal-04789292v1

Submitted on 18 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
ORIGINAL RESEARCH

The majority of observational studies in leading peer-reviewed medicine journals are not registered and do not have a publicly accessible protocol: a scoping review

Sophie Leducq<sup>a,b,c</sup>, Faaris Zaki<sup>d</sup>, Loes M. Hollestein<sup>e,f</sup>, Christian Apfelbacher<sup>f,g</sup>, Nikhil Prasanna Ponna<sup>h</sup>, Rishabh Mazmudar<sup>i</sup>, Sonia Gran<sup>a,*</sup>

Accepted 25 March 2024; Published online 29 March 2024

**Abstract**

**Objectives:** Observational studies are not subject to the same requirements as randomized controlled trials, such as registration or publishing a protocol. The aim of this scoping review was to estimate the registration rate of observational studies in leading peer-reviewed medicine journals and to evaluate whether protocols were available in the public domain.

**Study Design and Setting:** In March 2023, we searched OVID Medline for observational studies published in 2022 in the top five general medicine journals according to impact factor (*The Lancet*, *The British Medical Journal (BMJ)*, *The Journal of the American Medical Association*, *The New England Journal of Medicine*, and *Annals of Internal Medicine*). We defined an observational study as a cohort study, a case-control study, a cross-sectional study, or a case series. Information on i) the proportion of observational studies that have been registered and ii) the proportion of observational studies that have a protocol available in the public domain was extracted from a random sample of studies.

**Results:** Our search identified 699 studies; 290 studies were selected as full text, and a random sample of 200 studies was included. For half of the studies, the first author worked at a US institution. Most studies were cohort studies (n = 126, 63.0%) and used administrative healthcare records, electronic healthcare records, and registries. Of the 200 observational studies, 20 (10.0%) were registered. Among those, 14 were prospectively registered. Twenty-four studies (12.0%) had a protocol available in the public domain. Studies that were registered or had a protocol, were more frequently published in the BMJ (n = 12/28, 42.9%), had a first author working in the UK (n = 10/28, 35.7%) and used electronic health care records (n = 13/28, 46.4%) compared to studies with no registration and no protocol.

**Conclusion:** The rate of prospectively registered observational studies is worryingly low. Prospective registration of observational studies should be encouraged and standardized to ensure transparency in clinical research and reduce research waste. © 2024 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

**Keywords:** Observational study; Epidemiology; Transparency; Registration; Protocol; Cohort

Funding: No funding has been received for this review. Dr Sophie Leducq has received fees from the *Fonds de Dotation* of the French Society of Dermatology (on an educational grant funded by Amgen, unrelated to the submitted work) and Collège des Enseignants en Dermatologie de France for her fellowship in the Center of Evidence Based Dermatology.

* Corresponding author. Centre of Evidence Based Dermatology, School of Medicine, Applied Health Research Building, University Park, Nottingham, NG7 2RD UK.

E-mail address: sonia.gran@nottingham.ac.uk (S. Gran).

<footnote><sup>a</sup>Centre of Evidence Based Dermatology, School of Medicine, University of Nottingham, Nottingham, UK</footnote>

<footnote><sup>b</sup>Department of Dermatology, University Hospital of Tours, Tours, France</footnote>

<footnote><sup>c</sup>UMR INSERM 1246 - SPHERE, University of Tours, Tours, France</footnote>

<footnote><sup>d</sup>Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom</footnote>

<footnote><sup>e</sup>Department of Research, Netherlands Comprehensive Cancer Organisation (IKNL), Utrecht, The Netherlands</footnote>

<footnote><sup>f</sup>Department of Dermatology, Erasmus University Medical Center, Rotterdam, The Netherlands</footnote>

<footnote><sup>g</sup>Institute of Social Medicine and Health Systems Research, Otto von Guericke University Magdeburg, Magdeburg, Germany</footnote>

<footnote><sup>h</sup>Faculty of Medicine & Health Sciences, University of Nottingham, Nottingham, United Kingdom</footnote>

<footnote><sup>i</sup>Case Western Reserve University School of Medicine, Cleveland, OH, USA</footnote>
## What is new?

### Key findings

* 10% of the observational studies published in 2022 in leading general medicine journals were registered, and 12% had publicly accessible protocols.

* Studies that were registered or had a protocol were more frequently published in the BMJ, had a first author working in the UK, and used electronic health records compared to studies with no registration and no protocol.

### What this adds to what was known?

* This study confirms that there is a lack of transparency with regards to observational studies and quantifies the size of this problem.

* The heterogeneity of platforms used for registering observational studies.

## What is the implication and what should change now?

* There is a need for a common registration platform for researchers conducting observational studies to reduce research waste, selective outcome bias, and standardize information that should be registered. An international consultation process using a Delphi study is planned by our research group and will involve experts (researchers, journal editors, and funders) from different fields.

# 1. Introduction

Observational studies are defined as “studies that do not involve any intervention on the part of the investigator” [1]. They are mainly used to evaluate the effects of exposures or interventions that cannot be appropriately studied in a randomized controlled trial (RCT) or cannot be randomized [2,3]. Because of the high availability, in particular, of routinely collected data (registries, electronic health care records, and administrative databases), observational studies represent a large proportion of published scientific reports [4,5], which form the basis of systematic reviews and meta-analyses.

Despite the increase in observational studies involving participant data and the statement in the Declaration of Helsinki that “every research study involving human subjects must be registered” [6], as well as being vulnerable to bias and selective reporting, observational studies do not adhere to the same standards as other study designs do. For example, RCTs are subject to strict regulation. Mandatory registration of RCTs has reduced reporting bias and selective outcome reporting [7]. More recently, the registration

of systematic reviews has also been encouraged by journals and editors. In 2011, the International Prospective Register of Systematic Reviews (PROSPERO) was created [8], leading to reduced bias and research waste [9]. Therefore, there is a need for standardization and regulation of observational studies to prevent unreliable evidence and methodological shortcomings [10]. Some journals have encouraged the registration and publication of protocols [11–15].

The aim of this study was to determine the proportion of observational studies published in the last year (2022) that were registered or had a protocol across the top five highest impact factor peer-reviewed general medicine journals.

# 2. Materials and methods

## 2.1. Design

This scoping review of published papers was prospectively registered (28th March 2023) on the Center of Evidence Based Dermatology's registration portal and can be accessed online (https://www.nottingham.ac.uk/research/groups/cebd/resources/protocol-registration.aspx).

## 2.2. Identification of observational studies

We searched OVID MEDLINE (search date 28th March 2023) to identify observational studies that were published online between January 1st, 2022, and December 31st, 2022. A librarian helped to determine the search strategy (Appendix 1). We used search filters to identify observational studies (https://libguides.sph.uth.tmc.edu/search_filters/ovid_medline_filters). We looked at studies published in the five highest-impact factor general medicine journals (*The Lancet*, *The British Medical Journal* (BMJ), *The Journal of the American Medical Association*, *The New England Journal of Medicine*, and *Annals of Internal Medicine*). We chose high impact journals as we expected high methodological and reporting standards in these journals.

## 2.3. Study selection

We included observational studies (cohort study, case-control study, cross-sectional study, and case series). Case reports were excluded as protocols are not generally required [16]. The study selection was conducted by two reviewers independently (SL and FZ) using the software Rayyan®: titles and abstracts and then full texts were screened to exclude nonobservational studies, case reports, and reviews. Differences were resolved by a third reviewer (SG). Among observational studies identified, we randomly selected a sample of 200 observational studies, to accommodate resources available, using a computer-generated list of random numbers with the package random, R v4.2.2. We extracted the data for each of these studies.
## 2.4. Data extraction

The study extraction was conducted independently by three reviewers (SL, FZ, and NP) using a structured data extraction form developed to collect the relevant information from the selected studies (using the software Airtable®) (Appendix 2). The first 150 studies were blindly extracted by SL and FZ, and the next 50 studies were blindly extracted by SL and NP. Any discrepancies were resolved by a fourth reviewer (SG). Before the study started, the reviewers evaluated a set of 10 papers, resolved any differences in extraction, and ensured the interpretation of the data extraction tool was the same for all reviewers. The following data were extracted based on the included study and its protocol (if available), and supplementary data.

(i) Title, first author, country of the first author, journal, medical specialty, study design, data sources, and significant outcomes

(ii) Registration of the study: registration of the study, name of platform, date of registration, and prospective registration of the study (based on author's statement or dates of registration and study start)

(iii) Publication of the protocol: protocol in the public domain, date of publication of the protocol, prospective registration of the protocol (based on author's statement or dates of registration and study start), protocol adherence for primary outcome, protocol adherence for pre-specified subgroups analysis or sensitivity analysis, explanation and justification provided by the authors in the paper in case of deviation, and statistical analysis plan (SAP) or statistical paragraph included in the protocol.

For each study, we determined if the study or protocol had been prospectively registered as defined:

(iv) a study/protocol registered before collecting the data (ie, before the first participant was enrolled) for a prospective study,

(v) a study/protocol registered before assessing or analyzing all data for a retrospective study.

Data from the registration platform and protocol were extracted from the manuscript and its supplementary file. We did not contact the corresponding author or systematically search registration databases if no information was available in the paper published.

## 2.5. Primary outcome

We estimated (i) the proportion of observational studies that had been registered (no restriction on the type of platform); (ii) the proportion of observational studies that had a

protocol available in the public domain. Only registrations and protocols for the specific study were considered. Protocols for a related study or cohort were not included.

## 2.6. Secondary outcomes

We estimated (i) the proportion of published studies that adhere to the registered protocol, (ii) the proportion of protocols that included a SAP or statistical paragraph.

## 2.7. Modifications of this study from the original registered

No modifications were made regarding inclusion criteria or outcomes. A comparison of the general characteristics of the included studies according to the registration and protocol status was not initially planned and was added after the registration of this study. Modifications compared to the initial protocol are highlighted in supplementary material (Appendix 3).

## 2.8. Data synthesis

Relevant information from the included studies was narratively synthesised. Categorical data were expressed with numbers and percentages. Confidence intervals (95% CI) for proportions were estimated. A comparison of the general characteristics of the included studies according to their registration and protocol status was performed (studies that were registered or had a protocol vs studies that were not registered and did not have a protocol). We compared the studies' characteristics with chi-squared tests and Fisher's exact tests. We used R v4.2.2 (R Foundation for Statistical Computing, Vienna, Austria) to conduct the statistical analysis. A figure was created using the software Flourish (Canva UK Operations Ltd, https://flourish.studio).

# 3. Results

## 3.1. Study selection

Our search identified 699 studies. After screening titles and abstracts, 377 were assessed at the full-text stage. Among the 377 assessed, 290 studies were selected, and a random sample of 200 studies was included (Fig 1, Appendix 4).

## 3.2. Characteristics of included studies

The general characteristics of the selected studies are shown in Table 1. Most studies were published in Annals of Internal Medicine (n = 55, 27.5%) and the Journal of the American Medical Association (n = 55, 27.5%). Among the selected studies, the first authors worked mostly in high-income countries. Infectious diseases was the most common medical field (n = 86, 43.0%), and 83 studies were COVID-19 related (41.5%). Most studies were cohort
<figure><img src="image_2.png" /><figcaption>**Figure 1.** Flow-chart of studies screened and included. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)</figcaption></figure>

studies (n = 126, 63.0%) and used data from registry (n = 62, 31.0%), administrative health records (n = 62, 31.0%), or electronic health records (n = 60, 30.0%). Among the studies included, we observed a high rate of observational studies reporting a statistically significant outcome (n = 154, 77.0%).

### 3.3. Primary outcome

Of the 200 observational studies included, 20 (10.0%, 95% CI 6.4%–15.2%) were registered. Fourteen were prospectively registered (70.0%), three were not prospectively registered (15.0%), and for three studies (15.0%), the study start date and/or study data registration were not provided, and therefore, we were not able to determine if the study was prospectively or retrospectively registered (Table 2).

Twenty-four studies (12%, 95% CI 8.0%–17.5%) had a protocol available in the public domain. Among the studies with a protocol available, 10 (41.7%) were prospectively published, and 14 (58.3%) were not prospectively published or had no date (Fig 2). When considering only non-COVID-19- related studies (n = 117), the proportion of registered studies and studies with a protocol were similar, at n = 12 (10.3%) and n = 13 (11.1%), respectively.

Eight different platforms of registration were used, ClinicalTrials.gov (n = 5 studies) and Open Science Framework (n = 4 studies) were the most common (Table 3). Similarly, protocols were published on several platforms (eg, Clinicaltrials.gov, Github or Open Science Framework) or were publicly available within the published study.
<table><caption>**Table 1.** General characteristics of the studies included in the review (N = 200)</caption><thead><tr><th>Characteristics</th><th>N (%)</th></tr></thead><tbody><tr><td>Location of the first author</td><td></td></tr><tr><td>USA</td><td>109 (54.5)</td></tr><tr><td>Europe (Except UK)</td><td>26 (13.0)</td></tr><tr><td>UK</td><td>22 (11.0)</td></tr><tr><td>Canada</td><td>11 (5.5)</td></tr><tr><td>Other</td><td>32 (16.0)</td></tr><tr><td>Medical field<sup>a</sup></td><td></td></tr><tr><td>Infectious diseases</td><td>86 (43.0)</td></tr><tr><td>Cardiology</td><td>16 (8.0)</td></tr><tr><td>Endocrinology/Diabetology</td><td>11 (5.5)</td></tr><tr><td>Oncology</td><td>10 (5.0)</td></tr><tr><td>Public health</td><td>9 (4.5)</td></tr><tr><td>Nephrology</td><td>7 (3.5)</td></tr><tr><td>Geriatrics</td><td>6 (3.0)</td></tr><tr><td>Obstetrics and gynecology</td><td>6 (3.0)</td></tr><tr><td>Others</td><td>49 (24.5)</td></tr><tr><td>COVID-19-related study</td><td>83 (41.5)</td></tr><tr><td>Journal</td><td></td></tr><tr><td>Annals of Internal Medicine</td><td>55 (27.5)</td></tr><tr><td>JAMA</td><td>55 (27.5)</td></tr><tr><td>The BMJ</td><td>44 (22.0)</td></tr><tr><td>The NEJM</td><td>30 (15.0)</td></tr><tr><td>The Lancet</td><td>16 (8.0)</td></tr><tr><td>Study design<sup>b</sup></td><td></td></tr><tr><td>Cohort study</td><td>126 (63.0)</td></tr><tr><td>Case control study</td><td>16 (8.0)</td></tr><tr><td>Cross-sectional study</td><td>49 (24.5)</td></tr><tr><td>Self-controlled case series</td><td>3 (1.5)</td></tr><tr><td>Other<sup>c</sup></td><td>9 (4.5)</td></tr><tr><td>Data sources<sup>d</sup></td><td></td></tr><tr><td>Administrative healthcare records</td><td>62 (31.0)</td></tr><tr><td>Registry</td><td>62 (31.0)</td></tr><tr><td>Electronic healthcare records</td><td>60 (30.0)</td></tr><tr><td>Survey/questionnaire/interview</td><td>57 (28.5)</td></tr><tr><td>Biological samples</td><td>30 (15.0)</td></tr><tr><td>Medical record/review of patient notes</td><td>21 (10.5)</td></tr><tr><td>Clinical examination</td><td>9 (4.5)</td></tr><tr><td>Radiography/mri/etc</td><td>5 (2.5)</td></tr><tr><td>Other<sup>e</sup></td><td>9 (4.5)</td></tr><tr><td>Statistically significant primary outcome(s) or outcome(s) even if not specified as a primary outcome(s)</td><td></td></tr><tr><td>Yes</td><td>154 (77.0)</td></tr><tr><td>No</td><td>12 (6.0)</td></tr><tr><td>Not applicable/descriptive study</td><td>34 (17.0)</td></tr></tbody></table>

The data are n (%).

UK, United Kingdom; USA, United States of America.

a The most relevant medical field was extracted.

b An article may have more than one study design (eg, two articles combined two types of studies).

c Other: case time control study, modeling study, simulation study, ecological study, before and after study, contact tracing study, observational controlled interrupted time series, quasi experimental study with modelisation, time series analysis.

d A study may use more than one data source.

e Other: data from clinical trial (n = 2), non-clinical data (n = 5), Global Burden Disease (n = 1), Institute for Health Metrics and Evaluation's modeling database (n = 1).

### 3.4. Secondary outcomes

The proportion of observational studies with a statistically significant primary outcome(s) seemed similar for observational studies with a protocol that was prospectively registered and those with a protocol not prospectively published or those with no information on the date of publication (statistical testing was not performed due to the low sample size and lack of sample size calculation). Similarly, the proportion of studies that adhered to the protocol regarding the primary outcome and sensitivity analysis/subgroup analysis was similar in the two groups (Table 4). For 75% of observational studies with a protocol (n = 18/24), a SAP was included. Among the 16 studies that deviated from the protocol, only two provided justification by the authors.

### 3.5. Comparison of observational studies with vs without protocol or registration

When comparing the characteristics of observational studies according to the registration and protocol status, observational studies that have been registered or had a protocol were more frequently published in the BMJ, had a first author working in the UK more frequently, and used data from electronic health care records more frequently (Table 5).

### 3.6. Instructions for authors of included journal

Among the journals included, none require the registration of the observational studies on a registry, but The

<table><caption>**Table 2.** Characteristics regarding registration and protocol status (N = 200)</caption><thead><tr><th>Characteristics</th><th>N (%)</th></tr></thead><tbody><tr><td>Registration of the study</td><td></td></tr><tr><td>Yes</td><td>20 (10.0)</td></tr><tr><td>Prospective registration</td><td>14</td></tr><tr><td>No prospective registration</td><td>3</td></tr><tr><td>Lacking information</td><td>3</td></tr><tr><td>No</td><td>180 (90.0)</td></tr><tr><td>Protocol provided</td><td></td></tr><tr><td>Yes</td><td>24 (12.0)</td></tr><tr><td>Protocol prospectively published</td><td>10</td></tr><tr><td>Protocol not prospectively published</td><td>12</td></tr><tr><td>Lacking information</td><td>2</td></tr><tr><td>No</td><td>176 (88.0)</td></tr></tbody></table>

The data are n (%).
<figure><img src="image_3.png" /><figcaption>**Figure 2.** Registration and protocol status of included studies (N = 200). (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)</figcaption></figure>

Lancet and the BMJ have published editorials to encourage registration [9,10]. Publication of protocols within the original study is encouraged by four journals (Appendix 5).

# 4. Discussion

## 4.1. Principal findings

Although most of the journals (4 out of 5) we looked at do not require but encourage registration and/or publication of protocols, the rate among these journals was low in our study: 10% of observational studies were registered, and among them, less than half were not prospectively registered. Because there is no single internationally recognized platform, we also found several different platforms being used for registration. In addition, for most of the studies, the protocol was not publicly available.

## 4.2. Discussion of results and comparison with previous studies

To our knowledge, this is the first study to estimate the rate of registered observational studies. Several studies have reported the characteristics of registered observational studies and their impact [17,18]. Firstly, Dal Ré et al. reported that observational study registration was uncommon (survey made on Pubmed for papers published in 2011, no registration rate provided) and was more frequent if the observational studies used data from an RCT [17].

Boccia et al. found that, among observational studies registered on [ClinicalTrials.gov](https://ClinicalTrials.gov), registration of observational studies often occurs after studies have started (only 13% of studies were prospectively registered) [18].

Moreover, in RCTs, the study start date is defined by the inclusion of the first patient. For observational studies, the definition of the study start date is not standardized and will depend on the nature of the data and whether it is a prospective or retrospective study. Therefore, registration needs to be reported in a transparent and honest way [19], and it is the responsibility of researchers to ensure that published articles are an “unbiased, accurate representation of research” [20]. Other studies have found that prospectively registered studies are more likely to report null results and smaller effect sizes. Furthermore, they are more likely to be replicable [21], and registration could be an argument for limiting p-hacking (ie, manipulating data to obtain statistically significant results). In our study, we also observed a high rate of significant outcomes among studies, and this is consistent with previous publications that have found that, for both observational and RCTs, positive findings are more likely to be submitted and published compared to studies with negative results [22–24]. Lastly, we highlighted the heterogeneity of websites and platforms where study registrations were published. The most frequently used platform in our studies was [ClinicalTrials.gov](https://ClinicalTrials.gov), which is dedicated to the registration of RCTs rather than observational studies. Nonetheless, 23% of all registered studies on this platform in 2023 were observational studies [25]. Moreover, Dal Ré et al. have shown that there were large differences in the quantity and quality of information provided among and within registers for clinical observational research [26].

In our study, we found that a higher proportion of observational studies that have been registered or had a protocol had a first author working in the UK compared to studies that were not registered or did not have a protocol. Indeed,
<table><caption>**Table 3.** Platforms for registration and for diffusion of protocol</caption><thead><tr><th>Platforms</th><th>N</th></tr></thead><tbody><tr><td>Platforms for registration (N = 20)</td><td></td></tr><tr><td>Clinicaltrials.gov</td><td>5</td></tr><tr><td>OSF</td><td>4</td></tr><tr><td>Open Safely</td><td>1</td></tr><tr><td>Research structure platform</td><td>10</td></tr><tr><td>CPRD website</td><td>4</td></tr><tr><td>National of Health Research, ReSi</td><td>2</td></tr><tr><td>UK biobank</td><td>2</td></tr><tr><td>ISRTN</td><td>1</td></tr><tr><td>ENCePP</td><td>1</td></tr><tr><td>Platforms for protocol<sup>a</sup> (N = 24)</td><td></td></tr><tr><td>Protocol provided with the article (in the appendix/supplemental file)</td><td>10</td></tr><tr><td>Clinicaltrials.gov</td><td>5</td></tr><tr><td>OSF</td><td>4</td></tr><tr><td>Github</td><td>2</td></tr><tr><td>Study protocol published as an original article in a journal</td><td>3</td></tr><tr><td>Peer-review process<sup>b</sup></td><td>1</td></tr><tr><td>Research structure platform</td><td>6</td></tr><tr><td>CPRD website</td><td>2</td></tr><tr><td>ENCePP</td><td>1</td></tr><tr><td>ISRTN</td><td>1</td></tr><tr><td>National of Health Research, ReSi</td><td>1</td></tr><tr><td>The University of Edimburg website</td><td>1</td></tr></tbody></table>

CPRD, Clinical Practice Research Datalink; ENCePP, European Network of Centers for Pharmacoepidemiology and Pharmacovigilance; ISRTN, International Standard Randomised Controlled Trial Number; OSF, Open Science Framework.

<sup>a</sup> Some studies have several protocols available, for example, on Clinicaltrials.gov and Appendix/supplemental file.

<sup>b</sup> For one article published in the BMJ, the protocol was available online with the peer review process.

observational studies based on electronic health records or registries in the UK (eg, Clinical Practice Research Datalink, UK Biobank) are likely to be registered, as they publish protocols online for new studies. We also found that observational studies that had been registered or had a protocol were more likely to be published in the BMJ. Indeed, the BMJ encourages the registration of observational studies and the publication of protocols, which could explain this higher rate.

### 4.3. Pros and cons of compulsory observational study registration

Concerns about observational study registration have been raised and debated in the literature [27–30]. Concerns raised included an increased bureaucratic burden and the possibility that hypotheses could be stolen by rival researchers. There were also concerns that it could have a chilling effect on the exploration of large, previously collected datasets [31,32]. However, for observational studies using prospective data collection and some observational studies using retrospective data, institutional review boards or research ethics committees already require the submission of a protocol for authorizations [33]. Moreover, registration of the study does not preclude the exploration of additional hypotheses. A transparent approach is recommended, whereby researchers should document, report, and justify unplanned analyses and protocol changes for the benefit of readers and reviewers. The advantages of observational studies are multiple, and registration would lead to incontestable ethical and scientific benefits [17,34,35]. Indeed, registration of observational studies will minimize publication bias, increase transparency, protect against selective reporting, allow identification of unpublished studies, and reduce the likelihood of p-hacking [21,36]

<table><caption>**Table 4.** Studies' adherence to the protocol adherence (N = 24)</caption><thead><tr><th>Characteristics</th><th>Prospective protocol published N = 10 (%)</th><th>Not prospective protocol published or lacking information N = 14 (%)</th><th>Total N = 24 (%)</th></tr></thead><tbody><tr><td>Statistically significant primary outcome(s) or outcome(s) even if not specified as a primary outcome(s)</td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>7 (70)</td><td>9 (64)</td><td>16 (67)</td></tr><tr><td>No significant primary outcome</td><td>1 (10)</td><td>1 (7)</td><td>2 (8)</td></tr><tr><td>Not applicable/descriptive study</td><td>2 (20)</td><td>4 (29)</td><td>6 (25)</td></tr><tr><td>Protocol adherence (primary outcome)</td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>9 (90)</td><td>12 (86)</td><td>21 (88)</td></tr><tr><td>No</td><td>1 (10)</td><td>2 (14)</td><td>3 (13)</td></tr><tr><td>Protocol adherence (subgroups analysis or sensitivity analysis)</td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>3 (30)</td><td>2 (14)</td><td>5 (21)</td></tr><tr><td>No</td><td>5 (50)</td><td>10 (71)</td><td>15 (63)</td></tr><tr><td>Not applicable (no subgroups or sensitivity analysis performed)</td><td>2 (20)</td><td>2 (14)</td><td>4 (17)</td></tr></tbody></table>

The data are n (%).
<table><caption>**Table 5.** Comparison of general characteristics of included studies according to the registration and protocol status (N = 200)</caption><thead><tr><th>Characteristics</th><th>Studies that were registered or had a protocol (N = 28)</th><th>Studies that were not registered and did not have a protocol (N = 172)</th><th>P value</th></tr></thead><tbody><tr><td>First author's geographic region</td><td></td><td></td><td>&lt;0.001</td></tr><tr><td>USA</td><td>6 (21.4)</td><td>103 (59.9)</td><td></td></tr><tr><td>Europe (except UK)</td><td>2 (7.1)</td><td>24 (14.0)</td><td></td></tr><tr><td>UK</td><td>10 (35.7)</td><td>12 (7.0)</td><td></td></tr><tr><td>Canada</td><td>4 (14.3)</td><td>7 (4.1)</td><td></td></tr><tr><td>Other</td><td>6 (21.4)</td><td>26 (15.1)</td><td></td></tr><tr><td>Medical field<sup>a</sup></td><td></td><td></td><td>0.807</td></tr><tr><td>Infectious diseases</td><td>13 (46.4)</td><td>73 (42.4)</td><td></td></tr><tr><td>Cardiology</td><td>3 (10.7)</td><td>13 (7.6)</td><td></td></tr><tr><td>Endocrinology/diabetology</td><td>2 (7.1)</td><td>9 (5.2)</td><td></td></tr><tr><td>Oncology</td><td>1 (3.6)</td><td>9 (5.2)</td><td></td></tr><tr><td>Public health</td><td>0 (0.0)</td><td>9 (5.2)</td><td></td></tr><tr><td>Nephrology</td><td>0 (0.0)</td><td>7 (4.1)</td><td></td></tr><tr><td>Geriatrics</td><td>1 (3.6)</td><td>5 (2.9)</td><td></td></tr><tr><td>Obstetrics and gynaecology</td><td>0 (0.0)</td><td>6 (3.5)</td><td></td></tr><tr><td>Others</td><td>8 (28.6)</td><td>41 (23.8)</td><td></td></tr><tr><td>COVID-19 related study</td><td>13 (46.4)</td><td>70 (40.7)</td><td>0.716</td></tr><tr><td>Journal</td><td></td><td></td><td><strong>0.034</strong></td></tr><tr><td>Annals of Internal Medicine</td><td>6 (21.4)</td><td>49 (28.5)</td><td></td></tr><tr><td>JAMA</td><td>3 (10.7)</td><td>52 (30.2)</td><td></td></tr><tr><td>The BMJ</td><td>12 (42.9)</td><td>32 (18.6)</td><td></td></tr><tr><td>The NEJM</td><td>5 (17.9)</td><td>25 (14.5)</td><td></td></tr><tr><td>The Lancet</td><td>2 (7.1)</td><td>14 (8.1)</td><td></td></tr><tr><td>Study design<sup>b</sup></td><td></td><td></td><td>0.052</td></tr><tr><td>Cohort study</td><td>21 (75.0)</td><td>105 (61.0)</td><td></td></tr><tr><td>Case control study</td><td>4 (14.3)</td><td>12 (11.4)</td><td></td></tr><tr><td>Cross sectional study</td><td>2 (7.1)</td><td>47 (27.3)</td><td></td></tr><tr><td>Self-controlled case-series</td><td>1 (3.6)</td><td>2 (1.2)</td><td></td></tr><tr><td>Other<sup>c</sup></td><td>1 (3.6)</td><td>8 (4.7)</td><td></td></tr><tr><td>Data sources<sup>d</sup></td><td></td><td></td><td><strong>0.006</strong></td></tr><tr><td>Administrative healthcare records</td><td>4 (14.3)</td><td>58 (33.7)</td><td></td></tr><tr><td>Registry</td><td>11 (39.3)</td><td>51 (29.7)</td><td></td></tr><tr><td>Electronic healthcare records</td><td>13 (46.4)</td><td>47 (27.3)</td><td></td></tr><tr><td>Survey/questionnaire/interview</td><td>8 (28.6)</td><td>49 (28.5)</td><td></td></tr><tr><td>Biological samples</td><td>7 (25.0)</td><td>23 (13.4)</td><td></td></tr><tr><td>Medical record/review of patient notes</td><td>7 (25.0)</td><td>14 (8.1)</td><td></td></tr><tr><td>Clinical examination</td><td>5 (17.9)</td><td>4 (2.4)</td><td></td></tr><tr><td>Radiography/MRI/etc</td><td>2 (7.1)</td><td>3 (1.7)</td><td></td></tr><tr><td>Other<sup>e</sup></td><td>2 (7.1)</td><td>7 (4.1)</td><td></td></tr><tr><td>Statistically significant primary<br>outcome(s) or outcome(s) even if not<br>specified as a primary outcome(s)</td><td></td><td></td><td>0.389</td></tr><tr><td>Yes</td><td>19 (67.9)</td><td>135 (78.5)</td><td></td></tr><tr><td>No</td><td>2 (7.1)</td><td>10 (5.8)</td><td></td></tr><tr><td>Not applicable/descriptive study</td><td>7 (25.0)</td><td>27 (15.7)</td><td></td></tr></tbody></table>

The data are n (%)

Bold values = significant values (P < 0.05).

a The most relevant medical field was extracted.

b An article may have more than one study design (e.g., two articles combined two types of studies).

c Other: case time control study, modelling study, simulation study, ecological study, before and after study, contact tracing study, observational controlled interrupted time series, quasi experimental study with modelisation, time series analysis.

d A study may use more than one data source.

e Other: data from clinical trial (n = 2), non-clinical data (n = 5), Global Burden Disease (n = 1), Institute for Health Metrics and Evaluation's modelling database (n = 1).
and HARKing (ie, exploiting analytic flexibility to obtain statistically significant results) [37]. As Pocock et al. have reported, such phenomena are especially prevalent in epidemiological studies, where many associations between exposure and outcome are investigated, for example up to 264 estimates in one study [38]. Moreover, registration ensures that all the evidence is publicly available and reduces research waste by facilitating collaboration. Registration of observational studies will also improve the peer review process by providing additional information on the methods and therefore, improving clarity and transparency, similarly to systematic reviews [39]. Finally, there is an ethical obligation toward individuals who have consented to participate in research to register the methodology and aims of observational studies and RCTs [40–42].

### 4.4. Limitations

Our study has some limitations. Firstly, the main limitation is that we have limited this retrospective review to one single year (2022), which was also a pandemic year. We can hypothesize that the burden on researchers was greater during the pandemic and that they had limited time to register observational studies. Secondly, we have limited ourselves to the top five highest impact-factor general medicine journals. However, the proportion of registered studies is not likely to be higher in other journals, and therefore, the rate of registered observational studies found in our review might be overestimated if applied to all observational studies. Also, we have selected 200 studies from the 290 identified, as it was preplanned in the protocol. Indeed, we limited the number to 200 for feasibility, and we do not believe that including 290 articles would change the main conclusion of this paper. Furthermore, we also limited the extracted data to the data available in the paper and we did not systematically research the most commonly used registers. We also did not contact the corresponding author. Indeed, authors could have forgotten to mention study or protocol registration. However, the goal of this study was to evaluate the proportion of protocols available in the public domain and not the proportion of protocols that are available on request from the corresponding authors. Lastly, over-representation of COVID-19 studies in our review might also have impacted the results, as the methodological quality of COVID-19 studies has been demonstrated to be low [43]. However, if we considered only non-COVID-19 studies, the proportion of registered studies and observational studies with a protocol was similar.

### 4.5. Future research and solutions

There is a need for a common platform for the registration of observational studies, in order to improve the quality and rate of registration, and facilitate the peer-review process for all scholarly journals. This would be similar to PROSPERO, which is used for systematic reviews [44]. However, several challenges need to be addressed: i) the

minimum information to be published in a registration must be defined and standardized; ii) registration for secondary data analyses should be clarified [21]. This platform should meet the needs of both researchers/investigators who are registering and clinicians/researchers/reviewers who are retrieving the studies. An international consultation process using a Delphi study is planned and will involve experts (researchers, journal editors, and funders) from different fields and continents. Our research group then plan to develop a universal platform for registration of observational studies and their protocols.

## 5. Conclusions

In our study, we identified that less than 10% of observational studies were registered in leading general medicine journals, and 12% had publicly accessible protocols. Like the registration of systematic reviews, which is now widely accepted despite publishers do not mandating it, the registration of observational studies should be encouraged by journal editors, publishers, funders, and researchers. We agree that registration of observational studies would not guarantee their validity of observational studies but will definitely increase their quality. Registration will also enhance peer review and readers' confidence in the reported results. A structured and coordinated approach for observational studies is required, and a balance is necessary to ensure a rigorous and realistic process.

### Ethics approval

Not applicable.

### Data sharing

Data supporting the study findings will be available from the corresponding author on reasonable request.

### Registration

The study protocol has been prospectively registered (28th March 2023) on the Center of Evidence Based Dermatology registration portal (available at https://www.nottingham.ac.uk/research/groups/cebd/resources/protocol-registration.aspx)

### CRediT authorship contribution statement

**Sophie Leducq:** Conceptualization, Data curation, Formal analysis, Methodology, Software, Visualization, Validation, Writing — original draft. **Faaris Zaki:** Data curation, Writing — review & editing. **Loes M. Hollestein:** Conceptualization, Methodology, Writing — review &
editing. **Christian Apfelbacher:** Conceptualization, Methodology, Writing — review & editing. **Nikhil Prasanna Ponna:** Data curation, Writing — review & editing. **Rishabh Mazmudar:** Conceptualization, Methodology, Writing — review & editing. **Sonia Gran:** Conceptualization, Data curation, Methodology, Supervision, Writing — original draft, Writing — review & editing.

## Data availability

Data will be made available on request.

## Declaration of competing interest

The authors declare no competing interest.

## Acknowledgments

The authors thank Prof Hywel C. Williams, Centre of Evidence Based Dermatology, University of Nottingham, for his helpful comments on the study protocol, Mrs Sarah Beach, librarian, University of Nottingham, for her help in the equation research and Dr Natasha Rogers, Centre of Evidence Based Dermatology, University of Nottingham, for her help in the graphical abstract. We sincerely appreciate the insightful and constructive feedback of the reviewers, which have improved our manuscript.

## Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jclinepi.2024.111341.

## References

[1] Porta M. A dictionary of epidemiology. 6th ed. Oxford: Oxford University Press; 2014.

[2] Cochran WG, Chambers SP. The planning of observational studies of human populations. J Roy Stat Soc 1965;128:234–66.

[3] Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215–8.

[4] Ohn J, Eun SJ, Kim DY, Park HS, Cho S, Yoon HS. Misclassification of study designs in the dermatology literature. J Am Acad Dermatol 2018;79:315–9.

[5] Wilson M, Sampson M, Barrowman N, Doja A. Bibliometric analysis of neurology articles published in general medicine journals. JAMA Netw Open 2021;4:e215840.

[6] World Medical Association. WMA Declaration of Helsinki — ethical principles for medical research involving human subjects. 2023. Available at: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects. Accessed July 26, 2023.

[7] Lindsley K, Fusco N, Li T, Scholten R, Hooft L. Clinical trial registration was associated with lower risk of bias compared with non-registered trials among trials included in systematic reviews. J Clin Epidemiol 2022;145:164–73.

[8] Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev 2012;1:2.

[9] Ge L, Tian JH, Li YN, Pan JX, Li G, Wei D, et al. Association between prospective registration and overall reporting and methodological quality of systematic reviews: a meta-epidemiological study. J Clin Epidemiol 2018;93:45–55.

[10] Ricotta EE, Rid A, Cohen IG, Evans NG. Observational studies must be reformed before the next pandemic. Nat Med 2023;29:1903–5.

[11] Loder E, Groves T, MacAuley D. Registration of observational studies. BMJ 2010;340:c950.

[12] The Lancet. Should protocols for observational research be registered? Lancet 2010;375:48.

[13] PLOS Medicine. Observational studies: getting clear about transparency. PLoS Med 2014;11:e1001711.

[14] Agha R, Rosin D. The Research Registry - answering the call to register every research study involving human participants. Ann Med Surg (Lond) 2015;18(4):95–7.

[15] Eboreime EA, Abimbola S. A priori registration of global health research-necessity or absurdity? BMJ Glob Health 2021;6:e006199.

[16] Riley DS, Barber MS, Kienle GS, Aronson JK, von Schoen-Angerer T, Tugwell P, et al. CARE guidelines for case reports: explanation and elaboration document. J Clin Epidemiol 2017;89:218–35.

[17] Dal-Ré R, Ioannidis JP, Bracken MB, Buffler PA, Chan AW, Franco EL, et al. Making prospective registration of observational research a reality. Sci Transl Med 2014;6:224cm1.

[18] Boccia S, Rothman KJ, Panic N, Flacco ME, Rosso A, Pastorino R, et al. Registration practices for observational studies on ClinicalTrials.gov indicated low adherence. J Clin Epidemiol 2016;70:176–82.

[19] Altman DG, Moher D. Declaration of transparency for each research article. BMJ 2013;347:f4796.

[20] The PLoS Medicine. An unbiased scientific record should be everyone's agenda. PLoS Med 2009;6:e1000038.

[21] Baldwin JR, Pingault JB, Schoeler T, Sallis HM, Munafò MR. Protecting against researcher bias in secondary data analysis: challenges and potential solutions. Eur J Epidemiol 2022;37:1–10.

[22] Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 2009;2009:MR000006.

[23] Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337:867–72.

[24] Swaen GG, Teggeler O, van Amelsvoort LG. False positive outcomes and design characteristics in occupational cancer epidemiology studies. Int J Epidemiol 2001;30:948–54.

[25] Clinicaltrials.gov. Trends, charts, and maps. 2023. Available at: https://classic.clinicaltrials.gov/ct2/resources/trends#TypesOfRegiste redStudies. Accessed November 27, 2023.

[26] Dal-Ré R, Fauria K, Gramunt N, Molinuevo JL. Clinical observational research on Alzheimer's disease: what clinical trial registries can tell. J Alzheimers Dis 2013;34:183–90.

[27] Rushton L. Should protocols for observational research be registered? Occup Environ Med 2011;68:84–6.

[28] Pearce N. Registration of protocols for observational research is unnecessary and would do more harm than good. Occup Environ Med 2011;68:86–8.

[29] Boccia S. Credibility of observational studies: why public health researchers should care? Eur J Public Health 2015;25:554–5.

[30] Vandenbroucke JP. Preregistration: when shall we start the real discussion? Eur J Public Health 2015;25:555–6.

[31] Malmsjö D, Frost A, Hróbjartsson A. A scoping review finds that guides to authors of protocols for observational epidemiological studies varied highly in format and content. J Clin Epidemiol 2023; 154:156–66.

[32] de Jonge P, Conradi HJ, Thombs BD, Rosmalen JGM, Burger H, Ormel J. Prevention of false positive findings in observational
studies: registration will not work but replication might. J Epidemiol Community Health 2011;65:95-6.

[33] Chavers S, Fife D, Wacholtz M, Stang P, Berlin J. Registration of Observational Studies: perspectives from an industry-based epidemiology group. Pharmacoepidemiol Drug Saf 2011;20:1009-13.

[34] Gran S, Apfelbacher C, Hollestein LM. Observational studies: it's time to be transparent. Br J Dermatol 2023;188:449-51.

[35] Naudet F, Patel CJ, DeVito NJ, Le Goff G, Cristea IA, Braillon A, et al. Improving the transparency and reliability of observational studies through registration. BMJ 2024;384:e076123.

[36] Braillon A. Vitamin D and breast cancer: stop torturing the data. Cancer 2022;128:2999.

[37] Kerr NL. HARKing: hypothesizing after the results are known. Pers Soc Psychol Rev 1998;2:196-217.

[38] Pocock SJ, Collier TJ, Dandreo KJ, de Stavola BL, Goldman MB, Kalish LA, et al. Issues in the reporting of epidemiological studies: a survey of recent practice. BMJ 2004;329:883.

[39] Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev 2012;1:7.

[40] Dickersin K, Rennie D. The evolution of trial registries and their use to assess the clinical trial enterprise. JAMA 2012;307:1861-4.

[41] Haymond S. Realities of observational study registration. Clin Chem 2014;60:1128-9.

[42] Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: is it time? CMAJ 2010;182:1638-42.

[43] Jung RG, Di Santo P, Clifford C, Prosperi-Porta G, Skanes S, Hung A, et al. Methodological quality of COVID-19 clinical research. Nat Commun 2021;12:943.

[44] Sideri S, Papageorgiou SN, Eliades T. Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality. J Clin Epidemiol 2018;100:103-10.