Systematic reviews on the same topic are common but often fail to meet key methodological standards: a research-on-research study

Wilfred Kwok, Titiane Dallant, Guillaume Martin, Gabriel Fournier, Blandine Kervennic, Ophélie Pingeon, Agnès Dechartres

<figure><img src="image_1.png" /></figure>

► To cite this version:

Wilfred Kwok, Titiane Dallant, Guillaume Martin, Gabriel Fournier, Blandine Kervennic, et al. "Systematic reviews on the same topic are common but often fail to meet key methodological standards: a research-on-research study." Journal of Clinical Epidemiology, 2026, 189, pp.112018. 10.1016/j.jclinepi.2025.112018. hal-05505615

HAL Id: hal-05505615

https://hal.science/hal-05505615v1

Submitted on 11 Feb 2026

**HAL** is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire **HAL**, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
ORIGINAL RESEARCH

Systematic reviews on the same topic are common but often fail to meet
key methodological standards: a research-on-research study

Wilfred Kwok<fnref n="1" />, Titiane Dallant<fnref n="1" />, Guillaume Martin, Gabriel Fournier, Blandine Kervennic, Ophélie Pingeon, Agnès Dechartres*

Sorbonne Université, INSERM, Institut Pierre Louis d'Epidémiologie et de Santé Publique, AP-HP, Hôpital Pitié-Salpêtrière, Département de Santé Publique, Centre de Pharmacoépidémiologie de l'AP-HP (Cephepi), Paris, France

Accepted 9 October 2025; Published online 17 October 2025

Abstract

**Objectives:** To 1) assess the frequency of overlapping systematic reviews (SRs) on the same topic including overlap in outcomes, 2) assess whether SRs meet some key methodological characteristics, and 3) describe discrepancies in results.

**Study Design and Setting:** For this research-on-research study, we gathered a random sample of SRs with meta-analysis (MA) published in 2022, identified the questions they addressed and, for each question, searched all SRs with MA published from 2018 to 2023 to assess the frequency of overlap. We assessed whether SRs met a minimum set of six key methodological characteristics: protocol registration, search of major electronic databases, search of trial registries, double selection and extraction, use of the Cochrane Risk-of-Bias tool, and Grading of Recommendations, Assessment, Development, and Evaluation assessment.

**Results:** From a sample of 107 SRs with MA published in 2022, we extracted 105 different questions and identified 123 other SRs with MA published from 2018 to 2023. There were overlapping SRs for 33 questions (31.4%, 95% CI: 22.9–41.3), with a median of three overlapping SRs per question (IQR 2–6; range 2–19). Of the 230 SRs, 15 (6.5%) met the minimum set of six key methodological characteristics, and 12 (11.4%) questions had at least one SR meeting this criterion. Among the 33 questions with overlapping SRs, for 7 (21.2%), the SRs had discrepant results.

**Conclusion:** One-third of the SRs published in 2022 had at least one overlapping SR published from 2018 to 2023, and most did not meet a minimum set of methodological standards. For one-fifth of the questions, overlapping SRs provided discrepant results. © 2025 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

**Keywords:** Systematic reviews; Meta-analyses; Redundancy; Methodological quality; Waste of research; Research methodology

# 1. Introduction

Systematic reviews (SRs) of randomized controlled trials (RCTs) are considered to provide one of the highest levels of evidence [1]. These essential tools synthesize data from all relevant studies to provide reliable comprehensive summaries of the current state of knowledge. SRs may include one or more meta-analyses (typically one per outcome) to provide a summary estimate of the treatment effect. The number of SRs with meta-analysis (MA) has

increased substantially in recent years, showing the attractiveness of this type of study [2]. A PubMed search of the publication type “meta-analysis” in November 2024 retrieved 23,242 references in 2023 vs. 2166 in 2005. Overall, 25,721 references were indexed as RCTs in 2023 vs. 16,337 in 2005.

Among this profusion of SRs with MA, there is a possibility of redundancy, with overlapping SRs answering the same research question and more or less discrepancies in methods, results, and conclusions [3]. Previous studies have defined a redundant SR as having the same condition, population, interventions, comparison groups, and outcomes (PICO) as an index SR [4,5]. These redundant SRs must be distinguished from updates when new data or analyses are available [6]. Few studies have evaluated the problem

<footnote n="1">¹ Both authors contributed equally.</footnote>

<footnote>* Corresponding author. Sorbonne Université, INSERM U1136, équipe PEPITES, Institut Pierre Louis d'épidémiologie et de santé publique, Hôpital Pitié-Salpêtrière, 47-83 Boulevard de l'Hôpital, Paris 75013, France.</footnote>

<footnote>E-mail address: agnes.dechartres@aphp.fr (A. Dechartres).</footnote>
## What is new?

### Key findings

* Of 105 research questions investigated, 33 had overlapping systematic reviews (SRs) with meta-analysis (MA) (31.4%, 95% CI: 22.9–41.3), with a median number of three SRs per research question (IQR 2–6; range 2–19).

* Only 15 of the 230 (6.5%) SRs identified meet a minimum set of key methodological characteristics.

* Among the 33 research questions with overlapping SRs, for 7 (21.2%), the SRs had discrepant results.

### What this adds to what is known?

* Few studies have assessed the frequency of overlapping SRs on the same topic.

* Many SRs are overlapping and most lack key methodological characteristics.

### What is the implication and what should change now?

* All SRs should start with a search of existing SRs on the same topic and justify what they add to the existing literature.

* All SRs and MA should be registered

* SRs should follow a minimum set of methodological characteristics.

of redundant SRs on the same topic. In 2013, Siontis et al. [4] reported that two-thirds of MAs published in 2010 had at least one overlapping MA indexed in PubMed from inception to February 2013 (albeit without distinction between updates and redundancy). Recently, while preparing an update for a French guideline on the treatment of cancer-associated venous thromboembolism, Chapelle et al. [7] found 11 SRs with MA of the same four RCTs, all being published within the year of publication of the fourth RCT. Multiple SRs with MA addressing the same research question can lead to confusion for the clinician, who may not know which study to rely on, especially if conclusions differ. This situation also represents a significant waste of research because conducting an SR with MA is a time- and effort-consuming process.

This study aimed to 1) assess the frequency of overlapping SRs on a same topic including the overlap in outcomes, 2) assess whether SRs fulfill a minimum set of key methodological characteristics, and 3) describe the discrepancies in their results.

## 2. Methods

### 2.1. Study design

This is a research-on-research study. Two searches were conducted. The first search aimed to identify SRs with MA published in 2022 from which we constituted a random sample. We identified the research questions addressed by SRs in the obtained sample. Then, in the second step, for each research question, we searched for all SRs with MA published from 2018 to 2023 to assess the frequency of overlap during this period.

### 2.2. Identification of a random sample of SRs with MA published in 2022

We aimed to start from a sample of SRs with MA available in PubMed over a 1-year period, from January to December 2022. We searched MEDLINE (via PubMed) on March 6, 2023, using the terms meta-analysis[pt] AND randomi* AND trial* AND 2022 as a filter. We included SRs with MA of RCTs assessing the effectiveness of preventive or therapeutic interventions for any condition and having a binary primary outcome. Exclusion criteria were SRs without MAs, SRs including types of studies other than RCTs (eg, observational studies, animal, or in vitro studies), SRs with individual patient-data or network MAs, SRs assessing prognostic associations or diagnostic accuracy, meta-epidemiological studies or overviews, and SRs in languages other than English. One author (W.K.) screened all titles and abstracts to identify potentially eligible SRs with MA. We constituted a random sample of 200 SRs from those previously selected to have at least 100 SRs with MA finally included. These numbers were chosen for convenient reasons. Two reviewers (W.K. and B.K.) independently checked the eligibility of studies by reading titles, abstracts, and full texts. Any disagreement was resolved by discussion with a third senior reviewer (A.D.) to reach consensus.

### 2.3. Extraction of research questions

For each selected SR, two reviewers extracted the research question defined by the PICO framework: type of population/condition, intervention, control group, and outcomes. If several SRs addressed the same research question, they were grouped as one research question.

### 2.4. Identification of all SRs with MA published from 2018 to 2023 for each research question

For each research question, we searched PubMed on May 15, 2023, to identify all SRs with MA. We restricted the search to the period 2018–2023 to minimize the possibility of new published studies, thus reducing the risk of an update instead of a redundant SR. We used a dedicated search algorithm including both Medical Subject Headings terms and free-text words related to the “Population” and
"Intervention" categories from PICO for each research question, with "meta-analysis" as publication type or in titles. Whenever possible, we used the search strategy reported in SRs published in 2022 to identify relevant Medical Subject Headings terms and free-text words.

## 2.5. Assessment of overlapping SRs with MA

For each research question, two authors (W.K. and B.K.) selected overlapping SRs with MA using the following definition: SRs with MA published since 2018 that addressed the same condition/population, intervention, and control arm and having one of the outcomes matching the primary outcome of the SR published in 2022. We applied the same eligibility criteria as our first search and excluded SRs with MA including types of studies other than RCTs (eg, observational studies, animal or in vitro studies), SRs with individual patient-data or network MAs, SRs assessing prognostic associations or diagnostic accuracy, meta-epidemiological studies or overviews, and SRs in languages other than English. We did not consider as overlapping SRs those that were labeled as updates by the authors (ie, if the authors indicated in the rationale that an SR with MA had already been conducted but new studies were available).

## 2.6. Data extraction

Two authors (W.K. and T.D.) independently extracted the following data from the full text of each SR:

- Publication characteristics: name and impact factor of the journal, date of publication, country of affiliation of the corresponding author, and Cochrane or non-Cochrane review

- Funding sources (public, private, both, or not reported) and conflicts of interest

- PICO framework: population, intervention, control group and all outcomes reported

- Characteristics of included studies: main inclusion and exclusion criteria, number of RCTs, and participants

- Key methodological characteristics that should be present according to different tools, such as the Cochrane Handbook for Systematic Reviews [8,9], the ROBIS [10], or the User's Guide to the Medical Literature [11], and according to an expert in this field (A.D.). These characteristics are also required to be reported by the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 guidelines [12]:

- Protocol registration (on PROSPERO or other registration platform, or the Cochrane Library for Cochrane reviews)

- Search of major electronic databases: PubMed, Embase, and Cochrane Central Register of Controlled Trials (CENTRAL)

- Search of trial registries (as a way to identify gray literature)

- Double selection and extraction

- Assessment of risk of bias (RoB) using the Cochrane RoB tool [13].

- Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) for assessing the certainty of evidence; GRADE is the most widely used tool and is recommended by Cochrane [8].

- MA results: type of association measure (odds ratio [OR] or relative risk [RR]), OR/RR value and 95% CI

- Conclusion for the primary outcome

### 2.6.1. Minimum set of key methodological characteristics

We assessed whether each SR met the minimum set of six key methodological characteristics. This minimum set is defined as all six key methodological characteristics present: "Protocol registration" AND "search on the three major electronic databases (MEDLINE/PubMed, Embase and Cochrane CENTRAL)" AND "search of clinical trial registries (as a way to identify gray literature)" AND "double selection and extraction process" AND "use of Cochrane RoB tool" AND "GRADE assessment".

## 2.7. Data synthesis

### 2.7.1. Descriptive analyses

We described general and key methodological characteristics of all SRs with MA selected from the first and second searches with frequencies (percentages) for categorical variables and median (IQR) for quantitative variables. We estimated the prevalence of overlapping SRs with MA on the same topic, defined as the proportion of research questions addressed by at least two SRs with MA (that were not updates) during 2018–2023, with its 95% CI. To assess overlap in outcomes and therefore in MAs performed for each research question, we first grouped similar outcomes measured at similar time points (eg, 14-day and 28-day mortality, clinical improvement within 14 or 28 days) or using different scales (eg, quality of life measured by different questionnaires). We also combined adverse events and serious adverse events into single categories. Then, for each SR, we quantified the average proportion of redundancy in outcomes relative to other overlapping SRs on the same research question by calculating the overlap of its outcomes with those reported in each other SR addressing the same research question, averaging the overlap when there were multiple overlapping SRs. We provided the median (Q1-Q3) averaged outcome redundancy for each research question, as well as across all research questions.

### 2.7.2. Key methodological characteristics

We assessed the number and proportion of all SRs with MA selected from the first and second searches meeting our minimum set of key methodological characteristics and the
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Flowchart of the selection process. RCT, randomized controlled trial; SR, systematic review, PICO, population, interventions, comparison groups, and outcomes; MA, meta-analysis; NMA, network meta-analysis.</figcaption></figure>
number and proportion of research questions with at least 1 SR responding to this criterion.

### 2.7.3. Discrepancies in conclusions for each research question

For the research questions with overlapping SRs, we described the proportion of research questions with discrepancies in results for the same primary outcome, that is, whether all the SRs do not report the same result (significant or nonsignificant result for the MA of the same primary outcome). We also assessed discrepancies in terms of certainty of evidence from the GRADE assessment for the same primary outcome.

Analyses and plots involved using R 4.3.2.

## 3. Results

### 3.1. Selection process

The selection process is shown in Figure 1. Regarding the first search, after an initial screening of 6820 references published in 2022 and recorded on PubMed, 1732 studies met our eligibility criteria on the basis of the title and abstract. From a random sample of 200 of these potentially eligible SRs, 107 met our eligibility criteria and were included in our study. These 107 SRs corresponded to 105 different research questions (three SRs overlapped each other). The most represented research questions were cardiovascular (11.4%, n = 12), obstetric gynecology and

oncology (both 7.6%, n = 8), and COVID-19 and anesthesia (both 6.7%, n = 7).

### 3.2. Assessment of overlapping SRs

With the 105 research questions, our second search identified 2722 references published between January 1, 2018, and May 15, 2023. After applying the same eligibility criteria as for our first search, we selected a total of 123 overlapping SRs with MA that were not updates in addition to the three previously identified. Among the 105 research questions, 33 had overlapping SRs with MA, that is, at least two SRs with MA answered the same research question and were not updates. Therefore, the prevalence of overlapping SRs with MA was 33/105 (31.4%, 95% CI: 22.9–41.3). Among these 33 research questions, the median number of overlapping SRs per research question was 3 (IQR 2–6) and ranged from 2 to 19.

Among the 105 research questions, those with the highest number of overlapping SRs with MA (Fig 2) were “efficacy of catheter ablation compared to medical therapy in patients with atrial fibrillation and heart failure on mortality” (n = 19), “efficacy of early renal replacement therapy in patients with acute kidney injury compared to the delayed procedure on mortality at 1 month” (n = 16), and two research questions on COVID-19 therapies: “efficacy of convalescent plasma therapy (n = 15) and interleukin-6 antagonists (n = 15) in patients hospitalized for COVID-19 compared to standard of care or placebo on mortality”.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Number of overlapping systematic reviews with meta-analysis per research question. IL, interleukin; AF, atrial fibrillation; HF, heart failure; AKI, acute kidney injury; VAP, ventilator-associated pneumonia; IBS, irritable bowel syndrome; ARI, acute respiratory infections; JK1, Janus kinase inhibitors; GLP1RA, glucagon-like peptide-1 receptor agonists; TEAS, transcutaneous electrical acupoint stimulation; COPD, chronic obstructive pulmonary disease; CD, cardiovascular disease; EGFR TKI, epidermal growth factor receptor-tyrosine kinase inhibitor; NSC, non small cell; CETP, cholesteryl ester transfer protein inhibitors.</figcaption></figure>
The averaged redundancy in outcomes within an SR relative to other overlapping SRs on the same research question is described in Figure 3. For each question, the figure also reports the median averaged outcome redundancy and its IQR across all overlapping SRs on that question. Across the 33 research questions with overlapping SRs, the median proportion of redundancy in outcomes and therefore in MAs conducted was 55% (IQR: 50-68) indicating that overall, more than half of the outcomes assessed in a given SR had already been reported in other SRs on the same topic.

### 3.3. General and key methodological characteristics of the SRs

Table 1 shows the general characteristics of the 230 included SRs with MA (107 published in 2022 resulting from

the first search +123 overlapping SRs with MA on the same topics published between 2018 and 2023 resulting from the second search). Of these 230, 39.1% (n = 90) were published by authors from China. Other authors were from the United States (10.4%, n = 24), Taiwan (6.5%, n = 15), the United Kingdom (5.2%, n = 12), and 36 other countries. Most SRs had a public funding source (n = 90, 39.1%) or no funding (n = 87, 37.8%). Only 6 (2.6%) were Cochrane reviews.

Table 2 shows the key methodological characteristics of the 230 SRs with MA. In total, 53% (n = 122) did not register a protocol, 30.9% (n = 71) did not perform the literature search on the three major electronic databases (PubMed, Embase, and Cochrane CENTRAL), 68.7% (n = 158) did not search clinical trial registries and 60.9% (n = 140) did not assess the GRADE. Notably, no tool other than GRADE was used for assessing the certainty of evidence. Most (75.7%, n = 174) SRs reported both

<figure><img src="image_5.png" /></figure>

**Figure 3.** Averaged redundancy in outcomes relative to other overlapping systematic reviews on the same topic. Each row corresponds to a research question. Each box within a row (limited to 15 per question) represents an individual SR addressing that question. Boxes are color-coded according to the average proportion of outcome redundancy of that SR relative to the other overlapping SRs on the same question (green = low averaged outcome redundancy [0%–25%]; yellow = moderate [>25–50%]; orange = moderately high [>50–75%]; red = high [>75%]). SRs are ordered from highest to lowest averaged outcome redundancy. For each research question, the figure also reports the median averaged redundancy and its IQR, across all SRs even when more than 15 were available. For example, for RQ01 three overlapping SRs were identified: 1 with high averaged outcome redundancy (88%, red), 1 with moderate averaged outcome redundancy (44%, yellow), and 1 with low averaged outcome redundancy (25%, green). The median averaged redundancy across these three SRs was 44% (IQR 25–88). SR: systematic review. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.).
<table><caption>**Table 1.** General characteristics of systematic reviews with meta-analysis (N = 230)</caption><thead><tr><th>Characteristic</th><th>All SRs (N = 230)</th><th>2022 SRs (1st search) (N = 107)</th><th>2018–2023 SRs (2nd search) (N = 123)</th></tr></thead><tbody><tr><td>Cochrane reviews – n (%)</td><td>6 (2.6%)</td><td>2 (1.9%)</td><td>4 (3.3%)</td></tr><tr><td>Impact factor of the journal</td><td></td><td></td><td></td></tr><tr><td>Range</td><td>0.4–56.3</td><td>0.4–12.0</td><td>0.5–56.3</td></tr><tr><td>Med (IQR)</td><td>3.8 (2.6–5.7)</td><td>3.4 (2.3–5.1)</td><td>4.2 (2.9–6.3)</td></tr><tr><td>Medical condition – n (%)</td><td></td><td></td><td></td></tr><tr><td>COVID-19</td><td>46 (20.0%)</td><td>9 (8.4%)</td><td>37 (30.1%)</td></tr><tr><td>Cardiovascular</td><td>32 (13.9%)</td><td>12 (11.2%)</td><td>20 (16.3%)</td></tr><tr><td>Nephrology</td><td>22 (9.6%)</td><td>5 (4.7%)</td><td>17 (13.8%)</td></tr><tr><td>Obstetric gynecology</td><td>21 (9.1%)</td><td>8 (7.5%)</td><td>13 (10.6%)</td></tr><tr><td>Endocrinology</td><td>11 (4.8%)</td><td>5 (4.7%)</td><td>6 (4.9%)</td></tr><tr><td>Oncology</td><td>10 (4.3%)</td><td>8 (7.5%)</td><td>2 (1.6%)</td></tr><tr><td>Anesthesia</td><td>9 (3.9%)</td><td>7 (6.5%)</td><td>2 (1.6%)</td></tr><tr><td>Urology</td><td>8 (3.5%)</td><td>5 (4.7%)</td><td>3 (2.4%)</td></tr><tr><td>Neurology</td><td>7 (3.0%)</td><td>5 (4.7%)</td><td>2 (1.6%)</td></tr><tr><td>Dermatology</td><td>6 (2.6%)</td><td>5 (4.7%)</td><td>1 (0.8%)</td></tr><tr><td>Others</td><td>58 (25.2%)</td><td>38 (35.5%)</td><td>20 (16.3%)</td></tr><tr><td>Author country – n (%)</td><td></td><td></td><td></td></tr><tr><td>China</td><td>90 (39.1%)</td><td>52 (48.6%)</td><td>38 (30.9%)</td></tr><tr><td>USA</td><td>24 (10.4%)</td><td>8 (7.5%)</td><td>16 (13%)</td></tr><tr><td>Taiwan</td><td>15 (6.5%)</td><td>4 (3.7%)</td><td>11 (8.9%)</td></tr><tr><td>UK</td><td>12 (5.2%)</td><td>6 (5.6%)</td><td>6 (4.9%)</td></tr><tr><td>India</td><td>10 (4.3%)</td><td>5 (4.7%)</td><td>5 (4.1%)</td></tr><tr><td>Australia</td><td>9 (3.9%)</td><td>5 (4.7%)</td><td>4 (3.3%)</td></tr><tr><td>Italy</td><td>6 (2.6%)</td><td>1 (0.9%)</td><td>5 (4.1%)</td></tr><tr><td>Canada</td><td>6 (2.6%)</td><td>3 (2.8%)</td><td>3 (2.4%)</td></tr><tr><td>Korea</td><td>5 (2.2%)</td><td>2 (1.9%)</td><td>3 (2.4%)</td></tr><tr><td>Greece</td><td>4 (1.7%)</td><td>2 (1.9%)</td><td>2 (1.6%)</td></tr><tr><td>Germany</td><td>4 (1.7%)</td><td>1 (0.9%)</td><td>3 (2.4%)</td></tr><tr><td>Others</td><td>45 (19.6%)</td><td>18 (16.8%)</td><td>27 (22.0%)</td></tr><tr><td>Funding – n (%)</td><td></td><td></td><td></td></tr><tr><td>None</td><td>87 (37.8%)</td><td>41 (38.3%)</td><td>46 (37.4%)</td></tr><tr><td>Public</td><td>90 (39.1%)</td><td>45 (42.1%)</td><td>45 (36.6%)</td></tr><tr><td>Private</td><td>2 (0.9%)</td><td>1 (0.9%)</td><td>1 (0.8%)</td></tr><tr><td>Both</td><td>6 (2.6%)</td><td>1 (0.9%)</td><td>5 (4.1%)</td></tr><tr><td>Not reported</td><td>45 (19.6%)</td><td>19 (17.8%)</td><td>26 (21.1%)</td></tr><tr><td>Conflict of interest – n (%)</td><td></td><td></td><td></td></tr><tr><td>No</td><td>193 (83.9%)</td><td>89 (83.2%)</td><td>104 (84.6%)</td></tr><tr><td>Yes</td><td>27 (11.7%)</td><td>12 (11.2%)</td><td>15 (12.2%)</td></tr><tr><td>Not reported</td><td>10 (4.3%)</td><td>6 (5.6%)</td><td>4 (3.3%)</td></tr></tbody></table>

SR, systematic review.

double selection and extraction process, and almost all assessed the study RoB (98.3%, n = 226; 90%, n = 207 with the Cochrane RoB tool).

### 3.3.1. Minimum set of key methodological characteristics

Overall, 15 of the 230 SRs with MA (6.5%) meet our minimum set of key methodological characteristics. There was at least one SR with MA meeting the minimum set of key methodological characteristics for 12 (11.4%) of the 105 research

questions. This means that there were 93 research questions for which none of the SRs with MA conducted met the minimum set of key methodological characteristics.

### 3.4. Discrepancies in results within each research question

Among the 33 research questions with overlapping SRs, for 7 (21.2%), the MA had discrepant results in terms of statistical significance for the same primary outcome. There
<table><caption>**Table 2.** Key methodological characteristics of systematic reviews with meta-analyses (N = 230)</caption><thead><tr><th>Characteristic</th><th>All SRs (N = 230)</th><th>2022 SRs (1st search) (N = 107)</th><th>2018–2023 SRs (2nd search) (N = 123)</th></tr></thead><tbody><tr><td>Protocol registration – n (%)</td><td>108 (47.0)</td><td>49 (45.8)</td><td>59 (48.0)</td></tr><tr><td>Search on the 3 major electronic databases – n (%)</td><td>159 (69.1)</td><td>76 (71.0)</td><td>83 (67.4)</td></tr><tr><td>PubMed</td><td>227 (98.7)</td><td>106 (99.1)</td><td>121 (98.4)</td></tr><tr><td>Embase</td><td>184 (80.0)</td><td>91 (85.0)</td><td>93 (75.6)</td></tr><tr><td>Cochrane CENTRAL</td><td>192 (83.5)</td><td>86 (80.4)</td><td>106 (86.2)</td></tr><tr><td>Search of gray literature – n (%)</td><td>13 (5.7)</td><td>3 (2.8)</td><td>10 (8.1)</td></tr><tr><td>Clinical trial registries</td><td>72 (31.3)</td><td>25 (23.4)</td><td>47 (38.2)</td></tr><tr><td>Conference abstracts</td><td>19 (8.3)</td><td>7 (6.6)</td><td>12 (9.8)</td></tr><tr><td>Reproducible process – n (%)</td><td>174 (75.7)</td><td>80 (74.8)</td><td>94 (76.4)</td></tr><tr><td>Double selection</td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>198 (86.1)</td><td>94 (87.9)</td><td>104 (84.6)</td></tr><tr><td>No</td><td>5 (2.2)</td><td>2 (1.9)</td><td>3 (2.4)</td></tr><tr><td>Not reported</td><td>27 (11.7)</td><td>11 (10.3)</td><td>16 (13.0)</td></tr><tr><td>Double extraction</td><td></td><td></td><td></td></tr><tr><td>Yes</td><td>195 (84.8)</td><td>89 (83.2)</td><td>106 (86.2)</td></tr><tr><td>No</td><td>6 (2.6)</td><td>6 (5.6)</td><td>0 (0)</td></tr><tr><td>Not reported</td><td>29 (12.6)</td><td>12 (11.2)</td><td>17 (13.8)</td></tr><tr><td>Risk of bias assessment – n (%)</td><td>226 (98.3)</td><td></td><td></td></tr><tr><td>Cochrane RoB tool (with reported results)</td><td>207 (90.0)</td><td>95 (88.8)</td><td>112 (91.1)</td></tr><tr><td>Jadad scale</td><td>19 (8.3)</td><td>8 (7.5)</td><td>11 (8.9)</td></tr><tr><td>Pedro scale</td><td>2 (0.9)</td><td>2 (1.9)</td><td>0 (0)</td></tr><tr><td>Newcastle-Ottawa scale</td><td>3 (1.3)</td><td>1 (0.9)</td><td>2 (1.6)</td></tr><tr><td>NIHQ</td><td>2 (0.9)</td><td>1 (0.9)</td><td>1 (0.8)</td></tr><tr><td>GRADE Assessment – n (%)</td><td>90 (39.1)</td><td>40 (37.4)</td><td>50 (40.7)</td></tr><tr><td>Minimum set of 6 key methodological characteristics – n (%)</td><td>15 (6.5)</td><td>5 (4.7)</td><td>10 (8.1)</td></tr><tr><td>None of the six characteristics</td><td>1 (0.4)</td><td>1 (0.9)</td><td>0 (0)</td></tr><tr><td>1/6 characteristic</td><td>10 (4.3)</td><td>5 (4.7)</td><td>5 (4.1)</td></tr><tr><td>2/6 characteristics</td><td>43 (18.7)</td><td>22 (20.6)</td><td>21 (17.1)</td></tr><tr><td>3/6 characteristics</td><td>65 (28.3)</td><td>34 (31.8)</td><td>31 (25.2)</td></tr><tr><td>4/6 characteristics</td><td>51 (22.2)</td><td>16 (15.0)</td><td>35 (28.5)</td></tr><tr><td>5/6 characteristics</td><td>45 (19.6)</td><td>24 (22.4)</td><td>21 (17.1)</td></tr><tr><td>6/6 characteristics</td><td>15 (6.5)</td><td>5 (4.7)</td><td>10 (8.1)</td></tr></tbody></table>

SR, systematic review; RoB, risk of bias; GRADE, Grading of Recommendations, Assessment, Development, and Evaluations; NIHQ, National Institutes of Health Quality assessment tool.

The minimum set of methodological characteristics includes: Search on the three major databases + search of gray literature (at least clinical trial registries) + double selection and extraction + use of the Cochrane Risk of Bias tool + protocol registration on PROSPERO + GRADE assessment.

were 13 research questions with at least two SRs reporting a GRADE assessment. Of these, discrepancies were observed in the level of certainty for the same primary outcome across SRs for 9 (69.2%). Details on all disagreements in results and certainty of evidence are reported in eTable 1.

## 4. Discussion

This research-on-research study assessed the frequency of overlapping SRs with MA on the same research

question, with a detailed description of their general and methodological characteristics including whether they met a minimum set of key methodological characteristics and assessment of discrepancies in their conclusions. One-third of SRs with MA of RCTs published in 2022 overlapped with at least 1 other SR with MA published between 2018 and 2023, and redundancy in outcomes, and therefore in MAs performed, was common with a median proportion of redundancy of 55%. The topic COVID-19 had the highest number of overlapping SRs with MA, followed by questions in cardiovascular medicine and nephrology. Of the
105 research questions identified, only 11.4% had at least 1 SR meeting our minimum set of methodological characteristics that a good quality SR should have. Among the 33 research questions with overlapping SRs, for 21.2%, their MAs had discrepant results for the same primary outcome and for the 13 research questions with at least two SRs with a GRADE assessment, there were discrepancies in the level of certainty for the same primary outcome across SRs for 9 (69.2%).

Previous studies found a higher rate of overlapping SRs with MA but were not based on the same methodology [4,14]. In 2013, Siontis et al [4] found that 67% of MAs published in 2010 had at least 1 overlapping MA, but the authors searched MAs since PubMed inception and hence may have included updates. They considered only updates for Cochrane reviews and also considered MAs including both RCTs and observational studies, whereas our study included only MAs of RCTs. Katsura et al [14] later reported that 60.4% of SRs published in 2015 in the field of thoracoabdominal surgery had at least 1 overlapping SR in the preceding 5 years. Redundancy may be particularly high in specific domains, as was the case for COVID-19.

Overlapping SRs with MA on the same topic result in significant research waste when researchers engage in duplicative efforts that could be redirected toward other more relevant studies. Furthermore, our results suggest a poor methodological quality of these SRs, with only 6.5% meeting a minimum set of key methodological characteristics, which may also contribute to research waste. This leaves physicians and researchers uncertain about how to navigate through this multitude of redundant studies, especially when SRs with MA addressing the same research question do not draw the same conclusion.

As highlighted by Altman [15] 30 years ago, “we need less research, better research, and research conducted for the right reasons”. This is truer than ever, given the explosion in scientific literature and the profusion of poor-quality SRs with MA. All SRs should start with a search of existing SRs on the same topic and justify what they add to the existing literature. In addition, we strongly recommend that SRs follow our minimum set of methodological characteristics or more generally the Cochrane handbook. This minimum set should also be helpful for reviewers and editors when considering an SR for publication. One important perspective to limit the waste may be in the development of living SRs that aim to conduct a SR meeting methodological standards that is continuously updated as soon as new evidence is provided on a given research question [16,17].

The strengths of the study include its rigorous methodology. First, we screened all references indexed as “meta-analysis” in 2022 to constitute a random sample of SRs with MA published in 2022. Second, we elaborated a search equation for each research question to identify all SRs with MA published from 2018 to 2023. The selection

and data extraction process of all SRs were performed in duplicate. Finally, we assessed a minimum set of key methodological characteristics required by several major guidelines or methodological assessment tools for SRs and MAs.

Our study has limitations. We excluded SRs with individual patient-data or network MAs and as well as those including observational studies. Such types of studies add a substantial level of information and cannot be considered as a redundancy. We also excluded SRs with nonbinary primary outcomes for homogeneity purposes. We restricted our search of overlapping SRs on PubMed to “meta-analysis” as publication type or in titles for feasibility reasons, but we may have missed some SRs with MA because of inadequate reporting or indexing on PubMed. This may result in underestimating the rate of overlapping SRs. Conversely, we may have considered as overlapping SRs some updates because of difficulties in differentiating them. Our minimum set of key methodological characteristics, although based on several guidelines, has not been previously validated itself as a whole. Finally, the year of selection of the index SRs was 2022, and the COVID-19 context having been favorable to SR with MA profusion may affect the generalizability of our study to other time periods.

## 5. Conclusion

One-third of the SRs with MA published in 2022 had at least one overlapping SR with MA published from 2018 to 2023 and most of them did not meet a minimum set of key methodological standards. For one-fifth of the research questions, overlapping SRs provided discrepant results for the same outcome. For SRs reporting a GRADE assessment, the certainty of evidence was frequently different for a same outcome.

## CRediT authorship contribution statement

**Wilfred Kwok:** Writing — original draft, Investigation, Formal analysis, Data curation, Conceptualization.
**Titiane Dallant:** Writing — original draft, Investigation, Formal analysis, Data curation, Conceptualization.
**Guillaume Martin:** Writing — review & editing, Software, Formal analysis, Data curation.
**Gabriel Fournier:** Writing — review & editing, Investigation, Data curation.
**Blandine Kervennic:** Writing — review & editing, Investigation, Data curation.
**Ophélie Pingeon:** Writing — review & editing, Investigation, Data curation.
**Agnès Dechartres:** Writing — original draft, Supervision, Methodology, Conceptualization.

## Ethics statement

This study does not require ethical approval.
**Declaration of competing interest**

The authors declare that they have no conflict of interest.

**Supplementary data**

Supplementary data related to this article can be found at
https://doi.org/10.1016/j.jclinepi.2025.112018.

**Data availability**

Data will be made available on request.

**References**

[1] Oxford Centre for Evidence-Based Medicine. Levels of evidence (March 2009) — Centre for Evidence-Based Medicine (CEBM), University of Oxford [Internet]. Available at: https://www.cebm.ox.ac.uk/resources/levels-of-evidence/oxford-centre-for-evidence-based-medicine-levels-of-evidence-march-2009. Accessed March 21, 2023.

[2] Hernandez AV, Marti KM, Roman YM. Meta-analysis. Chest 2020; 158(1):S97-102.

[3] Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 2016; 94(3):485-514.

[4] Siontis KC, Hernandez-Boussard T, Ioannidis JPA. Overlapping meta-analyses on the same topic: survey of published studies. BMJ 2013;347:f4501.

[5] Hacke C, Nunan D. Discrepancies in meta-analyses answering the same clinical question were hard to explain: a meta-epidemiological study. J Clin Epidemiol 2020;119:47-56.

[6] Garner P, Hopewell S, Chandler J, MacLehose H, Akl EA, Beyene J, et al. When and how to update systematic reviews: consensus and checklist. BMJ 2016;354:i3507.

[7] Chapelle C, Ollier E, Girard P, Frere C, Mismetti P, Cucherat M, et al. An epidemic of redundant meta-analyses. J Thromb Haemost 2021; 19(5):1299-306.

[8] Cochrane handbook for systematic reviews of interventions. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. [Chapter 14]: Completing “Summary of findings” tables and grading the certainty of the evidence [Internet]. version 6.5 Available at: https://www.cochrane.org/authors/handbooksndmanuals/handbook/current/chapter-section-2. Accessed August 21, 2025.

[9] Cochrane Handbook for Systematic Reviews of Interventions. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. [Chapter 1]: Starting a review [Internet]. version 6.5. Available at: https://training.cochrane.org/handbook/current/chapter. Accessed April 10, 2024.

[10] Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016;69:225-34.

[11] Murad MH, Montori VM, Ioannidis JPA, Jaeschke R, Devereaux PJ, Prasad K, et al. How to read a systematic review and meta-analysis and apply the results to patient care: users' guides to the medical literature. JAMA 2014;312(2):171.

[12] Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021 Mar 29;372:n71.

[13] Higgins JPT, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.

[14] Katsura M, Kuriyama A, Tada M, Tsujimoto Y, Luo Y, Yamamoto K, et al. High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study. Br J Surg 2021;108(12):1521-9.

[15] Altman DG. The scandal of poor medical research. BMJ 1994;308 (6924):283-4.

[16] Living systematic reviews | Cochrane Community [Internet]. Available at: https://community.cochrane.org/review-development/resources/living-systematic-reviews. Accessed November 1, 2024.

[17] Simmonds M, Elliott JH, Synnot A, Turner T. Living systematic reviews. Methods Mol Biol 2022;2345:121-34.