# Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study

Amandine Dugard, Elsa Tavernier, Agnès Caille, Agnès Dechartres, Adeline

Hoang, Bruno Giraudeau, C. Dibao-Dina

<figure><img src="image_1.png" /></figure>

## To cite this version:

Amandine Dugard, Elsa Tavernier, Agnès Caille, Agnès Dechartres, Adeline Hoang, et al.. Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study. BMC Medical Research Methodology, 2022, 22 (1), pp.329. 10.1186/s12874-022-01815-2. hal-04027325

HAL Id: hal-04027325

https://hal.sorbonne-universite.fr/hal-04027325

Submitted on 29 May 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study

Amandine Dugard, Elsa Tavernier, Agnès Caille, Agnès Dechartres, Adeline Hoang, Bruno Giraudeau, Clarisse Dibao-Dina

<figure><img src="image_1.png" /></figure>

## To cite this version:

Amandine Dugard, Elsa Tavernier, Agnès Caille, Agnès Dechartres, Adeline Hoang, et al.. Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study. BMC Medical Research Methodology, 2022, 22 (1), pp.329. 10.1186/s12874-022-01815-2. doi:10.1186/s12874-022-01815-2

HAL Id: hal-04027325

https://hal.sorbonne-universite.fr/hal-04027325v1

Submitted on 29 May 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RESEARCH

Open Access

<figure><img src="image_3.png" /></figure>

Check for
updates

# Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study

Amandine Dugard<fnref n="1" />*, Elsa Tavernier<fnref n="1" />,², Agnès Caille<fnref n="1" />,², Agnès Dechartres³, Adeline Hoang⁴, Bruno Giraudeau<fnref n="1" /> and Clarisse Dibao-Dina<fnref n="1" />,⁴

**Abstract**

**Background:** Many clinical practice guidelines are based on randomised controlled trials conducted in secondary or tertiary care setting and general practitioners frequently question their relevance for primary care patients. Our aim was to compare the intervention effect estimates between primary care setting randomised controlled trials (PC-RCTs) and secondary or tertiary care setting randomised controlled trials (ST-RCTs).

**Methods:** Meta-epidemiological study of meta-analyses (MAs) of a binary outcome including at least one PC-RCT and one ST-RCT. PC-RCTs were defined as trials recruiting patients in general practices, primary care practices, family practices, community centers or community pharmacies. ST-RCTs were defined as trials recruiting in hospitals, including hospitalized patients, hospital outpatients and patients from emergency departments. For each MA, we estimated a ratio of odds ratio (ROR) by using random-effects meta-regression, with an ROR less than 1 indicating lower estimates of the intervention effect in PC-RCTs than ST-RCTs. Finally, we estimated a combined ROR across MAs by using a random-effects meta-analysis. We performed subgroup analyses considering the type of outcomes (objective vs subjective), type of experimental intervention (pharmacological vs non-pharmacological), and control group (active vs inactive) as well as analyses adjusted on items of the Cochrane Risk of Bias tool.

**Results:** Among 1765 screened reviews, 76 MAs with 230 PC-RCTs and 384 ST-RCTs were selected. The main medical fields were pneumology (13.2%) and psychiatry or addictology (38.2%). Intervention effect estimates did not significantly differ between PC-RCTs and ST-RCTs (ROR = 0.97, 95% confidence interval 0.88 to 1.08), with moderate heterogeneity across MAs (I² = 45%). Subgroup and adjusted analyses led to consistent results.

**Conclusion:** We did not observe any significant difference in intervention effect estimates between PC-RCTs and ST-RCTs. Nevertheless, most of the medical fields in this meta-epidemiological study were not representative of the pathologies encountered in primary care. Further studies with pathologies more frequently encountered in primary care are needed.

**Keywords:** Meta-epidemiological study, Cochrane systematic review, Primary care

**Background**

Many studies have highlighted the lack of adherence to clinical practice guidelines by general practitioners (GPs) [1, 2]. GPs generally argue a lack of relevance or

<footnote>*Correspondence: amandine.dugard@etu.univ-tours.fr</footnote>

<footnote n="1">¹ University of Tours, University of Nantes, INSERM 1246-SPHERE, Tours, France
Full list of author information is available at the end of the article</footnote>
RESEARCH

Open Access

<figure><img src="image_3.png" /></figure>

Check for
updates

# Intervention effect estimates in randomised controlled trials conducted in primary care versus secondary or tertiary care settings: a meta-epidemiological study

Amandine Dugard<fnref n="1" />*, Elsa Tavernier<fnref n="1" />,², Agnès Caille<fnref n="1" />,², Agnès Dechartres³, Adeline Hoang⁴, Bruno Giraudeau<fnref n="1" /> and Clarisse Dibao-Dina<fnref n="1" />,⁴

**Abstract**

**Background:** Many clinical practice guidelines are based on randomised controlled trials conducted in secondary or tertiary care setting and general practitioners frequently question their relevance for primary care patients. Our aim was to compare the intervention effect estimates between primary care setting randomised controlled trials (PC-RCTs) and secondary or tertiary care setting randomised controlled trials (ST-RCTs).

**Methods:** Meta-epidemiological study of meta-analyses (MAs) of a binary outcome including at least one PC-RCT and one ST-RCT. PC-RCTs were defined as trials recruiting patients in general practices, primary care practices, family practices, community centers or community pharmacies. ST-RCTs were defined as trials recruiting in hospitals, including hospitalized patients, hospital outpatients and patients from emergency departments. For each MA, we estimated a ratio of odds ratio (ROR) by using random-effects meta-regression, with an ROR less than 1 indicating lower estimates of the intervention effect in PC-RCTs than ST-RCTs. Finally, we estimated a combined ROR across MAs by using a random-effects meta-analysis. We performed subgroup analyses considering the type of outcomes (objective vs subjective), type of experimental intervention (pharmacological vs non-pharmacological), and control group (active vs inactive) as well as analyses adjusted on items of the Cochrane Risk of Bias tool.

**Results:** Among 1765 screened reviews, 76 MAs with 230 PC-RCTs and 384 ST-RCTs were selected. The main medical fields were pneumology (13.2%) and psychiatry or addictology (38.2%). Intervention effect estimates did not significantly differ between PC-RCTs and ST-RCTs (ROR = 0.97, 95% confidence interval 0.88 to 1.08), with moderate heterogeneity across MAs (I² = 45%). Subgroup and adjusted analyses led to consistent results.

**Conclusion:** We did not observe any significant difference in intervention effect estimates between PC-RCTs and ST-RCTs. Nevertheless, most of the medical fields in this meta-epidemiological study were not representative of the pathologies encountered in primary care. Further studies with pathologies more frequently encountered in primary care are needed.

**Keywords:** Meta-epidemiological study, Cochrane systematic review, Primary care

**Background**

Many studies have highlighted the lack of adherence to clinical practice guidelines by general practitioners (GPs) [1, 2]. GPs generally argue a lack of relevance or

<footnote>*Correspondence: amandine.dugard@etu.univ-tours.fr</footnote>

<footnote n="1">¹ University of Tours, University of Nantes, INSERM 1246-SPHERE, Tours, France
Full list of author information is available at the end of the article</footnote>
difficulties in applying guidelines [3, 4], and they par-
ticularly express concerns about the applicability of clini-
cal trial results used to establish guidelines to their own
patients [5, 6]. The first reason is that many randomised
controlled trials (RCTs) include highly selected patients,
which restricts the generalizability of their results [7].
As an example, a study found that less than half of pri-
mary-care patients would have been included in hyper-
tension trials [8]. The second, highly related to the first
one reason is that in most RCTs, patients are recruited
in secondary- or tertiary-care settings [9] rather than in
primary-care. However, as compared with secondary- or
tertiary-care setting patients, primary-care patients have
less severe disease and more undifferentiated symptoms
[10] but more multi-morbidity [11]. Such differences may
lead to differences in benefits and harms of the interven-
tion. Our hypothesis is that intervention effect estimates
differ between primary care setting randomised con-
trolled trials (PC-RCTs) and secondary or tertiary care
setting RCTs (ST-RCTs).

We performed a meta-epidemiological study to assess
whether intervention effect estimates differ between PC-
RCTs and ST-RCTs.

Methods

This was a meta-epidemiological study, a study design
used to compare intervention effect estimates between
trials with and without a characteristic of interest [12].
In this study, we focused on the setting, with the aim of
comparing PC-RCTs to ST-RCTs. A meta-epidemiol-
logical study is performed by generally using a two-step
approach and, contrary to traditional epidemiological
studies, units of analysis are studies rather than patients
[13]. First, for each selected MA, we assessed the dif-
ference in intervention effects between studies with the
characteristics of interest (i.e., primary care setting) and
without the characteristics of interest. This step involved
using a meta-regression for each selected MA (with the
intervention effect considered the outcome and the char-
acteristic of interest the independent variable). Second,
results from the meta-regression were meta-analysed
over the different MAs.

Search strategy

On July 9, 2020, we searched the Cochrane Database of Systematic Reviews to retrieve all systematic reviews with meta-analyses (MAs) that were published, with no restriction on time, by using the following text words in the full text: "primary care" OR "primary healthcare" OR "general practice" OR "family practice". We only used primary care keywords to ensure that we had PC-RCTs in each MA.

Selection of relevant MAs and trials

We screened the full text of potentially eligible system-
atic reviews to select MAs of binary outcomes that had at
least 3 trials (3 studies is the minimum to perform a meta-
regression) and at least one PC-RCT and one ST-RCT. PC-
RCTs were defined as trials recruiting patients in general
practices, primary care practices, family practices, com-
munity centers or community pharmacies according to the
definition by Afonso et al. [14]. Trials in nursing homes or
at home were not considered PC-RCTs and were excluded.
ST-RCTs were defined as trials recruiting in hospitals,
including hospitalized patients, hospital outpatients and
patients from emergency departments. Hospital-at-home
trials were not considered ST-RCTs and were excluded.
Trials with both primary and secondary or tertiary care
settings were excluded, as were trials with an unclear set-
ting or trials including patients in other than primary or
secondary or tertiary care settings, such as schools. For
this study, we selected only RCTs and excluded non-ran-
domised or quasi-randomised trials.

If more than one MA was eligible within the same
review, we selected the MA for the primary efficacy out-
come, then the MA with the highest number of trials.
MAs of adverse events were not included because of the
uncertainty in the direction of bias. We also discarded
MAs when it was impossible to determine which group
was the experimental and control group.

When a trial was included several times within the same
MA, we kept only the duplicate with the largest sample
size. When a trial was included in several selected MAs,
we kept the one in the most recent systematic review.

All this selection process was performed indepen-
dently by two reviewers (A.D., A.H.), with disagree-
ments resolved by discussion, referring to a third opinion
(C.D.D.) when necessary.

Data collection

Two independent reviewers (A.D., A.H.) collected data
from all selected MAs and trials by using a standardised
data collection form. Disagreements were resolved by
discussion, referring to a third opinion (C.D.D.) when
necessary. The following characteristics were extracted:

For each meta-analysis:

- General information: year of publication, first author name

- Medical field

- Type of intervention (pharmacological, non-pharmacological), type of comparator (active control or inactive control, with inactive control defined as placebo or no added intervention to usual care, sham or other)
difficulties in applying guidelines [3, 4], and they par-
ticularly express concerns about the applicability of clini-
cal trial results used to establish guidelines to their own
patients [5, 6]. The first reason is that many randomised
controlled trials (RCTs) include highly selected patients,
which restricts the generalizability of their results [7].
As an example, a study found that less than half of pri-
mary-care patients would have been included in hyper-
tension trials [8]. The second, highly related to the first
one reason is that in most RCTs, patients are recruited
in secondary- or tertiary-care settings [9] rather than in
primary-care. However, as compared with secondary- or
tertiary-care setting patients, primary-care patients have
less severe disease and more undifferentiated symptoms
[10] but more multi-morbidity [11]. Such differences may
lead to differences in benefits and harms of the interven-
tion. Our hypothesis is that intervention effect estimates
differ between primary care setting randomised con-
trolled trials (PC-RCTs) and secondary or tertiary care
setting RCTs (ST-RCTs).

We performed a meta-epidemiological study to assess
whether intervention effect estimates differ between PC-
RCTs and ST-RCTs.

Methods

This was a meta-epidemiological study, a study design
used to compare intervention effect estimates between
trials with and without a characteristic of interest [12].
In this study, we focused on the setting, with the aim of
comparing PC-RCTs to ST-RCTs. A meta-epidemi-
logical study is performed by generally using a two-step
approach and, contrary to traditional epidemiological
studies, units of analysis are studies rather than patients
[13]. First, for each selected MA, we assessed the dif-
ference in intervention effects between studies with the
characteristics of interest (i.e., primary care setting) and
without the characteristics of interest. This step involved
using a meta-regression for each selected MA (with the
intervention effect considered the outcome and the char-
acteristic of interest the independent variable). Second,
results from the meta-regression were meta-analysed
over the different MAs.

Search strategy

On July 9, 2020, we searched the Cochrane Database of Systematic Reviews to retrieve all systematic reviews with meta-analyses (MAs) that were published, with no restriction on time, by using the following text words in the full text: "primary care" OR "primary healthcare" OR "general practice" OR "family practice". We only used primary care keywords to ensure that we had PC-RCTs in each MA.

Selection of relevant MAs and trials

We screened the full text of potentially eligible system-
atic reviews to select MAs of binary outcomes that had at
least 3 trials (3 studies is the minimum to perform a meta-
regression) and at least one PC-RCT and one ST-RCT. PC-
RCTs were defined as trials recruiting patients in general
practices, primary care practices, family practices, com-
munity centers or community pharmacies according to the
definition by Afonso et al. [14]. Trials in nursing homes or
at home were not considered PC-RCTs and were excluded.
ST-RCTs were defined as trials recruiting in hospitals,
including hospitalized patients, hospital outpatients and
patients from emergency departments. Hospital-at-home
trials were not considered ST-RCTs and were excluded.
Trials with both primary and secondary or tertiary care
settings were excluded, as were trials with an unclear set-
ting or trials including patients in other than primary or
secondary or tertiary care settings, such as schools. For
this study, we selected only RCTs and excluded non-ran-
domised or quasi-randomised trials.

If more than one MA was eligible within the same
review, we selected the MA for the primary efficacy out-
come, then the MA with the highest number of trials.
MAs of adverse events were not included because of the
uncertainty in the direction of bias. We also discarded
MAs when it was impossible to determine which group
was the experimental and control group.

When a trial was included several times within the same
MA, we kept only the duplicate with the largest sample
size. When a trial was included in several selected MAs,
we kept the one in the most recent systematic review.

All this selection process was performed indepen-
dently by two reviewers (A.D., A.H.), with disagree-
ments resolved by discussion, referring to a third opinion
(C.D.D.) when necessary.

Data collection

Two independent reviewers (A.D., A.H.) collected data
from all selected MAs and trials by using a standardised
data collection form. Disagreements were resolved by
discussion, referring to a third opinion (C.D.D.) when
necessary. The following characteristics were extracted:

For each meta-analysis:

- General information: year of publication, first author name

- Medical field

- Type of intervention (pharmacological, non-pharmacological), type of comparator (active control or inactive control, with inactive control defined as placebo or no added intervention to usual care, sham or other)
- Outcome and whether it was subjective or objective following the classification provided by Savović et al. [15]

- All-cause mortality

- Objectively assessed (e.g., laboratory results)

- Objectively assessed but potentially influenced by a clinician or patient (e.g., smoking cessation)

- Subjectively assessed (e.g., pain)

Objective outcomes were the two first categories and subjective outcomes were the two last categories.

- Number of included studies

For each eligible RCT included in the MA:

- Author

- Year of publication

- First author name

- Trial type (PC-RCT or ST-RCT)

- Number of centers (for primary care, a center was defined as a practice, and for secondary and tertiary care, it was a hospital)

- Sample size

- For each group, the number of events and the number of patients analysed

- Domains of the Cochrane Risk of Bias tool [16] as was reported by the authors of the systematic reviews

For PC- RCTs, we extracted the following additional information if mentioned in the MA:

- Type of primary care setting

- Healthcare professionals who recruited patients (when mentioned)

- Whether the settings where patients are seen during the study for recruitment, follow-up and primary outcome assessment were the same or not

- Country(ies) where the study was performed because of existing differences in healthcare systems between countries in primary-care

All data were collected from the systematic review report except for the number of events, which was collected from the trial reports when missing in the MA report.

<figure><img src="image_4.png" /><figcaption>**Fig. 1** Flowchart of the selection process</figcaption></figure>
- Outcome and whether it was subjective or objective following the classification provided by Savović et al. [15]

- All-cause mortality

- Objectively assessed (e.g., laboratory results)

- Objectively assessed but potentially influenced by a clinician or patient (e.g., smoking cessation)

- Subjectively assessed (e.g., pain)

Objective outcomes were the two first categories and subjective outcomes were the two last categories.

- Number of included studies

For each eligible RCT included in the MA:

- Author

- Year of publication

- First author name

- Trial type (PC-RCT or ST-RCT)

- Number of centers (for primary care, a center was defined as a practice, and for secondary and tertiary care, it was a hospital)

- Sample size

- For each group, the number of events and the number of patients analysed

- Domains of the Cochrane Risk of Bias tool [16] as was reported by the authors of the systematic reviews

For PC- RCTs, we extracted the following additional information if mentioned in the MA:

- Type of primary care setting

- Healthcare professionals who recruited patients (when mentioned)

- Whether the settings where patients are seen during the study for recruitment, follow-up and primary outcome assessment were the same or not

- Country(ies) where the study was performed because of existing differences in healthcare systems between countries in primary-care

All data were collected from the systematic review report except for the number of events, which was collected from the trial reports when missing in the MA report.

<figure><img src="image_4.png" /><figcaption>**Fig. 1** Flowchart of the selection process</figcaption></figure>
## Statistical analysis

We estimated intervention effects as odds ratios (OR). Outcome events were re-coded so that an OR less than 1 indicated a beneficial effect of the experimental intervention. Randomised controlled trials with no event in both groups did not contribute to the analysis.

## Meta-epidemiological analysis

To compare PC-RCTs and ST-RCTs, we used the two-step approach described by Sterne et al. [12]. First for each meta-analysis, we estimated a ratio of odds ratio (ROR) by using random-effects meta-regression. In our study, the ROR was the ratio of the OR for PC-RCTs to the OR for ST-RCTs, An ROR less than 1 indicates lower intervention effects for PC-RCTs. Second, we estimated a combined ROR across meta-analyses and the 95% confidence interval (CI) by using a random-effects meta-analysis model. The heterogeneity across MAs was assessed with the I² statistic and its 95% CI and the between–meta-analysis variance τ².

## Subgroup and sensitivity analyses

Subgroup analyses were planned according to the objectivity of outcomes, type of intervention in the experimental group (pharmacological vs non-pharmacological), and control group (active vs inactive). We used an interaction test to assess whether the combined ROR varied between subgroups. We also performed sensitivity analyses by adjusting meta-regression models on each item of the Risk of Bias tool [16] (high or unclear risk vs low risk).

## Results

### Study selection and general characteristics

From 1765 identified systematic reviews, we selected 76 MAs with 1685 trials, from which 614 trials were selected (Fig. 1, full references of the 76 MAs in Additional file 1 and 2). Of these 76 MAs, 33 (43.4%) evaluated a pharmacological intervention, 15 (19.7%) had an active control and 32 (42.1%) concerned a subjective outcome. The median number of trials included per MA was 5 (interquartile range [IQR] 3.8–10). MA characteristics are detailed in Table 1.

No eligible MA refers to meta-analyses with several reasons for not selecting it: fewer than 3 trials, without both primary and secondary or tertiary care settings or without a primary efficacy outcome.

MA: meta-analysis.

MA: meta-analysis; IQR: interquartile range; PC-RCTs: primary care setting randomised controlled trials; ST-RCTs: secondary or tertiary care setting randomised controlled trials.

### Characteristics of selected trials

Among 614 trials selected, 230 were PC-RCTs and 384 ST-RCTs (Table 2). The median sample size was 276

<table><caption>**Table 1** Characteristics of the meta-analyses (MAs) included (n = 76)</caption><thead><tr><th colspan="2">MA characteristics</th></tr><tr><th>Year of publication, median (IQR)</th><th>2016 (2012–2018)</th></tr></thead><tbody><tr><td colspan="2"><strong>Medical fields, n (%)</strong></td></tr><tr><td>Psychiatry/addictology</td><td>29 (38.2)</td></tr><tr><td>Pneumology</td><td>10 (13.2)</td></tr><tr><td>Ear nose and throat</td><td>5 (6.6)</td></tr><tr><td>Cardiology</td><td>4 (5.3)</td></tr><tr><td>Dermatology</td><td>4 (5.3)</td></tr><tr><td>Pediatrics</td><td>4 (5.3)</td></tr><tr><td>Infectious diseases</td><td>3 (3.9)</td></tr><tr><td>Others</td><td>17 (22.4)</td></tr><tr><td colspan="2"><strong>Intervention in experimental group, n (%)</strong></td></tr><tr><td>Pharmacological</td><td>33 (43.4)</td></tr><tr><td>Non-pharmacological</td><td>43 (56.6)</td></tr><tr><td colspan="2"><strong>Intervention in control group, n (%)</strong></td></tr><tr><td>Active comparator</td><td>15 (19.7)</td></tr><tr><td>Placebo</td><td>15 (19.7)</td></tr><tr><td>No added intervention (i.e., usual care)</td><td>34 (44.7)</td></tr><tr><td>Sham comparator</td><td>1 (1.3)</td></tr><tr><td>Other</td><td>11 (14.6)</td></tr><tr><td colspan="2"><strong>Outcome objectivity, n (%)</strong></td></tr><tr><td>All-cause mortality</td><td>2 (2.6)</td></tr><tr><td>Objectively assessed</td><td>10 (13.2)</td></tr><tr><td>Objectively assessed but influenced by clinician or patient</td><td>31 (40.8)</td></tr><tr><td>Subjectively assessed</td><td>32 (42.1)</td></tr><tr><td>Unclear</td><td>1 (1.3)</td></tr><tr><td colspan="2"><strong>Number of trials, median (IQR) (range)</strong></td></tr><tr><td>Included</td><td>5 (3.8–10) (3 to 32)</td></tr><tr><td>PC-RCTs</td><td>2 (1–3) (1 to 17)</td></tr><tr><td>ST-RCTs</td><td>3 (1.8–6) (1 to 30)</td></tr></tbody></table>

(IQR 146–561.5) for PC-RCTs and 139 (IQR 66–284.3) for ST-RCTs. The median number of centers was 13 (IQR 3–40.5) and 1 (IQR 1–3), respectively. A total of 61 (62.9%) PC-RCTs were at low risk of bias for blinding of outcome assessors as compared with 83 (43.4%) ST-RCTs. More characteristics of the PC-RCTs included in this study are available in Additional file 3.

IQR: interquartile range, PC-RCT: primary care setting randomised controlled trial, ST-RCT: secondary or tertiary care setting randomised controlled trial.

### Differences in intervention effect estimates between PC-RCTs and ST-RCTs

We found no statistical difference in intervention effect estimates between PC-RCTs and ST-RCTs; the combined ROR was 0.97 (95% CI 0.88 to 1.08) (Fig. 2). The
## Statistical analysis

We estimated intervention effects as odds ratios (OR). Outcome events were re-coded so that an OR less than 1 indicated a beneficial effect of the experimental intervention. Randomised controlled trials with no event in both groups did not contribute to the analysis.

## Meta-epidemiological analysis

To compare PC-RCTs and ST-RCTs, we used the two-step approach described by Sterne et al. [12]. First for each meta-analysis, we estimated a ratio of odds ratio (ROR) by using random-effects meta-regression. In our study, the ROR was the ratio of the OR for PC-RCTs to the OR for ST-RCTs, An ROR less than 1 indicates lower intervention effects for PC-RCTs. Second, we estimated a combined ROR across meta-analyses and the 95% confidence interval (CI) by using a random-effects meta-analysis model. The heterogeneity across MAs was assessed with the I² statistic and its 95% CI and the between–meta-analysis variance τ².

## Subgroup and sensitivity analyses

Subgroup analyses were planned according to the objectivity of outcomes, type of intervention in the experimental group (pharmacological vs non-pharmacological), and control group (active vs inactive). We used an interaction test to assess whether the combined ROR varied between subgroups. We also performed sensitivity analyses by adjusting meta-regression models on each item of the Risk of Bias tool [16] (high or unclear risk vs low risk).

## Results

### Study selection and general characteristics

From 1765 identified systematic reviews, we selected 76 MAs with 1685 trials, from which 614 trials were selected (Fig. 1, full references of the 76 MAs in Additional file 1 and 2). Of these 76 MAs, 33 (43.4%) evaluated a pharmacological intervention, 15 (19.7%) had an active control and 32 (42.1%) concerned a subjective outcome. The median number of trials included per MA was 5 (interquartile range [IQR] 3.8–10). MA characteristics are detailed in Table 1.

No eligible MA refers to meta-analyses with several reasons for not selecting it: fewer than 3 trials, without both primary and secondary or tertiary care settings or without a primary efficacy outcome.

MA: meta-analysis.

MA: meta-analysis; IQR: interquartile range; PC-RCTs: primary care setting randomised controlled trials; ST-RCTs: secondary or tertiary care setting randomised controlled trials.

### Characteristics of selected trials

Among 614 trials selected, 230 were PC-RCTs and 384 ST-RCTs (Table 2). The median sample size was 276

<table><caption>**Table 1** Characteristics of the meta-analyses (MAs) included (n = 76)</caption><thead><tr><th colspan="2">MA characteristics</th></tr><tr><th>Year of publication, median (IQR)</th><th>2016 (2012–2018)</th></tr></thead><tbody><tr><td colspan="2"><strong>Medical fields, n (%)</strong></td></tr><tr><td>Psychiatry/addictology</td><td>29 (38.2)</td></tr><tr><td>Pneumology</td><td>10 (13.2)</td></tr><tr><td>Ear nose and throat</td><td>5 (6.6)</td></tr><tr><td>Cardiology</td><td>4 (5.3)</td></tr><tr><td>Dermatology</td><td>4 (5.3)</td></tr><tr><td>Pediatrics</td><td>4 (5.3)</td></tr><tr><td>Infectious diseases</td><td>3 (3.9)</td></tr><tr><td>Others</td><td>17 (22.4)</td></tr><tr><td colspan="2"><strong>Intervention in experimental group, n (%)</strong></td></tr><tr><td>Pharmacological</td><td>33 (43.4)</td></tr><tr><td>Non-pharmacological</td><td>43 (56.6)</td></tr><tr><td colspan="2"><strong>Intervention in control group, n (%)</strong></td></tr><tr><td>Active comparator</td><td>15 (19.7)</td></tr><tr><td>Placebo</td><td>15 (19.7)</td></tr><tr><td>No added intervention (i.e., usual care)</td><td>34 (44.7)</td></tr><tr><td>Sham comparator</td><td>1 (1.3)</td></tr><tr><td>Other</td><td>11 (14.6)</td></tr><tr><td colspan="2"><strong>Outcome objectivity, n (%)</strong></td></tr><tr><td>All-cause mortality</td><td>2 (2.6)</td></tr><tr><td>Objectively assessed</td><td>10 (13.2)</td></tr><tr><td>Objectively assessed but influenced by clinician or patient</td><td>31 (40.8)</td></tr><tr><td>Subjectively assessed</td><td>32 (42.1)</td></tr><tr><td>Unclear</td><td>1 (1.3)</td></tr><tr><td colspan="2"><strong>Number of trials, median (IQR) (range)</strong></td></tr><tr><td>Included</td><td>5 (3.8–10) (3 to 32)</td></tr><tr><td>PC-RCTs</td><td>2 (1–3) (1 to 17)</td></tr><tr><td>ST-RCTs</td><td>3 (1.8–6) (1 to 30)</td></tr></tbody></table>

(IQR 146–561.5) for PC-RCTs and 139 (IQR 66–284.3) for ST-RCTs. The median number of centers was 13 (IQR 3–40.5) and 1 (IQR 1–3), respectively. A total of 61 (62.9%) PC-RCTs were at low risk of bias for blinding of outcome assessors as compared with 83 (43.4%) ST-RCTs. More characteristics of the PC-RCTs included in this study are available in Additional file 3.

IQR: interquartile range, PC-RCT: primary care setting randomised controlled trial, ST-RCT: secondary or tertiary care setting randomised controlled trial.

### Differences in intervention effect estimates between PC-RCTs and ST-RCTs

We found no statistical difference in intervention effect estimates between PC-RCTs and ST-RCTs; the combined ROR was 0.97 (95% CI 0.88 to 1.08) (Fig. 2). The
<table><caption>**Table 2** Characteristics of the included trials</caption><thead><tr><th>Trial characteristics</th><th>PC-RCTs</th><th>ST-RCTs</th></tr></thead><tbody><tr><td><strong>N = 230</strong></td><td></td><td></td></tr><tr><td>Year of publication, median (IQR)</td><td>2003 (1994–2010)</td><td>2005 (1996–2011)</td></tr><tr><td>Sample size, median (IQR)</td><td>276 (146–561.5)</td><td>139 (166–284.3)</td></tr><tr><td>Number of centers, median (IQR)</td><td>13 (3–40.5)</td><td>1 (1–3)</td></tr><tr><td><strong>Medical fields</strong></td><td></td><td></td></tr><tr><td>Psychiatry/Addic-tology</td><td>109 (47.4)</td><td>184 (47.9)</td></tr><tr><td>Pneumology</td><td>15 (6.5)</td><td>66 (17.2)</td></tr><tr><td>Ear nose and throat</td><td>12 (5.2)</td><td>12 (3.1)</td></tr><tr><td>Cardiology</td><td>9 (3.9)</td><td>15 (3.9)</td></tr><tr><td>Dermatology</td><td>6 (2.6)</td><td>11 (2.9)</td></tr><tr><td>Pediatrics</td><td>11 (4.8)</td><td>21 (5.5)</td></tr><tr><td>Infectious diseases</td><td>8 (3.5)</td><td>4 (1.1)</td></tr><tr><td>Others</td><td>60 (26.1)</td><td>71 (18.5)</td></tr><tr><td><strong>Intervention in experimental group, n (%)</strong></td><td></td><td></td></tr><tr><td>Pharmacological</td><td>79 (34.3)</td><td>171 (44.5)</td></tr><tr><td>Non-pharmacological</td><td>151 (65.7)</td><td>213 (55.5)</td></tr><tr><td><strong>Intervention in control group, n (%)</strong></td><td></td><td></td></tr><tr><td>Active comparator</td><td>35 (15.2)</td><td>78 (20.3)</td></tr><tr><td>Placebo</td><td>29 (12.6)</td><td>39 (10.2)</td></tr><tr><td>No added intervention (i.e., usual care)</td><td>115 (50.0)</td><td>188 (49.0)</td></tr><tr><td>Sham comparator</td><td>2 (0.9)</td><td>2 (0.5)</td></tr><tr><td>Other</td><td>49 (21.3)</td><td>77 (20.0)</td></tr><tr><td><strong>Outcome objectivity, n (%)</strong></td><td></td><td></td></tr><tr><td>All-cause mortality</td><td>3 (1.3)</td><td>32 (8.3)</td></tr><tr><td>Objectively assessed</td><td>55 (23.9)</td><td>48 (12.5)</td></tr><tr><td>Objectively assessed but influenced by clinician or patient</td><td>95 (41.3)</td><td>170 (44.3)</td></tr><tr><td>Subjectively assessed</td><td>75 (32.6)</td><td>132 (34.4)</td></tr><tr><td>Unclear</td><td>2 (0.9)</td><td>2 (0.5)</td></tr><tr><td><strong>Cochrane Risk of Bias tool, n (%)</strong></td><td></td><td></td></tr><tr><td>Random sequence generation</td><td></td><td></td></tr><tr><td>Low</td><td>120 (57.4)</td><td>197 (57.4)</td></tr><tr><td>High</td><td>11 (5.3)</td><td>11 (3.2)</td></tr><tr><td>Unclear</td><td>78 (37.3)</td><td>135 (39.4)</td></tr><tr><td>Allocation concealment</td><td></td><td></td></tr><tr><td>Low</td><td>116 (50.4)</td><td>165 (43.0)</td></tr><tr><td>High</td><td>21 (9.1)</td><td>26 (6.8)</td></tr></tbody></table>

<table><caption>**Table 2** (continued)</caption><thead><tr><th>Trial characteristics</th><th>PC-RCTs</th><th>ST-RCTs</th></tr></thead><tbody><tr><td>Unclear</td><td>93 (40.4)</td><td>193 (50.3)</td></tr><tr><td>Blinding of participants and personnel</td><td></td><td></td></tr><tr><td>Low</td><td>68 (45.6)</td><td>74 (30.1)</td></tr><tr><td>High</td><td>33 (22.2)</td><td>62 (25.2)</td></tr><tr><td>Unclear</td><td>48 (32.2)</td><td>110 (44.7)</td></tr><tr><td>Blinding of outcome assessors</td><td></td><td></td></tr><tr><td>Low</td><td>61 (62.9)</td><td>83 (43.4)</td></tr><tr><td>High</td><td>14 (14.4)</td><td>50 (26.2)</td></tr><tr><td>Unclear</td><td>22 (22.7)</td><td>58 (30.4)</td></tr><tr><td>Incomplete data outcome</td><td></td><td></td></tr><tr><td>Low</td><td>135 (64.3)</td><td>215 (62.7)</td></tr><tr><td>High</td><td>34 (16.2)</td><td>53 (15.4)</td></tr><tr><td>Unclear</td><td>41 (19.5)</td><td>75 (21.9)</td></tr><tr><td>Selective reporting</td><td></td><td></td></tr><tr><td>Low</td><td>85 (63.9)</td><td>144 (58.5)</td></tr><tr><td>High</td><td>9 (6.8)</td><td>23 (9.4)</td></tr><tr><td>Unclear</td><td>39 (29.3)</td><td>79 (32.1)</td></tr></tbody></table>

heterogeneity across MAs was moderate ($I^2$ = 45, 95%CI 29 to 58%, between–meta-analysis variance $\tau^2$ = 0.07).

PC-RCTs: primary care setting randomised controlled trial.

ST-RCTs: secondary or tertiary care setting randomised controlled trial.

### Subgroup analyses

For the 33 MAs assessing pharmacological interventions, the combined ROR was estimated at 0.90 (95% CI 0.76 to 1.07), and for the 43 MAs assessing non-pharmacological interventions, the combined ROR was 1.01 (95% CI 0.90 to 1.14). The interaction test was not statistically significant ($P$ = 0.97). We did not find a significant interaction in the other subgroup analyses (Fig. 3).

ROR: ratio of odds ratio; CI: confidence interval; MA: meta-analysis; PC-RCTs: primary care setting randomised controlled trials; ST-RCTs: secondary or tertiary care setting randomised controlled trials.

### Sensitivity analyses

Results were consistent when adjusting on each item of the risk of bias (Fig. 4).

ROR: ratio of odds ratio; CI: confidence interval; MA: meta-analysis; PC-RCTs: primary care setting randomised controlled trials; ST-RCTs: secondary or tertiary care setting randomised controlled trials.
<table><caption>**Table 2** Characteristics of the included trials</caption><thead><tr><th>Trial characteristics</th><th>PC-RCTs</th><th>ST-RCTs</th></tr></thead><tbody><tr><td><strong>N = 230</strong></td><td></td><td></td></tr><tr><td>Year of publication, median (IQR)</td><td>2003 (1994–2010)</td><td>2005 (1996–2011)</td></tr><tr><td>Sample size, median (IQR)</td><td>276 (146–561.5)</td><td>139 (166–284.3)</td></tr><tr><td>Number of centers, median (IQR)</td><td>13 (3–40.5)</td><td>1 (1–3)</td></tr><tr><td><strong>Medical fields</strong></td><td></td><td></td></tr><tr><td>Psychiatry/Addic-tology</td><td>109 (47.4)</td><td>184 (47.9)</td></tr><tr><td>Pneumology</td><td>15 (6.5)</td><td>66 (17.2)</td></tr><tr><td>Ear nose and throat</td><td>12 (5.2)</td><td>12 (3.1)</td></tr><tr><td>Cardiology</td><td>9 (3.9)</td><td>15 (3.9)</td></tr><tr><td>Dermatology</td><td>6 (2.6)</td><td>11 (2.9)</td></tr><tr><td>Pediatrics</td><td>11 (4.8)</td><td>21 (5.5)</td></tr><tr><td>Infectious diseases</td><td>8 (3.5)</td><td>4 (1.1)</td></tr><tr><td>Others</td><td>60 (26.1)</td><td>71 (18.5)</td></tr><tr><td><strong>Intervention in experimental group, n (%)</strong></td><td></td><td></td></tr><tr><td>Pharmacological</td><td>79 (34.3)</td><td>171 (44.5)</td></tr><tr><td>Non-pharmacological</td><td>151 (65.7)</td><td>213 (55.5)</td></tr><tr><td><strong>Intervention in control group, n (%)</strong></td><td></td><td></td></tr><tr><td>Active comparator</td><td>35 (15.2)</td><td>78 (20.3)</td></tr><tr><td>Placebo</td><td>29 (12.6)</td><td>39 (10.2)</td></tr><tr><td>No added intervention (i.e., usual care)</td><td>115 (50.0)</td><td>188 (49.0)</td></tr><tr><td>Sham comparator</td><td>2 (0.9)</td><td>2 (0.5)</td></tr><tr><td>Other</td><td>49 (21.3)</td><td>77 (20.0)</td></tr><tr><td><strong>Outcome objectivity, n (%)</strong></td><td></td><td></td></tr><tr><td>All-cause mortality</td><td>3 (1.3)</td><td>32 (8.3)</td></tr><tr><td>Objectively assessed</td><td>55 (23.9)</td><td>48 (12.5)</td></tr><tr><td>Objectively assessed but influenced by clinician or patient</td><td>95 (41.3)</td><td>170 (44.3)</td></tr><tr><td>Subjectively assessed</td><td>75 (32.6)</td><td>132 (34.4)</td></tr><tr><td>Unclear</td><td>2 (0.9)</td><td>2 (0.5)</td></tr><tr><td><strong>Cochrane Risk of Bias tool, n (%)</strong></td><td></td><td></td></tr><tr><td>Random sequence generation</td><td></td><td></td></tr><tr><td>Low</td><td>120 (57.4)</td><td>197 (57.4)</td></tr><tr><td>High</td><td>11 (5.3)</td><td>11 (3.2)</td></tr><tr><td>Unclear</td><td>78 (37.3)</td><td>135 (39.4)</td></tr><tr><td>Allocation concealment</td><td></td><td></td></tr><tr><td>Low</td><td>116 (50.4)</td><td>165 (43.0)</td></tr><tr><td>High</td><td>21 (9.1)</td><td>26 (6.8)</td></tr></tbody></table>

<table><caption>**Table 2** (continued)</caption><thead><tr><th>Trial characteristics</th><th>PC-RCTs</th><th>ST-RCTs</th></tr></thead><tbody><tr><td>Unclear</td><td>93 (40.4)</td><td>193 (50.3)</td></tr><tr><td>Blinding of participants and personnel</td><td></td><td></td></tr><tr><td>Low</td><td>68 (45.6)</td><td>74 (30.1)</td></tr><tr><td>High</td><td>33 (22.2)</td><td>62 (25.2)</td></tr><tr><td>Unclear</td><td>48 (32.2)</td><td>110 (44.7)</td></tr><tr><td>Blinding of outcome assessors</td><td></td><td></td></tr><tr><td>Low</td><td>61 (62.9)</td><td>83 (43.4)</td></tr><tr><td>High</td><td>14 (14.4)</td><td>50 (26.2)</td></tr><tr><td>Unclear</td><td>22 (22.7)</td><td>58 (30.4)</td></tr><tr><td>Incomplete data outcome</td><td></td><td></td></tr><tr><td>Low</td><td>135 (64.3)</td><td>215 (62.7)</td></tr><tr><td>High</td><td>34 (16.2)</td><td>53 (15.4)</td></tr><tr><td>Unclear</td><td>41 (19.5)</td><td>75 (21.9)</td></tr><tr><td>Selective reporting</td><td></td><td></td></tr><tr><td>Low</td><td>85 (63.9)</td><td>144 (58.5)</td></tr><tr><td>High</td><td>9 (6.8)</td><td>23 (9.4)</td></tr><tr><td>Unclear</td><td>39 (29.3)</td><td>79 (32.1)</td></tr></tbody></table>

heterogeneity across MAs was moderate ($I^2$ = 45, 95%CI 29 to 58%, between–meta-analysis variance $\tau^2$ = 0.07).

PC-RCTs: primary care setting randomised controlled trial.

ST-RCTs: secondary or tertiary care setting randomised controlled trial.

### Subgroup analyses

For the 33 MAs assessing pharmacological interventions, the combined ROR was estimated at 0.90 (95% CI 0.76 to 1.07), and for the 43 MAs assessing non-pharmacological interventions, the combined ROR was 1.01 (95% CI 0.90 to 1.14). The interaction test was not statistically significant ($P$ = 0.97). We did not find a significant interaction in the other subgroup analyses (Fig. 3).

ROR: ratio of odds ratio; CI: confidence interval; MA: meta-analysis; PC-RCTs: primary care setting randomised controlled trials; ST-RCTs: secondary or tertiary care setting randomised controlled trials.

### Sensitivity analyses

Results were consistent when adjusting on each item of the risk of bias (Fig. 4).

ROR: ratio of odds ratio; CI: confidence interval; MA: meta-analysis; PC-RCTs: primary care setting randomised controlled trials; ST-RCTs: secondary or tertiary care setting randomised controlled trials.
<figure><img src="image_5.png" /><figcaption>**Fig. 2** Difference in the intervention effect estimates between randomised controlled trials in primary care versus secondary or tertiary settings</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Fig. 2** Difference in the intervention effect estimates between randomised controlled trials in primary care versus secondary or tertiary settings</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Fig. 3** Difference in intervention effect estimates for the subgroup analyses</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Fig. 4** Difference in intervention effect estimates when adjusting on risk of bias</figcaption></figure>

## Discussion

In this meta-epidemiological study of 76 MAs, we did not observe a significant difference in intervention effect estimates between randomised controlled trials conducted in primary care versus secondary or tertiary care settings. Consistent results were found in subgroup and sensitivity analyses. Our result and the way we worked to obtain it satisfy the 10 criteria as treated by Moustgaard et al. [17].

To our knowledge, our meta-epidemiological study is the first to compare intervention effect estimates between PC-RCTs and ST-RCTs. The number of MAs analysed is the major strength of this study. In 2016, Dechartres et al. found that the median number of MAs per meta-epidemiological study was 29 versus 76 MAs in our study [18]. Nevertheless, we restricted our study to Cochrane reviews, which could limit the generalizability of our results. Cochrane reviews were shown to have better methodological quality [19] and to be better reported [20] than non-Cochrane reviews. We performed an analysis of MAs of binary outcomes, so our results cannot be extrapolated to MAs with continuous outcomes. MAs with continuous outcomes have more subjective outcomes, are less frequently blinded

[21] and have higher heterogeneity [22] than those with binary outcomes. Finally, another limitation may be the definition of primary care settings. Primary care is often difficult and poorly defined in the literature and can include a large range of professions and professionals [14]. In Cochrane reviews, the description of the type of healthcare professionals in charge of including, following up and assessing patients was seldom reported. This omission could have led us to misclassify some trials or also to exclude some trials because of uncertainty. Nevertheless, the selection and the extraction process were performed in duplicate by health professionals (i.e., a pharmacy resident, a general medicine resident and a professor of general medicine in case of discrepancy), which helped limit the uncertainty of some terms and reduced the risk of errors.

Our result does not support GPs' general feeling that results from PC-RCTs differ from those derived from ST-RCTs [3, 5]. Our study is sufficiently powered, and the ROR point estimate was very close to 1, which indeed supports the conclusion of an absence of differences in intervention effect estimates between PC-RCTs and ST-RCTs. However, among the 76 MAs, 29 (38.2%) were related to psychiatry (mainly depression) or addictology
<figure><img src="image_6.png" /><figcaption>**Fig. 3** Difference in intervention effect estimates for the subgroup analyses</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Fig. 4** Difference in intervention effect estimates when adjusting on risk of bias</figcaption></figure>

## Discussion

In this meta-epidemiological study of 76 MAs, we did not observe a significant difference in intervention effect estimates between randomised controlled trials conducted in primary care versus secondary or tertiary care settings. Consistent results were found in subgroup and sensitivity analyses. Our result and the way we worked to obtain it satisfy the 10 criteria as treated by Moustgaard et al. [17].

To our knowledge, our meta-epidemiological study is the first to compare intervention effect estimates between PC-RCTs and ST-RCTs. The number of MAs analysed is the major strength of this study. In 2016, Dechartres et al. found that the median number of MAs per meta-epidemiological study was 29 versus 76 MAs in our study [18]. Nevertheless, we restricted our study to Cochrane reviews, which could limit the generalizability of our results. Cochrane reviews were shown to have better methodological quality [19] and to be better reported [20] than non-Cochrane reviews. We performed an analysis of MAs of binary outcomes, so our results cannot be extrapolated to MAs with continuous outcomes. MAs with continuous outcomes have more subjective outcomes, are less frequently blinded

[21] and have higher heterogeneity [22] than those with binary outcomes. Finally, another limitation may be the definition of primary care settings. Primary care is often difficult and poorly defined in the literature and can include a large range of professions and professionals [14]. In Cochrane reviews, the description of the type of healthcare professionals in charge of including, following up and assessing patients was seldom reported. This omission could have led us to misclassify some trials or also to exclude some trials because of uncertainty. Nevertheless, the selection and the extraction process were performed in duplicate by health professionals (i.e., a pharmacy resident, a general medicine resident and a professor of general medicine in case of discrepancy), which helped limit the uncertainty of some terms and reduced the risk of errors.

Our result does not support GPs' general feeling that results from PC-RCTs differ from those derived from ST-RCTs [3, 5]. Our study is sufficiently powered, and the ROR point estimate was very close to 1, which indeed supports the conclusion of an absence of differences in intervention effect estimates between PC-RCTs and ST-RCTs. However, among the 76 MAs, 29 (38.2%) were related to psychiatry (mainly depression) or addictology
(mainly tobacco use) and 10 (13.2%) to pneumology. Very few MAs were dedicated to cardiovascular conditions (n=4, 5.3%) or diabetes (n=1, 1.3%). So the medical fields represented in our study do not well match those mostly frequently encountered in primary care, typically cardiovascular diseases (13.6%), muskuloskeletal troubles (12.6%) and pneumology (12.5%), with psychiatric diseases representing only 8.5% of the consultations [23]. Therefore, how we selected the MAs for our study led to a sample of diseases that does not perfectly reflect the diseases mainly managed in primary care. Other studies should probably focus on specific medical fields or diseases rather than considering all MAs, whatever the medical field.

Another point is that we focused on binary outcomes, but in cardiovascular diseases or diabetes, most trials use surrogate or intermediate endpoints, which are mainly continuous ones, such as cholesterol level, blood pressure or HbA1c level [24, 25]. This situation may also have led to discarding some MAs focused on diseases that are highly prevalent in primary care.

Finally, although the present study focused on the settings, intervention effect estimates also depend on the target population, defined by selection criteria and/or the type/intensity of the intervention assessed. Due to the limited size of our study, no adjustment on these characteristics could be done while analyzing data. Therefore, one of the remaining questions, which is beyond the scope of the present work, is whether patients included in PC-RCTs are similar to patients included in ST-RCTs.

## Conclusion

We found no significant difference in intervention effect estimates between PC-RCTs and ST-RCTs. Nevertheless, the main medical fields selected are not fully representative of primary care consultations. Further studies focusing on medical fields highly prevalent in primary care are needed.

## Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12874-022-01815-2.

**Additional file 1.** List of included meta-analyses (n=76).

**Additional file 2.** Description of the 76 meta-analyses.

**Additional file 3.** Characteristics of the primary care setting randomised controlled trials included.

## Acknowledgements

The authors thank Laura Smales (BioMedEditing) for English review of the manuscript.

## Authors' contributions

ADu, CDD, BG, AC and ADe conceived the study; ADu, AH and CDD participated in data collection; ADu, AH and CDD extracted the data; ET analysed

data; ADu wrote the first draft of the manuscript, which was revised by CDD, BG, AC, ET, Ade, and AH. All coauthors read and approved the final version of the manuscript.

## Funding

The authors declare no specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

## Availability of data and materials

The datasets used and/or analysed during the current study are available in the Additional file.

## Declarations

### Ethics approval and consent to participate

This study involved analysis of data from published systematic reviews with meta-analyses; we did not collect participant data.

### Consent for publication

Not applicable.

### Competing interests

The authors declare no competing interests in relation with this study.

### Author details

¹University of Tours, University of Nantes, INSERM 1246-SPHERE, Tours, France.

²INSERM CIC1415, CHRU de Tours, Tours, France. ³Sorbonne Université, INSERM, Institut Pierre Louis d'Epidémiologie et de Santé Publique AP-HP. Sorbonne Université, Hôpital Pitié Salpêtrière, Département de Santé Publique, F75013 Paris, France. ⁴Department of General Practice, University of Tours, Tours, France.

Received: 15 March 2022 Accepted: 5 December 2022

Published online: 22 December 2022

## References

1. Ekesbo R, Midlöv P, Gerward S, et al. Lack of adherence to hypertension treatment guidelines among GPs in southern Sweden-a case report-based survey. BMC Fam Pract. 2012;13:34. https://doi.org/10.1186/1471-2296-13-34

2. Grol R, Dalhuijsen J, Thomas S, et al. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998;317:858–61. https://doi.org/10.1136/bmj.317.7162.858

3. Carlsen B, Norheim OF. « what lies beneath it all? » – an interview study of GPs' attitudes to the use of guidelines. BMC Health Serv Res. 2008;8:218. https://doi.org/10.1186/1472-6963-8-218

4. Lugtenberg M, Zegers-van Schaick JM, Westert GP, Burgers JS. Why don't physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009;4:54. https://doi.org/10.1186/1748-5908-4-54

5. Cranney M, Warren E, Barton S, et al. Why do GPs not implement evidence-based guidelines? A descriptive study Fam Pract. 2001;18(4):359–63. https://doi.org/10.1093/fampra/18.4.359

6. Steel N, Abdelhamid A, Stokes T, et al. A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients. J Clin Epidemiol. 2014;67:1251–7. https://doi.org/10.1016/j.jclinepi.2014.05.020

7. Rothwell PM. External validity of randomised controlled trials: "To whom do the results of this trial apply?". The Lancet. 2005;365(9453):82–93. https://doi.org/10.1016/S0140-6736(04)17670-8

8. Uijen AA, Bakx JC, Mokkink HGA, et al. Hypertension patients participating in trials differ in many aspects from patients treated in general practices. J Clin Epidemiol. 2007;6. https://doi.org/10.1016/j.jclinepi.2006.05.015

9. Fahey T. Applying the results of clinical trials to patients to general practice: perceived problems, strengths, assumptions, and challenges for the future. Br J Gen Pract J R Coll Gen Pract. 1998;48:1173–8.

10. Sonis J. Applicability of clinical trial results to primary care. JAMA. 1998;280(20):1746–a. https://doi.org/10.1001/jama.280.20.1746-a
(mainly tobacco use) and 10 (13.2%) to pneumology. Very few MAs were dedicated to cardiovascular conditions (n=4, 5.3%) or diabetes (n=1, 1.3%). So the medical fields represented in our study do not well match those mostly frequently encountered in primary care, typically cardiovascular diseases (13.6%), muskuloskeletal troubles (12.6%) and pneumology (12.5%), with psychiatric diseases representing only 8.5% of the consultations [23]. Therefore, how we selected the MAs for our study led to a sample of diseases that does not perfectly reflect the diseases mainly managed in primary care. Other studies should probably focus on specific medical fields or diseases rather than considering all MAs, whatever the medical field.

Another point is that we focused on binary outcomes, but in cardiovascular diseases or diabetes, most trials use surrogate or intermediate endpoints, which are mainly continuous ones, such as cholesterol level, blood pressure or HbA1c level [24, 25]. This situation may also have led to discarding some MAs focused on diseases that are highly prevalent in primary care.

Finally, although the present study focused on the settings, intervention effect estimates also depend on the target population, defined by selection criteria and/or the type/intensity of the intervention assessed. Due to the limited size of our study, no adjustment on these characteristics could be done while analyzing data. Therefore, one of the remaining questions, which is beyond the scope of the present work, is whether patients included in PC-RCTs are similar to patients included in ST-RCTs.

## Conclusion

We found no significant difference in intervention effect estimates between PC-RCTs and ST-RCTs. Nevertheless, the main medical fields selected are not fully representative of primary care consultations. Further studies focusing on medical fields highly prevalent in primary care are needed.

## Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12874-022-01815-2.

**Additional file 1.** List of included meta-analyses (n=76).

**Additional file 2.** Description of the 76 meta-analyses.

**Additional file 3.** Characteristics of the primary care setting randomised controlled trials included.

## Acknowledgements

The authors thank Laura Smales (BioMedEditing) for English review of the manuscript.

## Authors' contributions

ADu, CDD, BG, AC and ADe conceived the study; ADu, AH and CDD participated in data collection; ADu, AH and CDD extracted the data; ET analysed

data; ADu wrote the first draft of the manuscript, which was revised by CDD, BG, AC, ET, Ade, and AH. All coauthors read and approved the final version of the manuscript.

## Funding

The authors declare no specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

## Availability of data and materials

The datasets used and/or analysed during the current study are available in the Additional file.

## Declarations

### Ethics approval and consent to participate

This study involved analysis of data from published systematic reviews with meta-analyses; we did not collect participant data.

### Consent for publication

Not applicable.

### Competing interests

The authors declare no competing interests in relation with this study.

### Author details

¹University of Tours, University of Nantes, INSERM 1246-SPHERE, Tours, France.

²INSERM CIC1415, CHRU de Tours, Tours, France. ³Sorbonne Université, INSERM, Institut Pierre Louis d'Epidémiologie et de Santé Publique AP-HP. Sorbonne Université, Hôpital Pitié Salpêtrière, Département de Santé Publique, F75013 Paris, France. ⁴Department of General Practice, University of Tours, Tours, France.

Received: 15 March 2022 Accepted: 5 December 2022

Published online: 22 December 2022

## References

1. Ekesbo R, Midlöv P, Gerward S, et al. Lack of adherence to hypertension treatment guidelines among GPs in southern Sweden-a case report-based survey. BMC Fam Pract. 2012;13:34. https://doi.org/10.1186/1471-2296-13-34

2. Grol R, Dalhuijsen J, Thomas S, et al. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998;317:858–61. https://doi.org/10.1136/bmj.317.7162.858

3. Carlsen B, Norheim OF. « what lies beneath it all? » – an interview study of GPs' attitudes to the use of guidelines. BMC Health Serv Res. 2008;8:218. https://doi.org/10.1186/1472-6963-8-218

4. Lugtenberg M, Zegers-van Schaick JM, Westert GP, Burgers JS. Why don't physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009;4:54. https://doi.org/10.1186/1748-5908-4-54

5. Cranney M, Warren E, Barton S, et al. Why do GPs not implement evidence-based guidelines? A descriptive study Fam Pract. 2001;18(4):359–63. https://doi.org/10.1093/fampra/18.4.359

6. Steel N, Abdelhamid A, Stokes T, et al. A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients. J Clin Epidemiol. 2014;67:1251–7. https://doi.org/10.1016/j.jclinepi.2014.05.020

7. Rothwell PM. External validity of randomised controlled trials: "To whom do the results of this trial apply?". The Lancet. 2005;365(9453):82–93. https://doi.org/10.1016/S0140-6736(04)17670-8

8. Uijen AA, Bakx JC, Mokkink HGA, et al. Hypertension patients participating in trials differ in many aspects from patients treated in general practices. J Clin Epidemiol. 2007;6. https://doi.org/10.1016/j.jclinepi.2006.05.015

9. Fahey T. Applying the results of clinical trials to patients to general practice: perceived problems, strengths, assumptions, and challenges for the future. Br J Gen Pract J R Coll Gen Pract. 1998;48:1173–8.

10. Sonis J. Applicability of clinical trial results to primary care. JAMA. 1998;280(20):1746–a. https://doi.org/10.1001/jama.280.20.1746-a.
11. Barnett K, Mercer SW, Norbury M, et al. Epidemiology of multimorbid-
ity and implications for health care, research, and medical education: a
cross-sectional study. Lancet. 2012;380:37–43. https://doi.org/10.1016/
S0140-6736(12)60240-2.

12. Sterne JAC, Jüni P, Schulz KF, et al. Statistical methods for assessing the influence of study characteristics on treatment effects in « meta-epidemiological » research. Stat Med. 2002;21:1513-24.

13. Zhang W. Meta-epidemiology: building the bridge from research evidence to clinical practice. Osteoarthr Cartil. 2010;18(Suppl 2):S1.

14. Afonso MM, Bénard A, Chapron A, et al. Les soins primaires : une définition du champ pour développer la recherche. Epidemiol Public Health Rev Epidémiologie. Revue d'Épidémiologie et de Santé Publique. 2018;66:157-62 doi : https://doi.org/10.1016/j.respe.2017.09.004.

15. Savović J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157:429-38. https://doi.org/10.1002/sim.1184.

16. Cochrane Bias Methods Group Cochrane Statistical Methods Group, Hig-
gins Julian PT, Altman Douglas G, Gøtzsche Peter C, Peter J, David M, et al.
The Cochrane Collaboration's tool for assessing risk of bias in randomised
trials. BMJ. 2011;343:889–93. https://doi.org/10.1136/bmj.d5928.

17. Moustgaard H, Jones HE, Savović J, et al. Ten questions to consider when interpreting results of a meta-epidemiological study—the MetaBLIND study as a case. Res Synth Methods. 2020;11:260–74. https://doi.org/10.1002/jrsm.1392.

18. Dechartres A, Trinquart L, Faber T, et al. Empirical evaluation of which trial characteristics are associated with treatment effect estimates. J Clin Epidemiol. 2016;77:24–37. https://doi.org/10.1016/j.jclinepi.2016.04.005.

19. Moja LP, Telaro E, D'Amico R, et al. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ. 2005;330:1053. https://doi.org/10.1136/bmj.38414. 515938.8f.

20. Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting char-
acteristics of systematic reviews of biomedical research: a cross-sectional
study. PLoS Med. 2016;13:e1002028. https://doi.org/10.1371/journal.
pmed.1002028.

21. Leyrat C, Caille A, Eldridge S, Kerry S, et al. Intervention effect estimates in cluster randomized versus individually randomized trials: a meta-epide-miological study. Int J Epidemiol. 2019;609–19.

22. Alba AC, Alexander PE, Chang J, et al. High statistical heterogeneity is more frequent in meta-analysis of continuous than binary outcomes. J Clin Epidemiol. 2016;70:129–35.

23. Letrillart L, Supper I, Schuers M, et al. ECOGEN : étude des Éléments de la CONSultation en médecine GENérale. Exercer. 2014;114:148-57.

24. Gandhi GY, Murad MH, Fujiyoshi A, et al. Patient-important outcomes in registered diabetes trials. JAMA. 2008;299:2543-9. https://doi.org/10.1001/jama.299.21.2543.

25. Rahimi K, Malhotra A, Banning AP, et al. Outcome selection and role of patient reported outcomes in contemporary cardiovascular trials: systematic review. BMJ. 2010;341:c5707. https://doi.org/10.1136/bmj.c5707.

**Publisher’s Note**

Springer Nature remains neutral with regard to jurisdictional claims in pub-
lished maps and institutional affiliations.

Ready to submit your research? Choose BMC and benefit from:

* fast, convenient online submission

* thorough peer review by experienced researchers in your field

* rapid publication on acceptance

* support for research data, including large and complex data types

* gold Open Access which fosters wider collaboration and increased citations

* maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Learn more biomedcentral.com/submissions

<figure><img src="image_8.png" /></figure>
11. Barnett K, Mercer SW, Norbury M, et al. Epidemiology of multimorbid-
ity and implications for health care, research, and medical education: a
cross-sectional study. Lancet. 2012;380:37–43. https://doi.org/10.1016/
S0140-6736(12)60240-2.

12. Sterne JAC, Jüni P, Schulz KF, et al. Statistical methods for assessing the influence of study characteristics on treatment effects in « meta-epidemiological » research. Stat Med. 2002;21:1513-24.

13. Zhang W. Meta-epidemiology: building the bridge from research evidence to clinical practice. Osteoarthr Cartil. 2010;18(Suppl 2):S1.

14. Afonso MM, Bénard A, Chapron A, et al. Les soins primaires : une définition du champ pour développer la recherche. Epidemiol Public Health Rev Epidémiologie. Revue d'Épidémiologie et de Santé Publique. 2018;66:157-62 doi : https://doi.org/10.1016/j.respe.2017.09.004.

15. Savović J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157:429-38. https://doi.org/10.1002/sim.1184.

16. Cochrane Bias Methods Group Cochrane Statistical Methods Group, Hig-
gins Julian PT, Altman Douglas G, Gøtzsche Peter C, Peter J, David M, et al.
The Cochrane Collaboration's tool for assessing risk of bias in randomised
trials. BMJ. 2011;343:889-93. https://doi.org/10.1136/bmj.d5928.

17. Moustgaard H, Jones HE, Savović J, et al. Ten questions to consider when interpreting results of a meta-epidemiological study—the MetaBLIND study as a case. Res Synth Methods. 2020;11:260–74. https://doi.org/10.1002/jrsm.1392.

18. Dechartres A, Trinquart L, Faber T, et al. Empirical evaluation of which trial characteristics are associated with treatment effect estimates. J Clin Epidemiol. 2016;77:24–37. https://doi.org/10.1016/j.jclinepi.2016.04.005.

19. Moja LP, Telaro E, D'Amico R, et al. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ. 2005;330:1053. https://doi.org/10.1136/bmj.38414. 515938.8f.

20. Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting char-
acteristics of systematic reviews of biomedical research: a cross-sectional
study. PLoS Med. 2016;13:e1002028. https://doi.org/10.1371/journal.
pmed.1002028.

21. Leyrat C, Caille A, Eldridge S, Kerry S, et al. Intervention effect estimates in cluster randomized versus individually randomized trials: a meta-epide- miological study. Int J Epidemiol. 2019;609–19.

22. Alba AC, Alexander PE, Chang J, et al. High statistical heterogeneity is more frequent in meta-analysis of continuous than binary outcomes. J Clin Epidemiol. 2016;70:129-35.

23. Letrillart L, Supper I, Schuers M, et al. ECOGEN : étude des Éléments de la CONSultation en médecine GENérale. Exercer. 2014;114:148-57.

24. Gandhi GY, Murad MH, Fujiyoshi A, et al. Patient-important outcomes in registered diabetes trials. JAMA. 2008;299:2543-9. https://doi.org/10.1001/jama.299.21.2543.

25. Rahimi K, Malhotra A, Banning AP, et al. Outcome selection and role of patient reported outcomes in contemporary cardiovascular trials: systematic review. BMJ. 2010;341:c5707. https://doi.org/10.1136/bmj.c5707.

**Publisher’s Note**

Springer Nature remains neutral with regard to jurisdictional claims in pub-
lished maps and institutional affiliations.

Ready to submit your research? Choose BMC and benefit from:

* fast, convenient online submission

* thorough peer review by experienced researchers in your field

- rapid publication on acceptance

* support for research data, including large and complex data types

* gold Open Access which fosters wider collaboration and increased citations

* maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Learn more biomedcentral.com/submissions

<figure><img src="image_8.png" /></figure>