# Added Value of Medical Subject Headings Terms in Search Strategies of Systematic Reviews: Comparative Study

Victor Leblanc, Raphael Bentegeac, Bastien Le Guellec, Remi Lenain,
Emmanuel Chazard

► To cite this version:

Victor Leblanc, Raphael Bentegeac, Bastien Le Guellec, Remi Lenain, Emmanuel Chazard. Added Value of Medical Subject Headings Terms in Search Strategies of Systematic Reviews: Comparative Study. Journal of Medical Internet Research, 2024, Journal of Medical Internet Research, 26, pp.e53781. 10.2196/53781. hal-04911069

HAL Id: hal-04911069

https://lilloa.hal.science/hal-04911069v1

Submitted on 24 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Original Paper

# Added Value of Medical Subject Headings Terms in Search Strategies of Systematic Reviews: Comparative Study

Victor Leblanc<fnref n="1" />, MD; Aghiles Hamroun<fnref n="1" />, MD, PhD; Raphaël Bentegeac<fnref n="1" />, MD; Bastien Le Guellec<fnref n="1" />, MD; Rémi Lenain<fnref n="1" />, MD; Emmanuel Chazard<fnref n="1" />,<fnref n="2" />, MD, PhD

**Corresponding Author:**

Victor Leblanc, MD
Public Health Department
CHU Lille
Université de Lille
42 Rue Paul Duez
Lille, 59000
France
Phone: 33 637004971
Email: leblancvictor59@gmail.com

Abstract

**Background:** The massive increase in the number of published scientific articles enhances knowledge but makes it more complicated to summarize results. The Medical Subject Headings (MeSH) thesaurus was created in the mid-20th century with the aim of systematizing article indexing and facilitating their retrieval. Despite the advent of search engines, few studies have questioned the relevance of the MeSH thesaurus, and none have done so systematically.

**Objective:** The objective of this study was to estimate the added value of using MeSH terms in PubMed queries for systematic reviews (SRs).

**Methods:** SRs published in 4 high-impact medical journals in general medicine over the past 10 years were selected. Only SRs for which a PubMed query was provided were included. Each query was transformed to obtain 3 versions: the original query (V1), the query with free-text terms only (V2), and the query with MeSH terms only (V3). These 3 queries were compared with each other based on their sensitivity and positive predictive values.

**Results:** In total, 59 SRs were included. The suppression of MeSH terms had an impact on the number of relevant articles retrieved for 24 (41%) out of 59 SRs. The median (IQR) sensitivities of queries V1 and V2 were 77.8% (62.1%-95.2%) and 71.4% (42.6%-90%), respectively. V1 queries provided an average of 2.62 additional relevant papers per SR compared with V2 queries. However, an additional 820.29 papers had to be screened. The cost of screening an additional collected paper was therefore 313.09, which was slightly more than triple the mean reading cost associated with V2 queries (88.67).

**Conclusions:** Our results revealed that removing MeSH terms from a query decreases sensitivity while slightly increasing the positive predictive value. Queries containing both MeSH and free-text terms yielded more relevant articles but required screening many additional papers. Despite this additional workload, MeSH terms remain indispensable for SRs.

(J Med Internet Res 2024;26:e53781) doi: 10.2196/53781

KEYWORDS

Medical Subject Headings; MeSH; MeSH thesaurus; systematic review; PubMed; search strategy; comparative analysis; literature review; positive predictive value; PPV; review; scientific knowledge; medical knowledge; utility; systematic literature review

Introduction

The number of articles published in scientific and medical journals have been increasing exponentially since the late 20th century. In 2021 alone, over 1,700,000 indexed, full-text articles

were included in the PubMed database. In response to the massive production of scientific knowledge, the need for access to synthetic scientific data has been driven by the emergence of evidence-based medicine [1] and the establishment of national

<footnote n="1">¹Public Health Department, CHU Lille, Université de Lille, Lille, France</footnote>

<footnote n="2">²ULR 2694 Metrics, CERIM, Université de Lille, Lille, France</footnote>
regulatory bodies, medical associations, and learned societies
that provide guidelines on best practice.

In this context, systematic reviews (SRs; a type of analysis
developed in the 1970s) are becoming more important. Given
that quality of an SR depends largely on the research
methodology, building search queries is a crucial part of the
review process. The challenge of constructing a query for a SR
lies in the absolute necessity of being as sensitive as possible,
despite the fact that this query will return at most a few tens of
thousands of articles among the hundreds of millions that make
up the scientific literature [2].

In the mid-20th century, researchers started to develop a
common vocabulary that facilitated article indexing and retrieval
and helped to avoid misunderstandings [3-5]. These efforts led
to the creation of the Medical Subject Headings (MeSH)
thesaurus in the 1960s by the US National Library of Medicine
(NLM) [6]. PubMed (the NLM's search engine), which is one
of the most widely used search engines [7], heavily relies on
MeSH terms to assist users in their literature searches. The
MeSH thesaurus is intended to facilitate literature searches by
limiting term permutations [8,9]. In other words, it assigns a
unique term to a concept—regardless of the language used or
the time period concerned.

Subsequent improvements in search engine performance have
enabled researchers to query databases with simple free-text
terms, rather than MeSH terms. Furthermore, the massive influx
of publications and the emergence of many new scientific and
medical topics have led to delays in MeSH indexing and
difficulties in updating the thesaurus [10]. In addition, although
frequently recommended [11-13], the value of using the MeSH
thesaurus in queries for literature reviews has never been
systematically assessed. The few studies to have tested the utility
of MeSH terms in SRs have limitations, such as a small sample
size or a lack of generalizability [14-20]. Finally, some studies
simply compared the numbers of results retrieved for a given
query but did not evaluate the results' relevance [21].

To the best of our knowledge, only 1 study has extensively
explored the relevance of MeSH terms with regard to the results
of SRs [22]. The study concluded that the use of queries based
on free-text alone (ie, free-text terms) appeared to decrease the
retrieval of articles of interest, relative to queries based on both
free-text terms and MeSH terms. However, this study included
SRs from a single research center, which limited the

generalizability of the findings. Furthermore, the MEDLINE
database was queried with the Ovid search engine, rather than
PubMed. We therefore decided to evaluate this question in more
detail. The objective of this work was to estimate the added
value of using MeSH terms in PubMed queries for SRs.

Methods

Paper Selection

We first selected the top 6 journals in the “Medicine. General & Internal” Journal Citation Reports category, according to the impact factors computed by Clarivate [23,24]. Next, we selected all the PubMed-indexed SRs published in the 6 journals between 2012 and 2021 and for which the free full text was available on PubMed Central. The time period was chosen arbitrarily, with the objective of obtaining at least 60 SRs. The following PubMed query was used: ‘(“The New England Journal of Medicine”[Journal] OR “Lancet London England”[Journal] OR “JAMA”[Journal] OR “Nature Reviews Disease Primers”[Journal] OR “BMJ Clinical Research Ed”[Journal] OR “Annals of Internal Medicine”[Journal]) AND “loattrfree full text”[Filter] AND 2012/01/01:2021/12/31[Date - Publication] AND systematic review[Filter]’.

The exclusion criteria were as follows: (1) articles other than
an SR, (2) the absence of a published search query, (3) the use
of queries in multiple parts that had to be assembled, (4) the
absence of a query specifically built for PubMed, (5) a query
that did not return any results, (6) a query that returned more
than 100,000 results, and (7) a query with only MeSH terms or
without MeSH terms. The sorting was carried out by a single
researcher (VL).

Analysis of the PubMed Results

The query was extracted from each included SR and inserted
into the PubMed search bar. PubMed has a feature called
automatic term mapping (ATM) [25]; when terms not enclosed
in quotation marks are inserted in the search bar, they are
automatically transformed into a query segment that contains
several descriptors, such as [MeSH terms], [tiab], and [all fields].
To ensure greater reproducibility, we checked for the automatic
transformation of queries. This step was important because
PubMed's ATM feature might add MeSH terms to query initially
considered to be free of such terms. Hence, we always retrieved
the query formatted by PubMed's ATM (henceforth referred to
as V1; Figure 1).
<figure><img src="image_2.png" /><figcaption>**Figure 1.** Example of an automated transformation of queries. MeSH: Medical Subject Headings.</figcaption></figure>

For each SR, V1 was transformed into a V2 query by replacing each MeSH term in the query with a free-text term that had to be present in the title or in the abstract. To do this, we simply replaced the [MeSH] tag with a [Title/Abstract] tag. Hence, the resulting V2 did not contain any explicit [MeSH] tags (Figure 1). Lastly, the V3 (MeSH-only) query was obtained by transforming all free-text terms into MeSH terms. It should be noted that terms stated as MeSH terms in the query but that do not actually exist in the MeSH thesaurus are ignored by the PubMed engine; this is equivalent to deleting the terms (Figure 1 [26]).

The transformations from V1 to V2 and V3 were the same for all queries, regardless of whether they contained MeSH terms only or free-text terms only. However, we noted that some PubMed filters are based on MeSH terms [27]. It would therefore not be relevant to convert these terms into free-text terms. We drew up a list of these terms so that they were not transformed and were still able to serve as filters. Those 14 terms are “80 and over,” “adolescent,” “adult,” “aged,” “animals,” “child,” “female,” “humans,” “infant,” “male,” “middle aged,” “newborn,” “preschool,” and “young adult.”

Hence, each SR had a query written by the SR's authors (a combination of MeSH and free-text terms; V1), a free-text-only query (V2), and a MeSH-only query (V3). Therefore, we intend to interpret the comparison of V2 with V1 as the added value of MeSH terms, and we intend to interpret the comparison of V3 with V1 as the added value of free-text terms.

Each query was submitted to the PubMed search engine, and the results were retrieved and sorted by the “Best Match” option. If there were more than 10,000 results, only the first 10,000 results were retained; in fact, PubMed does not allow more than

10,000 results to be extracted. The results were identified by their PubMed Identifier (PMID).

For each SR, the “gold standard” (GS) consisted of the articles selected by the authors of the SR. Each SR was read in order to extract the list of PMIDs selected by the authors. This work was done “by hand” by 4 researchers (VL, RB, BLG, and AH). Publications cited in the SR but not indexed in MEDLINE were not analyzed. If the reference section did not contain the items selected in the SR, data extraction from supplementary files allowed for the completion of the GS.

## Data Analysis

For each SR, we obtained 4 lists of PMIDs: the GS, those retrieved by V1 (MeSH and free-text terms), those retrieved by V2 (free-text terms only), and those retrieved by V3 (MeSH terms only). For each list, we computed the sensitivity (also referred to as “recall”) and the positive predictive value (PPV; also referred to as “precision”) with respect to the GS. We then computed the $F_1$-score, which is the harmonic mean of the sensitivity and the PPV.

For each query $i$ (V1, V2, and V3), the odds for the PPV was defined as the ratio between 2 numbers:

$$ \text{odds}(\text{PPV}, \text{query}_i) = \frac{\#(\text{query}_i \cap \text{GS})}{\#(\text{query}_i \cap \overline{\text{GS}})} $$

Next, for a given SR and using the same GS, the odds ratio (OR) of query$_2$ to query$_1$ for the PPV was defined as:

$$ \text{OR}(\text{PPV}, \text{query}_2, \text{query}_1) = \frac{\text{odds}(\text{PPV}, \text{query}_2)}{\text{odds}(\text{PPV}, \text{query}_1)} $$
Likewise, the odds for the sensitivity of each query i (V1, V2, and V3) was defined as the ratio between 2 numbers:

$$odds(Se, query_i) = \frac{\#(GS \cap query_i)}{\#(GS \cap query_i)}$$

Hence, for a given SR and using the same GS, the OR for query2 versus query1 with regard to sensitivity was:

$$OR(Se, query_2, query_1) = \frac{odds(Se, query_2)}{odds(Se, query_1)}$$

We computed the respective ORs for V2 versus V1 and V3 versus V1 for the PPV and the sensitivity:

$$\begin{align*} OR(Se, query_2, query_1) \\ OR(Se, query_3, query_1) \\ OR(VPP, query_2, query_1) \\ OR(VPP, query_3, query_1) \end{align*}$$

An OR of 1 means that the queries have the same level of performance with regard to the chosen indicator. An OR<1 denotes worse performance, and an OR>1 denotes better performance.

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Flowchart for the selection of systematic reviews. MeSH: Medical Subject Heading; SR: systematic review.</figcaption></figure>

## Description of the Included Systematic Reviews

A total of 59 SRs were selected for analysis, which contained both MeSH terms and free-text terms (Table 1 and Multimedia Appendices 1 and 2 [26,31-88]).

## Statistical Analysis

Qualitative variables, binary variables, or discrete variables with very few modalities were expressed as the frequency (percentage). Quantitative variables were expressed as the mean (SD) when symmetrically distributed and the median (IQR) when not. The independence of 2 qualitative variables was probed in a chi-square test.

All statistical tests were 2-sided. The threshold for statistical significance was set to P<.05. The 95% CI of a proportion was calculated using the Wald method. Statistical analyses were performed with R software (R Core Team), RStudio software (Posit PBC), and the R *metafor* package [28-30].

## Ethical Considerations

The research was performed using publicly available documents. It did not involve individuals or personal data. Approval by an institutional review board was not required.

## Results

### Flowchart

The SRs used to compile the set of queries were selected by a single researcher (VL; Figure 2).
<table><caption>**Table 1.** General description of the attributes for each included systematic review.</caption><thead><tr><th rowspan="2">SR's<sup>a</sup> PMID<sup>b</sup></th><th colspan="3">Items, n</th><th colspan="3">Number of GS found</th><th colspan="3">V1<sup>c</sup></th><th colspan="3">V2<sup>d</sup></th><th colspan="3">V3<sup>e</sup></th></tr><tr><th>V1</th><th>V2</th><th>V3</th><th>GS<sup>f</sup></th><th>V1</th><th>V2</th><th>V3</th><th>Se<sup>g</sup></th><th>PPV<sup>h</sup></th><th>F-Sc<sup>i</sup></th><th>Se</th><th>PPV</th><th>F-Sc</th><th>Se</th><th>PPV</th><th>F-Sc</th></tr></thead><tbody><tr><td>33472813 [46]</td><td>297</td><td>294</td><td>147</td><td>30</td><td>27</td><td>27</td><td>24</td><td>0.900</td><td>0.091</td><td>0.165</td><td>0.900</td><td>0.092</td><td>0.167</td><td>0.800</td><td>0.163</td><td>0.271</td></tr><tr><td>33441384 [50]</td><td>4408</td><td>2265</td><td>901</td><td>16</td><td>14</td><td>13</td><td>11</td><td>0.875</td><td>0.003</td><td>0.006</td><td>0.812</td><td>0.006</td><td>0.011</td><td>0.688</td><td>0.012</td><td>0.024</td></tr><tr><td>33186535 [49]</td><td>145</td><td>153</td><td>1</td><td>6</td><td>4</td><td>4</td><td>1</td><td>0.667</td><td>0.028</td><td>0.053</td><td>0.667</td><td>0.026</td><td>0.050</td><td>0.167</td><td>1.000</td><td>0.286</td></tr><tr><td>33148618 [40]</td><td>9024</td><td>10000</td><td>3706</td><td>66</td><td>16</td><td>18</td><td>3</td><td>0.242</td><td>0.002</td><td>0.004</td><td>0.273</td><td>0.002</td><td>0.004</td><td>0.045</td><td>0.001</td><td>0.002</td></tr><tr><td>32909814 [59]</td><td>349</td><td>349</td><td>40</td><td>9</td><td>1</td><td>1</td><td>0</td><td>0.111</td><td>0.003</td><td>0.006</td><td>0.111</td><td>0.003</td><td>0.006</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>32496521 [47]</td><td>94</td><td>162</td><td>0</td><td>24</td><td>0</td><td>0</td><td>0</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>32459529 [54]</td><td>1950</td><td>812</td><td>1441</td><td>9</td><td>7</td><td>2</td><td>7</td><td>0.778</td><td>0.004</td><td>0.007</td><td>0.222</td><td>0.002</td><td>0.005</td><td>0.778</td><td>0.005</td><td>0.010</td></tr><tr><td>32442035 [73]</td><td>10000</td><td>10000</td><td>842</td><td>15</td><td>8</td><td>7</td><td>2</td><td>0.533</td><td>0.001</td><td>0.002</td><td>0.467</td><td>0.001</td><td>0.001</td><td>0.133</td><td>0.002</td><td>0.005</td></tr><tr><td>32371466 [64]</td><td>4185</td><td>3320</td><td>324</td><td>50</td><td>49</td><td>48</td><td>21</td><td>0.980</td><td>0.012</td><td>0.023</td><td>0.960</td><td>0.014</td><td>0.028</td><td>0.420</td><td>0.065</td><td>0.112</td></tr><tr><td>32199484 [70]</td><td>6347</td><td>6111</td><td>1231</td><td>128</td><td>0</td><td>0</td><td>0</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>31255301 [43]</td><td>1164</td><td>1023</td><td>456</td><td>61</td><td>41</td><td>35</td><td>34</td><td>0.672</td><td>0.035</td><td>0.067</td><td>0.574</td><td>0.034</td><td>0.065</td><td>0.557</td><td>0.075</td><td>0.132</td></tr><tr><td>30884526 [33]</td><td>770</td><td>716</td><td>30</td><td>79</td><td>0</td><td>0</td><td>0</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>30617123 [75]</td><td>1086</td><td>788</td><td>292</td><td>29</td><td>28</td><td>26</td><td>14</td><td>0.966</td><td>0.026</td><td>0.050</td><td>0.897</td><td>0.033</td><td>0.064</td><td>0.483</td><td>0.048</td><td>0.087</td></tr><tr><td>30326495 [77]</td><td>4360</td><td>2212</td><td>951</td><td>158</td><td>123</td><td>117</td><td>62</td><td>0.778</td><td>0.028</td><td>0.054</td><td>0.741</td><td>0.053</td><td>0.099</td><td>0.392</td><td>0.065</td><td>0.112</td></tr><tr><td>30158148 [41]</td><td>10000</td><td>2673</td><td>395</td><td>45</td><td>33</td><td>24</td><td>28</td><td>0.733</td><td>0.003</td><td>0.007</td><td>0.533</td><td>0.009</td><td>0.018</td><td>0.622</td><td>0.071</td><td>0.127</td></tr><tr><td>29049756 [80]</td><td>3155</td><td>2858</td><td>913</td><td>20</td><td>16</td><td>16</td><td>2</td><td>0.800</td><td>0.005</td><td>0.010</td><td>0.800</td><td>0.006</td><td>0.011</td><td>0.100</td><td>0.002</td><td>0.004</td></tr><tr><td>28903922 [35]</td><td>687</td><td>610</td><td>95</td><td>24</td><td>23</td><td>23</td><td>16</td><td>0.958</td><td>0.033</td><td>0.065</td><td>0.958</td><td>0.038</td><td>0.073</td><td>0.667</td><td>0.168</td><td>0.269</td></tr><tr><td>27893131 [60]</td><td>10000</td><td>10000</td><td>2439</td><td>48</td><td>34</td><td>33</td><td>27</td><td>0.708</td><td>0.003</td><td>0.007</td><td>0.688</td><td>0.003</td><td>0.007</td><td>0.562</td><td>0.011</td><td>0.022</td></tr><tr><td>27802505 [74]</td><td>2299</td><td>2227</td><td>195</td><td>21</td><td>19</td><td>19</td><td>14</td><td>0.905</td><td>0.008</td><td>0.016</td><td>0.905</td><td>0.009</td><td>0.017</td><td>0.667</td><td>0.072</td><td>0.130</td></tr><tr><td>27802478 [26]</td><td>3847</td><td>3525</td><td>2526</td><td>89</td><td>45</td><td>43</td><td>43</td><td>0.506</td><td>0.012</td><td>0.023</td><td>0.483</td><td>0.012</td><td>0.024</td><td>0.483</td><td>0.017</td><td>0.033</td></tr><tr><td>27548070 [63]</td><td>1634</td><td>910</td><td>626</td><td>26</td><td>20</td><td>10</td><td>9</td><td>0.769</td><td>0.012</td><td>0.024</td><td>0.385</td><td>0.011</td><td>0.021</td><td>0.346</td><td>0.014</td><td>0.028</td></tr><tr><td>27142267 [78]</td><td>10000</td><td>10000</td><td>10000</td><td>10</td><td>7</td><td>9</td><td>5</td><td>0.700</td><td>0.001</td><td>0.001</td><td>0.900</td><td>0.001</td><td>0.002</td><td>0.500</td><td>0.000</td><td>0.001</td></tr><tr><td>26903336 [81]</td><td>903</td><td>414</td><td>115</td><td>92</td><td>47</td><td>47</td><td>0</td><td>0.511</td><td>0.052</td><td>0.094</td><td>0.511</td><td>0.114</td><td>0.186</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>26349907 [53]</td><td>2675</td><td>265</td><td>1728</td><td>8</td><td>0</td><td>0</td><td>0</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>26199070 [58]</td><td>554</td><td>303</td><td>0</td><td>20</td><td>17</td><td>15</td><td>0</td><td>0.850</td><td>0.031</td><td>0.059</td><td>0.750</td><td>0.050</td><td>0.093</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>26109551 [66]</td><td>298</td><td>297</td><td>14</td><td>14</td><td>9</td><td>9</td><td>5</td><td>0.643</td><td>0.030</td><td>0.058</td><td>0.643</td><td>0.030</td><td>0.058</td><td>0.357</td><td>0.357</td><td>0.357</td></tr><tr><td>25770113 [42]</td><td>3046</td><td>2761</td><td>1956</td><td>7</td><td>7</td><td>5</td><td>7</td><td>1.000</td><td>0.002</td><td>0.005</td><td>0.714</td><td>0.002</td><td>0.004</td><td>1.000</td><td>0.004</td><td>0.007</td></tr><tr><td>25569206 [39]</td><td>746</td><td>691</td><td>0</td><td>49</td><td>49</td><td>49</td><td>0</td><td>1.000</td><td>0.066</td><td>0.123</td><td>1.000</td><td>0.071</td><td>0.132</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>25556126 [67]</td><td>212</td><td>206</td><td>50</td><td>9</td><td>8</td><td>8</td><td>4</td><td>0.889</td><td>0.038</td><td>0.072</td><td>0.889</td><td>0.039</td><td>0.074</td><td>0.444</td><td>0.080</td><td>0.136</td></tr><tr><td>25006006 [52]</td><td>834</td><td>395</td><td>407</td><td>25</td><td>24</td><td>23</td><td>20</td><td>0.960</td><td>0.029</td><td>0.056</td><td>0.920</td><td>0.058</td><td>0.110</td><td>0.800</td><td>0.049</td><td>0.093</td></tr><tr><td>24727842 [62]</td><td>2046</td><td>1989</td><td>0</td><td>69</td><td>66</td><td>66</td><td>0</td><td>0.957</td><td>0.032</td><td>0.062</td><td>0.957</td><td>0.033</td><td>0.064</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>24157497 [87]</td><td>10000</td><td>81</td><td>8636</td><td>61</td><td>61</td><td>0</td><td>61</td><td>1.000</td><td>0.006</td><td>0.012</td><td>0.000</td><td>0.000</td><td>0.000</td><td>1.000</td><td>0.007</td><td>0.014</td></tr><tr><td>24046285 [48]</td><td>978</td><td>978</td><td>812</td><td>12</td><td>12</td><td>12</td><td>12</td><td>1.000</td><td>0.012</td><td>0.024</td><td>1.000</td><td>0.012</td><td>0.024</td><td>1.000</td><td>0.015</td><td>0.029</td></tr><tr><td>23935058 [69]</td><td>628</td><td>20</td><td>887</td><td>5</td><td>3</td><td>0</td><td>1</td><td>0.600</td><td>0.005</td><td>0.009</td><td>0.000</td><td>0.000</td><td>0.000</td><td>0.200</td><td>0.001</td><td>0.002</td></tr><tr><td>23900314 [51]</td><td>499</td><td>300</td><td>195</td><td>6</td><td>5</td><td>5</td><td>4</td><td>0.833</td><td>0.010</td><td>0.020</td><td>0.833</td><td>0.017</td><td>0.033</td><td>0.667</td><td>0.021</td><td>0.040</td></tr><tr><td>23529983 [65]</td><td>283</td><td>160</td><td>0</td><td>8</td><td>8</td><td>2</td><td>0</td><td>1.000</td><td>0.028</td><td>0.055</td><td>0.250</td><td>0.013</td><td>0.024</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>23420235 [37]</td><td>6848</td><td>2305</td><td>3434</td><td>27</td><td>25</td><td>20</td><td>10</td><td>0.926</td><td>0.004</td><td>0.007</td><td>0.741</td><td>0.009</td><td>0.017</td><td>0.370</td><td>0.003</td><td>0.006</td></tr><tr><td>23033409 [86]</td><td>434</td><td>258</td><td>0</td><td>16</td><td>11</td><td>11</td><td>0</td><td>0.688</td><td>0.025</td><td>0.049</td><td>0.688</td><td>0.043</td><td>0.080</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>22986378 [76]</td><td>8633</td><td>7169</td><td>3884</td><td>40</td><td>26</td><td>38</td><td>19</td><td>0.650</td><td>0.003</td><td>0.006</td><td>0.950</td><td>0.005</td><td>0.011</td><td>0.475</td><td>0.005</td><td>0.010</td></tr><tr><td>22422870 [55]</td><td>1010</td><td>980</td><td>0</td><td>4</td><td>3</td><td>3</td><td>0</td><td>0.750</td><td>0.003</td><td>0.006</td><td>0.750</td><td>0.003</td><td>0.006</td><td>0.000</td><td>0.000</td><td>0.000</td></tr></tbody></table>
<table><thead><tr><th rowspan="2">SR's<sup>a</sup> PMID<sup>b</sup></th><th colspan="3">Items, n</th><th colspan="3">Number of GS found</th><th colspan="3">V1<sup>c</sup></th><th colspan="3">V2<sup>d</sup></th><th colspan="3">V3<sup>e</sup></th></tr><tr><th>V1</th><th>V2</th><th>V3</th><th>GS<sup>f</sup></th><th>V1</th><th>V2</th><th>V3</th><th>Se<sup>g</sup></th><th>PPV<sup>h</sup></th><th>F-Sc<sup>i</sup></th><th>Se</th><th>PPV</th><th>F-Sc</th><th>Se</th><th>PPV</th><th>F-Sc</th></tr></thead><tbody><tr><td>22323502 [32]</td><td>10000</td><td>10000</td><td>9784</td><td>5</td><td>3</td><td>3</td><td>2</td><td>0.600</td><td>0.000</td><td>0.001</td><td>0.600</td><td>0.000</td><td>0.001</td><td>0.400</td><td>0.000</td><td>0.000</td></tr><tr><td>22226047 [44]</td><td>1911</td><td>1239</td><td>1807</td><td>49</td><td>38</td><td>32</td><td>35</td><td>0.776</td><td>0.020</td><td>0.039</td><td>0.653</td><td>0.026</td><td>0.050</td><td>0.714</td><td>0.019</td><td>0.038</td></tr><tr><td>33176180 [84]</td><td>87</td><td>87</td><td>0</td><td>4</td><td>3</td><td>3</td><td>0</td><td>0.750</td><td>0.034</td><td>0.066</td><td>0.750</td><td>0.034</td><td>0.066</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>32479176 [68]</td><td>303</td><td>266</td><td>64</td><td>29</td><td>28</td><td>26</td><td>2</td><td>1.000</td><td>0.092</td><td>0.169</td><td>0.931</td><td>0.098</td><td>0.177</td><td>0.069</td><td>0.031</td><td>0.043</td></tr><tr><td>32427305 [57]</td><td>2035</td><td>2004</td><td>0</td><td>13</td><td>12</td><td>12</td><td>0</td><td>0.923</td><td>0.006</td><td>0.012</td><td>0.923</td><td>0.006</td><td>0.012</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>31727627 [56]</td><td>1884</td><td>1878</td><td>610</td><td>132</td><td>127</td><td>127</td><td>29</td><td>0.985</td><td>0.067</td><td>0.126</td><td>0.985</td><td>0.068</td><td>0.127</td><td>0.227</td><td>0.048</td><td>0.079</td></tr><tr><td>31585960 [36]</td><td>10000</td><td>10000</td><td>9514</td><td>227</td><td>186</td><td>206</td><td>139</td><td>0.819</td><td>0.019</td><td>0.036</td><td>0.907</td><td>0.021</td><td>0.040</td><td>0.612</td><td>0.015</td><td>0.029</td></tr><tr><td>30383109 [72]</td><td>4723</td><td>4716</td><td>570</td><td>38</td><td>36</td><td>36</td><td>2</td><td>0.947</td><td>0.008</td><td>0.015</td><td>0.947</td><td>0.008</td><td>0.015</td><td>0.053</td><td>0.004</td><td>0.007</td></tr><tr><td>28348110 [79]</td><td>27</td><td>27</td><td>0</td><td>24</td><td>1</td><td>1</td><td>0</td><td>0.042</td><td>0.037</td><td>0.039</td><td>0.042</td><td>0.037</td><td>0.039</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>28114600 [71]</td><td>85</td><td>35</td><td>25</td><td>68</td><td>9</td><td>3</td><td>0</td><td>0.132</td><td>0.106</td><td>0.118</td><td>0.044</td><td>0.086</td><td>0.058</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>26868137 [34]</td><td>10000</td><td>10000</td><td>10000</td><td>32</td><td>2</td><td>7</td><td>2</td><td>0.062</td><td>0.000</td><td>0.000</td><td>0.219</td><td>0.001</td><td>0.001</td><td>0.062</td><td>0.000</td><td>0.000</td></tr><tr><td>26830221 [82]</td><td>6167</td><td>1166</td><td>5102</td><td>76</td><td>74</td><td>28</td><td>72</td><td>0.974</td><td>0.012</td><td>0.024</td><td>0.368</td><td>0.024</td><td>0.045</td><td>0.947</td><td>0.014</td><td>0.028</td></tr><tr><td>26830055 [45]</td><td>6167</td><td>1166</td><td>5102</td><td>29</td><td>29</td><td>19</td><td>27</td><td>1.000</td><td>0.005</td><td>0.009</td><td>0.655</td><td>0.016</td><td>0.032</td><td>0.931</td><td>0.005</td><td>0.011</td></tr><tr><td>26420598 [83]</td><td>8405</td><td>8387</td><td>2421</td><td>57</td><td>47</td><td>47</td><td>30</td><td>0.825</td><td>0.006</td><td>0.011</td><td>0.825</td><td>0.006</td><td>0.011</td><td>0.526</td><td>0.012</td><td>0.024</td></tr><tr><td>26420387 [38]</td><td>8405</td><td>8387</td><td>2421</td><td>78</td><td>59</td><td>59</td><td>30</td><td>0.756</td><td>0.007</td><td>0.014</td><td>0.756</td><td>0.007</td><td>0.014</td><td>0.385</td><td>0.012</td><td>0.024</td></tr><tr><td>25059938 [61]</td><td>1524</td><td>1503</td><td>33</td><td>21</td><td>14</td><td>14</td><td>1</td><td>0.667</td><td>0.009</td><td>0.018</td><td>0.667</td><td>0.009</td><td>0.018</td><td>0.048</td><td>0.030</td><td>0.037</td></tr><tr><td>24592495 [31]</td><td>873</td><td>798</td><td>100</td><td>16</td><td>9</td><td>9</td><td>0</td><td>0.562</td><td>0.010</td><td>0.020</td><td>0.562</td><td>0.011</td><td>0.022</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>23460092 [85]</td><td>2426</td><td>2411</td><td>0</td><td>20</td><td>18</td><td>18</td><td>0</td><td>0.900</td><td>0.007</td><td>0.015</td><td>0.900</td><td>0.007</td><td>0.015</td><td>0.000</td><td>0.000</td><td>0.000</td></tr><tr><td>22777524 [88]</td><td>4645</td><td>3048</td><td>2838</td><td>37</td><td>37</td><td>35</td><td>31</td><td>1.000</td><td>0.008</td><td>0.016</td><td>0.946</td><td>0.011</td><td>0.023</td><td>0.838</td><td>0.011</td><td>0.022</td></tr></tbody></table>

Of the 59 selected SRs, 29 (49%) came from *The BMJ*, 19 (32%) came from the *Annals of Internal Medicine*, 6 (10%) came from *The Lancet*, and 5 (9%) came from the *Journal of the American Medical Association*. The publication dates were evenly distributed; the mean publication year and the median publication year were both 2016.

The countries of origin of the first authors were available for 49 (83%) SRs. The 3 most frequent countries of origin were the United States (21/49, 43%), the United Kingdom (5/49, 10%), and Canada (5/49, 10%).

## Quantification of the Utility of Medical Subject Headings Terms

The queries contained a median (IQR) of 43 (17.0-98) terms. The median (IQR) number of MeSH terms in the V1 queries was 6.0 (3.0-19.5). The median (IQR) proportion of MeSH terms relative to all terms in queries was 18.5% (13.7-25.5).

The V1 queries returned a total of 206,095 items, of which 1628 (0.79%) were included in the GS (Table 1). The V2 queries returned a total of 157,698 items, of which 1473 (0.93%) were included in the GS. In other words, an average of 820.29 additional articles per SR had to be screened for V1, relative to V2. Furthermore, V1 retrieved an average of 2.62 additional relevant articles, when compared with V2.

The median (Q1-Q3) sensitivities of queries V1 and V2 were 77.8% (62.1%-95.2%) and 71.4% (42.6%-90%), respectively (Table 2). The median (Q1-Q3) PPV of queries V1 and V2 were 0.9% (0.3%-2.8%) and 1.1% (0.3%-3.4%), respectively. The median (Q1-Q3) F₁-scores of queries V1 and V2 were 1.8% (0.7%-5.4%) and 2.2% (0.7%-6.1%), respectively. A graphic visualization of the sensitivity and PPV per SR showed that the addition of MeSH terms to a query typically increased the sensitivity but decreased the PPV (Figure 3). Furthermore, it can be seen that the transition from V2 to V1 had no effect for many SRs.

<footnote><sup>a</sup>SR: systematic review.</footnote>

<footnote><sup>b</sup>PMID: PubMed Identifier.</footnote>

<footnote><sup>c</sup>V1: original query.</footnote>

<footnote><sup>d</sup>V2: query with free-text terms only.</footnote>

<footnote><sup>e</sup>V3: query with Medical Subject Headings terms only.</footnote>

<footnote><sup>f</sup>GS: gold standard.</footnote>

<footnote><sup>g</sup>Se: sensitivity.</footnote>

<footnote><sup>h</sup>PPV: positive predictive value.</footnote>

<footnote><sup>i</sup>F-Sc: F₁-score.</footnote>
<table><caption>**Table 2.** Comparison of the performance levels of queries V1, V2, and V3.</caption><thead><tr><th>Query</th><th>Sensitivity, median (IQR)</th><th>PPV<sup>a</sup>, median (IQR)</th><th>F<sub>1</sub>-score, median (IQR)</th><th>Number of results, median (IQR)</th><th>Number of GS<sup>b</sup> items found, median (IQR)</th><th>Number of results per GS item found, median (IQR)</th></tr></thead><tbody><tr><td>Query V1 (MeSH<sup>c</sup> and FTTs<sup>d</sup>)</td><td>77.8 (62.1-95.2)</td><td>0.9 (0.3-2.8)</td><td>1.8 (0.7-5.4)</td><td>1950 (657.50-6167.00)</td><td>17 (7.00-36.50)</td><td>108.857 (35.062-298.574)</td></tr><tr><td>Query V2 (FTTs only)</td><td>71.4 (42.6-90)</td><td>1.1 (0.3-3.4)</td><td>2.2 (0.7-6.1)</td><td>1166 (301.50-2953.00)</td><td>15 (4.50-32.50)</td><td>88.667 (29.682-314.848)</td></tr><tr><td>Query V3 (MeSH only)</td><td>35.7 (0-61.7)</td><td>0.5 (0-2.6)</td><td>1 (0-3.9)</td><td>456 (31.50-2188.50)</td><td>4 (0-22.50)</td><td>81.305 (20.99-564.125)</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>**Figure 3.** Contribution of Medical Subject Headings (MeSH) terms to the queries. The orange circles correspond to V2 (free-text terms only), and the blue dots correspond to V1 (free-text terms and MeSH terms). PPV: positive predictive value.</figcaption></figure>

Overall, V1 provided 8.49% more of the GS's items than V2 and 35.55% more of the GS's items than V3. V2 provided 27.06% more of the GS's items than V3. The ratio between the number of GS references retrieved by V1 and the number retrieved by V2 was within the interval (0-1.05) in 66% (39/59) cases (Figure 4). In 59% (35/59) of cases, the ratio was 1 or less. In other words, the transition from V1 to V2 did not have a marked effect on the number of relevant articles retrieved for more than half of the SRs.

We also calculated the ORs for the number of relevant articles retrieved by V2 relative to V1 (Figure 5 [26,31-88]). Overall, the OR (95% CI) for V2 versus V1 was 0.55 (0.38-0.78) for sensitivity and 1.26 (1.03-1.54) for the PPV (Figure 5). The OR (95% CI) for V3 versus V1 was 0.31 (0.23-0.41) for sensitivity and 3.11 (2.15-4.48) for the PPV (Multimedia Appendix 3 [26,31-88]).

<footnote><sup>a</sup>PPV: positive predictive value.</footnote>

<footnote><sup>b</sup>GS: gold standard.</footnote>

<footnote><sup>c</sup>MeSH: Medical Subject Headings.</footnote>

<footnote><sup>d</sup>FTT: free-text terms (n=59).</footnote>
<figure><img src="image_5.png" /><figcaption>**Figure 5.** Forest plot of the odds ratio (OR) for V2 versus V1. An OR>1 means that V2 was better than V1 and so that inclusion of the Medical Subject Headings (MeSH) terms was harmful. An OR<1 means that V2 was worse than V1 and so that MeSH terms were useful. FPV1: false positive V1; FPV2: false positive V2; GS: gold standard; PMID: PubMed Identifier; PPV: positive predictive value; TPV1: true positive V1; TPV2: true positive V2.</figcaption></figure>

<figure><img src="image_6.png" /></figure>
# Discussion

## Key Results

The objective of this work was to quantify the utility of MeSH terms in SR queries. To this end, we retrieved the queries drafted by the authors of 59 SRs published in 4 prestigious medical journals. We then modified the V1 query to give a free-text terms only query and a MeSH-only query. Finally, we calculated the 3 queries' sensitivities, PPVs, and $F_1$-scores.

Our first key observation was that MeSH terms typically accounted for a nonnegligible proportion (on average, 20.4%) of the terms in the query. Second, the removal of MeSH terms from SR queries decreased the sensitivity (by 6.4%, on the median) and increased the PPV (by 0.2%, on the median). In other words, queries containing both MeSH terms and free-text terms yield an average of 2.62 additional relevant papers per SR, necessitating the screening of an additional 820.29 papers. The cost of screening an additional collected paper was therefore 313.09, which was slightly more than triple the mean reading cost associated with free-text terms only queries (88.67). Third, our results indicated that the deletion of MeSH terms had no effect on the number of relevant articles retrieved for 35 (59%) of the 59 reviews.

## Discussion of the Literature Data

The results of a previous study were similar to those found here; 95% of the relevant articles were retrieved in 67% (49/73) of the analyzed SRs when the query contained free-text terms alone (relative to the V1 query with a mixture of MeSH terms and free-text terms) [22]. Another study with a similar objective gave significantly different results; the free-text terms-only query was 25% less sensitive than MeSH-only query [15]. However, it should be noted that (1) the latter findings were based on a single query, and (2) the MeSH terms were converted to free-text terms manually, with a relatively limited set of synonyms used in the free-text terms strategy.

Furthermore, 3 messages should be highlighted. First, MeSH terms remain an indispensable tool for SRs despite the significant advancements in free-text search engines, especially in an era where the quality of SRs is declining [89]. Second, free-text terms appear to contribute more effectively to the retrieval of relevant articles compared with MeSH terms. Third, mixed queries (combining free-text and MeSH terms) exhibit poor PPV; for rapid literature reviews, it is preferable to use either MeSH terms or free-text terms exclusively.

Our study involved queries developed by experienced researchers; choosing free-text terms can be challenging and requires expertise. It is possible that clinicians with limited experience in literature searching struggle to choose free-text terms effectively, and yet, bibliographic research among clinicians is essential [90]. MeSH terms offer a distinct advantage over free-text terms by covering a broad range of vocabulary, which can be particularly beneficial for clinicians, early-career researchers, or nonnative English speakers. In such cases, incorporating MeSH terms can help clinicians construct more comprehensive and effective queries.

## Discussion of the Method

The GS comprised solely MEDLINE-indexed documents with a PMID. This choice was restrictive but technically essential, given that the 3 queries were submitted to the PubMed search engine. However, our restriction to documents with a PMID increased the queries' sensitivities and decreased their PPVs. We expect this bias to be nondifferential, insofar as it should affect the 3 types of queries in the same way.

The publications with PMIDs 26420387 [38] and 26420598 [83] were written by the same authors and were based on the same search query. This was also the case for PMIDs 26830055 [45] and 26830221 [82]. However, we considered these publications to be independent SRs, insofar as the corresponding GSs were different.

## Strengths and Weaknesses

### Strengths

One strength of our study is that we used queries from a number of different researchers and research centers; this should mean that our results are more representative of currently used search strategies. Furthermore, the automatic transformation of V1 to V2 probably helped us to avoid any bias associated with the differences in an individual's knowledge of the MeSH thesaurus.

### Weaknesses

Interpreting the results of V3 is delicate because the authors' queries are not designed to remain viable when ignoring all [tiab] and [all fields], etc. Indeed, after transformation to V3, a total of 11 queries become nonviable and return zero items.

In addition, it is important to note that the use of MeSH terms by the authors of the included SRs may be suboptimal and depends on each author's level of expertise. We assessed the quality of the MeSH selected by the authors of the included SR, not the actual utility of the MeSH as a feature. Finally, we are not able to measure the free-text terms retrieved from initial PubMed searches using only MeSH terms. However, the initial queries using MeSH terms alone may have enriched the search by helping to identify relevant free-text terms. It represents a potentially valuable contribution of MeSH terms that we do not measure here.

## Perspectives

Our results and the literature data provide quantitative information on the use and value of MeSH terms in the queries used for SRs. MeSH terms still appear to be important for achieving a comprehensive SR. Our results also emphasized how difficult it is to build a query for an SR and highlighted the significant variability in the results obtained; the search strategies are a matter of concern for researchers [91-93]. With a view to gaining insights into the possible benefits of MeSH terms for use by less experienced researchers, it would be interesting to conduct a similar study of literature searches performed by clinicians. Finally, our study also highlights that any bibliographic research involves a tedious process of sifting through articles, akin to finding a needle in a haystack. While the authors of SRs perform this task efficiently, inexperienced clinicians might find it discouraging to search for scientific
articles. New tools based on network analysis [94] could help
these clinicians find relevant articles more quickly.

Conclusion

The objective of this study was to estimate the utility of MeSH
terms, selected by authors, in SR queries by analyzing the
queries from 59 SRs published in 4 high-impact medical journals
in general medicine. Our results revealed that removing MeSH
terms from a query decreases sensitivity while slightly

increasing the PPV. Queries containing both MeSH and free-text
terms yielded more relevant articles but required screening many
additional papers. Despite this additional workload, MeSH terms
remain indispensable for SRs and can be particularly beneficial
for inexperienced clinicians or nonnative English speakers,
aiding in constructing more comprehensive queries. However,
mixed queries combining MeSH and free-text terms show poor
PPV, suggesting the exclusive use of either MeSH terms or
free-text terms for rapid reviews.

Acknowledgments

This research did not receive any specific funding from agencies or organizations in the public, commercial, or not-for-profit sectors.

Conflicts of Interest

None declared.

Multimedia Appendix 1

List of literature reviews analyzed.

[DOCX File, 22 KB-Multimedia Appendix 1]

Multimedia Appendix 2

All systematic reviews' queries: V1, V2, and V3.

[XLSX File (Microsoft Excel File), 100 KB-Multimedia Appendix 2]

Multimedia Appendix 3

Forest plot of the OR for V3 versus V1. An OR>1 means that V3 was better than V1. FPV1: false positive V1; FPV2: false positive V2; GS: gold standard; PMID: PubMed Identifier; OR: odds ratio; PPV: positive predictive value; TPV1: true positive V1; TPV2: true positive V2.

[PNG File, 803 KB-Multimedia Appendix 3]

References

1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-72. [FREE Full text] [doi: 10.1136/bmj.312.7023.71] [Medline: 8555924]

2. Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. J Med Libr Assoc. 2018;106(4):531-541. [FREE Full text] [doi: 10.5195/jmla.2018.283] [Medline: 30271302]

3. Brodman E, Field HG. Librarians' symposia and problems in Medical Subject Headings. Bull Med Libr Assoc. 1947;35(4):287-295. [FREE Full text] [Medline: 16016776]

4. Brodman E. Practical or service aspects of Medical Subject Headings. Bull Med Libr Assoc. 1948;36(2):102-107. [FREE Full text] [Medline: 16016805]

5. Larkey SV. Introduction to the problem of medical subject headings. Bull Med Libr Assoc. 1948;36(2):69-81. [FREE Full text] [Medline: 16016801]

6. Rogers FB. Medical subject headings. Bull Med Libr Assoc. 1963;51(1):114-116. [FREE Full text] [Medline: 13982385]

7. Doherty C, Joorabchi A, Megyesi P, Flynn A, Caulfield B. Physiotherapists' use of web-based information resources to fulfill their information needs during a theoretical examination: randomized crossover trial. J Med Internet Res. 2020;22(12):e19747. [FREE Full text] [doi: 10.2196/19747] [Medline: 33331826]

8. Lipscomb CE. Medical subject headings (MeSH). Bull Med Libr Assoc. 2000;88(3):265-266. [FREE Full text] [Medline: 10928714]

9. Rennesson M, Georget M, Paillard C, Perrin O, Pigeotte H, Tête C. Le thésaurus, un vocabulaire contrôlé pour parler le même langage. Médecine Palliative. 2020;19(1):15-23. [doi: 10.1016/j.medpal.2019.09.003]

10. Salgado TM, Fernandez-Llimos F. Missing pharmacy-specific medical subject headings (MeSH) terms: problems and solutions. Res Social Adm Pharm. 2019;15(9):1189-1190. [doi: 10.1016/j.sapharm.2019.04.008] [Medline: 31000335]

11. Clarke M, Greaves L, James S. MeSH terms must be used in Medline searches. BMJ. 1997;314(7088):1203. [FREE Full text] [doi: 10.1136/bmj.314.7088.1203a] [Medline: 9146422]
12. Richter R, Austin T. Using MeSH (Medical Subject Headings) to enhance PubMed search strategies for evidence-based practice in physical therapy. Phys Ther. 2012;92(1):124-132. [doi: 10.2522/ptj.20100178] [Medline: 21979271]

13. Klerings I, Robalino S, Booth A, Escobar-Liquitay CM, Sommer I, Gartlehner G, et al. Cochrane Rapid Reviews Methods Group. Rapid reviews methods series: guidance on literature search. BMJ Evid Based Med. 2023;28(6):412-417. [FREE Full text] [doi: 10.1136/bmjebm-2022-112079] [Medline: 37076268]

14. Jenuwine ES, Floyd JA. Comparison of Medical Subject Headings and text-word searches in MEDLINE to retrieve studies on sleep in healthy individuals. J Med Libr Assoc. 2004;92(3):349-353. [FREE Full text] [Medline: 15243641]

15. DeMars MM, Perruso C. MeSH and text-word search strategies: precision, recall, and their implications for library instruction. J Med Libr Assoc. 2022;110(1):23-33. [FREE Full text] [doi: 10.5195/jmla.2022.1283] [Medline: 35210959]

16. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309(6964):1286-1291. [FREE Full text] [doi: 10.1136/bmj.309.6964.1286] [Medline: 7718048]

17. Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994;1(6):447-458. [FREE Full text] [doi: 10.1136/jamia.1994.95153434] [Medline: 7850570]

18. Bachmann LM, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc. 2002;9(6):653-658. [FREE Full text] [doi: 10.1197/jamia.m1124] [Medline: 12386115]

19. Kassaï B, Sonié S, Shah NR, Boissel J. Literature search parameters marginally improved the pooled estimate accuracy for ultrasound in detecting deep venous thrombosis. J Clin Epidemiol. 2006;59(7):710-714. [FREE Full text] [doi: 10.1016/j.jclinepi.2005.09.013] [Medline: 16765274]

20. Golder S, McIntosh HM, Duffy S, Glanville J, Centre for Reviews and Dissemination and UK Cochrane Centre Search Filters Design Group. Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE. Health Info Libr J. 2006;23(1):3-12. [FREE Full text] [doi: 10.1111/j.1471-1842.2006.00634.x] [Medline: 16466494]

21. Chang AA, Heskett KM, Davidson TM. Searching the literature using medical subject headings versus text word with PubMed. Laryngoscope. 2006;116(2):336-340. [doi: 10.1097/01.mlg.0000195371.72887.a2] [Medline: 16467730]

22. Bramer WM, Giustini D, Kleijnen J, Franco OH. Searching embase and MEDLINE by using only major descriptors or title and abstract fields: a prospective exploratory study. Syst Rev. 2018;7(1):200. [FREE Full text] [doi: 10.1186/s13643-018-0864-9] [Medline: 30458825]

23. An introduction to journal impact factor. Clarivate. 2023. URL: https://clarivate.com/products/scientific-and-academic-research/research-analytics-evaluation-and-management-solutions/journal-citation-reports/publishers/first-time-publishers/ [accessed 2023-08-24]

24. Garfield E. Journal impact factor: a brief review. CMAJ. 1999;161(8):979-980. [FREE Full text] [Medline: 10551195]

25. How PubMed works: automatic term mapping (ATM). PubMed User Guide. 2023. URL: https://pubmed.ncbi.nlm.nih.gov/help/#automatic-term-mapping [accessed 2023-06-01]

26. Shekelle PG, Newberry SJ, FitzGerald JD, Motala A, O'Hanlon CE, Tariq A, et al. Management of gout: a systematic review in support of an American College of Physicians Clinical Practice Guideline. Ann Intern Med. 2017;166(1):37-51. [FREE Full text] [doi: 10.7326/M16-0461] [Medline: 27802478]

27. McKeever L, Nguyen V, Peterson SJ, Gomez-Perez S, Braunschweig C. Demystifying the search button: a comprehensive PubMed search strategy for performing an exhaustive literature review. JPEN J Parenter Enteral Nutr. 2015;39(6):622-635. [FREE Full text] [doi: 10.1177/0148607115593791] [Medline: 26129895]

28. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria. R Foundation for Statistical Computing; 2022.

29. RStudio Team. RStudio: Integrated Development Environment for R. Boston, MA. RStudio, PBC; 2020.

30. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J. Stat. Soft. 2010;36(3):1-48. [doi: 10.18637/jss.v036.i03]

31. Anglemyer A, Horvath T, Rutherford G. The accessibility of firearms and risk for suicide and homicide victimization among household members: a systematic review and meta-analysis. Ann Intern Med. 2014;160(2):101-110. [FREE Full text] [doi: 10.7326/M13-1301] [Medline: 24592495]

32. Asbridge M, Hayden JA, Cartwright JL. Acute cannabis consumption and motor vehicle collision risk: systematic review of observational studies and meta-analysis. BMJ. 2012;344:e536. [FREE Full text] [doi: 10.1136/bmj.e536] [Medline: 22323502]

33. Balk EM, Rofeberg VN, Adam GP, Kimmel HJ, Trikalinos TA, Jeppson PC. Pharmacologic and nonpharmacologic treatments for urinary incontinence in women: a systematic review and network meta-analysis of clinical outcomes. Ann Intern Med. 2019;170(7):465-479. [FREE Full text] [doi: 10.7326/M18-3227] [Medline: 30884526]

34. Bangalore S, Fakheri R, Toklu B, Messerli FH. Diabetes mellitus as a compelling indication for use of renin angiotensin system blockers: systematic review and meta-analysis of randomized trials. BMJ. 2016;352:i438. [FREE Full text] [doi: 10.1136/bmj.i438] [Medline: 26868137]

35. Batelaan NM, Bosman RC, Muntingh A, Scholten WD, Huijbregts KM, van Balkom AJLM. Risk of relapse after antidepressant discontinuation in anxiety disorders, obsessive-compulsive disorder, and post-traumatic stress disorder:
systematic review and meta-analysis of relapse prevention trials. BMJ. 2017;358:j3927. [FREE Full text] [doi: 10.1136/bmj.j3927] [Medline: 28903922]

36. Bellou V, Belbasis L, Konstantinidis AK, Tzoulaki I, Evangelou E. Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: systematic review and critical appraisal. BMJ. 2019;367:15358. [FREE Full text] [doi: 10.1136/bmj.15358] [Medline: 31585960]

37. Boland MV, Ervin AM, Friedman DS, Jampel HD, Hawkins BS, Vollenweider D, et al. Comparative effectiveness of treatments for open-angle glaucoma: a systematic review for the U.S. preventive services task force. Ann Intern Med. 2013;158(4):271-279. [FREE Full text] [doi: 10.7326/0003-4819-158-4-201302190-00008] [Medline: 23420235]

38. Bolland MJ, Leung W, Tai V, Bastin S, Gamble GD, Grey A, et al. Calcium intake and risk of fracture: systematic review. BMJ. 2015;351:h4580. [FREE Full text] [doi: 10.1136/bmj.h4580] [Medline: 26420387]

39. Carter JL, Coletti RJ, Harris RP. Quantifying and monitoring overdiagnosis in cancer screening: a systematic review of methods. BMJ. 2015;350:g7773. [FREE Full text] [doi: 10.1136/bmj.g7773] [Medline: 25569206]

40. Chersich MF, Pham MD, Areal A, Haghighi MM, Manyuchi A, Swift CP, et al. Climate ChangeHeat-Health Study Group. Associations between high temperatures in pregnancy and risk of preterm birth, low birth weight, and stillbirths: systematic review and meta-analysis. BMJ. 2020;371:m3811. [FREE Full text] [doi: 10.1136/bmj.m3811] [Medline: 33148618]

41. Chowdhury R, Ramond A, O'Keeffe LM, Shahzad S, Kunutsor SK, Muka T, et al. Environmental toxic metal contaminants and risk of cardiovascular disease: systematic review and meta-analysis. BMJ. 2018;362:k3310. [FREE Full text] [doi: 10.1136/bmj.k3310] [Medline: 30158148]

42. Conway R, Low C, Coughlan RJ, O'Donnell MJ, Carey JJ. Methotrexate use and risk of lung disease in psoriasis, psoriatic arthritis, and inflammatory bowel disease: systematic literature review and meta-analysis of randomised controlled trials. BMJ. 2015;350:h1269. [FREE Full text] [doi: 10.1136/bmj.h1269] [Medline: 25770113]

43. Drolet M, Bénard É, Pérez N, Brisson M, HPV Vaccination Impact Study Group. Population-level impact and herd effects following the introduction of human papillomavirus vaccination programmes: updated systematic review and meta-analysis. Lancet. 2019;394(10197):497-509. [FREE Full text] [doi: 10.1016/S0140-6736(19)30298-3] [Medline: 31255301]

44. Edmond KM, Kortsalioudaki C, Scott S, Schrag SJ, Zaidi AKM, Cousens S, et al. Group B streptococcal disease in infants aged younger than 3 months: systematic review and meta-analysis. Lancet. 2012;379(9815):547-556. [FREE Full text] [doi: 10.1016/S0140-6736(11)61651-6] [Medline: 22226047]

45. Eng J, Wilson RF, Subramaniam RM, Zhang A, Suarez-Cuervo C, Turban S, et al. Comparative effect of contrast media type on the incidence of contrast-induced nephropathy: a systematic review and meta-analysis. Ann Intern Med. 2016;164(6):417-424. [FREE Full text] [doi: 10.7326/M15-1402] [Medline: 26830055]

46. Ferreira GE, McLachlan AJ, Lin CC, Zadro JR, Abdel-Shaheed C, O'Keeffe M, et al. Efficacy and safety of antidepressants for the treatment of back pain and osteoarthritis: systematic review and meta-analysis. BMJ. 2021;372:m4825. [FREE Full text] [doi: 10.1136/bmj.m4825] [Medline: 33472813]

47. Ferreyro BL, Angriman F, Munshi L, Del Sorbo L, Ferguson ND, Rochwerg B, et al. Association of noninvasive oxygenation strategies with all-cause mortality in adults with acute hypoxemic respiratory failure: a systematic review and meta-analysis. JAMA. 2020;324(1):57-67. [FREE Full text] [doi: 10.1001/jama.2020.9524] [Medline: 32496521]

48. Gagne JJ, Bykov K, Choudhry NK, Toomey TJ, Connolly JG, Avorn J. Effect of smoking on comparative efficacy of antiplatelet agents: systematic review, meta-analysis, and indirect comparison. BMJ. 2013;347:f5307. [FREE Full text] [doi: 10.1136/bmj.f5307] [Medline: 24046285]

49. Gencer B, Marston NA, Im K, Cannon CP, Sever P, Keech A, et al. Efficacy and safety of lowering LDL cholesterol in older patients: a systematic review and meta-analysis of randomised controlled trials. Lancet. 2020;396(10263):1637-1643. [FREE Full text] [doi: 10.1016/S0140-6736(20)32332-1] [Medline: 33186535]

50. Goldenberg JZ, Day A, Brinkworth GD, Sato J, Yamada S, Jönsson T, et al. Efficacy and safety of low and very low carbohydrate diets for type 2 diabetes remission: systematic review and meta-analysis of published and unpublished randomized trial data. BMJ. 2021;372:m4743. [FREE Full text] [doi: 10.1136/bmj.m4743] [Medline: 33441384]

51. Goto A, Arah OA, Goto M, Terauchi Y, Noda M. Severe hypoglycaemia and cardiovascular disease: systematic review and meta-analysis with bias analysis. BMJ. 2013;347:f4533. [FREE Full text] [doi: 10.1136/bmj.f4533] [Medline: 23900314]

52. Haycock PC, Heydon EE, Kaptoge S, Butterworth AS, Thompson A, Willeit P. Leucocyte telomere length and risk of cardiovascular disease: systematic review and meta-analysis. BMJ. 2014;349:g4227. [FREE Full text] [doi: 10.1136/bmj.g4227] [Medline: 25006006]

53. Hendriksen JMT, Geersing GJ, Lucassen WAM, Erkens PMG, Stoffers HEJH, van Weert HCPM, et al. Diagnostic prediction models for suspected pulmonary embolism: systematic review and independent external validation in primary care. BMJ. 2015;351:h4438. [FREE Full text] [doi: 10.1136/bmj.h4438] [Medline: 26349907]

54. Hernandez AV, Roman YM, Pasupuleti V, Barboza JJ, White CM. Hydroxychloroquine or chloroquine for treatment or prophylaxis of COVID-19: a living systematic review. Ann Intern Med. 2020;173(4):287-296. [FREE Full text] [doi: 10.7326/M20-2496] [Medline: 32459529]

55. Hu EA, Pan A, Malik V, Sun Q. White rice consumption and risk of type 2 diabetes: meta-analysis and systematic review. BMJ. 2012;344:e1454. [FREE Full text] [doi: 10.1136/bmj.e1454] [Medline: 22422870]
56. Huang SW, Tsai CY, Tseng CS, Shih MC, Yeh YC, Chien KL, et al. Comparative efficacy and safety of new surgical treatments for benign prostatic hyperplasia: systematic review and network meta-analysis. BMJ. 2019;367:15919. [FREE Full text] [doi: 10.1136/bmj.l5919] [Medline: 31727627]

57. Hughes D, Judge C, Murphy R, Loughlin E, Costello M, Whiteley W, et al. Association of blood pressure lowering with incident dementia or cognitive impairment: a systematic review and meta-analysis. JAMA. 2020;323(19):1934-1944. [FREE Full text] [doi: 10.1001/jama.2020.4249] [Medline: 32427305]

58. Imamura F, O'Connor L, Ye Z, Mursu J, Hayashino Y, Bhupathiraju SN, et al. Consumption of sugar sweetened beverages, artificially sweetened beverages, and fruit juice and incidence of type 2 diabetes: systematic review, meta-analysis, and estimation of population attributable fraction. BMJ. 2015;351:h3576. [FREE Full text] [doi: 10.1136/bmj.h3576] [Medline: 26199070]

59. Juraschek SP, Hu JR, Cluett JL, Ishak A, Mita C, Lipsitz LA, et al. Effects of intensive blood pressure treatment on orthostatic hypotension: a systematic review and individual participant-based meta-analysis. Ann Intern Med. 2021;174(1):58-68. [FREE Full text] [doi: 10.7326/M20-4298] [Medline: 32909814]

60. Kavalieratos D, Corbelli J, Zhang D, Dionne-Odom JN, Ernecoff NC, Hanmer J, et al. Association between palliative care and patient and caregiver outcomes: a systematic review and meta-analysis. JAMA. 2016;316(20):2104-2114. [FREE Full text] [doi: 10.1001/jama.2016.16840] [Medline: 27893131]

61. Kerrigan D, Kennedy CE, Morgan-Thomas R, Reza-Paul S, Mwangi P, Win KT, et al. A community empowerment approach to the HIV response among sex workers: effectiveness, challenges, and considerations for implementation and scale-up. Lancet. 2015;385(9963):172-185. [FREE Full text] [doi: 10.1016/S0140-6736(14)60973-9] [Medline: 25059938]

62. Kim CA, Rasania SP, Afilalo J, Popma JJ, Lipsitz LA, Kim DH. Functional status and quality of life after transcatheter aortic valve replacement: a systematic review. Ann Intern Med. 2014;160(4):243-254. [FREE Full text] [doi: 10.7326/M13-1316] [Medline: 24727842]

63. Kim DH, Kim CA, Placide S, Lipsitz LA, Marcantonio ER. Preoperative frailty assessment and outcomes at 6 months or later in older adults undergoing cardiac surgical procedures: a systematic review. Ann Intern Med. 2016;165(9):650-660. [FREE Full text] [doi: 10.7326/M16-0652] [Medline: 27548070]

64. Kisely S, Warren N, McMahon L, Dalais C, Henry I, Siskind D. Occurrence, prevention, and management of the psychological effects of emerging virus outbreaks on healthcare workers: rapid review and meta-analysis. BMJ. 2020;369:m1642. [FREE Full text] [doi: 10.1136/bmj.m1642] [Medline: 32371466]

65. Kramer CK, Zinman B, Gross JL, Canani LH, Rodrigues TC, Azevedo MJ, et al. Coronary artery calcium score prediction of all cause mortality and cardiovascular events in people with type 2 diabetes: systematic review and meta-analysis. BMJ. 2013;346:f1654. [FREE Full text] [doi: 10.1136/bmj.f1654] [Medline: 23529983]

66. Lamont K, Scott NW, Jones GT, Bhattacharya S. Risk of recurrent stillbirth: systematic review and meta-analysis. BMJ. 2015;350:h3080. [FREE Full text] [doi: 10.1136/bmj.h3080] [Medline: 26109551]

67. Liao WC, Tu YK, Wu MS, Lin JT, Wang HP, Chien KL. Blood glucose concentration and risk of pancreatic cancer: systematic review and dose-response meta-analysis. BMJ. 2015;350:g7371. [FREE Full text] [doi: 10.1136/bmj.g7371] [Medline: 25556126]

68. Lyles CR, Nelson EC, Frampton S, Dykes PC, Cemballi AG, Sarkar U. Using electronic health record portals to improve patient engagement: research priorities and best practices. Ann Intern Med. 2020;172(11 Suppl):S123-S129. [FREE Full text] [doi: 10.7326/M19-0876] [Medline: 32479176]

69. Långström N, Enebrink P, Laurén EM, Lindblom J, Werkö S, Hanson RK. Preventing sexual abusers of children from reoffending: systematic review of medical and psychological interventions. BMJ. 2013;347:f4630. [FREE Full text] [doi: 10.1136/bmj.f4630] [Medline: 23935058]

70. Martinez L, Cords O, Horsburgh CR, Andrews JR, Pediatric TB Contact Studies Consortium. The risk of tuberculosis in children after close exposure: a systematic review and individual-participant meta-analysis. Lancet. 2020;395(10228):973-984. [FREE Full text] [doi: 10.1016/S0140-6736(20)30166-5] [Medline: 32199484]

71. Mendelson A, Kondo K, Damberg C, Low A, Motúapuaka M, Freeman M, et al. The effects of pay-for-performance programs on health, health care use, and processes of care: a systematic review. Ann Intern Med. 2017;166(5):341-353. [FREE Full text] [doi: 10.7326/M16-1881] [Medline: 28114600]

72. Naik RP, Smith-Whitley K, Hassell KL, Umeh NI, de Montalembert M, Sahota P, et al. Clinical outcomes associated with sickle cell trait: a systematic review. Ann Intern Med. 2018;169(9):619-627. [FREE Full text] [doi: 10.7326/M18-1161] [Medline: 30383109]

73. Schünemann HJ, Khabsa J, Solo K, Khamis AM, Brignardello-Petersen R, El-Harakeh A, et al. Ventilation techniques and risk for transmission of coronavirus disease, including covid-19: a living systematic review of multiple streams of evidence. Ann Intern Med. 2020;173(3):204-216. [FREE Full text] [doi: 10.7326/M20-2306] [Medline: 32442035]

74. Newberry SJ, FitzGerald JD, Motala A, Booth M, Maglione MA, Han D, et al. Diagnosis of gout: a systematic review in support of an American College of Physicians Clinical Practice Guideline. Ann Intern Med. 2017;166(1):27-36. [FREE Full text] [doi: 10.7326/M16-0462] [Medline: 27802505]
75. Ochen Y, Beks RB, van Heijl M, Hietbrink F, Leenen LPH, van der Velde D, et al. Operative treatment versus nonoperative treatment of achilles tendon ruptures: systematic review and meta-analysis. BMJ. 2019;364:k5120. [FREE Full text] [doi: 10.1136/bmj.k5120] [Medline: 30617123]

76. Prvu Bettger J, Alexander KP, Dolor RJ, Olson DM, Kendrick AS, Wing L, et al. Transitional care after hospitalization for acute stroke or myocardial infarction: a systematic review. Ann Intern Med. 2012;157(6):407-416. [FREE Full text] [doi: 10.7326/0003-4819-157-6-201209180-00004] [Medline: 22986378]

77. Rotenstein LS, Torre M, Ramos MA, Rosales RC, Guille C, Sen S, et al. Prevalence of burnout among physicians: a systematic review. JAMA. 2018;320(11):1131-1150. [FREE Full text] [doi: 10.1001/jama.2018.12777] [Medline: 30326495]

78. Salvo F, Moore N, Arnaud M, Robinson P, Raschi E, de Ponti F, et al. Addition of dipeptidyl peptidase-4 inhibitors to sulphonylureas and risk of hypoglycaemia: systematic review and meta-analysis. BMJ. 2016;353:i2231. [FREE Full text] [doi: 10.1136/bmj.i2231] [Medline: 27142267]

79. Schandelmaier S, Kaushal A, Lytvyn L, Heels-Ansdell D, Siemieniuk RAC, Agoritsas T, et al. Low intensity pulsed ultrasound for bone healing: systematic review of randomized controlled trials. BMJ. 2017;356:j656. [FREE Full text] [doi: 10.1136/bmj.j656] [Medline: 28348110]

80. Selby K, Baumgartner C, Levin TR, Doubeni CA, Zauber AG, Schottinger J, et al. Interventions to improve follow-up of positive results on fecal blood tests: a systematic review. Ann Intern Med. 2017;167(8):565-575. [FREE Full text] [doi: 10.7326/M17-1361] [Medline: 29049756]

81. Shankar-Hari M, Phillips GS, Levy ML, Seymour CW, Liu VX, Deutschman CS, et al. Sepsis Definitions Task Force. Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):775-787. [FREE Full text] [doi: 10.1001/jama.2016.0289] [Medline: 26903336]

82. Subramaniam RM, Suarez-Cuervo C, Wilson RF, Turban S, Zhang A, Sherrod C, et al. Effectiveness of prevention strategies for contrast-induced nephropathy: a systematic review and meta-analysis. Ann Intern Med. 2016;164(6):406-416. [FREE Full text] [doi: 10.7326/M15-1456] [Medline: 26830221]

83. Tai V, Leung W, Grey A, Reid IR, Bolland MJ. Calcium intake and bone mineral density: systematic review and meta-analysis. BMJ. 2015;351:h4183. [FREE Full text] [doi: 10.1136/bmj.h4183] [Medline: 26420598]

84. Thomalla G, Boutitie F, Ma H, Koga M, Ringleb P, Schwamm LH, et al. Intravenous alteplase for stroke with unknown time of onset guided by advanced imaging: systematic review and meta-analysis of individual patient data. Lancet. 2020;396(10262):1574-1584. [FREE Full text] [doi: 10.1016/S0140-6736(20)32163-2] [Medline: 33176180]

85. Weaver SJ, Lubomksi LH, Wilson RF, Pfoh ER, Martinez KA, Dy SM. Promoting a culture of safety as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 Pt 2):369-374. [FREE Full text] [doi: 10.7326/0003-4819-158-5-201303051-00002] [Medline: 23460092]

86. Wehner MR, Shive ML, Chren MM, Han J, Qureshi AA, Linos E. Indoor tanning and non-melanoma skin cancer: systematic review and meta-analysis. BMJ. 2012;345:e5909. [doi: 10.1136/bmj.e5909] [Medline: 23033409]

87. Wu HY, Huang JW, Lin HJ, Liao WC, Peng YS, Hung KY, et al. Comparative effectiveness of renin-angiotensin system blockers and other antihypertensive drugs in patients with diabetes: systematic review and bayesian network meta-analysis. BMJ. 2013;347:f6008. [FREE Full text] [doi: 10.1136/bmj.f6008] [Medline: 24157497]

88. Yeh HC, Brown TT, Maruthur N, Ranasinghe P, Berger Z, Suh YD, et al. Comparative effectiveness and safety of methods of insulin delivery and glucose monitoring for diabetes mellitus: a systematic review and meta-analysis. Ann Intern Med. 2012;157(5):336-347. [FREE Full text] [doi: 10.7326/0003-4819-157-5-201209040-00508] [Medline: 22777524]

89. Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485-514. [FREE Full text] [doi: 10.1111/1468-0009.12210] [Medline: 27620683]

90. McKibbon KA, Lokker C, Keepanasseril A, Wilczynski NL, Haynes RB. Net improvement of correct answers to therapy questions after PubMed searches: pre/post comparison. J Med Internet Res. 2013;15(11):e243. [FREE Full text] [doi: 10.2196/jmir.2572] [Medline: 24217329]

91. Lazarus JV, Palayew A, Rasmussen LN, Andersen TH, Nicholson J, Norgaard O. Searching PubMed to retrieve publications on the COVID-19 pandemic: comparative analysis of search strings. J Med Internet Res. 2020;22(11):e23449. [FREE Full text] [doi: 10.2196/23449] [Medline: 33197230]

92. Agoritsas T, Merglen A, Courvoisier DS, Combescure C, Garin N, Perrier A, et al. Sensitivity and predictive value of 15 PubMed search strategies to answer clinical questions rated against full systematic reviews. J Med Internet Res. 2012;14(3):e85. [FREE Full text] [doi: 10.2196/jmir.2021] [Medline: 22693047]

93. Kastner M, Wilczynski NL, Walker-Dilks C, McKibbon KA, Haynes B. Age-specific search strategies for medline. J Med Internet Res. 2006;8(4):e25. [FREE Full text] [doi: 10.2196/jmir.8.4.e25] [Medline: 17213044]

94. BibliZap: an open-source and non-profit tool for reference mining that helps find similar articles. BibliZap Team. URL: https://biblizap.org/ [accessed 2024-09-18]

Abbreviations

ATM: automatic term mapping
**GS:** gold standard

**MeSH:** Medical Subject Headings

**NLM:** US National Library of Medicine

**OR:** odds ratio

**PMID:** PubMed Identifier

**PPV:** positive predictive value

**SR:** systematic review

**V1:** original query

**V2:** query with free-text terms only

**V3:** query with MeSH terms only

Edited by G Tsafnat; submitted 18.10.23; peer-reviewed by M Rethlefsen, M Arab-Zozani, F Fernandez-Limós; comments to author 06.04.24; revised version received 17.04.24; accepted 07.07.24; published 19.11.24

Please cite as:

Leblanc V, Hamroun A, Bentegeac R, Le Guellec B, Lenain R, Chazard E

Added Value of Medical Subject Headings Terms in Search Strategies of Systematic Reviews: Comparative Study

J Med Internet Res 2024;26:e53781

URL: https://www.jmir.org/2024/1/e53781

doi: 10.2196/53781

PMID:

©Victor Leblanc, Aghiles Hamroun, Raphaël Bentegeac, Bastien Le Guellec, Rémi Lenain, Emmanuel Chazard. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.11.2024. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.