Abstract: We identified two eligible studies reporting 18 methodological filters. All methodological filters in these two studies were developed using terms from the reference standard records. The first study evaluated six filters for retrieving observational studies of surgical interventions. The study reported on six filters: one Precision Terms Filter (comprising terms with higher precision while maximum sensitivity was maintained) and one Specificity Terms Filter (comprising terms with higher specificity while maximum sensitivity was maintained), both of which were adapted for MEDLINE, for Embase, and for combined MEDLINE/Embase searches. The study reported one reference standard consisting of 217 articles from one systematic review of which 83.9% of the included studies were case series The second study reported on 12 filters for retrieving comparative non-randomised studies (cNRSs) including cohort, case-control, and cross-sectional studies. This study reported on 12 filters using four different approaches: Fixed method A (comprising of a fixed set of controlled vocabulary (CV) words), Fixed method B (comprising a fixed set of CV words and text words (TW)), Progressive method (CV) (a random choice of study design-related CV terms), and Progressive method (CV or TW) (a random choice of study design-related CV terms, and title and abstracts-based TWs). The study reported four reference standards consisting of 89 cNRSs from four systematic reviews. The six methodological filters developed from the first study reported sensitivity of 99.5% to 100% and precision of 16.7% to 21.1%. The Specificity Terms Filter for combined MEDLINE/Embase was preferred because it had higher precision and equal sensitivity to the Precision Terms Filter. The 12 filters from the second study reported lower sensitivity (48% to 100%) and much lower precision (0.09% to 4.47%). The Progressive method (CV or TW) had the highest sensitivity. There were methodological limitations in both included studies. The first study used one surgical intervention-focused systematic review thus limiting the generalizability of findings. The second study used four systematic reviews but with less than 100 studies. The external validation was performed only on Specificity Terms Filter from the first study Both studies were published 10 years ago and labelling and indexing of observational studies has changed since then. We found 18 methodological filters across two eligible studies. Search strategies from the first study had higher sensitivity and precision, underwent external validation and targeted observational studies. Search strategies from the second study had lower sensitivity and precision, focused on cNRSs, and were not validated externally. Given this limited and heterogeneous evidence, and its methodological limitations, further research and better indexation are needed.

Plain-language summary: We found two eligible studies reporting on 18 methodological filters, including six MEDLINE, six Embase and six combined MEDLINE/Embase filters. The firsts study focused on filters on observational studies of surgical interventions. The second study focused on filters for a specific subtype of observational studies: comparative non-randomised studies. Six filters from the first study showed sensitivity of 99.5% to 100% and precision of 16.7% to 21.1%. One type of filter was evaluated by two additional systematic reviews (i.e. externally validated) and found that this retrieved 85.2% to 100% of the articles in the reference standard. Twelve filters from the second study had lower sensitivity (48% to 100%) and much lower precision (0.09% to 4.47%). The included studies had several limitations. The first study used only one systematic review for search strategy development and focused on observational studies of surgical interventions, which might limit the generalizability of the findings to other literature searches. The reference standard in the second study, although encompassing four different systematic reviews, included a limited number of studies, which might affect the accuracy of the performance assessment. Both studies were published 10 years ago and labelling and indexing of observational studies has changed since then.