Title: Injury rates in Female and Male Military Personnel: A Systematic Review and Meta-
Analysis
Version: 2 Date: 03 Jun 2022
Reviewer:  Thane Chambers
Reviewer's report:
Thank you very much for revising your article and for your thoughtful responses to the previous
round of peer review comments. I continue to believe that this paper is well written and will be
of interest to readers of BMC Women's Health. However, I do continue to have concerns about
the way in which the search was developed and managed for this review.I hope the following
comments will help to to improve your study or any future reviews you decide to conduct.
Methods
TABLE 1: SEARCH TERMS USED IN EACH DATABASE
Please change the database labelled Ovid to Ovid Medline. Ovid is a database platform not a
database.
There is an error in your search strategy for Ovid Medline
((((((female.ti,ab. OR women.ti,ab. OR woman.ti,ab.)))) AND injur*.ti,ab.)) AND
((defence.ti,ab. OR defence.ti,ab. OR military.ti,ab. OR army.ti,ab. OR air force.ti,ab. OR
navy.ti,ab. OR marines.ti,ab. OR tactical.ti,ab. OR recruit.ti,ab. OR soldier.ti,ab. OR cadet.ti,ab.
OR trainee.ti,ab.))
The query term defence has been duplicated. I notice in your PubMed and CINAHL searches
that you have a different query related to this term: defence OR defense.
Also, especially for your Ovid Medline search which does only exact searching (it only searches
for the terms as they are presented), you should have truncated the term female to also retrieve
articles that use the word females.
The rest of my feedback concerns your responses to my previous peer review comments. And
before I begin, thank you for taking the time to answer my questions and for the revisions you
have done so far based on this feedback.
ORIGINAL PEER REVIEW COMMENT:
It is unclear who was responsible for the search used in this meta-analysis. None of the authors
appear to be librarians or information scientists and no one took credit for the search strategies
in the Author Contribution section. But there are some things about the search that should be
corrected.
AUTHORS' RESPONSE:
Thank you for your comment, the search was created, adjusted by all the authors collaboratively.
All authors are experienced researchers who have been involved in the design and conduct of
many systematic reviews and have developed many search strategies for such reviews, alongside
experienced University librarians. The final search was run by the first author and replicated
by the second.
NEW QUESTION/COMMENT:
Are you able to add some of this information to your Authors' Contribution statement at the end
of your article? As it currently stands, it's not clear who designed and conducted the searches.
ORIGINAL PEER REVIEW COMMENTS:
Sorry if this is a bit confusing, I'm just combining these 2 comments and answers because they
are essentially about the same thing--the decision not to use controlled vocabularies.
In the search strategies, no controlled vocabulary terms were used in any of your searches.
Considering that the searches were run in PubMed, MEDLINE, and CINAHL this seems quite
strange as PubMed and MEDLINE use MeSH as their controlled vocabulary and CINAHL uses
CINAHL terms. Both of these controlled vocabularies are well-known, established, and deciding
not to use them should, at the very least, be justified. MeSH terms such as Military Personnel
and Wounds and Injuries (or perhaps some of its narrower terms) should have been included
in your searches in PubMed and MEDLINE and I am certain that there are equivalent relevant
terms in the CINAHL headings.
The decision to search only the title and abstract fields also seems very strange and should
be justified. This means that no subject headings (either controlled vocabulary terms or author
keywords) were included in your search strategy. I would recommend re-running your searches
with your query terms in the keyword fields to see if any relevant studies were missed.
AUTHORS' RESPONSE:
Thank you for this query and for the subsequent queries 3 and 4, for which we have provided
additional responses below. The tool ‘systematic review accelerator’ (sr-accelerator.com) was
used when developing the final search strategy, to refine and optimise initial proposed search
strategies and select databases. We have added this detail to the methods to make this clear to
the reader.
This tool was developed by the Institute of Evidence Based Healthcare and is an automated
tool which enables optimisation of search strategies based on success of search strategies in
identifying relevant articles; the tool indicates terms within search strategies which are redundant
(i.e. their inclusion in the search strategy does not result in identification of additional, relevant
search results), so they can be removed. The initial search strategy, which was subsequently
optimised using this tool, included a larger number of databases and search terms, but some
were removed because they were found to be redundant and, in this way, the finalised list was
developed.
Thank you for this comment. We took this approach of limiting the search to title and abstract
(inclusive of key words) because, regardless of where else in an article or citation a search term
occurred, if the search terms did not appear in the title and/or abstract it was highly unlikely
the article would meet the eligibility criteria for inclusion, since the title and abstract together
provide a summary of the focus and content of the respective article. This approach reduced the
burden of screening thousands of articles that were clearly irrelevant based on the fact search
terms did not appear in the title or abstract but were identified simply because the search terms
appeared somewhere else in the article – often with just a cursory mention.
NEW QUESTIONS/COMMENTS
I'm glad that you added the information about using sr-accelerator.com which is a very useful
tool and it's good to see that it will receive credit and recognition in your article. However, I still
continue to have some major reservations about this decision. For systematic reviews and meta-
analyses it is very important to be thorough in your search strategies to ensure that the widest
range of data is available for synthesis and analysis. This is well described in the literature.
Please refer to any of the following:
Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane
handbook for systematic reviews of interventions. John Wiley & Sons; 2019 Sep 23.
Greyson, D., Rafferty, E., Slater, L. et al. Systematic review searches must be systematic,
comprehensive, and transparent: a critique of Perman et al. BMC Public Health 19, 153 (2019).
https://doi.org/10.1186/s12889-018-6275-y
Akers J, Aguiar-Ibáñez R, Baba-Akbari A. Systematic reviews: CRD’s guidance for undertaking
reviews in health care. York, UK: Centre for Reviews and Dissemination, University of York;
2009.
Bibliographic databases are made up of several fields. For reviews and meta-analysis the most
important search fields are usually the  title, abstract, and keywords fields. The decision to
not search for terms in the keyword fields (either the author keyword fields or the controlled
vocabulary keywords field) is not justified in your responses to my comments. I definitely do
agree that it would have increased the number of citations that would need to be retrieved,
however, as you only had 3 databases in your search and the searches in PubMed and Ovid
MEDLINE have a huge amount of overlap (please see https://wkhealth.force.com/ovidsupport/s/
article/PubMed-vs-Ovid-s-Medline-1489081398582 for more information), the number of total
new results should have been manageable for the size of the listed authors on this paper.  Also,
I note strongly that I was not asking that these terms be found in all fields, but only the keyword
fields and as senior researchers who have worked with librarians in the past, I feel I shouldn't
have to explain to you the importance of controlled vocabularies or author keywords. However,
pleaser refer to any of the following references for more information about the important role
played by controlled vocabularies in systematic review :
Lefebvre C, Glanville J, Briscoe S, Featherstone R, Littlewood A, Marshall C, Metzendorf M-
I, Noel-Storr A, Paynter R, Rader T, Thomas J, Wieland LS. Chapter 4: Searching for and
selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch
VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated
February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.
Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, Glanville JM.
Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane
Database Syst Rev. 2013 Sep 11;2013(9):MR000022. doi: 10.1002/14651858.MR000022.pub3.
PMID: 24022476; PMCID: PMC7390022.
Chang AA, Heskett KM, Davidson TM. Searching the literature using medical subject
headings versus text word with PubMed. Laryngoscope. 2006 Feb;116(2):336-40. doi:
10.1097/01.mlg.0000195371.72887.a2. PMID: 16467730.
Jenuwine ES, Floyd JA. Comparison of Medical Subject Headings and text-word searches
in MEDLINE to retrieve studies on sleep in healthy individuals. J Med Libr Assoc. 2004
Jul;92(3):349-53. PMID: 15243641; PMCID: PMC442177.
Lefebvre C, Glanville J, Wieland LS, Coles B, Weightman AL. Methodological developments
in searching for studies for systematic reviews: past, present and future? Syst Rev. 2013 Sep
25;2:78. doi: 10.1186/2046-4053-2-78. PMID: 24066664; PMCID: PMC4015986.
Martin-Rodero H, Sanz-Valero J, Galindo-Villardon P. The methodological quality of
systematic reviews indexed in the MEDLINE database: A multivariate approach. The Electronic
Library. 2018 Feb 5;36(1):146-58.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review
of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016 Jul;75:40-6.
doi: 10.1016/j.jclinepi.2016.01.021. Epub 2016 Mar 19. PMID: 27005575.
Another serious issue with an approach that relies exclusively on title and abstract searching is
that it makes a very large assumption that the metadata supplied in these data fields accurately
and comprehensively describes what an article is about. Of course, no descriptive data can be
as complete as the actual article itself, but searching the additional keywords fields does correct
for incorrect or incomplete information in the title and abstract fields. Please refer to any of the
following about issues with the data supplied in the abstract field:
Cohen JF, Korevaar DA, Bossuyt PM. Diagnostic accuracy studies need more informative
abstracts. Eur J Clin Microbiol Infect Dis. 2019 Aug;38(8):1383-1385. doi: 10.1007/
s10096-019-03570-7. PMID: 31049744.
Harris AH, Standard S, Brunning JL, Casey SL, Goldberg JH, Oliver L, Ito K, Marshall JM.
The accuracy of abstracts in psychology journals. J Psychol. 2002 Mar;136(2):141-8. doi:
10.1080/00223980209604145. PMID: 12081089.
Korevaar DA, Cohen JF, Hooft L, Bossuyt PM. Literature survey of high-impact journals
revealed reporting weaknesses in abstracts of diagnostic accuracy studies. J Clin Epidemiol.
2015 Jun;68(6):708-15. doi: 10.1016/j.jclinepi.2015.01.014. Epub 2015 Jan 28. PMID:
25703213.
Nascimento DP, Costa LOP, Gonzalez GZ, Maher CG, Moseley AM. Abstracts of low back
pain trials are poorly reported, contain spin of information and are inconsistent with the full
text: An overview study. Arch Phys Med Rehabil. 2019 Jun 14:1976-1985.e18. doi: 10.1016/
j.apmr.2019.03.024. Epub ahead of print. PMID: 31207219.
Fontelo P, Gavino A, Sarmiento RF. Comparing data accuracy between structured abstracts
and full-text journal articles: implications in their use for informing clinical decisions. Evid
Based Med. 2013 Dec;18(6):207-11. doi: 10.1136/eb-2013-101272. Epub 2013 Jun 20. PMID:
23786759.
Pagkalidou E, Anastasilakis DA, Kokkali S, Doundoulakis I, Tsapas A, Dardavessis T,
Haidich AB. Reporting completeness in abstracts of systematic reviews of diagnostic test
accuracy studies in cardiovascular diseases is suboptimal. Hellenic J Cardiol. 2022 Feb
15:S1109-9666(22)00021-5. doi: 10.1016/j.hjc.2022.02.001. Epub ahead of print. PMID:
35181563.
Ward LG, Kendrach MG, Price SO. Accuracy of abstracts for original research articles in
pharmacy journals. Ann Pharmacother. 2004 Jul-Aug;38(7-8):1173-7. doi: 10.1345/aph.1D416.
Epub 2004 May 18. PMID: 15150375.
Indeed sr-accelerator.com does show you the fields where terms appear for a select group of
references or for a specific search strategy, but did you use it to test what was lost by expanding
your search to the keyword fields?
I redid your search as you have described it in OVID Medline and did an additional search for the
MeSH term female and searched in the author keywords for the term female*. I then subtracted
the references that your search would have retrieved. I scanned ONLY the first 50 references
and found the following articles. I do not know if these articles would have made it through your
inclusion/exclusion criteria especially as you were only interested in general injuries, rather than
specific ones, but based on a very superficial scan they look like they may be somewhat relevant.
So this makes me wonder what I would have found if I had gone past the first 50 references?
Asgari M, Alizadeh S, Sendt A, Jaitner T. Evaluation of the Functional Movement Screen (FMS)
in Identifying Active Females Who are Prone to Injury. A Systematic Review. Sports Med Open.
2021;7(1):85.
Campbell KE, Parent EC, Crumback DJ, Hebert JS. Predicting Upper Quadrant Musculoskeletal
Injuries in the Military: A Cohort Study. Med Sci Sports Exerc. 2022;54(2):337–44.
Chimenti RL, Pacha MS, Glass NA, Frazier M, Bowles AO, Valantine AD, et al. Elevated
Kinesiophobia Is Associated With Reduced Recovery From Lower Extremity Musculoskeletal
Injuries in Military and Civilian Cohorts. Phys Ther. 2022;102(2).
Ericksen HM, Reckelberg R, Earl-Boehm J. The injury risk profile of an Army Reserve
Officers’ Training Corps population: A preliminary descriptive study. Phys Ther Sport.
2021;51(100940513):65–70.
Loverro KL, Hasselquist L, Lewis CL. Females and males use different hip and knee mechanics
in response to symmetric military-relevant loads. J Biomech. 2019;95(0157375, hjf):109280.
Nesterovica D, Vaivads N, Stepens A. Relationship of footwear comfort, selected size, and lower
leg overuse injuries among infantry soldiers. BMC Musculoskelet Disord. 2021;22(1):952.
Por ED, Akers KS, Chung KK, Livezey JR, Selig DJ. Population Pharmacokinetic Modeling
and Simulations of Imipenem in Burn Patients With and Without Continuous Venovenous
Hemofiltration in the Military Health System. J Clin Pharmacol. 2021;61(9):1182–94.
Sandler AB, Hoyt BW, Klahs KJ, Scanaliato JP, Nesti LJ, Dunn JC. Epidemiology and
Long-Term Outcomes of Wrist Sprains in Military Academy Cadets. Am J Sports Med.
2021;49(8):2085–9.
Shaw II TW, Chung KK, Wilson RL, April MD, Pamplin JC, Gillespie KR, et al. An
Analysis of Patient Movements during Sustained Combat Operations in the US Central
Command: Implications for Remote Support Capabilities. Med J (Ft Sam Houst Tex). 2022;Per
22-04-05-06(Per 22-04-05-06):73–7.
I'm guessing that you probably do not want to redo the search and essentially have to start the
study over again? In which case, I would recommend adding something to your limitations about
how possible relevant studies may have been missed by searching for query terms in only the
title and abstract fields. And I do realize I am sounding a bit harsh here and I know from a
lot of experience that there is no such thing as a perfect search. But I do want to refer you to
the latest best practices for reporting literature searching for systematic reviews which contains
some good recommendations, including having your search peer reviewed prior to running it:
Rethlefsen, M.L., Kirtley, S., Waffenschmidt, S. et al. PRISMA-S: an extension to the PRISMA
Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev 10, 39 (2021).
https://doi.org/10.1186/s13643-020-01542-z
ORIGINAL PEER REVIEW COMMENTS:
I see in your search queries that you searched terms about women but did not search for any
terms about men. I'm wondering why that decision was made. Perhaps it would have made more
sense to have just searched for terms about the military and injuries and then screened those
retrieved studies to see if they included sex/gender related data? Other possible terms that could
have been included in your search strategies could have been "sex" or "gender."
AUTHORS' RESPONSE:
Thank you for this comment. We conduct a lot of research in this field and on that basis recognise
that most of the military injury research is focused on men, and this is reflective of the fact that the
majority of military personnel are male. It is rare to find injury studies focused solely on female
military personnel and most studies that include women also include men. For our systematic
review and meta-analysis, we wished to ensure that we identified every available study that
had some focus on rates of injury in female military personnel, so they could be assessed for
eligibility for inclusion. We knew from past experience that injury studies comparing rates of
injury between the sexes would be identified based on their mention of involving women but
searching for studies involving men would yield a great many studies that did not compare rates
of injury between the sexes. It is for this reason we searched for studies using search terms
relating to women.
NEW QUESTIONS/COMMENTS
Thank you for providing me with more information about this. It totally makes sense. I am just
wondering if it is possible to add something about this to your paper. There may be readers of
this paper who may have the same questions that I did.