Developing Guidelines Before, During, and After the COVID-19 Pandemic
Authors: Zachary Munn, Sara Twaddle, Duncan Service, Elaine Harrow, Patrick Mbah Okwen, Holger Schünemann, Per Olav Vandvik
Journal: Annals of Internal Medicine (2020)

Ideas and Opinions15 September 2020Developing Guidelines Before, During, and After the COVID-19 PandemicFREEZachary Munn, PhD, Sara Twaddle, MSc, PhD, Duncan Service, MA (Hons), DipLis, Elaine Harrow, MA, Patrick Mbah Okwen, MD, Holger Schünemann, MD, PhD, and Per Olav Vandvik, MD, PhDZachary Munn, PhDGuidelines International Network, Pitlochry, Scotland, and Joanna Briggs Institute, University of Adelaide, Adelaide, Australia (Z.M.)Search for more papers by this author, Sara Twaddle, MSc, PhDGuidelines International Network, Pitlochry, Scotland (S.T., E.H., P.O.V.)Search for more papers by this author, Duncan Service, MA (Hons), DipLisGuidelines International Network, Pitlochry, and Scottish Intercollegiate Guidelines Network, Edinburgh, Scotland (D.S.)Search for more papers by this author, Elaine Harrow, MAGuidelines International Network, Pitlochry, Scotland (S.T., E.H., P.O.V.)Search for more papers by this author, Patrick Mbah Okwen, MDGuidelines International Network, Pitlochry, Scotland, and Effective Basic Services for Africa (eBASE), Bamenda, Cameroon (P.M.O.)Search for more papers by this author, Holger Schünemann, MD, PhDGuidelines International Network, Pitlochry, Scotland, Michael G. DeGroote Cochrane Canada Centre, McMaster GRADE Centre, and McMaster University, Hamilton, Ontario, Canada, and University of Oslo, Oslo, Norway (H.S.)Search for more papers by this author, and Per Olav Vandvik, MD, PhDGuidelines International Network, Pitlochry, Scotland (S.T., E.H., P.O.V.)Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/M20-4907 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail The coronavirus disease 2019 (COVID-19) pandemic has resulted in unprecedented human, social, and economic impacts across the globe. It has prompted decision makers to demand evidence-based solutions to reduce the risk for transmission and the associated morbidity and mortality. Such demands have resulted in substantial challenges for all involved in the generation and translation of knowledge, including guideline developers.Guideline Development Before COVID-19Producing and updating trustworthy guidance is a deliberative process that requires substantial investment of time and resources and often takes years to complete. Well-accepted guidance and standards for creating and assessing the methodology used to develop trustworthy guidelines, including rapid guidelines (1–3), exist and are widely used.Despite this, the process of developing trustworthy recommendations has often been plagued with inefficiencies, redundancies, duplication, and a lack of collaboration and sharing (Figure, top) (4). Members of the guideline and evidence-based health care communities have been working for years to improve this poorly functioning evidence ecosystem, yet the underlying collaborations, tools, and processes for wholescale efficiencies in the translation of evidence into practice were not widely established when the COVID-19 pandemic began (4).Figure. The current poorly functioning evidence ecosystem.This figure shows the current inefficiencies at the various stages of the evidence ecosystem and the enhanced evidence ecosystem through coronavirus disease 2019 initiatives. EHR = electronic health record; HTA = health technology assessment; RCT = randomized controlled trial. (Reproduced with permission from MAGIC Evidence Ecosystem Foundation [www.magicevidence.org].) Download figure Download PowerPoint Challenges COVID-19 Has ProvidedThe sudden emergence and scale of the COVID-19 pandemic have led to substantial pressure and heightened expectations for accelerated systematic reviews and rapid guidelines. Researchers, systematic reviewers, and guideline developers have been tasked with resolving uncertainty in much shorter time frames, in an emerging field, and with types of evidence that they may not have used previously. The evidence base for COVID-19 is characterized by many studies and systematic reviews that are poorly designed and conducted (5), presenting numerous complications for guideline developers. Only a few high-quality studies exist that directly address the problem and could be used as the basis of guideline development work, and many recommendations have to rely on lower-quality evidence. The use of indirect evidence from other viral infections is controversial, and experts often disagree on its use, which exacerbates the problems.There has been duplication of effort, some of which is entirely wasteful (6); a proliferation of poorer-quality guidance; and omissions in focusing on structured processes for evidence assessment and guidance development (7, 8).Restrictions on the movement of individuals and pressure faced by those delivering health services have changed the way guidance is produced. Face-to-face meetings where evidence can be discussed and challenged and judgments made explicit are no longer possible, and many stakeholders are facing overwhelming demands on their time and attention. As a result, many guideline development processes have become remote, less diverse, and less rigorous (7).Solutions and Changes for the FutureDespite the enormous challenges that COVID-19 presents, the research, evidence, and guidance communities have reacted swiftly. Emerging work offers the potential to provide sustainable solutions to increasing efficiency and reducing duplication in guideline development, subject to ongoing rigorous evaluation. These innovations fall within a more trustworthy, efficient, and well-integrated evidence ecosystem (Figure, bottom) (4).The pandemic has resulted in a breakthrough in a long-standing quest for living evidence and guidance: the dynamic updating of systematic reviews and guideline recommendations in the face of new evidence (9) through the application of systematic and trustworthy methods. The Australian COVID-19 Clinical Evidence Taskforce (https://covid19evidence.net.au/#living-guidelines) revises and adds recommendations each week as priority clinical questions are identified and new research is synthesized. Where research is available to address priority questions, living systematic reviews are done as the basis of evidence summaries, and evidence-based recommendations are updated following GRADE (Grading of Recommendations Assessment, Development and Evaluation) methods using an electronic platform for authoring and publication. Similarly, the living World Health Organization guideline on drugs for COVID-19 demonstrates how global guidance is developed in a trusted collaboration, including trialists sharing their data early (10).Repositories and registries of guidelines are available, and many provide tailored resources for COVID-19. The ECRI Guidelines Trust (www.ecri.org/covid-19-clinical-guidelines) provides a centralized repository of evidence-based guidance developed by nationally and internationally recognized medical organizations and medical specialty societies; it includes quality assessment of guidelines using TRUST (Transparency and Rigor Using Standards of Trustworthiness) Scorecards. The Pan American Health Organization (https://covid19-evidence.paho.org) provides a searchable database of guidelines in English, Spanish, Portuguese, and French. The International Practice Guideline Registry (www.guidelines-registry.org) is available for developers to register prospective guidelines, and the Guidelines International Network library is being developed into both a registry and a repository.A new concept is recommendation mapping (RecMap); this will include a listing of quality-appraised individual recommendations on COVID-19 and tools to adapt recommendations to various contexts, which is required to ensure trustworthiness (https://covid19.evidenceprime.com).Although willingness to collaborate has increased, vested interests, bureaucracy, and inability to change remain limiting factors. Around the globe, organizations have set up networks, task forces, and working groups to coordinate efforts and overcome some of these challenges. The COVID-19 Evidence Network to support Decision-making (www.mcmasterforum.org/networks/covid-end) is a group of representatives from the communities of evidence synthesis, guideline development, and health technology assessment. They work together to support researchers, policymakers, and decision makers to find and use evidence to support decision making while improving coordination and reducing duplication in the development of evidence resources. The Guidelines International Network also fosters communications and advocates collaboration in guideline development.ConclusionMuch more work is still required to support the guideline development communities during COVID-19. Although the evidence maps, repositories, and collections of systematic reviews and guidelines are welcomed, some critical features are missing that would greatly assist with addressing challenges. Many of the entries in these repositories and maps have not been assessed for their risk of bias or trustworthiness, and it is difficult to identify the trustworthy information and guidance. To better foster reuse, adoption, adaptation, and development of recommendations, we need the ability and commitment to share the output of systematic reviews and guidelines in formats that will enable reuse and reproduction.The recent advances in collaboration, efficiencies in guideline development, and increased focus on and commitment to sharing and coordination must be maintained after the pandemic. These efforts should include representatives from across the globe—particularly low- and middle-income countries, which have both ongoing risk for new pandemics and wide experience dealing with past pandemics. Lessons must be learned, and solutions initiated must be sustained and usable in diverse settings; this will create preparedness for subsequent pandemics while also improving the inefficiencies in the evidence ecosystem into the future.References1. Qaseem A, Forland F, Macbeth F, et al; Board of Trustees of the Guidelines International Network. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156:525-31. doi:10.7326/0003-4819-156-7-201204030-00009 LinkGoogle Scholar2. Graham R, Mancher M, Wolman DM, et al, eds. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Institute of Medicine. Clinical Practice Guidelines We Can Trust. National Academies Pr; 2011. Google Scholar3. Morgan RL, Florez I, Falavigna M, et al. Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations. Health Res Policy Syst. 2018;16:63. [PMID: 30005679] doi:10.1186/s12961-018-0330-0 CrossrefMedlineGoogle Scholar4. Vandvik PO, Brandt L. Future of evidence ecosystem series: evidence ecosystems and learning health systems: why bother? J Clin Epidemiol. 2020;123:166-170. [PMID: 32145365] doi:10.1016/j.jclinepi.2020.02.008 CrossrefMedlineGoogle Scholar5. Glasziou PP, Sanders S, Hoffmann T. Waste in covid-19 research [Editorial]. BMJ. 2020;369:m1847. [PMID: 32398241] doi:10.1136/bmj.m1847 CrossrefMedlineGoogle Scholar6. Page MJ, Welch VA, Haddaway NR, et al. “One more time”: why replicating some syntheses of evidence relevant to COVID-19 makes sense. J Clin Epidemiol. 2020;125:179-182. [PMID: 32464320] doi:10.1016/j.jclinepi.2020.05.024 CrossrefMedlineGoogle Scholar7. Dagens A, Sigfrid L, Cai E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the covid-19 pandemic: rapid review. BMJ. 2020;369:m1936. [PMID: 32457027] doi:10.1136/bmj.m1936 CrossrefMedlineGoogle Scholar8. Schünemann HJ, Santesso N, Vist GE, et al. Using GRADE in situations of emergencies and urgencies: certainty in evidence and recommendations matters during the COVID-19 pandemic, now more than ever and no matter what. J Clin Epidemiol. 2020. [PMID: 32512187] doi:10.1016/j.jclinepi.2020.05.030 CrossrefMedlineGoogle Scholar9. Akl EA, Meerpohl JJ, Elliott J, et al; Living Systematic Review Network. Living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol. 2017;91:47-53. [PMID: 28911999] doi:10.1016/j.jclinepi.2017.08.009 CrossrefMedlineGoogle Scholar10. Lamontagne F, Agoritsas T, Macdonald H, et al. A living WHO guideline on drugs for covid-19. BMJ. 2020;370:m3379. [PMID: 32887691] doi:10.1136/bmj.m3379 CrossrefMedlineGoogle Scholar Comments0 CommentsSign In to Submit A Comment Boris BikbovIstituto di Ricerche Farmacologiche Mario Negri IRCCS (Bergamo, Italy)16 September 2020 Shortage of guidelines for management of vulnerable groups at the beginning of COVID-19 epidemic At the beginning of COVID-19 pandemic the major focus on guidelines development has been done on clinical management of patients. However, the management and communication guidelines play a crucial role in the definition of appropriate public health policy and protection of persons during epidemics or natural disasters. Regarding this, I would like to mention the results of a systematic review(1) of the literature (published up to April 28) that summarized only public health management and communication but not clinical aspects of COVID-19 infection. The vulnerable social groups were defined as children, persons with low health literacy, international migrant workers, international students, persons in assisted living and nursing homes, persons with intellectual or sensorial disabilities, refugees, undocumented migrants, homeless people, prisoners.(2) Sieving of data sources from PubMed and Google searches with full-text evaluation identified 36 relevant publications.(1) Among them, 21 represented "Brief communications" (letter to editor, short comment), 8 could be considered as "Extended articles" (original article, extended comment), 6 were "Guidance/plan of international organizations" (accounting to only international intergovernmental organizations [WHO, UNICEF, etc] but not to country-level or non-intergovernmental organizations), and only one could be considered as "Practice guidelines" (comprehensive guidelines produced by professional societies, national or international organizations). The most informative materials also included 11 web-resources that contain description of communication or management strategies for vulnerable groups. Results of this rapid systematic review performed at the beginning of the COVID-19 pandemic highlight an initial lack of practice guidelines and severe shortage of extended articles which could provide reliable information in the compact form and facilitate provision of care to vulnerable groups. Production of such guidelines focused on vulnerable group management and communication, or adaptation of previous guidelines, should be among priorities during current and possible future epidemics. References Bikbov B. Literature summary on vulnerable groups’ management and communication during COVID-19 epidemic. Open Sci Found. 2020; DOI: 10.31219/osf.io/mn7rd. https://doi.org/10.31219/osf.io/mn7rd Bikbov B, Bikbov A. Communication on COVID-19 to community – measures to prevent a second wave of epidemic. Open Sci Found. 2020; DOI: 10.31235/osf.io/ea9jm. https://doi.org/10.31219/osf.io/ea9jm Author, Article, and Disclosure InformationAuthors: Zachary Munn, PhD; Sara Twaddle, MSc, PhD; Duncan Service, MA (Hons), DipLis; Elaine Harrow, MA; Patrick Mbah Okwen, MD; Holger Schünemann, MD, PhD; Per Olav Vandvik, MD, PhDAffiliations: Guidelines International Network, Pitlochry, Scotland, and Joanna Briggs Institute, University of Adelaide, Adelaide, Australia (Z.M.)Guidelines International Network, Pitlochry, Scotland (S.T., E.H., P.O.V.)Guidelines International Network, Pitlochry, and Scottish Intercollegiate Guidelines Network, Edinburgh, Scotland (D.S.)Guidelines International Network, Pitlochry, Scotland, and Effective Basic Services for Africa (eBASE), Bamenda, Cameroon (P.M.O.)Guidelines International Network, Pitlochry, Scotland, Michael G. DeGroote Cochrane Canada Centre, McMaster GRADE Centre, and McMaster University, Hamilton, Ontario, Canada, and University of Oslo, Oslo, Norway (H.S.)Acknowledgment: The authors thank the Guidelines International Network membership for sharing examples of guidance and initiatives. They also thank Allison Smith for secretarial support.Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M20-4907.Corresponding Author: Zachary Munn, PhD, Joanna Briggs Institute, The University of Adelaide, Adelaide, SA 5005, Australia; e-mail, Zachary.Munn@adelaide.edu.au.Current Author Addresses: Dr. Munn: Joanna Briggs Institute, The University of Adelaide, Adelaide, SA 5005, Australia.Drs. Twaddle, Okwen, Schünemann, and Vandvik and Ms. Harrow: Guidelines International Network (G-I-N), c/o J & H Mitchell WS, 51 Atholl Road, Pitlochry PH16 5BU, Scotland.Mr. Service: Scottish Intercollegiate Guidelines Network, Gyle Square, 1 South Gyle Crescent, Edinburgh EH12 9EB, Scotland.Author Contributions: Conception and design: Z. Munn, S. Twaddle, D. Service, E. Harrow, P.M. Okwen, H. Schünemann, P.O. Vandvik.Analysis and interpretation of the data: D. Service, P.M. Okwen.Drafting of the article: Z. Munn, S. Twaddle, D. Service, P.M. Okwen, P.O. Vandvik.Critical revision of the article for important intellectual content: Z. Munn, S. Twaddle, P.M. Okwen, H. Schünemann, P.O. Vandvik.Final approval of the article: Z. Munn, S. Twaddle, D. Service, E. Harrow, P.M. Okwen, H. Schünemann, P.O. Vandvik.Administrative, technical, or logistic support: Z. Munn, E. Harrow.Collection and assembly of data: D. Service.This article was published at Annals.org on 15 September 2020. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoThe COVID-NMA Project: Building an Evidence Ecosystem for the COVID-19 Pandemic Isabelle Boutron , Anna Chaimani , Joerg J. Meerpohl , Asbjørn Hróbjartsson , Declan Devane , Gabriel Rada , David Tovey , Giacomo Grasselli , and Philippe Ravaud , for the COVID-NMA Consortium* Metrics Cited byEvidence surveillance for a living clinical guideline: Case study of the Australian stroke guidelinesEvaluation of the quality of COVID-19 guidance documents in anaesthesia using the Appraisal of Guidelines for Research and Evaluation II instrumentConversational System for Clinical Communication Training Supporting User-defined TasksThe Impact of the COVID-19 Pandemic on the Anxiety of Adolescents in QuébecMost published systematic reviews of remdesivir for COVID-19 were redundant and lacked currencyFrom “getting things right” to “getting things right now”: Developing COVID ‐19 guidance under time pressure and knowledge uncertaintyGuidance in the Ongoing StormA comparison and evaluation of international guidelines on the treatment of severe SARS-CoV-2 infectionA taxonomy and framework for identifying and developing actionable statements in guidelines suggests avoiding informal recommendationsQUALITY APPRAISAL OF CLINICAL PRACTICE GUIDELINES ON PROVISION OF DENTAL SERVICES DURING THE FIRST MONTHS OF THE COVID-19 PANDEMICThe CAPACITI Decision-Support Tool for National Immunization ProgramsCOVID-19 Pandemic—Frontline Experiences and Lessons Learned From a Tertiary Care Teaching Hospital at a Suburban Location of Southeastern IndiaClinical Practice Guidelines Registry: Toward Reducing Duplication, Improving Collaboration, and Increasing TransparencyYaolong Chen, MD, MSc, Gordon H. Guyatt, MD, MSc, Zachary Munn, PhD, Ivan D. Florez, MD, MSc, PhD, Ana Marušić, MD, PhD, Susan L. Norris, MD, MPH, MSc, Tamara Kredo, MBChB, MMed, PhD, and Amir Qaseem, MD, PhD, MHASystematic reviews on interventions for COVID-19 have rarely graded the certainty of the evidenceGuidelines rarely used GRADE and applied methods inconsistently: A methodological study of Australian guidelines 15 December 2020Volume 173, Issue 12Page: 1012-1014KeywordsCOVID-19Decision makingDisclosureElectronic medical recordsGrading of Recommendations Assessment Development and EvaluationMorbidityMortalityMotivationPrevention, policy, and public healthSystematic reviews ePublished: 15 September 2020 Issue Published: 15 December 2020 Copyright & PermissionsCopyright © 2020 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...