Advance Access Publication 18 January 2008 Review eCAM 2009;6(4)441–448 doi:10.1093/ecam/nem186 Evaluating the Quality of Randomized Controlled Trials that Examine the Efficacy of Natural Health Products: A Systematic Review of Critical Appraisal Instruments Anne Marie Whelan, Tannis M. Jurgens and Lindsay Lord College of Pharmacy, Dalhousie University, Dalhousie University, 5968 College Street, Halifax NS, Canada B3H 3J5 The purpose of this project was to conduct a systematic review to identify instruments designed to evaluate the quality of randomized controlled trials (RCTs) of natural health products (NHPs). Instruments were examined for inclusion of items assessing methods, identity and content of the NHP, generalizability of results and instructions for use. Online databases, websites, textbooks and reference lists were searched to identify instruments. Relevance assessment and data extraction of articles were completed by two investigators and disagreements were settled by the third investigator. Data were analyzed using descriptive statistics. Of the 4442 citations identified, 29 were potentially relevant with 16 meeting the criteria for inclusion. None of the instruments stated they were validated; content in the four areas of interest varied considerably. The most common items included randomization sequence generation (100%), blinding (100%), allocation concealment (75%) and participant flow (75%). Only nine of the NHP instruments included at least one item to appraise the specific content of the NHP. The CONSORT Statement for Herbal Interventions most closely addressed the four areas of interest; however, this instrument was specific for herbs. There is a need for the development of a validated instrument for assessment of the quality of RCTs that would be useful for herbs as well as other NHPs. Keywords: Checklists – herbs – quality assessment Introduction Natural sourced medicines, such as herbal products, have become increasingly popular among consumers who search for ‘natural’ ways to maintain their health. In Canada, natural sourced medicines are called natural health products (NHPs) and are classified by Health Canada to include all herbal remedies, traditional medicines, homeopathic medicines, probiotics, amino acids, essential fatty acids, vitamins and minerals that are appropriate for over the counter selection (1). For reprints and all correspondence: Dr Anne Marie Whelan, B.Sc. (Pharm), Pharm D, College of Pharmacy, Dalhousie University, 5968 College Street, Halifax NS, Canada B3H 3J5. Tel: 902-494-3503; Fax: 902-494-1396; E-mail: anne.marie.whelan@dal.ca Consumers often ask health care providers for guidance on the safe and effective use of NHPs. When providing recommendations to consumers, health care providers are accustomed to following principles of evidence based medicine, which includes, in part, con- sidering data from research reports that have been assessed on their quality of reporting of research methodology, design and analysis of data (2). Increasingly, randomized controlled trials (RCTs), considered the ‘gold standard’ when evaluating the efficacy of conventional medicines (3), are being used to evaluate the efficacy of NHPs (4). Therefore, it is essential for health care providers and their patients that reports of RCTs of NHPs be critically appraised with the same rigor as is applied to the assessment of the quality of RCTs of conventional medications. 2008 The Author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/ licenses/by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.