eCAM 2009;6(4) 445 s m e t I 1. Title and Abstract 2. Background 3. Participants 4a. Herbal Medicinal Product Name 4b. Characteristics of Herbal Product 4c. Dosage Regimen and Quantitative Description 4d. Qualitative Testing 4e. Placebo/Control Group 4f. Practitioner 5. Objectives 6. Outcomes 7. Sample Size 8. Sequence Generation 9. Allocation Concealment 10. Implementation 11a. Blinding (Masking) 11b. Success of Blinding 12. Statistical Methods 13. Participant Flow 14. Recruitment 15. Baseline Data 16. Numbers Analyzed 17. Outcomes and Estimation 18. Ancillary Analysis 19. Adverse Effects 20. Interpretation 21. Generalizability 22. Overall Evidence 0 2 4 6 8 10 12 14 16 # of Instruments Checklists Numeric scales Rating Scales Combined scale & checklist Figure 2. Frequency of items included in instruments for assessing quality of RCTs of NHPs. content of some NHPs can vary significantly between preparations, thus affecting the pharmacological effects of the product (9,10). Most importantly, documenting the detailed identity of the NHP used in the RCT allows an interested person to select a product of similar content and be able to expect similar outcomes. Since there did not appear to be a gold standard appraisal instrument for RCTs of NHPs, it was important to determine how instruments were developed and how items were chosen for inclusion in instruments. Approximately 75% of the articles that were identified provided information about how the instrument was developed. Of those who reported their development techniques, almost 70% were adapted or modified from existing instruments. The instrument that was published by Jadad et al. (40). was most commonly used as a template for the development of other instruments. Some of the references cited were used to develop only one other instrument. This may suggest that authors are selecting their preferred existing instrument and then modifying it to meet their needs. However, in the new instruments developed from previous instruments, there was little specific information provided as to why an item was included/excluded or how/why the item was modified. Thus there appeared to be minimal empirical basis for the development of many of the instruments. Although none of the instruments stated that they were validated, six of the instruments were adapted from the validated instrument published by Jadad et al. (40). However, in most cases, it was the non-validated instruments that were used as a starting point for the development of new instruments, meaning that potential problems with existing instruments are being perpetuated in the newly created instruments. It was discouraging to see that only 25% of the identified instruments included some instructions for use. Guidance in using the instrument as the developers intended should provide some consistency in the interpretation of items and therefore less variability in results obtained from the use of the instrument. Strengths and Limitations To our knowledge this is the first article to systematically review the literature to identify instruments for the critical appraisal of RCTs evaluating efficacy of NHPs.