4 22 23 24 25 26 27 Item number Results Results Paper section Results Discussion Discussion Discussion Evidence-Based Complementary and Alternative Medicine Table 1: Continued. For each primary and (if applicable) secondary outcome, is (a) a summary of results presented for each group, (b) the estimated eﬀect size presented for each between-group diﬀerence (e.g., SD), and (c) the precision of the eﬀect size presented for each between-group diﬀerence (e.g., conﬁdence interval (CI))? If additional subgroup analyses and/or adjusted analyses are reported, is it stated whether they were prespeciﬁed or exploratory, that is, not prespeciﬁed? Question Are all important adverse events or side eﬀects presented for each intervention group? Is an interpretation of the results presented that takes into account (a) study hypotheses, (b) sources of potential bias or imprecision, and (c) the potential dangers associated with multiple analyses and outcomes? Is the generalizability (external validity) of the trial ﬁndings discussed? Is a general interpretation of the results presented, in the context of current evidence? Instructions. OCSI evaluates how well an “item” is reported, not whether it was appropriate or adequate. When scoring each question, consider the following. (i) If you were a reviewer of the paper, would you be satisﬁed with what is reported? (ii) If you were attempting to reproduce the ﬁndings, is there suﬃcient reporting of details to allow you to do so? Note. Items 4–9 from STRICTA (MacPherson et al., 2002) substitute for item 4 of CONSORT (Altman et al., 2001) [17, 18]. as educational, behavioral, physical, or pharmaceutical. RCTs were excluded if acupuncture points were stimulated by means other than ﬁliform needles, for example, ear tacks, intradermal needles, TENS, or laser. Databases that were searched to identify articles included MEDLINE, the Cochrane Central Register of Controlled Trials, Alt HealthWatch, AMED, University of Maryland CAMPAIN, and the Oregon College of Oriental Medicine library database, which includes 16 non-MEDLINE journals of acupuncture and Oriental medicine. In addition, hand searches were performed of the reference lists from the WHO report on controlled clinical trials of acupuncture [37] and from 71 systematic reviews of acupuncture published between October 1997 and October 2007. 2.3. Application of OCSI. Seven individuals with experience in acupuncture research formed the OCSI group. Two members served as alternates, ensuring a group of 5 OCSI- raters over the course of the project. The group comprised a diverse range of backgrounds, all with experience in assessing RCTs of acupuncture, including three licensed acupunctur- ists, two nonpractitioners, a medical acupuncturist, and an acupuncture student. As a means of eliminating systematic bias, it was initially suggested to blind reviewers of RCTs to article information such as journal, author, and publication year [11]. An attempt to validate this recommendation was unsuccessful [38], and a recent assessment in acupuncture trials demonstrated limited diﬀerence in outcomes [34]. Based on these results, the OCSI Group was not blinded to article information. From the RCTs that met the selection criteria, eight arti- cles were initially chosen at random for scoring by the OCSI group (each of ﬁve raters) to assess face validity of the OCSI questions. Several group meetings were held to compare raters’ scores per question on each of the eight articles and to seek consensus on scoring. The latter process included devel- opment of an OCSI manual that outlines criteria for scoring each question as yes, partial, no, or (Appendix A). Following the consensus process, the remaining articles were randomly distributed among ﬁve raters for individual scoring. Each article’s score was divided by the total possible score (2 × [27 – n questions scored N/A]) and converted to percent. Scores for each OCSI question were entered into a database from which total OCSI scores per article were calculated. A second database contained extracted demographics from each article (e.g., country of origin, journal, year, condition). Results for the ﬁrst eight articles were group consensus scores; results for the remaining articles were from single raters. 2.4. Statistics. For descriptive purposes, we computed and displayed averages of the individual question scores. For select questions, regression analysis was performed, and binominal 95% conﬁdence intervals were computed for change of reporting over time. The distribution of all RCTs across OCSI score by decile is presented as a histogram. Interrater agreement was assessed at a point in the study when low, middle, and high scoring articles could be dis- cerned. Nine RCTs were randomly selected, three from each of these categories, in order to estimate agreement across the spectrum of quality. Conventional intraclass coeﬃcients (ICCs) were computed from results with ﬁve reviewers each rating all nine articles. Since China contributed the largest number of articles and had the lowest mean OCSI score by country, we plotted the mean OSCI scores by year for all countries, for China alone, and for all countries other than China, and ﬁtted linear trends. Regression analysis was initially performed for all countries combined. A second regression analysis included a binary variable for China or all others as a covariate.