Improving Publication Standards for Psychotherapy Outcome Studies (2014)

The manifesto on salvaging psychotherapy research has much to commend it and I compliment Coyne and Kok for their unflinching criticism and proposed stringent solutions. Almost all the methodological, peer review and publication shortcomings they mention are legitimate concerns. In responding let me state that I am a part of the problem. For the past 25 years I have served as the Editor of a peer-reviewed journal titled Research on Social Work Practice (RSWP), produced by Sage Publications. I founded RSWP to serve as an outlet primarily for outcome studies on psychosocial interventions used by social workers, and attempted to model it after the prestigious Journal of Consulting and Clinical Psychology, with which I was very familiar. Clinical social workers are the largest provider of mental health services in the United States, surpassing the combined numbers of psychologists and psychiatrists (Hamp, Stamm, Christidis & Nigrinis, 2014). RSWP has been a success. For two of the past three years it has had the highest impact factor of all social work journals tabulated by the Journal Citation Reports, and it is received by over 8000 subscribers. Over 150,000 articles are downloaded from the journal's website each year. Yet RSWP regularly publishes the type of underpowered and methodologically weak outcome studies Coyne and Kok fulminate against! RSWP has the most rigorous submission requirements of all social work research journals, having adopted the APA's Journal Article Reporting Standards some years ago, shortly after they first appeared. The journal has already published an article describing the importance of registering randomized clinical trials in an appropriate registry (Harrison & Mayo-Wilson, 2014), and years ago we began using structured abstracts containing the subheadings of Purpose, Method, Results, and Discussion. Here are some additional specific editorial policies of RSWP which I have been planning for some time and will take effect in 2015, steps which will move the field of psychotherapy research forward along the lines suggested by Coyne & Kok.The journal will clearly state that the authors of RCTs should have pre- registered their study and provide a citation as to where this description can be located. RCT submissions must be accompanied by a completed CONSORT Checklist. Articles reporting the results of a quasi-experimental outcome study must follow the standards found in the Transparent Reporting of Evaluation Studies using Nonrandomized Designs (TREND) checklist. Analogous to CONSORT, the TREND checklist will greatly facilitate the timely peer-review of submissions and their transparency upon publication (see http://www.cdc.gov/trendstatement/). All nomothetic outcome reports must have a participant flow chart, depicting attrition at each stage of the study. All reports of statistically significant differences or changes much be accompanied by an appropriate effect size, and the meaning of this ES must be discussed. Causal inferences, if any, should be made conservatively and not go beyond the limits imposed by the presented methods and data. The authors of outcome studies evaluating nonpharmacological interventions (e.g., psychotherapies and other psychosocial treatments) are urged to familiarize themselves with relevant guidelines useful for reporting such studies. Grant et al. (2013) will be a recommended resource for authors to consult, as is Boutron, Ravaud and Moher (2012).Also effective for RSWP in 2015, articles claiming to be a Systematic Review must adhere to the guidelines for preparing systematic reviews developed by the Cochrane Collaboration (Higgins & Green, 2009) or the Campbell Collaboration (2014). In addition, the authors of systematic reviews and meta- analyses must follow the guidelines found in the PRISMA Statement (Preferred Reporting Items for Systematic Reviews and Meta-analyses), found at: http://www.prisma-statement.org/, and include a copy of a completed PRISMA Checklist with the submitted manuscript. …