Over-simplification and exclusion of non-conforming studies can demonstrate absence of effect: a lynching party?

A number of meta-analyses of cognitive behavioural therapy (CBT) in severe mental illness have been published by its advocates (Wykes et al. 2008), independent commentators (Jones et al. 2004) and now finally by its opponents (McKenna, 2001, 2006 ; Turkington & McKenna, 2003). Lynch et al. (2009) conclude that there is a small effect of CBT in severe depression but no effect in bipolar disorder and schizophrenia. In bipolar disorder, there have been relatively few studies and the largest reported a negative finding which has overshadowed promising earlier findings. This may be related to the treatment group selected in that study or the therapeutic intervention used. Lam et al. (2003) have described an adaptation of CBT, whereas the intervention in the larger study seems to have been more behavioural in form, concentrating on early intervention, treatment adherence and relapse prevention (Scott et al. 2006). The findings on schizophrenia are more interesting. Meta-analyses and reviews published so far have found favourably for CBT. Lynch et al. (2009) find differently. For relapse prevention, this may be because a large new study has produced negative findings in this area and also because of the use of studies that do not fit their inclusion criteria. For example, Hogarty’s Personal Therapy has very different origins and practice from CBT and was not described as such by him (Hogarty et al. 1997). Most importantly, the total exclusion of studies using hospitalization as a proxy for relapse led to a substantial underestimate of effect. Lynch et al. (2009) review studies which have an active control but wrongly conclude that such controls ‘ lacked any specific therapeutic effects ’. Befriending, for example, appears to have positive effects on delusions including paranoia but not hallucinations (Samarasekera et al. 2007). Active controls allow for differentiation of effects from the non-specific but very important effects of developing a relationship with patients. Studies comparing with treatment as usual are summarily dismissed but these do have relevance in assessing generalizability in effectiveness studies. Studies in this area have used different time scales and target symptoms and previous meta-analyses have made allowance for this. Lynch et al. (2009) do not, focusing only on end-of-treatment scoring. This has allowed them to claim that ‘perhaps the best study published to date . conducted by Sensky and colleagues ’ (Beck et al. 2008) was one which failed. It did not indeed show a difference on most measures at the treatment end-point – apart from suicidality (Bateman et al. 2007) – but such effects were apparent at the 9-month follow-up and maintained at 5 years (Turkington et al. 2008). Lynch et al. (2009) justify excluding these beneficial effects because some other studies did not show this despite others – not quoted – which did (Drury et al. 2000 ; Turkington et al. 2006). It is also disingenuous to hold up double-blind placebo trials of medication as if they were infallible : the effects of bias and other factors are still clearly a major concern as has emerged with evaluation of the latest generation of antipsychotics (Tyrer & Kendall, 2009). Blindness is rarely assessed in these trials. Yet the side-effects of drugs such as haloperidol compared with olanzapine and quetiapine are markedly different such that it would be expected that patient and rater would frequently be aware of which drug was * Address for correspondence : D. Kingdon, Professor of Mental Health Care Delivery, University of Southampton, Department of Psychiatry, Royal South Hants Hospital, Brintons Terrace, Southampton SO14 0YG, UK. (Email : d.kingdon@soton.ac.uk) Psychological Medicine (2010), 40, 25–27. f Cambridge University Press 2009 doi:10.1017/S0033291709990201 COMMENTARY