Can Assessors in a Psychotherapy Trial Be Successfully Blinded Analysis of a Randomized Controlled Trial on Psychotherapy for Refractory Insomnia in Residual Depression
Authors: Norio Watanabe, Toshi A. Furukawa, Shinji Shimodera, Fujika Katsuki, Hirokazu Fujita, Megumi Sasaki, Yousuke Suga, Kyoko Kakeda, Michael L. Perlis
Journal: Psychotherapy and Psychosomatics (2013)

Blinding is accepted to be a key element of treatment evaluation in randomized controlled trials. The recent systematic review of Baethge et al. [1] focuses on the proportion of recent randomized controlled trials that present a method of blinding procedures in psychiatric disorders. They report that only 4 of 78 papers in psychotherapy trials do so. It is considered to be more challenging to achieve successful blinding in psychotherapy trials than in pharmacotherapy trials [2]. A placebo pill can be easily prepared for pharmacotherapy trials, whereas in psychotherapy trials, a psychological placebo such as quasi-desensitization [3] is often used; however, the differences in the contents of the intervention and control arms may easily be perceived by the participants, and it is probably impossible to blind the participants or the therapists. To make the assessment in psychotherapy trials as blind as possible, objective measures are included among primary outcomes, and the raters are kept blinded to the group allocation. However, the degree of successful blinding and its associated factors in psychotherapy trials are rarely reported. We aimed to explore this by analyzing data in a randomized controlled trial on brief behavioral therapy for insomnia in depression [4]. The patients suffered from major depressive disorder diagnosed with DSM-IV and from significant insomnia, and were individually randomized to psychotherapy plus treatment as usual (TAU) or TAU alone. The psychotherapy involved four weekly individual sessions, each lasting approximately 50 min. The patients were assessed at baseline, and at 4 and at 8 weeks. An objective measure for depression, the 17-item GRID-HAMD [5], was evaluated through a face-to-face semistructured interview by raters at each assessment.The patients, the psychotherapists, or the physicians of TAU were not blind to allocation. However, all the six raters using the GRID-HAMD, all of whom were psychiatrists, were kept blind. The raters evaluated the patient upon request from a research coordinator. All patients were asked not to reveal their allocated treatment to the raters. After each assessment at 4 and 8 weeks, the rater filled in a questionnaire guessing which group the patient had been assigned to.The outcome of the present paper depended upon the agreement between the actual and the guessed allocations. The questionnaire also included questions about the degree of confidence in guessing (slightly, moderately, or definitely) and about cues for guessing including the atmosphere during the conversation with the patient, test results of the GRID-HAMD, behaviors of the research coordinator, just intuition or others, all of which were answered as 'yes' or 'no'. In addition, remission of insomnia defined by the total Insomnia Severity Index (ISI) score [6] and remission of depression defined by the total GRID-HAMD score [5] were included as possibly associated factors with agreement. The sample size was calculated a priori, based on a power analysis for the ISI scores.Agreement rates and kappa values between the actual and the guessed allocations were calculated. The χ2 statistic was used to test a difference of agreement rates between assessment at 4 weeks and that at 8 weeks, and between the active intervention and the control groups. To explore factors associated with agreement, the variables mentioned above were entered in logistic regression models. Multivariate models started from all variables with p < 0.25 in univariate analyses with the enter method using SPSS for Windows 21.0. p values were two-tailed, and statistical significance was defined by the alpha level of 0.05. The protocol was approved by the ethics committees of all the recruiting centers and registered in ClinicalTrials.gov (NCT00610259).Thirty-seven patients satisfied the eligibility criteria, with 20 participants assigned to receive the active treatment and 17 TAU alone. All the patients completed all the study assessments at 4 and 8 weeks. Thus, 74 questionnaires were collected from the raters without any missing data.The agreement rates and kappa values for a whole sample were 63.5% and 0.28 (95% CI 0.08-0.49). Dividing the sample regarding the time point of assessment, these values were 56.7% and 0.15 (0.00-0.45) at 4 weeks and 70.2% and 0.41 (0.13-0.69) at 8 weeks, respectively (p = 0.227). The agreement rate was higher in the TAU at 76.5% than in the psychotherapy groups at 52.5% (p = 0.033). Regarding factors associated with agreement in the multivariate analyses, only the intuition of the raters was significant with an OR of 0.23 (95% CI 0.07-0.77, p = 0.018, table 1), which meant that only intuition led to incorrect guessing. The other factors led neither to correct nor to incorrect guessing. The agreement rate of the present study at 63.5% seems very similar to that described in the recent review at 62% [1]. The kappa value at 0.28 has led us to conclude that blinding was successful.In psychiatry trials, there are several cases where blindness was not fully maintained [1]. Successful blinding in our study may be due to employing not pharmacotherapy but psychotherapy as the active intervention. In pharmacotherapy trials, taste or smell, or the emergence of adverse events may be different between active and placebo pills, and this may lead to a failure in blinding [7]. Another reason for successful blinding in the study may include a massive effort by research coordinators to keep assessors blind.The study is not without some methodological limitations. First, previous studies recommended that subjects should be allowed to express uncertainty ('do not know') when guessing treatment assignment [8,9] so that they do not merely hazard a guess. We did not include this option; however, the degree of confidence in guessing was not associated with agreement. We believe that we have successfully avoided what we consider to be a hazard. Second, the sample sizes were relatively small and concerns about the generalizability of the results may be raised. Further replication studies with a larger sample are needed to evaluate the outcomes with more confidence. On the other hand, the strength of our study is that no data for our analyses are missing and that it investigates the certainty of and cues for guessing as previously recommended [8].In conclusion, raters in a psychotherapy trial can be successfully blinded, when a procedure is carefully planned and sufficient effort in conducting it is made.We thank all the assessors and the therapists (Nao Shiraishi, MD, Yoshie Murata, MSc, Jun Mayumi, MD, Mako Morikawa, MD, and Yumi Nakano, MD, PhD at Nagoya City University Graduate School of Medical Sciences, and Ryosuke Fujito, MD at Kochi Medical School) and the administrative staff (Kozue Maki at Nagoya City University Graduate School of Medical Sciences) for their contribution to this study. We also thank Mitsuhiko Yamada, MD, PhD, Naohiro Yonemoto, MPH, and Masatoshi Inagaki, MD, PhD at National Center of Neurology and Psychiatry, Japan for their advice regarding the design and statistical issues, and Izuru Miyoshi, MD, PhD at National Center of Neurology and Psychiatry for providing the central allocation.This study was funded by a Grant-in-Aid for Scientific Research (No. 19230201) from the Ministry of Health, Labor and Welfare, Japan. The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data, and preparation, review, or approval of the manuscript.N.W. received research funds from the Japanese Ministry of Health Labor and Welfare and the Japanese Ministry of Education, Science, and Technology. He also received lecture fees and research funds from Asahi Kasei, Dai-Nippon Sumitomo, Eli Lilly, GlaxoSmithKline, Janssen, Otsuka, Pfizer and Schering-Plough. T.A.F. received honoraria for speaking at CME meetings sponsored by Astellas, Dai-Nippon Sumitomo, Eli Lilly, GlaxoSmithKline, Janssen, Kyorin, MDS, Meiji, Otsuka, Pfizer, Shionogi, and Yoshitomi. He is on the advisory board for Sekisui Chemicals and Takeda Science Foundation. He received royalties from Igaku-Shoin, Seiwa-Shoten, Nihon Bunka Kagaku-sha and American Psychiatric Publication. The Japanese Ministry of Education, Science, and Technology and the Japanese Ministry of Health, Labor, and Welfare funded his research. S.S. received research funds from the Japanese Ministry of Education, Science, and Technology. He also received lecture fees from Asahi Kasei, Dai-Nippon Sumitomo, GlaxoSmithKline, Astellas, Eli Lilly, Janssen, Meiji, Otsuka, Pfizer, and Schering-Plough. All the other authors report no conflicts of interest.