Developing and evaluating valid, reliable and usable measures of assessing competence in Cognitive Behavioural Therapy

Tools for measuring competence in delivering Cognitive Behavioural Therapy (CBT) provide a means of assessing the training of new CBT therapists and ensuring the quality of treatment provision within routine practice, provide a framework for delivering formative feedback, promote ongoing self-reflection, and are essential to establishing treatment integrity in research trials. As such, identifying an optimal strategy for assessing the competence with which CBT is delivered is crucial to the continued progression of the field. However, research in this area has been somewhat limited to date. Thus, there are at present no evidence-based best practice guidelines outlining the way CBT competence should be assessed. Furthermore, many of the assessment measures currently available have been widely criticised, indicating a need for improved tools for assessing CBT competence. To begin addressing this issue, the first two chapters of this thesis focus on reviewing and evaluating current assessment methods. Chapter one provides a systematic review of current methods of assessing CBT competence and chapter two outlines a qualitative exploration of experts’ understandings and experiences of assessing CBT competence. Findings from these studies provide tentative recommendations for practitioners and researchers assessing CBT competence. These initial studies also highlight ways in which the assessment of CBT competence could be improved and therefore provide a platform for guiding subsequent thesis chapters which focus on further developing existing assessment measures. Specifically, chapters three to six focus on the development and evaluation of a novel CBT competence rating scale: the Assessment of Core CBT Skills (ACCS). The ACCS builds upon currently available scales (especially the Cognitive Therapy Scale- Revised: CTS-R) to provide an assessment framework for assessors to deliver formative and summative feedback regarding therapists’ performance within observed CBT treatment sessions and for therapists to rate and reflect on their own performance. Development of the ACCS involved three key stages: 1- theory-driven scale development (chapter three), 2- an ‘expert’ review of the content validity, face validity, and usability of the scale (chapter four), and 3- an evaluation of the scale involving a pilot study examining its psychometric properties (chapter five) and a focus group examining its usability and utility (chapter six). Results from these studies indicate that the ACCS is a useful learning tool, is easy to use, has good psychometric properties, and offers an acceptable alternative to the CTS-R. Finally, chapter seven examines whether assessors require training in how to use the ACCS, concluding that simply reading the ACCS manual may be sufficient to achieve acceptable levels of reliability and usability. The results from the thesis are then drawn together in the final concluding comments in chapter eight, which discusses the findings within the broader context of the assessment of CBT competence.

Keywords: CBT
cognitive behavioural therapy
clinical skill
competence
assessment
Advisor: McManus, F
Rakovshik, S