E-SMART: Examining Strategy Monitoring and Remediation Training for Schizophrenia
Phase: NA
Conditions: Psychotic Disorders, Executive Dysfunction
Sponsor: University of Toronto

 
STUDY PROTOCOL: COMPUTERIZED PRACTICE VS. STRATEGY 
MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
Version 1 – May 12, 2020 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Principal Investigator:  
Michael W. Best, Ph.D. 
Assistant Professor 
Departments of Psychology and Psychological Clinical Science 
University of Toronto Scarborough  
 

RESEARCH STUDY PROTOCOL 
 
 Page 1 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
Table of Contents 
STUDY INTRODUCTION ............................................................................................................................... 2 
Background ............................................................................................................................................. 2 
Justification for Research ........................................................................................................................ 2 
Purpose and Hypotheses ........................................................................................................................ 3 
STUDY DESIGN ............................................................................................................................................ 3 
Sample Size/Subject Selection ................................................................................................................ 3 
Inclusion Criteria ..................................................................................................................................... 3 
Exclusion Criteria..................................................................................................................................... 3 
Method of Recruitment .......................................................................................................................... 3 
Limitation of the study ............................................................................................................................ 3 
Informed Consent ................................................................................................................................... 3 
STUDY PROCEDURE ..................................................................................................................................... 3 
Randomization (if applicable) ................................................................................................................. 3 
Study Intervention/Standard of Care ..................................................................................................... 4 
Primary Outcome Measures ................................................................................................................... 4 
Visits and Rationale ................................................................................................................................. 5 
Data Collection, Storage, Privacy, Destruction ....................................................................................... 5 
STUDY ANALYSIS ......................................................................................................................................... 5 
Sample size Statistics .............................................................................................................................. 5 
Statistical Plan ......................................................................................................................................... 6 
STUDY ETHICS ............................................................................................................................................. 6 
Potential Benefits .................................................................................................................................... 6 
Potential Risks ......................................................................................................................................... 6 
Safety Provisions ..................................................................................................................................... 6 
Ethics Approval ....................................................................................................................................... 6 
Dissemination ......................................................................................................................................... 6 
Statement of Right to withdraw from the study .................................................................................... 7 
REFERENCES ................................................................................................................................................ 7 
 
 
 

RESEARCH STUDY PROTOCOL 
 
 Page 2 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
STUDY INTRODUCTION 
Background 
Schizophrenia-spectrum disorders are the most persistent, debilitating, and economically 
burdensome mental illnesses worldwide, and are associated with the greatest per-patient expense 
of all mental health conditions.[1] Schizophrenia is associated with a 15-20 year decrease in life 
expectancy, 5-fold increase in likelihood of death by suicide,[2] and a significant decrease in quality 
of life.[3] Antipsychotic medications are the first line treatment for individuals with schizophrenia-
spectrum disorders and are prescribed to nearly every service-user. However, in the Clinical 
Antipsychotic Trials of Intervention Effectiveness (CATIE) trial (one of the largest antipsychotic trials 
in 1493 individuals with schizophrenia), medication effects on psychosocial functioning were small 
(d = 0.25).[4] Thus, the primary treatment available to all individuals with schizophrenia does little to 
improve community functioning. This may partially be a result of the limited efficacy of 
antipsychotic medication to improve neurocognitive abilities, widely recognized as a core feature of 
schizophrenia, and one recommendation stemming from the CATIE trial was that “more intensive 
psychosocial rehabilitative services, including cognitive rehabilitation, may be needed to affect 
more substantial gains in functioning.”[4]  
Cognitive remediation (CR) is a psychological intervention based on principles of learning and 
neuroplasticity to improve neurocognitive abilities with the ultimate goal of improving community 
functioning. The neuroplastic effects of CR are well established with evidence for increased gray 
matter volume in the hippocampus and amygdala,[5] increased activation of the medial prefrontal 
cortex,[6] and increased amplitude of the mismatch negativity event-related potential[7] following 
CR. In two recent randomized controlled trials (RCTs), I also demonstrated that CR improves 
synchronization of neural networks in the alpha and theta frequency bands.[7,8] Meta-analyses 
support moderate transfer of these neurophysiological improvements to neurocognitive abilities (d 
= 0.45) and community functioning (d = 0.37).[9] In a recent systematic review, I reported that CR 
approaches vary widely, but approaches that incorporate training of executive functions are 
generally the most effective.[10] Based on these findings I developed a novel CR intervention 
specifically targeting executive functions and conducted two double-blind RCTs, in which targeted 
executive function training (ET) produced greater improvements in neurophysiology, 
neurocognition, functional skills and real-world community functioning compared to other leading 
forms of CR.[7,8] This intervention is approximately half the duration of other CR programs, yet 
produces larger effect size improvements in community functioning.  
Justification for Research 
In order to further increase the efficiency and effectiveness of ET it is necessary to determine 
the active component of treatment. There are two primary components of ET and neither have ever 
been examined in isolation: 1) computerized cognitive training, and 2) strategy development. It is 
unclear whether both components are necessary to stimulate neurophysiological changes and 
improve functioning or whether one component drives the observed treatment effects. Dismantling 
ET will provide critical information regarding whether a single component is necessary or whether it 
is the interaction of these components that results in effective CR. 

RESEARCH STUDY PROTOCOL 
 
 Page 3 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
This will be the first study to dismantle CR and examine components necessary to effectively 
improve neurocognition and functioning for individuals with schizophrenia. These findings will 
directly inform methods of improving CR efficacy. 
 
Purpose and Hypotheses 
The primary goal of this study is to: 
(1) Examine the efficacy of computerized training alone, compared to strategy monitoring alone, 
compared to full ET on neurocognition, and functioning. 
STUDY DESIGN 
Sample Size/Subject Selection 
90 participants with schizophrenia-spectrum disorders will be recruited. 
Inclusion Criteria 
The target population for this study are those who meet the criteria for schizophrenia based on 
the DSM-V diagnosis. The inclusion criteria is anyone who meets the criteria of schizophrenia, 
schizoaffective disorder or any other psychotic disorder, are also 18-65 years of age, know how to 
use a computer, are not abusing drugs or alcohol and can read and speak English. 
Exclusion Criteria 
Exclusion criteria include anyone enrolled in a cognitive training program in the last 6 months, 
anyone with a neurological disease or neurological damage, medical illnesses that can change 
neurocognitive function, medical history of head injury with loss of consciousness and those with 
physical handicaps. 
Method of Recruitment 
The participants will be recruited from various sources including mental health agencies and 
referrals from mental health providers. 
Limitation of the study  
The primary limitation of the study will be attrition if participants do not complete the study or 
the follow-up assessment.  
Informed Consent 
A consent form will be given to all participants to sign. The language of the consent form will be 
written at an elementary school level. A research assistant will read out the consent form to 
participants and will clarify anything and answer any participant questions.  They will also ask the 
participant questions during the consenting process to ensure that the participant understands. 
Participants will also be reminded that it is their right to refuse to participate or withdraw from the 
study and that doing so will not affect their existing treatments or services. 
STUDY PROCEDURE 
Randomization 
 
Participants will be randomized to receive Computerized Cognitive Training only, 
Strategy Development only, or both (ET). 

RESEARCH STUDY PROTOCOL 
 
 Page 4 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
Study Intervention/Standard of Care 
All interventions will involve 4 weeks of group treatment consisting of two 1-hour group 
sessions per week and additional practice at home between sessions. The Executive Training 
condition will consist of the ET intervention that I previously developed and evaluated.[7] ET sessions 
consist of 50% of the session practicing computerized cognitive training exercises, and 50% of the 
session developing cognitive strategies to use in the computerized exercises. Participants are 
encouraged to complete 40 minutes of computerized training per day, and complete strategy 
worksheets, at home between sessions. In Computerized Cognitive Training only participants will 
spend the entire one-hour session practicing computerized training exercises. Between sessions 
participants will be encouraged to practice the computerized exercises at home for 40 minutes per 
day. There will be no strategy development in this condition. In Strategy Development only 
participants will engage in cognitive strategy discussions to develop new executive function 
strategies that can be used in daily life. Between sessions, participants will be encouraged to 
practice their cognitive strategies in their daily life and track their strategies using the strategy 
worksheet. There will be no computerized cognitive training in this condition. All interventions will 
be delivered virtually in the participant’s home and group sessions will be conducted using the 
online platform Zoom. 
Outcome Measures 
The primary outcome measure is community functioning measured using the Specific Levels of 
Functioning scale which will be rated by a participant’s case manager who is familiar with their 
everyday activities and independent from the trial. I have successfully used this method of 
objectively measuring functioning in several previous studies.[7,8]  
The secondary outcome measures include the Cambridge Neuropsychological Test Automated 
Battery (CANTAB), the Virtual Reality Functional Capacity Assessment Tool (VRFCAT), the Wide 
Range Achievement Test (WRAT), the Questionnaire About the Process of Recovery (QPR), the Brief 
Psychiatric Rating Scale (BPRS), the Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-
Q), the Dysfunctional Attitudes Scale (DAS), the Brief Core Schema Scale (BCSS), the Generalized 
Self-Efficacy Scale (GSES), the Cognitive Failures Questionnaire (CFQ), the Need for Cognition Scale 
(NCS), the Davos Assessment of Cognitive Biases (DACOBS), and the Motivation and Pleasure Scale - 
Self-Report (MAP-SR).[11-23] 
The CANTAB is a battery consisting of highly sensitive, precise and objective measures of 
cognitive function. It includes tests of working memory, learning and executive function; visual, 
verbal and episodic memory; attention, information processing and reaction time; social and 
emotion recognition, decision making and response control. The VRFCAT is a measure of functional 
capacity that simulates activities of daily living in a virtual environment and improves clinical trials 
by detecting functionally meaningful improvements in patients’ everyday lives. The WRAT is an 
academic skills assessment which measures reading skills, math skills, spelling, and comprehension. 
The QPR was developed from service users’ accounts of recovery from psychosis in collaboration 
with local service users. It asks people living with psychosis about aspects of recovery that are 
meaningful to them, and is strongly associated with general psychological wellbeing, quality of life 
and empowerment. The BPRS measures psychopathology and symptom severity and is sensitive to 
changes in symptom levels. The Q-LES-Q is a sensitive measure of the degree of enjoyment and 
satisfaction experienced by subjects in various areas of daily functioning. The DAS measures self-

RESEARCH STUDY PROTOCOL 
 
 Page 5 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
defeating attitudes theorized to underlie clinical depression and anxiety. The BCSS assesses four 
dimensions of self and other evaluation: negative-self, positive-self, negative-other, and positive-
other. The GSES assesses optimistic self-beliefs to cope with a variety of difficult demands in life. 
The CFQ was designed to measure perception, memory, and motor lapses in daily life. The NCS 
measures “the tendency for an individual to engage in and enjoy thinking." The DAVOS measures 
cognitive biases and discriminates between schizophrenia spectrum patients and normal control 
subjects. The MAP-SR assesses the motivation and pleasure domains of negative symptoms. 
Visits and Rationale 
Visits: All interventions will involve 4 weeks of group treatment consisting of two 1-hour group 
sessions per week and additional practice at home between sessions. 
Rationale: In order to further increase the efficiency and effectiveness of ET it is necessary to 
determine the active component of treatment. It is unclear whether both components are 
necessary to stimulate neurophysiological changes and improve functioning or whether one 
component drives the observed treatment effects. Dismantling ET will provide critical information 
regarding whether a single component is necessary or whether it is the interaction of these 
components that results in effective CR. Therefore, the proposed number and length of sessions are 
justifiable because the proposed number and length of sessions are congruent with ET and will 
allow the present research questions to be addressed. 
Data Collection, Storage, Privacy, Destruction 
Privacy: Information provided will not have any identifying features on it. A random code will 
be assigned to each participant and those numbers will be used to refer to a participant's data. The 
document linking names to codes will be kept in Dr. Best’s locked laboratory on a password 
protected computer. 
Limits to Confidentiality: Limitations that will require confidentiality to be broken include: if a 
research member is aware of child abuse where the child might still be a risk, if a participant 
expresses that they are risk of harming themselves or others, and if a participant expresses that 
another health professional has engaged in sexual abuse.  
Data Storage: All de-identified forms will be kept in a locked cabinet where only members of 
the research team have access to it. Information kept on a computer will be password protected. All 
filing cabinets and computers will be located in Dr. Best's locked laboratory that only research staff 
have access to. All research staff receive data protection and confidentiality training and sign a 
confidentiality form. 
Data Retention/Destruction: De-identified data will be kept for seven years before being 
destroyed. Only members of Dr. Best's lab will have access to the data. 
STUDY ANALYSIS 
Sample size Statistics 
90 participants with schizophrenia-spectrum disorders will be recruited. Power analyses, 
conducted with GPower, indicate that 90 participants (30 per treatment condition), accounting for 
an upper limit of 25% attrition observed in my previous trials of ET,[7,8] provides 80% power to 
detect a medium effect size (cohen’s f = 0.2) difference between conditions. 

RESEARCH STUDY PROTOCOL 
 
 Page 6 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
Statistical Plan 
Primary and secondary outcomes will be examined using Linear Mixed Models on the Intent-to-
Treat sample with missing data interpolated using maximum likelihood estimation. The primary 
endpoint is the 3-month follow-up assessment, and secondary endpoint of post-treatment will also 
be examined. 
STUDY ETHICS 
Potential Benefits 
The direct benefit to the participant includes improvements in attention, memory and problem 
solving, caused by the treatment. Also, if the results are released to the participant's clinical 
treatment team, they can use it to help with the ongoing care. There is also a potential benefit to 
the scientific community because the results may help in understanding individual differences. 
Potential Risks 
Psychological/emotional risks may arise during assessment procedures due to frustration or 
embarrassment during the neurocognitive assessment trials. Participants may feel uncomfortable 
discussing personal topics related to their symptoms or they may become tired. These risks are 
equal to those that occur within the clinical contest. Participants will be reminded that they are able 
to stop the interview at any time to take a break or to postpone the procedure to a different day. 
Participants will also be given the phone number of a mental health professional if discomfort does 
occur. 
Safety Provisions 
All therapists and lab staff will be trained in suicide risk assessment procedures. Therapists will 
follow established protocols to conduct a risk assessment if there is any indication of suicide risk. 
Therapists will create a safety plan with the participant if risk is low or moderate. If risk is high, then 
therapists will collaboratively discuss the risk with the participant and escort participants to the 
emergency department at the respective hospital site (CAMH, Ontario Shores, or Scarborough 
Health Network). If a risk assessment is required during an assessment, one of the assessors will 
conduct an initial assessment with the participant. If risk is above minimal, assessor will immediately 
consult with Dr. Best on the best course of action before continuing the assessment. If risk is 
determined to be high then the participant will be collaboratively escorted to the emergency 
department. 
During the consenting process participants will be asked comprehension questions about the 
consent form to ensure participants are aware of what they are consenting to. 
Ethics Approval 
This protocol is currently under review at the University of Toronto Research Ethics Board. 
Approval from the University of Toronto REB will be forwarded to Ontario Shores Centre for Mental 
Health Sciences upon approval. 
Dissemination 
Results from this research will be presented at academic conferences and submitted for 
publication at peer-reviewed journals. 

RESEARCH STUDY PROTOCOL 
 
 Page 7 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
Statement of Right to withdraw from the study 
Participants can withdraw from the study at any time. If participants decide that they would like 
to withdraw their data from the study, then their data will be destroyed. 
REFERENCES 
[1] Wu, E. Q. et al. The Economic Burden of Schizophrenia in the United States in 2002. J. Clin. 
Psychiatry 66, 1122–1129 (2005). 
 
[2] Palmer, B. A., Pankratz, V. S. & Bostwick, J. M. The Lifetime Risk of Suicide in Schizophrenia: A 
Reexamination. Arch Gen Psychiatry 62, 247–253 (2005). 
 
[3] Eack, S. M. & Newhill, C. E. Psychiatric Symptoms and Quality of Life in Schizophrenia: A Meta-
Analysis. Schizophr. Bull. 33, 1225–1237 (2007). 
 
[4] Swartz, M. et al. Effects of Antipsychotic Medications on Psychosocial Functioning in 
Patients with Chronic Schizophrenia: Findings from the NIMH CATIE Study. Am J 
Psychiatry 164, 428–436 (2007). 
 
[5] Eack, S. M. et al. Neuroprotective Effects of Cognitive Enhancement Therapy Against Gray Matter 
Loss in Early Schizophrenia. Arch Gen Psychiatry 67, 674–682 (2010). 
 
[6] Subramaniam, K. et al. Computerized cognitive training restores neural activity within the reality 
monitoring network in schizophrenia. Neuron 73, 842–853 (2012). 
 
[7] Best, M. W., Milanovic, M., Iftene, F. & Bowie, C. R. A randomized controlled trial of executive 
functioning training compared with perceptual training for schizophrenia spectrum disorders: 
Effects on neurophysiology, neurocognition, and functioning. Am. J. Psychiatry 176, 297–306 (2019). 
 
[8] Best, M. W., Gale, D., Tran, T., Haque, M. K. & Bowie, C. R. Brief executive function training for 
individuals with severe mental illness : Effects on EEG synchronization and executive functioning. 
Schizophr. Res. 203, 32–40 (2019). 
 
[9] Wykes, T. & Spaulding, W. D. Thinking about the future cognitive remediation therapy - what 
works and could we do better? Schizophr. Bull. 37, s80–s90 (2011). 
 
[10] Best, M. W. & Bowie, C. R. A review of cognitive remediation approaches for schizophrenia: 
from top-down to bottom-up, brain training to psychotherapy. Expert Rev. Neurother. 17, 713–723 
(2017). 
 
[11] CANTAB® [Cognitive assessment software]. Cambridge Cognition (2019). All rights reserved. 
www.cantab.com 
 

RESEARCH STUDY PROTOCOL 
 
 Page 8 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
[12] Ruse, S. A., Harvey, P. D., Davis, V. G., Atkins, A. S., Fox, K. H., & Keefe, R. S. (2014). Virtual 
reality functional capacity assessment in schizophrenia: Preliminary data regarding feasibility and 
correlations with cognitive and functional capacity performance. Schizophrenia Research: Cognition, 
1(1), e21–e26. 
 
[13] Wilkinson, G. S., & Robertson, G. J. (2017). WRAT-5: Wide Range Achievement Test Professional 
Manual. Psychological Assessment Resources. https://books.google.ca/books?id=amQXtAEACAAJ 
 
[14] Neil, S. T., Kilbride, M., Pitt, L., Nothard, S., Welford, M., Sellwood, W., & Morrison, A. P. (2009). 
The questionnaire about the process of recovery (QPR): A measurement tool developed in 
collaboration with service users. Psychosis, 1(2), 145–155. 
https://doi.org/10.1080/17522430902913450 
 
[15] Faustman, W. O., & Overall, J. E. (1999). Brief Psychiatric Rating Scale. In M. E. Maruish (Ed.), 
The use of psychological testing for treatment planning and outcomes assessment (p. 791–830). 
Lawrence Erlbaum Associates Publishers. 
 
[16] Endicott, J., Nee, J., Harrison, W., & Blumenthal, R. (1993). Quality of life enjoyment and 
satisfaction questionnaire: A new measure. Psychopharmacology Bulletin, 29(2), 321–326. 
 
[17] Weissman AN, Beck AT. Development and validation of the dysfunctional Attitude Scale: A 
preliminary investigation. Paper presented at the Association for the Advancement of Behavior 
Therapy; Chicago. 1978. 
 
[18] Fowler, D., Freeman, D., Smith, B. E. N., Kuipers, E., Bebbington, P., Bashforth, H., ... & Garety, 
P. (2006). The Brief Core Schema Scales (BCSS): psychometric properties and associations with 
paranoia and grandiosity in non-clinical and psychosis samples. Psychol Med, 36(6), 749-759. 
 
[19] Schwarzer, R., & Jerusalem, M. (1995). Generalized Self-Efficacy scale. In J. Weinman, S. Wright, 
& M. Johnston, Measures in health psychology: A user’s portfolio. Causal and control beliefs (pp. 35-
37). Windsor, UK: NFER-NELSON. 
 
[20] Broadbent, D.E., Cooper, P.F., FitzGerald, P., & Parkes, K.R. (1982). The Cognitive Failures 
Questionnaire (CFQ) and its correlates. British Journal of Clinical Psychology, 21, 1-16. 
 
[21] Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social 
Psychology, 42, 116–131. 
 
[22] van der Gaag, M., Schütz, C., ten Napel, A., Landa, Y., Delespaul, P., Bak, M., ... & De Hert, M. 
(2013). Development of the Davos assessment of cognitive biases scale (DACOBS). Schizophrenia 
Research, 144(1-3), 63-71. 
 

RESEARCH STUDY PROTOCOL 
 
 Page 9 
COMPUTERIZED PRACTICE VS. STRATEGY MONITORING IN COGNITIVE TRAINING FOR PSYCHOSIS 
 
[23] Llerena, K., Park, S. G., McCarthy, J. M., Couture, S. M., Bennett, M. E., & Blanchard, J. J. (2013). 
The Motivation and Pleasure Scale–Self-Report (MAP-SR): Reliability and validity of a self-report 
measure of negative symptoms. Comprehensive psychiatry, 54(5), 568-574.