Enhancing Evidence-Based Practice for Youth and Emerging Adults With Early Psychosis: Implementation and Evaluation in Diverse Service Settings
Phase: NA
Conditions: First Episode Psychosis, Schizophrenia, Childhood, Psychosis, Schizophrenia, Schizoaffective Disorder, Schizophreniform Disorders, Bipolar I Disorder, Schizophrenia Spectrum and Other Psychotic Disorders, Affective Disorders, Psychotic, Psychotic Disorder Due to Unspecified Substance
Sponsor: Centre for Addiction and Mental Health

 
 
Version #5: 26-Jun-2019 
1
STUDY PROTOCOL 
 
(V#5: 26-June-2019) 
 
Study Title:  
Enhancing Evidence-Based Practice for Youth and Emerging Adults with 
Early Psychosis: Implementation and Evaluation in Diverse Service Settings 
 
Lead Principal Investigator: 
Aristotle Voineskos MD, PhD, FRCPC 
Institution: 
 
 
 
Centre for Addiction and Mental Health  
 
 
Principal Investigators: 
 
George Foussias MD, PhD, FRCPC 
Paul Kurdyak MD, PhD, FRCPC 
Janet Durbin PhD, MSc 
Sanjeev Sockalingam MD, FRCPC, MHPE  
 
Co-Investigators: 
 
 
Jean Addington PhD  
Alexia Jaouich PhD  
Nicole Kozloff MD 
Sarah Bromley OT Reg (Ont) 
Dielle Miranda MA, MHSc (candidate) 
Allison Crawford MD, PhD 
 
 
 
 
 
Eva Serhal PhD 
Sophie Soklaridis PhD 
Joanna Henderson PhD 
Kelly Anderson PhD 
Melanie Barwick PhD, CPsych 
Diana Urajnik PhD 
Claire de Oliveira PhD 
Juveria Zaheer MD 
Simone Dahrouge PhD  
John Haltigan PhD 
Lillian Duda, MPA 
Donald Addington MD 
Augustina Ampofo, BA 
 
 
 
 
 
Site Leads: 
 
 
 
Valerie Primeau, North Bay Regional Health Centre 
 
 
 
 
 
Krista Whittard, Niagara Region Public Health 
Sheila Gallagher, Durham Amaze Early Psychosis Intervention-
Lakeridge Health 
 
Source of funding: 
 
 
Canadian Institutes of Health Research 
CAMH Foundation 
Provincial System Support Program 
 
 
 
 
 
Extension of Community Healthcare Outcomes (ECHO) 

 
 
Version #5: 26-Jun-2019 
2
1. 
Abstract  
 
Implementation of ‘NAVIGATE’ in Ontario aims to help youth and emerging adults (YEA) suffering from a 
first episode of psychosis (FEP). Although Ontario already has early psychosis intervention (EPI) programs, 
our team’s recent work has identified major challenges of delivering coordinated care, particularly those 
elements of care that enhance recovery. These challenges also exist nationally and internationally. By 
building on the already existing EPI community of practice through the Early Psychosis Intervention Ontario 
Network, we will implement NAVIGATE with the help of CAMH’s Provincial System Support Program 
facilitators. The use of tele-videoconferencing through ECHO Mental Health Ontario, and the processes and 
protocols that ECHO provide us with an opportunity to ensure sustainability. Using health administrative 
data held at the Institute for Clinical Evaluative Sciences (ICES), we can examine system-level outcomes, 
including hospitalizations, emergency department visits, and outpatient physician visits of YEA suffering 
from a FEP who are treated with NAVIGATE compared with those treated in EPI programs without 
NAVIGATE and those who are not treated in EPI programs. In addition, we can also evaluate health care 
costs. Prior to initiating this project, we obtained the input of YEA with an FEP and family members. We 
will also continue to measure engagement across the study. This work fulfills the innovative clinical trial 
mandate through its implementation and measurement plan, along with the strategy for patient-oriented 
research through the involvement of patients, family members, and policy makers as equal partners. 
 
2. 
Introduction and Background/Rationale 
 
2.1. Overview 
 
First Episode Psychosis (FEP) as a Public Health Priority:  
An FEP is one of the most frightening and disabling experiences from which a young person can suffer. 
Onset occurs in nearly all cases in youth and emerging adults (YEA), a crucial transition time, when major 
developmental milestones (high school graduation, first intimate relationship, maturation of executive 
function) occur1,2,3. An FEP typically occurs in the context of diagnoses of schizophrenia, schizoaffective 
disorder, psychotic disorder NOS, substance-induced psychosis, and psychotic bipolar disorder, which when 
taken together are the most disabling disorders (among all medical illnesses) for YEA in OECD countries 
(causing over 10% of the total disability adjusted life years (DALYs) in this population4). Recent evidence 
from the U.S. shows the lethality of an FEP: associated mortality rates are 24 times greater in the 12 months 
after diagnosis compared to the general population5. Suicide, the second leading cause of death among youth 
in Canada, remains much higher in early psychosis than in the general population. Even after accounting for 
suicide, persons suffering from these illnesses have a shortened lifespan that is attributed to early mortality, 
primarily due to downward socioeconomic drift and poorly treated medical disorders, a reflection of the 
gross inequity experienced by those with an FEP in relation to the general population6,7,8. 
 
Early Psychosis Intervention (EPI) Programs: Life-Saving, but Lack of High Quality Care:  
The emergence of EPI programs has provided tremendous hope by providing early intervention9,10,11. Work 
by our team, recently published in the American Journal of Psychiatry12, shows that youth who access an EPI 
program in Ontario, Canada receive faster psychiatric follow-up, better coordination of care between 
inpatient and outpatient services, reduced burden on the emergency department, and reduced all-cause 
mortality. Our system-level data support EPI programs as a critical life-saving intervention. Internationally, 
EPI has demonstrated significant benefits compared to treatment as usual with respect to engagement, 
service utilization, and suicide9,13,14,15.  

 
 
Version #5: 26-Jun-2019 
3
Despite the initial life-saving benefits of EPI services, provincial, national and international data show that 
consistent delivery of high-quality, evidence-based care in EPI programs is a major challenge16. Recovery 
rates in EPI programs remain low17, and associated disability has not improved under routine clinical care18. 
One explanation for these disappointing facts is that a low proportion of patients receive recovery-based 
services, namely, case management including individualized psychosocial interventions, family education 
and intervention, and supported education and employment. Even in clinical service delivery trials (where 
service quality is typically superior to real-world settings), recovery-based service is received by 15-56% of 
patients9,13 with only 18% receiving comprehensive EPI services9. While EPI standards across jurisdictions 
recommend coordinated and comprehensive recovery-based care19, effective implementation and 
sustainability of such care in real-world settings remains poor20.  
 
Our own work in Ontario, detailed below, has uncovered a startling gap between the evidence-based standard 
of care, and real-world delivery of care, even in a jurisdiction (Ontario) that has prioritized EPI services. The 
results of our Ontario survey21 were mirrored in a national survey of 11 Canadian EPI programs22. Similarly, 
ten years after Australia (a leader in EPI care) published national early psychosis guidelines, surveys 
demonstrated that implementation of recovery-based care was varied and disappointingly low23. This study is 
designed to improve the delivery of recovery-oriented evidence-based EPI care in Ontario. If successful, we 
will improve person-, system-, and economic-level outcomes for YEA suffering from an FEP, and offer a 
potential roadmap for the rest of Canada. 
 
EPI in Ontario: Shows Early Success Through a Community of Practice But Challenges Continue in 
Delivering Consistent and Sustained High Quality, Evidence-Based Care: 
In 2004, the province of Ontario allocated funding for EPI programs based on early evidence of efficacy. 
Between 2005 and 2007, this new provincial funding led to a major EPI program expansion in Ontario24, but 
the absence of provincial standards led to program development in an ad-hoc manner, drawing upon general 
service delivery experience and advice from established programmes24. Concomitantly, the Early Psychosis 
Intervention Ontario Network (EPION) was established, which has now grown to 52 EPI sites. The 
establishment of EPI program standards25 in Ontario in 2011, informed by existing international and national 
standards, but also by input from clinicians, patients, family-members, and policy makers was a crucial first 
step toward standardizing care. However, the release of standards alone is not sufficient to ensure effective 
implementation, practice change, and ongoing quality practice.  
 
The Ontario Ministry of Health and Long Term Care established a Standards Implementation Steering 
Committee (SISC) within EPION that is supported by the Provincial System Support Program (PSSP), based 
at the Centre for Addiction and Mental Health (CAMH). The SISC was established due to increasing 
recognition that active support is required to implement and sustain evidence-based care in routine practice26. 
Key findings from the first survey by the Ontario SISC of 52 EPI program sites in Ontario (92% 
participation) demonstrated important challenges in delivering evidence-based EPI care27. A follow-up 
survey also identified opportunities for addressing these challenges21. Building on the two surveys, we 
conducted a study28 to measure fidelity to current EPI standards using the First Episode Psychosis Service – 
Fidelity Scale (FEPS-FS)29. This fidelity project trained EPI staff as peer assessors to document current 
service delivery approaches, challenges and support needs in self-selected EPION programs. Our most 
notable finding was lack of delivery of consistent recovery-oriented care, with no structured or manualized 
process for these elements of care. Through site visits, and in-person interviews with nine Ontario EPI 
programs, we obtained a richer and clearer picture of the current state, creating an opportunity to implement 
solutions that can address the major challenges identified in the fidelity study:  
 

 
 
Version #5: 26-Jun-2019 
4
Challenge 1. There is variability in the type and frequency of EPI services offered. Variability was 
particularly noted in aspects of recovery-oriented care, including comprehensive care planning and 
monitoring, use of psychotherapies (CBT), family support, and employment support. A lack of protocols to 
guide such delivery and inconsistent documentation was a common theme.  
 
Challenge 2. Programs expressed a need for more tools and protocols to guide service delivery, more 
consistent access to training, and lack of time for training in the implementation of new practices.  
 
Challenge 3. Programs desired opportunities to learn from peers in a community of practice, and noted 
resource challenges such as accessing expertise and travel time. 
 
How Will this Study Address the Identified Challenges?  
We want to address these challenges in Ontario EPI settings by implementing NAVIGATE, a coordinated 
and comprehensive multidisciplinary treatment program for FEP that is deliverable in community mental 
health settings. NAVIGATE is manualized and measurement-based. To maximize affordability, adaptability, 
spread, and sustainability, the key elements of success for an implementation project as defined by CIHR, we 
will utilize a) the already-established EPION community of practice, b) the implementation science expertise 
of PSSP, and c) the sustainability and capacity building approach of The Extension of Community Health 
Outcomes (ECHO) Ontario Mental Health at CAMH and the University of Toronto (ECHO-ONMH). These 
are detailed point-by-point below. We first briefly explain, and summarize the evidence for NAVIGATE.  
 
In 2008, the National Institute of Mental Health (NIMH) issued a request for proposals entitled: Recovery 
After an Initial Schizophrenia Episode (RAISE), with a goal to change the trajectory and prognosis of 
FEP30. NAVIGATE was developed in consultation with clinical and research experts, biostatisticians, health 
economists, consumers, family members, advocacy groups, and government officials. It is a form of 
coordinated specialty care for FEP consisting of 4 key intervention components: (i) individualized 
medication management using a decision support tool, (ii) a package of psychoeducation and a blend of 
evidence-based psychotherapies called “individual resiliency training” (IRT); (iii) supported employment 
and education (SEE); and (iv) a family education program. NAVIGATE was evaluated from 2009-2014 in a 
cluster randomized controlled trial involving 404 individuals with an FEP in 34 community mental health 
centers across the United States20. Notably, it was delivered by re-allocating existing community mental 
health resources with no new funding for clinical care. Compared to usual care, NAVIGATE treatment 
provided greater improvement in symptoms, but more importantly, as prioritized by patients, significantly 
greater improvement in real-world functioning, including social functioning and engagement in educational 
and vocational training20,31. NAVIGATE was also readily implementable across a broad range of 
community-based mental health settings, with enhanced engagement and delivery of multidisciplinary care 
to YEA presenting with psychosis and their families compared to standard care, and with longer and more 
consistent receipt of mental health services20. A recent economic analysis revealed that NAVIGATE 
treatment was more cost-effective compared to standard community care, driven by the anticipated enhanced 
health benefits and improvements in quality of life32. 
 
Challenge 1. Need to Standardize Care, Particularly Recovery-Oriented Care and Manualized 
Protocols: NAVIGATE operationalizes current EPI standards using manualized protocols, ensuring 
consistency and reducing variability in care. The four NAVIGATE components (individualized medication 
management, IRT, SEE, and family education program) are systematically applied in collaboration with the 
patient33. There is an overarching emphasis on the coordinated delivery of these elements of care. Every 
patient is offered these elements of care, and modules are completed in a systematic time-oriented fashion 

 
 
Version #5: 26-Jun-2019 
5
that reduces variability in care among sites and team-members within a site. For instance, goal-setting and 
identification of personal strengths are consistently discussed at the first visit. This ensures that care is 
systematically oriented and aligned towards patient preference. At each patient visit, a contact/progress note 
is completed, including the modules delivered, that the team reviews to assess patient progress, fidelity, and 
determine need for adjustments. 
 
Challenge 2. Need for Consistent Access to Training, To Investigate and Implement New Practices:  
Ontario EPI sites have learned about NAVIGATE through didactic sessions and conferences offered through 
EPION. As new evidence emerges, network members are interested in how to translate new research to daily 
practice. Because translation and implementation of evidence-based practice remains challenge, training, 
implementation expertise, and other resources are required34. Implementation science supports a systematic, 
deliberate and evidence-based process of implementing new practices, and requires key factors and processes 
for sustainable practice change. The mandate of the CAMH PSSP is to support system change in Ontario by 
providing implementation, evaluation, knowledge exchange, engagement and information management 
expertise to organizations and networks across the province. PSSP operates regional offices with 
implementation teams throughout Ontario, and includes regional oversight of the sites participating in the 
study. Regional implementation teams work closely with local communities and key partners to implement 
and sustain system improvements. This approach is aligned with implementation science research showing 
that a guided implementation process by a group of individuals with specialized skills, dedicated time, and 
accountability for overall guidance is critical to good implementation and clinical outcomes35. PSSP 
facilitators will guide the implementation process in the study. PSSP has a strong relationship with EPION, 
providing in-kind support through annual knowledge exchange and training activities. PSSP supported the 
implementation of the EPI surveys conducted by the SISC. For this project, regional PSSP teams will 
support implementation and work closely with NAVIGATE content experts from the Slaight Centre 
(CAMH’s EPI program), and community site leads, who are members of EPION. 
 
Challenge 3. Need to Bolster a Community of Practice and Provide Ongoing Access to Expertise:  
EPION has identified use of technology to transfer knowledge, and equitable application of the provincial 
standards as key priorities. ECHO consists of specialist hubs that connect with multiple spoke (learner) 
teams in remote areas through televideo-conferencing technology, and thus can bridge the geographic gap 
required to bolster the Ontario EPI community of practice. The goal of ECHO clinics is to extend the reach 
of best practices in academic settings to the chronic and complex illnesses seen in local settings, thereby 
reducing variation, increasing access to specialist mentoring and services, and monitoring and improving 
patient outcomes36. Project ECHO is designed to facilitate a community of practice, provide access to 
expertise, and overcome geographic barriers in relation to specialty coordinated care. In a landmark study, 
the original Project ECHO demonstrated treatment outcomes for hepatitis C that were comparable to referral 
to an academic centre for specialist management37. As a result of this growing evidence base, the ECHO Act 
was adopted by the United States Senate in 2016, stating that technology-enabled capacity building 
interventions, such as ECHO, be used to bridge the gap between specialist care in academic hubs to more 
rural and remote communities38. Initial evaluation data in Ontario has shown high provider satisfaction and 
engagement with ECHO, and increased knowledge and self-efficacy in managing mental health and 
addictions in rural and remote settings39. 
 
How is this Study Further Enhanced? From Patient Level to System Level:  
i). This project meets the SPOR Mandate through Patient and Family Engagement (using both full 
participation as equal partners, and more limited commitment, i.e. interviews). Most recently CAMH’s youth 
advisory committee recently selected functioning (i.e. functional outcome) as their preferred outcome 

 
 
Version #5: 26-Jun-2019 
6
measure in YEA mental health intervention research44. In addition, in the direct planning and design of the 
present study, our own previous Principal Investigator with lived experience was present at each team 
meeting, and was an equal partner in decisions regarding all of our outcome measures. Following success at 
the letter of intent stage, our co-investigator family member (her son experienced an FEP) joined our team, 
and she provided input, further refining our research objectives. Since this time, a co-investigator with lived 
experience of psychosis has also joined our team, in order to provide input and continue to refine research 
objectives and logistics. These collective efforts were predicated on SPOR patient engagement guiding 
principles of inclusiveness, support, mutual respect, and co-building These SPOR principles will be applied 
across sites, such that our Advisory Committee will grow to include patient/family representation from each 
site. The Advisory Committee will provide ongoing guidance to the research team in implementation, 
evaluation, analyses, and dissemination. In addition, at CAMH’s Slaight Centre, we undertook a qualitative 
study45 that placed an emphasis on experiential knowledge, a desired patient engagement outcome. This 
study included interviews with 15 patients (men and women) and eight family members (mothers and 
fathers). Respondents identified that highly integrated coordinated multidisciplinary care that includes the 
core components of NAVIGATE is essential to engagement and recovery. Patients and families had positive 
attitudes toward scales and measures, and felt that formalized assessment tools provided them and their care 
providers an approach to track progress. We will also continue our qualitative work with patients and 
families to further evaluate NAVIGATE at the participating sites. To date, qualitative research in FEP has 
focused on patient experiences of hospitalization and medication management, narratives of illness and 
recovery, or transitions in care46-49 rather than a program of care50. Therefore, the present study provides an 
opportunity to build on our novel work. 
 
ii). Institute for Clinical and Evaluative Sciences (ICES) Data: A Unique Opportunity to Evaluate System-
Level Impact – Meeting the Mandate of the Innovative Clinical Trial (ICT): Using data from ICES, which 
captures all physician and hospital-provided insured services in Ontario, we will compare population-based 
outcomes (hospitalizations, emergency department visits, suicide attempts and mortality) with two 
comparison groups: 1) all EPI program FEP patients who have not been part of the NAVIGATE trial; and 2) 
all FEP patients who are not attached to EPI programs. All EPI programs in Ontario report their service 
utilization data through the Ontario Common Assessment of Need (OCAN) dataset (OCAN is linked to ICES 
data). While accuracy of data within OCAN is variable, we have confirmed that we can accurately ascertain 
the initiation date of all EPI patients by sex within Ontario through ICES linkage. The ability to link 
NAVIGATE’s primary clinical and implementation data with ICES data, and the capacity to identify all FEP 
patients in Ontario (whether treated in EPI programs or not) allows for a meaningful, valid, and highly 
generalizable comparison of outcomes. The use of propensity score methods will address confounding 
associated with observational studies, and mimics some of the characteristics of a randomized controlled 
trial51. Since outcome data across all comparison groups are routinely collected by ICES, this dramatically 
reduces the time and cost of a randomized controlled trial (RCT) (thus fulfilling the ICT mandate), because 
the study does not require costly, extensive follow-up. 
 
ICES has developed a costing macro that we have used to study health care costs among individuals in 
Ontario with schizophrenia52. The costing macro ensures that we have the capacity to determine costs 
associated with NAVIGATE relative to EPI usual care and no EPI care, a key implementation outcome. Cost 
estimation adds to the feasibility and policy-relevance of scaling up. 
 
 
 
 

 
 
Version #5: 26-Jun-2019 
7
 
3. 
Statement of Objectives and Hypotheses 
 
3.1. Primary Objectives 
 
1. Assess whether implementation of NAVIGATE leads to improvement in fidelity to the EPI standard 
(using the FEPS-FS). 
 
2. Compare system-level outcomes, i.e. days in hospital, emergency department visits, suicide attempts, 
mortality, and system costs among patients receiving NAVIGATE compared with patients at other EPI 
sites not receiving NAVIGATE and patients with psychotic disorders who are not enrolled in EPI. 
 
3. Determine longitudinal change in functioning and symptoms in NAVIGATE patients. 
 
4. Evaluate patient and family member engagement according to the SPOR framework. 
 
3.2. Hypotheses 
 
1. Following the implementation of NAVIGATE, program fidelity to the Ontario EPI standard will 
improve. 
 
2. Compared to patients not receiving NAVIGATE, those who receive NAVIGATE through this 
implementation study will have fewer days in hospital, fewer ED visits, fewer suicide attempts, lower 
mortality, and lower costs. 
 
3. Improvements in functioning and symptoms will be comparable to the RAISE study; improvement may 
be influenced by demographic, socio-economic, geographic, and clinical factors. 
 
4. Our engagement approach will demonstrate that we used the full range of patient engagement based on 
objectively assessed engagement metrics. 
 
3.3. Study Design/Approach 
 
Description of Sites: 
We will include EPI programs, with representation from different geographic regions of Ontario  that 
collectively cover 45% of the province’s land area, which includes the north-east region. The North-east is 
considered one EPI program but does consist of smaller sub-sites due to geographic spread. All programs are 
EPION members, participated in the PSSP/SISC survey and in the PSSP/EPION fidelity study. Each site 
includes health professionals that provide case management support as well as psychiatric care. Each site has 
unique characteristics or factors that strengthen the diversity and generalizability of this work - site 
differences may offer the opportunity to evaluate subpopulations, e.g. ethnic/racial minority including 
Indigenous populations, urban vs. rural location, and variation in service resources.   
 
Assessing outcomes of NAVIGATE patients (obj. #3) 
Duration of Study, Recruitment Plan; Inclusion/Exclusion Criteria, Assessment Schedule:  
Recruitment: We will recruit patients from EPI programs participating in the study, with representation from 
diverse geographic regions of Ontario. Based on 2016 program data, the total number of new patients across 

 
 
Version #5: 26-Jun-2019 
8
sites was 411. In the RAISE study, 423 patients were randomized to NAVIGATE (n=223) vs. usual care 
(n=181). We anticipate that we will recruit n=400 patients into the study (based on EPI service sex ratios, we 
anticipate n=150 women, and n=250 men), with a two year follow-up plan mirroring RAISE. From months 
6-12, when we expect 200+ new patients across sites, we will recruit n=100, since we will gradually phase in 
recruitment (i.e. some sites will be ready earlier than others during this time frame), and since some patients 
might decline participation. All sites will proceed with full recruitment (i.e. consecutive referral series) from 
months 12-24. During this time, of the 400+ new patients across sites, we will aim to recruit n=300 
(accounting for ~ 25% who might decline or be lost to follow-up prior to first visit). Follow-Up: Of the 
n=400 who will be recruited between months 6-24, we will complete assessments across one to two year 
period. The first patient will complete the study in month 30 (with n=100 completing by month 36), while 
the final patient will complete 2 year follow-up at month 48.  
 
Inclusion Criteria: All of the EPI sites follow people experiencing an FEP: 
 Age range of 14-35 years;  
 any DSM-diagnosis that can manifest as early psychosis (schizophrenia, schizoaffective disorder, 
schizophreniform disorder, bipolar I disorder, major depressive disorder with psychotic features, 
substance induced psychotic disorder, or unspecified psychotic disorder);  
 
Exclusion Criteria:  
 Absence of  psychosis  
 Inability to provide informed consent to participate in the research study 
 
Clinical Assessments: Patients will be evaluated at/after baseline, 6 months, 12 months, 18 months, and 24 
months of their EPI program with comprehensive assessments of symptom and overall illness severity, and 
functional outcomes. For example, a patient entering the study at the beginning of their EPI program will 
complete all study visits (baseline, 6, 12, 18, and 24 months), while a patient entering the study after 1 year 
of their EPI program would complete only some study visits (12, 18, and 24 months, plus all baseline 
measurements at first study visit). Patients may enter the study up until the point where they have been in 
their EPI program for more than one year. All patient-level assessment tools chosen are those used in the 
original NAVIGATE study, with the exception of the AADIS. At the time of entry all patients will be 
administered the Structured Clinical Interview for DSM-5 (SCID-5) for formal diagnostic assessment. 
Functioning will be assessed with the Quality of Life Scale (QLS)53, consistent with the original RAISE 
clinical trial20. The QLS is the most comprehensive measure of community functioning in schizophrenia 
populations54. Because the QLS is psychosis-specific, we will augment assessment of functioning using the 
self-report WHO Disability Assessment Schedule 2.055 to provide an assessment of health and disability that 
is utilized across diverse populations and illnesses, and aligns with recent recommendations from the DSM-5 
for the use of this instrument across mental illnesses56. In order to measure patient satisfaction with their 
care, patients will complete the Client Ontario Perception of Care Tool (OPOC)97 and the Youth Services 
Survey98. To measure the therapeutic relationship with their care team, patients will complete the Scale to 
Assess Therapeutic Relationships – Patient Version (STAR-P)99. To measure patient perspective of the 
quality of their care, patients will complete the Perception of Recovery Orientation and Care Quality of 
Mental Health Services (Recovery Self Assessment; RSA) Scale100. Our project Youth Advisory Committee 
identified perception of care quality, satisfaction, and therapeutic relationships as important factors that 
should be assessed when evaluating longitudinal patient outcomes. In order to capture caregiver/family 
member perspective on the patient’s functioning, we will also have one of the patient’s caregivers/family 
members complete the proxy-administered version of the WHODAS, as well as the Life Skills Profile96 (see 

 
 
Version #5: 26-Jun-2019 
9
below for further explanation). Family members will also complete proxy-versions of the care satisfaction, 
quality of care, and therapeutic relationship measures identified above. At the symptom level, psychosis, 
negative symptoms, and general psychopathology will be assessed using the Brief Psychiatric Rating Scale 
(BPRS), and the Patient Health Questionnaire – 9 (PHQ-9). The Clinical Global Impression scale (CGI) will 
be administered to characterize overall illness severity. The Adolescent Alcohol and Drug Involvement Scale 
(AADIS)95 will be administered to characterize current substance use.  
 
Using videoconferencing software consistent with Project-ECHO infrastructure, all interviewer-rated 
assessments will be administered via live two-way video conferencing by trained interviewers, in a manner 
identical to the approach utilized in the original NAVIGATE trial20.  This method of remote assessment is 
comparable to in-person assessments for both patient acceptability and reliability57. 
 
As mentioned above, in order to capture caregiver perspective on the patient’s functioning, we will have one 
of their caregivers/family members complete the proxy-administered version of the WHODAS and Life 
Skills Profile. Family members will also complete the OPOC, Youth Services Survey, and RSA. Family 
members will complete these at similar time points as patients complete their assessments (i.e. At baseline 
and after 6, 12, 18, and 24 months of the EPI program). They will complete these in self-report survey form.  
 

 
 
Version #5: 26-Jun-2019 
10
Schedule of Assessments (Timeframe relates to the participants’ 2 year timeframe in the clinic) 
Assessments 
Admission to 
Clinic  
Month 6 
Month 
12 
Month 
18 
Month 
24 
Screening 
 
 
 
 
 
Informed Consent 
 
 
** 
** 
** 
** 
Demographic form/SES 
 
 
** 
** 
** 
** 
Eligibility/Termination Checklist 
 
 
** 
** 
** 
** 
SCID 5 
Baseline: Full SCID 
Month 12 and 24: Mood, Psychosis, 
Substance Use Disorders, Anxiety, and 
OCD Modules 
 
 
** 
 
 
** 
 
 
Medical History 
 
 
** 
** 
** 
** 
 
 
 
 
 
 
Clinical Assessments 
 
 
 
 
 
CGI 
 
 
 
 
 
 
 
 
 
 
PHQ-9 
 
 
 
 
 
 
 
 
 
 
BPRS 
 
 
 
 
 
 
 
 
 
 
AADIS 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Service Utilization 
 
 
 
 
 
SURF 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Functional Assessments 
 
 
 
 
 
QLS 
 
 
 
 
 
 
 
 
 
 
WHO DAS 2.0 
 
 
 
 
 
 
 
 
 
 
WHO DAS 2.0 – proxy version, 
completed by family member/care 
giver 
 
 
 
 
 
 
 
 
 
 
Life Skills Profile – proxy version, 
completed by family member/care 
giver 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Satisfaction, Care Quality, and 
Therapeutic Relationship 
 
 
 
 
 
OPOC (participant and family member 
complete) 
 
 
 
 
 
 
 
 
 
 
Youth Services Survey (participant 
and family member complete) 
 
 
 
 
 
 
 
 
 
 
STAR-P (participant completes) 
 
 
 
 
 
 
 
 
 
 
RSA (participant and family member 
complete) 
 
 
 
 
 
 
 
 
 
 
Note: **  indicates measure completed 
at this time point if necessary 
 
 
 
 
 

 
 
Version #5: 26-Jun-2019 
11
 
Assessment for diagnosis 
 
1. The Structured Clinical Interview for Axis I DSM-5 Disorders. Diagnosis will be confirmed using the 
Structured Clinical Interview for DSM-5 (SCID-5). Information from the SCID will be supplemented by 
information from,  a current psychiatrist and/or other clinician, if necessary. Demographics and a medical 
history will also be recorded at baseline. The SCID is a semi-structured diagnostic interview designed to 
assist clinicians, researchers, and trainees in making reliable DSM-5 psychiatric diagnoses.  
 
 
Scale to measure clinical psychopathology  
 
1. The Brief Psychiatric Rating Scale (BPRS) (24 item) will be used to assess the severity of positive 
symptoms, psychosis, negative symptoms, and general psychopathology. 
2. The self-report Patient Health Questionnaire – 9 (PHQ-9) will be used to characterize the presence and 
severity of depressive symptoms. 
3. The Intrinsic Motivation Factor of the Quality of Life Scale (QLS) will serve as a specific measure of 
motivation to augment the above psychopathology measures. 
4. The Adolescent Alcohol and Drug Involvement Scale (AADIS) will serve as a measure of current 
substance use. 
 
Scale to assess illness severity and improvement 
 
1. The Clinical Global Impressions Scale (CGI) will be administered to characterize overall illness severity. It 
is designed to rate both illness severity and improvement. It takes into account all available information, 
including knowledge of the patient’s history, psychological circumstances, symptoms, behaviour, and the 
impact of the symptoms on the patient’s ability to function.  
 
Scales to assess functioning 
 
1. Quality of Life Scale (QLS). Functioning will be assessed with the Quality of Life Scale (QLS), which is 
the most comprehensive measure of community functioning in schizophrenia populations. The scale has 
been used successfully in studies in prodromal, first episode, and chronically ill participants. It is 
designed to evaluate the current functioning of nonhospitalized individuals with schizophrenia apart from 
the presence or absence of florid psychotic symptomatology or need for hospitalization. It is a semi-
structured interview. It has 4 subscales: Interpersonal Relations, Instrumental Role, Intrapsychic 
Foundations and Common Objects and Activities.  For the evaluation of community functioning, the 
Intrapsychic Foundations subscale (which comprises the Intrinsic Motivation factor) will be excluded in 
order to minimize overlap between assessments of schizophrenia psychopathology and community 
functioning.  
 
2. WHO Disability Assessment Schedule 2.0: Because the QLS is psychosis-specific, we will augment 
assessment of functioning using the self-report WHO Disability Assessment Schedule 2.0 to provide an 
assessment of health and disability that is utilized across diverse populations and illnesses, and aligns with 
recent recommendations from the DSM-5 for the use of this instrument across mental illnesses. 
 
Scales to measure caregiver/family member perspectives of patient functioning 
 

 
 
Version #5: 26-Jun-2019 
12
1. We will use the proxy-administered version of the WHO Disability Assessment Schedule 2.0, which 
aligns with the patient self-report version, as described above. This will give us insight into family 
member/caregiver 
perspectives 
of 
the 
patient’s 
functioning 
(health 
and 
disability). 
 
2. We will also use the Life Skills Profile, rated by a caregiver or family member. The Life Skills Profile is 
a measure of functioning for individuals with severe mental illness that includes domains encompassing 
communication, social contact, non-turbulence, self-care, and responsibility.  
 
Scales to measure patient and family member perspectives of care quality, satisfaction, and therapeutic 
relationships 
 
1. Client Ontario Perception of Care Tool: A Patient- and caregiver-reported measure of satisfaction 
consisting of 38 items on a Likert scale with additional open-ended questions. It has been used in other 
Canadian clinical trials in youth, and is used by PSSP to evaluate services across Ontario. Patients and 
family members will complete this measure. 
2. Youth Services Survey: A patient- and caregiver-reported 25-item measure of service satisfaction, 
validated in kids. It is used in PhenX (a toolkit of measures for EPI programs). Patients and family 
members will complete this measure. 
3. Scale to Assess Therapeutic Relationships – Patient Version (STAR-P): A 12-item patient-reported 
measure of the therapeutic relationship in community mental health, based on items from the Working 
Alliance Inventory and other measures of the therapeutic alliance with input from patients and clinicians. 
It has been used in a number of trials of community mental health interventions. Factors include: positive 
collaboration, emotional difficulty, positive clinician factor. Only patients (and not family members) will 
complete this measure. 
4. Perception of Recovery Orientation and Care Quality of Mental Health Services: A Patient- and 
caregiver-reported 32-item measure of perceptions of recovery principles and overall quality of services, 
including determination, staff helpfulness, and staff responsiveness. It includes six subscales: life goals, 
consumer involvement, diversity of treatment options, consumer choice, individually-tailored services, 
and inviting environment. Patients and family members will complete this measure. 
 
Scale to determine parental socio-economic status 
 
1. Parental and participant education to be used as indicator of SES. 
 
Measurement of service utilization 
 
1. The Service Use and Resource Form (SURF) will be used to measure utilization of mental health and 
other medical services across residential, inpatient, and outpatient treatment settings. This will be 
administered by research staff either over the phone or via email survey, based on participant preference. 
 
   
Other Measures  
 
1. Demographic questionnaire  
 
4. 
NAVIGATE Implementation Approach 
 

 
 
Version #5: 26-Jun-2019 
13
Although each EPI program has existing processes for providing the components of early psychosis care, 
these are not necessarily delivered in the systematic and intentional manner outlined through NAVIGATE. 
Building on a large body of practice change evidence supporting a ‘make it happen’ rather than ‘let it 
happen’ approach34,58, we will guide each program site through a facilitated, staged, change process informed 
by the Active Implementation Framework and the Quality Implementation Framework34,59. A regional PSSP 
facilitator will have primary responsibility for the implementation, working closely with the CAMH 
NAVIGATE experts, and EPI program staff lead at each site. The CAMH NAVIGATE experts and PSSP 
facilitator will bring expertise and momentum to the work. The PSSP facilitator, EPI program staff lead, and 
ECHO team will assist with site engagement, understanding and preparing the program context, and local 
problem-solving60.  
 
Although sites will vary somewhat in their pace of implementation, we expect that installation (see below) 
will occur across the first six months, that trialing and refining will occur during months 7-12, and that 
practice will become embedded after month 12. Thus, PSSP involvement and facilitation will be most 
intense during early and mid-implementation stages as sites plan, trial and refine, and will decrease as 
delivery practices stabilize and become routine. The facilitators will monitor program progress in a task log 
based on Meyers59. Achievement of key implementation milestones will be reported and areas flagged where 
more support may be needed. Correspondingly, involvement of ECHO will increase over time to support 
NAVIGATE as part of routine practice and this will also be monitored. The stages of our implementation 
process include:  
 
Stage 1. Exploration (to assess site capacity and need, build engagement): This stage is key to successful 
practice change. As EPION members who are committed to this project, there is already considerable 
engagement among participating sites. Additionally, having participated in the prior sector surveys and 
fidelity reviews, NAVIGATE was identified as a strategy to improve upon current practice at each site. In 
this stage the CAMH NAVIGATE experts, PSSP facilitators, and ECHO team will met with each site to 
explain NAVIGATE, learn about their current staffing and service delivery processes, and discuss how 
NAVIGATE could be integrated into their practice. Staff will complete a site readiness assessment survey 
and a program fidelity review will be conducted to learn more about site capacities and needs for 
NAVIGATE implementation.     
 
Stage 2. Installation (to create structures/build capacity for implementation): Intensive staff training over 
several days will include a didactic component, role playing, and discussions with individual sites to best fit 
NAVIGATE with each site’s current processes. The training will initially require three days at the 
coordinating site (CAMH, Slaight Centre), co-led by NAVIGATE trainers from the RAISE study, or 
equivalent training. Regular site visits from PSSP facilitators and CAMH Slaight Centre content experts will 
occur to develop the local implementation plan working with each site lead to prepare staff.  Preparation for 
practice change will include discussions of: role allocation among staff (i.e. IRT vs. SEE vs. family 
psychoeducation program); whether the program needs to partner with an external service to deliver some 
elements (particularly the smaller sites); any changes to current patient flow required to deliver NAVIGATE 
(the care path); how contact and progress note data will be collected and used in meetings to review progress 
in NAVIGATE delivery; and how to prepare for ECHO sessions (topics and cases). The ECHO team will 
also work with each site to ensure set-up and ongoing functioning of infrastructure for live video-
conferencing.  
 
Stage 3. Initial implementation (trialing and refining) and Evaluation of Implementation Quality: During this 
stage, each site will begin NAVIGATE delivery and use feedback from various sources, including ECHO 

 
 
Version #5: 26-Jun-2019 
14
training and coaching, contact and progress notes, and staff meetings to make refinements to the 
implementation and service delivery processes and to address staff support/skill building needs. To increase 
the quality and consistency of the implementation process across sites (fidelity to the implementation 
process), we will build on experience of implementing NAVIGATE at the CAMH Slaight Centre, using the 
Quality Implementation Framework60 to track implementation steps that have been tailored to our study. The 
PSSP facilitator will document progress, strategies, and challenges in relation to these tasks in a structured 
log that they will share in regular meetings with the NAVIGATE experts and other facilitators for continuous 
improvement. Here, we will be mindful of site-specific factors and population-specific factors (e.g. 
biological sex, gender, ethnicity, rural vs. urban, LGBTQ status) that may influence implementation. In 
addition to implementation process tracking data, staff feedback will be used to refine the implementation 
process.   
 
Stage 4. Full Implementation and Sustainability: At this stage NAVIGATE is fully embedded into the 
organization and can be sustained with internal resources. The focus on sustainability runs through all stages 
to ensure capacity building along the way so that additional implementation supports provided by PSSP are 
eventually no longer needed. The ECHO team will work in collaboration with EPION study sites to create 
and sustain a community of practice for NAVIGATE implementation and spread beyond the duration of this 
study, such that it becomes routine practice (care paths, feedback processes). After each ECHO session, 
questionnaires will be used to evaluate satisfaction and inform ECHO modifications. Additionally, cases 
discussed during the sessions will generate implementation recommendations, and surveys at 2-3-months 
post-case discussion will be used to evaluate adherence to these recommendations. 
 
Evaluation of Implementation outcomes (Objective 1):  
Proctor’s implementation outcomes61 will guide evaluation of implementation outcomes: three of the four 
CIHR implementation science metrics (adaptability, scalability, sustainability). Specifically we will assess i) 
fidelity (i.e. adaptability) to examine the extent to which programs operating in different contexts with 
different resources are able to implement NAVIGATE as intended. Second we will assess ii) penetration (i.e. 
scalability) the extent to which all eligible program patients receive NAVIGATE. Using the Consolidated 
Framework for Implementation Research (CFIR)62, we will assess contextual factors (stakeholder 
perceptions of acceptability, appropriateness, feasibility) that help or hinder use of NAVIGATE. Learning 
about these factors including how they relate to unique aspects across settings will inform wider efforts to 
adapt NAVIGATE and tailor implementation (adaptability and scalability). We will also assess iii) 
sustainability through uptake of the ECHO model. We address the fourth CIHR implementation science 
metric (affordability) in the System-Level outcome section further below.  
 
i) Fidelity (ADAPTABILITY): We will use the First Episode Psychosis Service–Fidelity Scale (FEPS-
FS)29 to assess fidelity of service delivery to the current standard of EPI evidence-based practice in relation 
to 31 program-specific items (individual and team practices) on a 5-point scale from “not implemented” to 
“fully implemented”. Ratings for each site will be made by centrally located trained assessors through a 
remote assessment process that includes a review of site administrative data, data abstracted from client 
health records by trained abstractors, and phone interviews with site staff. The FEPS-FS was developed with 
a formal knowledge synthesis process rather than a single program model. It was used in the Ontario pilot 
and is relevant to the Ontario EPI standards. Reviews will be conducted at three points in time:  during the 
exploration stage, after full implementation is reached and at the study end to assess sustainability. To 
examine fidelity results, site will be the unit of analysis. Descriptive statistics (percentages, means, medians, 
range) will be reported for the total scale score and for subscale scores that align with NAVIGATE 

 
 
Version #5: 26-Jun-2019 
15
components. With a small number of sites, we cannot quantitatively test changes over time and across sites, 
but will describe and qualitatively compare findings.  
 
For additional validation of intervention delivery, fidelity scores will be calculated per site in relation to the 
four core NAVIGATE interventions (medication, IRT, SEE, family education) using the service utilization 
measure (see description in previous section). The Clinician Contact and Progress Note (a standard 
documentation template within NAVIGATE for IRT, SEE, and FEP components of treatment, captured in 
client health records, which captures information about which module was covered during each treatment 
session) will provide model adherence data for each patient across the core interventions. This will be 
augmented by the Service Use and Resource Form (SURF) that research staff will complete with each 
patient. We will mirror the original study, calculating the percentage of core modules completed per 
intervention per patient.   
 
ii.a.) Penetration (SCALABILITY): Penetration assesses the extent to which eligible program patients 
receive the NAVIGATE treatments, and identifies any biases in delivery61,63. This is important because a 
core aim of NAVIGATE is to increase consistency of model delivery across in patients across all programs. 
Fidelity to NAVIGATE will be calculated from the contact forms (clinician report) and the SURF (client 
report), and will be used to assess penetration and to identify variations in delivery across patients within 
sites and across sites. When assessing scalability, the unit of analysis will be the patient (n=400 patients). 
The outcome will be percentage of modules received for each of the four interventions and will be reported 
using percentages, means, medians, standard deviations, and interquartile ranges. A linear regression model 
will be used to identify predictors, with patient baseline characteristics such as sex, age, and illness 
complexity. The nesting of patients within each site will be accounted for in the analysis. All statistical tests 
will be two-tailed and statistical significance will be defined as p-values less than 0.05. 
 
ii.b.) Contextual Factors (ADAPTABILITY & SCALABILITY): The Consolidated Framework for 
Implementation Research (CFIR)62 will be used to systematically assess contextual factors that are associated 
with effective implementation. The CFIR offers a comprehensive menu of constructs from published 
frameworks that influence implementation. The constructs are organized within five major domains: 
intervention characteristics (e.g., complexity, relative advantage); outer setting (e.g., external policy, patient 
needs); inner setting (e.g., resources, fit, leadership); staff characteristics (e.g., knowledge, beliefs); and 
implementation process (e.g., facilitation, planning, coaching). Similar to the Damschroder approach64,65, the 
CFIR will be used to develop a semi-structured interview to guide data collection. Interviews will be 
conducted with stakeholders at each site (EPI staff, organization leaders) at the end of the active 
implementation, recorded, and transcribed. The CFIR will provide the organizing framework for qualitative 
data coding and analysis, while being open to new, emergent themes. At each analysis stage, coding and 
development of themes will be conducted by multiple coders, with consensus achieved through discussion 
and deliberation. We will develop summary statements of relevant constructs for each site. In combination 
with fidelity results, we will look for patterns within and across sites related in implementation facilitators 
and challenges. Qualitative analysis will be supported with NVivo software.  
 
iii) SUSTAINABILITY: ECHO coaching is expected to build staff competencies in delivery of 
NAVIGATE and to support longer-term sustainability of high quality NAVIGATE delivery. Our evaluation 
of ECHO will adhere to an established evaluation framework led previously by the ECHO team, building on 
continuing education program evaluation66,67. At each study site we will assess staff participation 
(engagement and retention); staff satisfaction with ECHO support; and perceived changes in competence to 
perform NAVIGATE. Staff attendance during ECHO sessions will indicate ECHO engagement and 

 
 
Version #5: 26-Jun-2019 
16
retention. Competency assessment questionnaires will be administered to staff prior to participation in 
ECHO sessions and at the conclusion of each ECHO cycle to assess changes in attitudes, knowledge and 
self-efficacy (self-reported competence) in delivering NAVIGATE components. These questionnaires will be 
based on pre-existing evaluation questionnaires used for ECHO-ONMH39.  
 
ECHO pre-post questionnaires will assess participants before and after each ECHO cycle. The staff 
knowledge and skills questionnaire is based on survey design literature and Bandura’s self-efficacy question 
framework. The questionnaire has been previously used to assess ECHO Ontario Mental Health programs 
and will be adapted to assess participants’ confidence and self-reported capability in delivering NAVIGATE 
components. Question domains will include Likert scales assessing self-reported knowledge and skill 
domains specific to the NAVIGATE model. Self-efficacy questions will focus on key competency domains 
of NAVIGATE and assess confidence in delivering these components in their local settings.  
 
Site implementation of NAVIGATE into practice will be assessed at the CFIR interviews on factors 
associated with successful implementation will explore the role of ECHO as a support for sustainability, 
including the formation of a community of practice. Additionally the readiness assessment68 will assess site 
capacity at the end of the study to continue NAVIGATE. Site readiness scores will be combined with CFIR 
and fidelity results for a qualitative examination of patterns within and across sites related in implementation 
facilitators and challenges. 

 
 
Version #5: 26-Jun-2019 
17
* includes perceptions of ECHO support 
 
System Level Outcomes + Analytic Plan (Objective 2):   
Implementation evaluation measures and timeline 
Implementation stage 
 
Explore 
(stage 1) 
Install 
(stage 2) 
Initial 
imp. 
(stage 3) 
Full 
implementation 
& sustainability 
(stage 4) 
Measurement domain 
Measure 
Capacity 
& needs 
Ax 
Plan & 
prepare 
Trial & 
refine 
Stabiliz
e 
practic
e 
Practice 
is 
routine 
Months 
1-3 
Months 
4-6  
Months 
7-12 
Months 
13-24 
Months 
25-42 
Implementation 
process 
 
 
 
 
 
 
Implementation log 
(milestones, risk, 
action) 
QIF (adapted)  
 
 
 
 
 
 
 
 
 
 
ECHO implementation 
(clinical coaching) 
Post-session 
questionnaires, 
surveys,  
and/or phone calls 
 
 
 
 
 
 
 
 
 
 
Implementation 
outcomes  
(organizational 
capacity) 
 
 
 
 
 
 
Staff survey  
(readiness to 
implement)  
RMT adapted 
 
 
 
 
 
 
 
 
Fidelity to EPI model  
FEPS-FS 
 
 
 
 
 
 
 
 
 Fidelity to 
NAVIGATE  
(clinician report) 
Clinician contact 
form 
 
 
 
 
 
 
 
 
 
Fidelity to NAVIGATE  
(client report) 
SURF 
 
 
 
 
 
 
 
 
 
Staff interview  
(value and feasibility)* 
CFIR adapted 
 
 
 
 
 
 
Implementation 
outcomes (staff 
capacity) 
 
 
 
 
 
 
Staff survey  
(knowledge and skills)  
ECHO survey 
 
 
 
 
 
 
 
 

 
 
Version #5: 26-Jun-2019 
18
Data Sources: The primary data collected for patients of the NAVIGATE protocol will be linked 
deterministically to data sources held at the Institute for Clinical Evaluative Sciences (ICES) via their unique 
health card number. The patient-level outcomes are enhanced by routine system-level outcome measurement 
and the capacity to compare outcomes among NAVIGATE subjects with two control populations: 1) FEP 
subjects in Ontario who are not part of the NAVIGATE trial but are receiving EPI services; and 2) FEP 
patients with no EPI contact. Only NAVIGATE patients who have consented will have their program-level 
data linked to ICES. The following ICES data sources will be used: the Ontario Health Insurance Plan 
(OHIP) which covers data for physician billings; the Canadian Institute for Health Information Discharge 
Abstract Database (CIHI-DAD) which covers all hospitalizations; the Ontario Mental Health Reporting 
System (OMHRS), covering adult mental health hospitalizations; the National Ambulatory Care Reporting 
System (NACRS) for emergency department (ED) visits; the Ontario Drug Benefits (ODB) which provides 
information on all prescriptions covered by the ODB (based on financial need for those under 65 years of age 
and universally for young people up to age 25); and the Registered Persons Database (RPDB) containing 
information on patient demographics and deaths. The Ontario Community Assessment of Need (OCAN) 
database captures all first episode patients attached to EPI programs in Ontario. OCAN will be used to 
develop a control population of FEP patients who are attached to non-NAVIGATE EPI programs. Statistics 
Canada estimates will be used to derive neighbourhood income and rurality. These data sources can be 
linked via the encrypted health card number such that all the information is available for each individual, and 
de-identified. 
 
Matched Controls: Each NAVIGATE subject will be matched to two types of subjects: 1) FEP patients 
attached to EPI programs who are not participating in the NAVIGATE trial; and 2) FEP patients with no EPI 
program attachment. Control cases will be matched to NAVIGATE subjects based on OCAN admission date 
for patients attached to EPI programs and at time of first diagnosis for those patients not attached to EPI 
programs. Patients with no EPI attachment will be identified using a validated algorithm that identifies all 
incident cases of psychotic disorders69. The diagnostic codes vary by database and included ICD-9, ICD-10, 
DSM-IV codes. In order to be a FEP case among the non-EPI attached control group, subjects will not have 
had a non-affective psychotic disorder diagnosis within five years of the matching date. We will use 
propensity score matching to create the 2 comparison groups of non-NAVIGATE subjects. Propensity score 
matching mimics the characteristics of a randomized trial using observational data. A propensity scores 
defined as the probability that a person is in the “exposed” category. In this case, the exposure condition is 
access to the NAVIGATE protocol. The propensity score is developed using logistic regression to model 
exposure to NAVIGATE as a function of observed covariates to yield a probability of NAVIGATE access 
for each subject. The propensity score model will include sociodemographic characteristics, clinical factors, 
and prior service use (defined below). We will match non-NAVIGATE to NAVIGATE subjects based on 
year of index diagnosis and caliper of the propensity score. Non-NAVIGATE Subjects will be assigned the 
same admission date as NAVIGATE subjects.  
 
System Outcomes (Hospitalization, ED visits, suicide attempts): Our primary outcome is number of 
psychiatric hospitalization days in the year following NAVIGATE admission. We will include secondary 
outcomes, such as ED visits and suicide attempts. Hospitalization-based outcomes will include number of 
psychiatric hospitalizations and time to first psychiatric hospitalization. We will measure psychiatric 
emergency department visits with a year of admission as well as ED visits for suicide attempts. Visits to 
psychiatrists and primary care physicians will also be counted70,71. Visits to primary care physicians will be 
stratified as mental health related versus non-mental health related based on a previously validated 
algorithm72.  
 

 
 
Version #5: 26-Jun-2019 
19
 
 
iv) AFFORDABILITY System Outcome – Cost (i.e. Implementation Evaluation Outcome iv): 
We will employ a costing algorithm developed in SAS®, and available at ICES73, to estimate all direct 
patient-level health care costs incurred by the public third-party payer (Ontario Ministry of Health and Long-
Term Care) across the three comparison groups. We will include costs of hospitalizations (both non-
psychiatric and psychiatric); ED visits; physician services (i.e. primary care, psychiatry and other care) and 
diagnostics tests; outpatient prescription drugs for individuals covered under the provincial public drug 
insurance plan only (for individuals under 65 who receive social assistance and currently universally for 
individuals under age 25); home care; long-term care; and other care (this includes other ambulatory care, 
such as same-day  surgery/procedures, cancer and dialysis clinic visits, and other hospital-based care, such as 
rehabilitation and complex continuing care). The costing methodology used in the algorithm will include a 
bottom-up/micro-costing approach to cost services at the individual level. This approach will make use of 
individual episodes of care or utilization in the health care system and attached prices (or costs or amounts 
paid) to each one. A top-down approach, which allocates corporate aggregate (i.e. institutional) costs to 
individual visits or cases/episodes of care, will be applied in cases where individual unit costs are not 
available (e.g. for institutional care settings). Assuming that all subjects will incur health care costs, we will 
use a generalized linear model with a gamma distribution and log link to model health care costs. Actual 
model parameters will be determined by the nature of the cost distributions.  
 
Covariates: We will extract information on sociodemographic characteristics, including age, sex, 
neighborhood-level income quintile, and rurality of residence. Clinical covariates will include type of 
diagnosis (schizophrenia, schizoaffective disorder, etc.), source of index diagnosis (hospitalization, 
outpatient visit), and history of visits with alcohol- or substance-related diagnoses. We will also measure 
prior service use for mental disorders, including the number of prior visits for mental health reasons with the 
primary care physician, visits to psychiatrist, emergency department visits related to mental health and 
addiction conditions, and previous psychiatric hospitalizations.  
 
Statistical Analysis: Our primary outcome is days in hospital in the year following NAVIGATE admission. 
We will compare the three groups (NAVIGATE, non-NAVIGATE EPI, and non-EPI FEP patients) across all 
covariates listed above. We will subsequently model total hospital days adjusting for covariates. The 
regression model will be determined by the distribution of the dependent variables. If normally distributed, 
linear regression; if the distribution assumes a Poisson distribution, we will use Poisson regression. 
Secondary outcomes will be modeled using logistic regression (for binary outcomes) or Cox Proportional 
Hazard modelling (for time-to-event outcomes).  
 
Patient-Level Outcomes – Analytic/Statistical Plan (Objective 3):  
Since this is an implementation study, based on the already established effectiveness of NAVIGATE, we do 
not have a comparator arm. We will compare patient level outcome data from our study with the aggregate 
data from the original NAVIGATE study to assess comparability of the intervention in different 
jurisdictions, using a matching-adjusted indirect comparison (MAIC) model. In this approach we adjust the 
population receiving the intervention to match the average baseline characteristics with a reference 
population. We then compare outcomes across balanced populations. This is facilitated by the use of 
identical patient-level outcome measures as in the RAISE study, and the collection of similar baseline 
characteristics that might influence outcome (e.g. age, education, sex, race, baseline illness severity). With a 
large number of patients, we will also explore sex-stratified analyses. Matching will be based on propensity 
score weighting. We will compare the primary patient-level outcome (total quality of life score) between our 

 
 
Version #5: 26-Jun-2019 
20
NAVIGATE and the U.S. RAISE study NAVIGATE groups over two years (baseline, 6 months, 12 months, 
18 and 24 months). At each time point, we will use percentage change in total QLS score, along with 95% 
confidence interval from the RAISE study and the present study. Using the MAIC approach, mean results 
with overlapping 95% confidence intervals are taken to indicate that the interventions produced equivalent 
results.   
 
With a sample size n=400 and repeated measures across five time-points (for QLS and BPRS), we will have 
the opportunity to identify subgroups of patients with different functional outcome and symptom trajectories 
using complex mixture modeling, namely latent class growth analysis (LCGA) and Latent Growth Mixture 
Modelling (LGMM). We can also model factors associated with outcomes in key subpopulations of interest. 
Given that 30% of individuals with first-episode psychosis disengage from services74, measuring the 
effectiveness of implementation in specific at-risk subpopulations can help inform future adaptations of the 
intervention to better serve YEA with an FEP. These at-risk subpopulations include YEA without family 
involvement74, those who use substances74, homeless youth75, youth from ethno-racial minority 
populations76-78, Indigenous youth79,80, and sexual minority youth81. Service in more remote areas (especially 
at our north-eastern sites) may be an additional vulnerability factor. We have engaged the OSSU’s Centre for 
Rural and Northern Health Research to incorporate some of these considerations. In addition, we have 
engaged OSSU Women’s Xchange given sex-based differences in prevalence and illness severity.  
 
Authentic Engagement of people with lived experience (Objective 4):  
Engagement process: Consistent with best practices for both youth82-84 and patient85-90 engagement, 
opportunities for youth and family participation will range from ad hoc, limited commitment (e.g., surveys or 
interviews) to full, ongoing participation with opportunities for research mentorship, and research team 
membership. Regarding full, ongoing participation, we will establish an advisory committee that will meet 
monthly (compensated), and provide guidance to the research team. This committee will guide: recruitment 
strategies; assessment and treatment protocols; outcome measures; interpretations of findings; knowledge 
translation goals and strategies to reach youth, families and other stakeholders; dissemination of the trial 
learnings to knowledge users. This committee will consist of patients and family members from each site and 
will initially be co-led by our CAMH Slaight Centre patient with lived experience and family member, but 
where leadership will be determined on an annual basis, to give participating site patient members an 
opportunity to lead. Leadership of this committee by a patient and family member with lived experience will 
help facilitate ongoing interaction among research, implementation and patient engagement leads to ensure 
that project activities are continuously informed by patient priorities.   
 
Preparation for Advisory Group Membership and Participation: In order to prepare patients and family 
members for advisory group membership, the co-leaders will discuss their readiness for becoming part of the 
NAVIGATE team (which includes NAVIGATE experts, local site leads, the PSSP facilitator, and an ECHO 
hub member). Special and specific NAVIGATE training will be held for patients and family representatives 
that is tailored to their learning needs. In addition, scientific, clinical, and program leaders will receive 
training on how to include patients and family members on a team in a meaningful manner. Patient and 
family representatives, i.e. the advisory committee will meet via monthly ECHO-style teleconference or in 
person.  
 
Evaluation of Engagement of Youth and Family Advisory Members: Following consultation with the OSSU 
Support Unit Innovations Strengthening Primary HealthCare through Research (INSPIRE-PHC) & Patient 
Engagement Resource Centre (PERC), we reviewed public and patient engagement evaluation tools92. We 
identified the Public and Patient Engagement Evaluation Tool (PPEET)93 and the PCORI engagement 

 
 
Version #5: 26-Jun-2019 
21
activity inventory94 as those that might most comprehensively evaluate engagement. In particular, the PPEET 
had the highest aggregate score on all categories, and includes a patient questionnaire (i.e. for service users, 
citizens, community representatives, and other stakeholders), a project questionnaire (i.e. for project lead, 
manager, or director), and an organization questionnaire (i.e. for senior leadership). We will administer a 
simplified PPEET and PCORI engagement activity inventory in addition to semi-structured interviews on a 
regular basis (e.g., monthly for PPEET and PCORI, less frequently for interviews).. 
 
Building on Experiential Knowledge to Further Evaluate NAVIGATE from the Patient and Family 
Perspective: This particular approach will engage patients and family members to build on our qualitative 
work regarding acceptability, feasibility, and preference for NAVIGATE, i.e. coordinated, manualized EPI 
care, and will serve to engage those patients and family members who prefer more limited involvement (i.e. 
through interviews). Patients and family members participating in this process will be different than those 
participating on the advisory committee. Semi-structured interviews will be conducted 1-3 times during the 
NAVIGATE implementation process. By working alongside patients and families through engagement 
activities focused on the content and purpose of NAVIGATE, all stakeholders have the opportunity to 
contribute to the dialogue on its development. We will use a combination of purposive and convenience 
sampling strategies for the study91. Each site will help facilitate the recruitment of approximately three 
patient and two family member representatives, to ensure recruitment of approximately 20 male and female 
patients and family members in total. We will aim to include patients and families from diverse 
demographic, geographic, or socio-economic backgrounds: e.g. LGBTQ, rural, Indigenous, urban socio-
economically disadvantaged and newly-immigrated populations.   
 
5. Study Duration 
 
The estimated length of time needed to complete the entire study (from enrolment of the first participant to 
completion of the last participant) is four years.  
 
Termination of Participants from the Study: 
Reasons for withdrawing individual participants from the study may include one or more of the following: 
 
a) Participant lost to follow-up. 
b) Withdrawal of consent: As this is a two part study, participants are free to withdraw consent in part 
or in entirety. Their participation in part or the entire study would be terminated accordingly. 
c) inability to continue giving informed consent to participate in the research study 
 
Any participant may be discontinued from the study at the discretion of the investigators if this is deemed to 
be in the best interest of the participant. The decision may be made either to protect the participant’s health 
and safety, or because it is part of the research plan that people who develop certain conditions may not 
continue to participate.  
 
Any research information recorded for, or resulting from, participation in this research study prior to the date 
that the participant formally withdrew their consent will be retained and may continue to be used and 
disclosed by the investigators for research purposes; however, no new data will be collected. 
 
 
6. Risks/Benefits 
 

 
 
Version #5: 26-Jun-2019 
22
6.1. Risks 
 
Assessments can involve emotional discomfort and possibly fatigue. Some of the assessment, in particular 
the questionnaires, may be upsetting to study participants. We will minimize this by having assessments 
performed by well-trained research staff with clinical experience and appropriate skills to maximize the 
participants’ comfort and keep their distress to a minimum during each visit.  
 
6.2. Benefits 
 
Past studies have shown that implementation of a team-based model of first episode psychosis care 
demonstrated significant benefits compared to treatment as usual. This study may benefit participants in that 
it might help with patient engagement, and greater quality of life. Ontario research has shown that EPI 
programs in general are associated with more contact with outpatient psychiatry, fewer ED visits, and 
reduced mortality12. The RAISE study found that the NAVIGATE intervention was associated with greater 
treatment retention, improvement in quality of life and psychopathology and involvement in work or school 
compared with standard community care20. The benefits of this study include the long-term contribution to 
determine whether the NAVIGATE will lead to improved health outcomes for youth and emerging adults 
experiencing early psychosis. For the longitudinal assessments, there is a reasonable likelihood that quality 
of care delivered through NAVIGATE will be superior. Also the use of measurement to assess progress, 
symptoms and functioning over time has been associated with improved outcomes. 
 
7. Innovation/Impact/KT/Future Plan 
 
Building on Existing Structures: We established new relationships and built on existing ones over the past 
several months and years to set the stage for this project. Efforts and substantial investment of time with 
persons with lived experience, family members, front-line clinicians, members and leaders of invested 
organizations and networks within Ontario, and across Canada along with U.S. EPI and NAVIGATE 
leadership now position us uniquely to conduct the study.  
 
i) Engagement with Persons with Lived Experience and Family Members: Relationships have been 
established with men and women with lived experience and family members. The Principal Knowledge User 
with lived experience has worked closely with the other investigators from the initial conception of this 
project, through the weekly planning calls, the teleconferences with community EPION sites, and in the 
writing and review of the study. Our family member with lived experience has also contributed significantly 
to the conception of this project based on her own experience across different care settings in Ontario in 
relation to her son who has experienced psychosis. Her desire to see reduction in variability of care, and 
better quality care across settings, all crucial patient-oriented goals for this project, is a driving force behind 
her participation. In addition, the aggregate participation of several persons with lived experience and family 
members, along with front-line clinicians in our qualitative research study that examined the acceptability 
and feasibility of manualized, measurement-based care in EPI, further informed our design of the study. 
 
ii) Engagement with Clinician/Front-Line Stakeholders and Community Sites: Key members of the 
study team arranged meetings with EPION-member EPI programs that participated in the Ontario fidelity 
study to introduce the concept of the study, obtained feedback on the proposal, and set the stage for ongoing 
collaborations. Communication was conducted via two-way video conferencing to ensure both verbal and 
non-verbal relational aspects were evident for all team members. During these meetings, each site asked 
questions, provided criticisms, and feedback. Our co-investigator with lived experience also participated in 

 
 
Version #5: 26-Jun-2019 
23
all of these meetings to provide his perspective on NAVIGATE, measurement-based care, and other aspects 
of the study.  
 
Operational Plan 
 
iii) Organizational Engagement - Bringing EPION (and community sites), PSSP, and ECHO-ONMH 
together: Our team includes the EPION co-Chair (Sarah Bromley) who has already received NAVIGATE 
training and led NAVIGATE implementation at CAMH. Ms. Bromley is the Principal Knowledge User 
Clinician. Dr. Janet Durbin, Principal Investigator, scientist at PSSP, and lead scientist for EPION will lead 
the fidelity implementation and evaluation at each site. Alexia Jaouich, Director of Implementation and 
Innovation at PSSP, will broadly oversee implementation efforts for this project, consistent with her 
expertise. Dr. Sockalingam, Principal Investigator, is the co-lead for ECHO-ONMH, and Dr. Crawford, co-
lead, ECHO is a co-investigator. Ms. E. Serhal, Project Director, ECHO, Knowledge User, will oversee the 
continued use of ECHO across sites. Through weekly grant planning meetings with other Principal and co-
investigators, along with meetings with the community sites, EPION, PSSP, and ECHO-ONMH leadership 
crystallized their respective roles, in-kind support, and motivation to pursue and succeed in the project. 
 
iv) Canadian and U.S. EPI Leadership: Dr. Voineskos, Director, Slaight Centre, Nominated Principal 
Investigator, will oversee all aspects of the study, and Dr. George Foussias will provide leadership in clinical 
training and expertise for participating sites, as well as analyses of the longitudinal patient-level data 
working with Dr. Haltigan. Dr. Jean Addington, co-investigator, will provide additional leadership and 
oversight of training. Dr. John Kane, Principal Investigator of the NIMH-Funded RAISE study who is a 
collaborator on the study, and along with Dr. J. Addington, will ensure RAISE-level NAVIGATE training 
for all community sites. Dr. Don Addington, chief author of the Canadian Schizophrenia Guidelines, is a 
collaborator and will oversee use of his FEPS-FS scale to evaluate fidelity to the EPI standard. Dr. Phil 
Tibbo, President of the CCEIP, also a collaborator, has interest to scale nationally. The presence of EPI 
leadership from western and eastern regions of Canada ensures opportunity for future work in other 
provinces, and unified national leadership. 
 
v) Support from the Ontario Health System: Dr. Kurdyak, one of the Principal Investigators, and Lead of the 
ICES Mental Health and Addictions Program, will oversee the system-level outcome evaluation, and will 
work closely with Dr. Anderson (co-investigator) and Dr. de Oliveira for data and cost analyses. Mr. Patrick 
Mitchell, Principal Knowledge User, and Director of the Mental Health branch within the Ontario Ministry 
of Health and Long-term Care (MOHLTC) will ensure that our implementation is aligned with ongoing 
provincial Mental Health Strategy goals. He has participated in all steps of planning for this work. His team 
is interested to translate any findings from the present work into policy, and specifically articulated a need 
for data analyses across the duration of the study, in order to inform policy as early as possible. Mr. Mitchell 
will also provide advice on shaping the knowledge generated by the project to maximize uptake, and will 
liaise with other MOHLTC divisions to facilitate discussion on scaling and spreading innovation and 
evidence from this project. Finally, as described previously, we have engaged the OSSU, and OSSU Units 
INSPIRE-PHC/PERC (Dahrouge, co-investigator), CRaNHR (Urajnik, co-investigator), and Women’s 
Xchange (Mason, collaborator), to assist with study design and analysis. 
 
8. Data Management and Integrity 
 
The basic protection against risk in this study will be provided by the Principal Investigators.  The PIs will 
have primary responsibility for monitoring of participants during the entire time they participate in the study. 

 
 
Version #5: 26-Jun-2019 
24
The PIs, scientific committee, and study team will meet regularly to review accrued data, data 
confidentiality, adherence to protocol design, recruitment and implementation. During meetings the scientific 
committee will also review the enrollment data, the accrual and integrity of clinical data, implementation and 
fidelity of NAVIGATE and any adverse event associated with the various components of the study.  
 
All data pertaining to a participant’s involvement in this study will be coded and stored in locked offices or 
in password-protected databases. This information will only be accessible to the research team. In unusual 
cases, a participant’s research records may be released in response to a court order. If the research team 
learns that a participant or someone with whom the participant is involved with is in serious danger or harm, 
an investigator will inform the appropriate agencies as per legal or regulatory requirements. 
 
The hard data are stored in a locked filing cabinet stored in a locked office to further protect participant 
anonymity. Data auditing, entry and quality control will be carried out regularly. Regularly scheduled, and as 
needed, communications between the study team and the Study PI will clarify any inconsistencies and 
ambiguities in the data. 
 
Study data will be entered in a secure database (REDCap) web-based electronic data capture platform 
offered at CAMH and managed by the CAMH Information Management Group and Research IT. At point-
of-entry, data values will undergo consistency edits (e.g., ID validation, range verification, duplicate 
detection) and personnel will be required to correct errors. Data management staff will run logic error 
programs to check for accuracy and irregularities within and across data structures and within and across 
sites. Quality assurance checks will be conducted daily and weekly by site personnel, as well as by data 
management staff. No personal identifiers will be included in study measure data entry forms, only initials 
and participant ID. Separate, unlinked REDCap databases will be maintained to study measures, participant 
consent forms, and patient referral information.   
 
System-level data will be stored at Institute for Clinical Evaluative Sciences (ICES) which has its own 
protocols for data management. All patient-level records at ICES are de-identified and linked across datasets 
by a unique identifier number following procedures approved by Ontario’s Information and Privacy 
Commissioner. 
 
9. Study Management and materials 
 
CAMH Investigators will retain a participant identification code list if they need to contact participants after the 
study. This list will contain the complete name, identification number, address, phone number, and emails (if 
provided) of all participants and will be held confidentially at the investigator’s site after completion of the 
study. 
 
An eCRF will be completed for each participant enrolled in the study. A participant screening log, noting 
reasons for screen failure, where applicable, will be maintained for all participants. The investigator will 
document the obtained informed consent and record all medications, medical history, and efficacy data in the 
eCRF. Psychopathology scales will be considered source documents and will be incorporated into the eCRF in a 
confidential manner. Participating sites will send the coordinating site (CAMH) numbers quarterly (numbers of 
new intakes in services/clinics); this will be set up as a survey in REDCAP and all of the data would be stored 
there. 
 
10. Recruitment and Consent 

 
 
Version #5: 26-Jun-2019 
25
 
a) Patient participation 
Recruitment for the study will be initiated by the clinical team at each participating site who is treating the 
potential study participant. The treating physician/clinical care team will not obtain consent to reduce the risk 
of undue influence in the consent process. They may identify potential research participants and obtain 
verbal permission from these potential participants for a member of the research team to approach them. 
They will enter the potential research participant’s name and contact information (phone number, email 
address) into a separate REDCap database (not linked with their study ID or study data) in order to securely 
transmit it to the CAMH research team 
 
Potential participants who indicate a further interest in hearing more about the study and provide 
assent/verbal consent to be contacted by a member of the research team will then be contacted by a member 
of the research team who will engage them in an informed consent process. Informed Consent will be 
obtained in-person or via 2-way videoconference by CAMH research personnel; participants who consent via 
2-way videoconference will read the consent form online and will provide digital consent signatures (by 
clicking a checkbox to indicate their consent). A copy of the signed consent will be emailed (or mailed, if 
requested) to the participant. PII data collected via 2-way videoconference consent process will not be linked 
to any study data. User rights will also be configured to limit access to any PII containing form to essential 
research staff only. 
 
All patients will be given the option to participate. Participation in the study is voluntary. The decision to 
participate will not affect patients’ receipt of treatment or clinical services including NAVIGATE. 
Participants will be informed that they have the option of terminating their participation at any time, without 
consequence and that no new data will be collected on them. Any existing data will be de-identified. 
 
Participants will be provided with a clear explanation of the objectives, procedures, risks and benefits of the 
study and all questions will be answered. Questions will be asked of participants to ensure that they 
understand the nature of the research, the risks and potential benefits of study participation, and their rights 
as research participants prior to obtaining their signature on the informed consent document. Because we 
believe that consent is an ongoing process in any study, we will continue to educate participants about the 
nature of the research and address any questions that may arise throughout the course of the study. We are 
not planning to use proxy consent.  
 
The delegated research staff will obtain collateral information (if necessary) from the sites’ clinical team to 
confirm diagnosis or obtain other clinically relevant information in the research study.  
b) Patient’s family member participation 
 
As a part of the informed consent process with patients, the research team will ask the patient if we can reach 
out to one of their family members in order to tell them about the study. The research team will also briefly 
explain that the family member participation involves the completion of brief surveys about their (the 
patient’s) functioning. If the patient consents, we will ask them to provide the contact information of a family 
member or caregiver. The research team will then contact the family member, introduce themselves, and 
read a brief consent script and ask them to provide an email address and/or cell phone number for the 
purpose of receiving a full consent form electronically. The research team will document this phone call in 
the study records, including whether email addresses/cell phone numbers were provided for research 
purposes. The electronic (web- or SMS-based, depending on what they provided) consent form will be sent 
immediately. If the participant (family member) clicks indicating that they do not consent, they will not be 

 
 
Version #5: 26-Jun-2019 
26
contacted again about the study. If they click indicating consent, they will immediately receive an electronic 
survey. The survey will contain the proxy-administered version of the WHODAS and Life Skills questionnaires, 
which ask about the patient’s recent level of functioning. The survey will also ask about basic demographic, 
clinical, and service use factors. The survey will take approximately 15 minutes to complete. This survey will be 
re-sent to the participants (family members) every six months, up to a total of 5 times, mirroring the patient study 
visits. 
 
 
11. Confidentiality 
 
There is a potential risk of breach of confidentiality that is inherent in all research protocols. Breach of 
confidentiality will be minimized by the staff who will maintain research data (identified only by participant 
code number not related to name, or date of birth) in separate charts and a dedicated password protected 
electronic database. A list of participant names, their ID numbers, their health card numbers, their birth dates, 
and information about how they can be reached will be kept in a separate password-protected database with 
access only to study personnel authorized by the PI. Procedures have been established, and will be followed, 
to minimize the risk of breach of confidentiality. Procedures to maintain confidentially include: (1) formal 
training sessions for all research staff emphasizing the importance of confidentiality; (2) specific procedures 
developed to protect participants’ confidentiality, and (3) formal mechanisms limiting access to information 
that can link data to individual participants. All information obtained from participants will be kept as 
confidential as possible. Computer based files/data will be entered into password-secured databases and 
paper-based files will be stored in a secure location. These data will only be accessible to personnel involved 
in the study and they will abide by confidentiality regulations of the REB. The ethics committee will be 
granted direct access to the study participants’ original study records for verification of study procedures and/or 
data, without violating the confidentiality of the participants, to the extent permitted by the law and regulations. 
 
Participants will not be identified by name in any publication of research results. Results will be published as 
group data without the use of characteristics that would identify individual participants. 
 
12. Financial Compensation 
 
Longitudinal Assessments: Participants will receive a stipend of $65 to 100 to compensate them for their 
time after they complete all the baseline and screening procedures of the study ($30 to 50 for diagnostic 
assessments and $35 to 50 for symptom assessments). At the 6 and 18 month follow-up visit participants will 
receive $35 to 50 for the completion of assessments. At the 12 and 24 month follow-up visits participants 
will receive $50 to 100 ($25 to 50 for diagnostic assessments and $25 to 50 for symptom assessments). Not 
all participants will complete all study visits, depending on when they are enrolled into the study. 
Participants will be compensated for the study visits they complete. They will also receive travel 
compensation at each study visit. However, if they drop out of the study or their participation is terminated, 
they will be reimbursed at the time of the last scheduled visit based on what they have completed. 
Compensation amounts within these ranges will be decided according to guidance from site-specific research 
ethics boards.  Compensation will be in the form of a Visa gift card, given in-person OR mailed to the 
participant’s home address. Participants will need a piece of government-issued identification in order to sign 
for this delivery, and the research team will record the participant’s address only for this purpose. Delivery 
confirmation will be used in lieu of participant signature confirming the compensation was received. 
Additionally, first and last names of all participants will be given to the CAMH research finance department 
as a part of standard research financial procedures for reimbursement reconciliation.  

 
 
Version #5: 26-Jun-2019 
27
 
CAMH Patient with Lived Experience: Patient of CAMH, who has received early psychosis 
intervention services, will participate in weekly scientific meetings, and monthly advisory group 
meetings. He/she will provide input into study design and objectives planning the proposed work and 
will receive compensation of $25/per hour. Their first and last name will be given to the CAMH research 
finance department as a part of standard research financial procedures. 
 
Family Member of CAMH Patient with Lived experience: Parent of a current CAMH patient, who is 
currently receiving early psychosis intervention at CAMH’s Slaight Centre, will provide input and 
feedback across the duration of the proposed study. She/he will receive compensation of $25/per hour. 
Her/his first and last name will be given to the CAMH research finance department as a part of standard 
research financial procedures. 
 
Advisory committee (youth and family members with lived experience): In addition to the CAMH youth 
with lived experience and family members, the team members of the advisory will provide input and 
feedback for the NAVIGATE program. This advisory group will include one or two patient or family 
members per site, to ensure that site-specific considerations related to the patient or family member are 
taken into account across the duration of the study. The advisory committee will meet on a monthly 
basis, and will receive up to $89 after each meeting ($25 for time and effort, $10 for travel expenses if 
applicable, and $20 for meal costs if applicable.  Compensation will be in the form of cash or a Visa gift 
card, given in-person OR mailed to the advisory committee member’s home address. Participants will need a 
piece of government-issued identification in order to sign for this delivery, and the research team will record 
the member’s address only for this purpose. Delivery confirmation will be used in lieu of member signature 
confirming the compensation was received. Additionally, first and last names of all advisory group members 
will be given to the CAMH research finance department as a part of standard research financial procedures. 
 
Qualitative interviews (patients and family members): Separate from the advisory committee we will 
conduct interviews to obtain experiential knowledge from patients and family members at each site 
related to the implementation of the NAVIGATE program. Approximately 20 interviews will be 
conducted at 1-3 time points.. The interviews will involve approximately 3 clients and 2 family 
members from each site and compensation of $40/per interview ($30 for time and $10 for travel). 
Compensation will be in the form of a Visa gift card, given in-person OR mailed to the participant’s home 
address. Participants will need a piece of government-issued identification in order to sign for this delivery, 
and the research team will record the member’s address only for this purpose. Delivery confirmation will be 
used in lieu of participant signature confirming the compensation was received. Additionally, first and last 
names of all participants will be given to the CAMH research finance department as a part of standard 
research financial procedures. 
 
 
References 
 
1. McGlashan TH. Early Detection and Intervention of Schizophrenia: Rationale and Research. British 
Journal of Psychiatry. Supplement. 1998;172(33):3-6.  
2. Birchwood M, Todd P, Jackson C. Early Intervention in Psychosis. The Critical Period Hypothesis. 
British Journal of Psychiatry. Supplement. 1998;172(33):53-59.  
3. Arnett JJ. Emerging Adulthood. A Theory of Development from the Late Teens Through the Twenties. 
The American Psychologist. 200 May;55(5):469-480.  

 
 
Version #5: 26-Jun-2019 
28
4. World Economic Forum and the Harvard School of Public Health. (2011) Methodological Appendix: 
The Global Economic Burden of Non-Communicable Diseases.  
5. Schoenbaum M, Sutherland JM, Chappel A, Azrin S, Goldstein AB, Rupp A, Heinssen RK. Twelve-
Month Health Care Use and Mortality in Commercially Insured Young People With Incident Psychosis 
in the United States. Schizophrenia Bulletin. 2017 Oct 21;43(6):1262-1272.  
6. Gatov E, Rosella L, Chiu M, Kurdyak PA. Trends in Standardized Mortality Among Individuals with 
Schizophrenia, 1993-2014: A Population-Based, Repeated Cross-Sectional Study. CMAJ. 2017 Sep 
18;189(37):E1177-E1187.  
7. Correl CU, Solmi M, Veronese N, Bortolato B, Rosson S, Santonastaso P, Thapa-Chhetri N, Fornaro M, 
Gallicchio D, Collantoni E, Pigato G, Favaro A, Monaco F, Kohler C, Vancampfort D, Ward PB, 
Gaughran F, Carvalho AF, Stubbs B. Prevalence, Incidence and Mortality from Cardiovascular Disease 
in Patients with Pooled and Specific Severe Mental Illness: A Large-Scale Meta-Analysis of 3,211,768 
patients and 113,383,368 controls. World Psychiatry. 2017 Jun; 16(2): 163-180.  
8. Hjorthoj C, Sturup AE, McGrath JJ, Nordentoft M. Years of Potential Life Lost and Life Expectancy in 
Schizophrenia: A Systemic Review and Meta-Analysis. The Lancet. 2017 Apr;4(4):295-301.  
9. Craig TK, Garety P, Power P, Rahaman N, Colbert S, Fornells-Ambrojo M, Dunn G. The Lambeth Early 
Onset (LEO) Team: randomised Controlled Trial of the Effectiveness of Specialised Care for Early 
Psychosis. BMJ. 2004 Nov 6;329(747):1067.  
10. Kuipers E, Holloway F, Rabe-Hesketh S, Tennakoon L; Croydon Outreach and Assertive Support Team 
(COAST). An RCT of Early Intervention in Psychosis: Croydon Outreach and Assertive Support Team 
(COAST). Social Psychiatry and Psychiatric Epidemiology. 2004 May;39(5):358-363.  
11. Grawe RW, Falloon IR, Widen JH, Skogvoll E. Two Years of Continued Early Treatment for Recent-
Onset Schizophrenia: A Randomised Controlled Study. Acta Psychiatrica Scandinavica. 2006 
Nov;114(5):328-336.  
12. Anderson KK, Norman R, Macdougall A, Edwards J, Palaniyappan L, Lau C and Kurdyak P. The Real 
World Effectiveness of Early Psychosis Intervention: Comparison of Service Users and Non-Users in 
Population-Based Health Administrative Data. American Journal of Psychiatry, in press.  
13. Petersen L, Jeppesen P, Thorup A, Abel MB, Ohlenschlaeger J, Christensen TO, Krarup G, Jorgensen P, 
Nordentoft M. A Randomised Multicentre Trial of Integrated Versus Standard Treatment for Patients 
with a First Episode of Psychotic Illness. BMJ. 2005 Sep 17;331(7517):602.  
14. Malla A, Schmitz N, Norman R, Archie S, Windell D, Roy P, Zipursky RB. A Multisite Canadian Study 
of Outcome of First-Episode Psychosis Treated in Publicly Funded Early Intervention Services. 
Canadian Journal of Psychiatry. 2007 Sep;52(9):563-571.  
15. Bond GR, Drake RE, Luciano A. Employment and Educational Outcomes in Early Intervention 
Programmes for Early Psychosis: A Systemic Review. Epidemiology and Psychiatric Sciences. 2015 
Oct;24(5):446-457.  
16. Fusar-Poli P, McGorry PD, Kane JM. Improving Outcomes of First-Episode Psychosis: An Overview. 
World Psychiatry. 2017 Oct;16(3):251-265.  
17. Jaaskelainen E, Juola P, Kirvonen N, McGrath JJ, Saha S, Isohanni M, Veijola J, Miettunen J. A 
Systematic Review and Meta-Analysis of Recovery in Schizophrenia. Schizophrenia Bulletin. 2013 
Nov;39(6):1296-1306.  
18. Global Burden of Disease Study 2013 Collaborators. Global, Regional, and National Incidence, 
Prevalance, and Years Lived with Disability for 301 Acute and Chronic Diseases and Injuries in 188 
Countries, 1990-2013: A Systemic Analysis for the Global Burden of Disease Study 2013. Lancet. 2015 
Aug 22;386(9995):743-800.  

 
 
Version #5: 26-Jun-2019 
29
19. Addington D, Anderson E, Kelly M, Lesage A, Summerville C. Canadian Practice Guideline for 
Comprehensive Community Treatment for Schizophrenia and Schizophrenia Spectrum Disorders. 
Canadian Journal of Psychiatry. 2017 Sep;62(9):662672.  
20. Kane JM, Robinson DG, Schooler NR, et al. Comprehensive Versus Usual Community Care for First-
Episode Psychosis: 2-Year Outcomes From the NIMH RAISE Early Treatment Program. The American 
Journal of Psychiatry. 2016;173(4):362-372.  
21. Durbin J, Selick A, Hierlihy D, Moss S, Cheng C. A First Step in System Improvement: A Survey of 
Early Psychosis Intervention Programmes in Ontario. Early Intervention in Psychiatry. 2016 
Dec;10(6):485-493.  
22. Nolin M, Malla A, Tibbo P, Norman R, Abdel-Baki A. Early Intervention for Psychosis in Canada: What 
is the State of Affairs? Canadian Journal of Psychiatry. 2016 Mar;61(3):186-194.  
23. Catts SV, Evans RW, O’Toole BI, Carr VJ, Lewin T, Neil AL, Harris MG, Frost AD, Crissman BR, 
Eadie K. Is a National Framework for Implementing Early Psychosis Services Necessary? Results of a 
Survey of Australian Mental Health Service Directors. Early Intervention in Psychiatry. 2010 
Feb;4(1):25-30.  
24. Cheng C, Dewa CS, Goering P. Matryoshka Project: Lessons Learned About Early Intervention in 
Psychosis Programme Development. Early Intervention in Psychiatry. 2011 Feb;5(1):64-69.  
25. Ministry of Health and Long-Term Care (2011). Early Psychosis Intervention Program Standards. 
Retrieved from http://www.health.gov.on.ca/english/providers/pub/mental/epi_program_standards.pdf  
26. Fixsen DL, Blasé KA, Naoom SF, Wallace F. Core Implementation Components. Research on Social 
Work Practice. 2009 Sep;19(5):531-540.  
27. Standards Implementation Steering Committee. (2012). Implementation of Early Psychosis Intervention 
Program Standards in Ontario: Results from a Provincial Survey: Centre for Addiction and Mental 
Health and the Ontario Working Group for Early Psychosis Intervention, Toronto, Ontario.  
28. Durbin J, Cheng C, Langill G, Addington D, Archie S, Selick A. Conducting Fidelity Assessments in 
Ontario Early Psychosis Intervention Programs: Evaluation of a Pilot Implementation Initiative. In 
Preparation.  
29. Addington DE, Norman R, Bond GR, Sale T, Melton R, McKenzie, Wang J. Development and Testing 
of the First-Episode Psychosis Services Fidelity Scale. Psychiatric Services. 2016 Sep 1;67(9):1023-25.  
30. Kane JM, Schooler NR, Marcy P, Correll CU, Brunette MF, Mueser KT, Rosenheck RA, Addington J, 
Estroff SE, Robinson J, Penn DL, Robinson DG. The RAISE Early Treatment Program for First-Episode 
Psychosis: Background, Rationale, and Study Design. Journal of Clinical Psychiatry. 2015 
Mar;76(3):240-246.  
31. Rosenheck R, Mueser KT, Sint K, Lin H, Lynde DW, Glynn SM, Robinson DG, Schooler NR, Marcy P, 
Mohamed S, Kane JM. Supported Employment and Education in Comprehensive, Integrated care for 
First Episode Psychosis: Effects on Work, School, and Disability Income. Schizophrenia Research. 2017 
Apr;182:120-128.  
32. Rosenheck R, Leslie D, Sint K, Lin H, Robinson DG, et al. Cost-Effectiveness of Comprehensive, 
Integrated Care for First Episode Psychosis in the NIMH RAISE Early Treatment Program. 
Schizophrenia Bulletin. 2016 Jul;42(2):896-906.  
33. Mueser KT, Penn DL, Addington J, Brunette MF, Gingerich S, Glynn SM, Lynde DW, Lynde DW, 
Gottlieb, Meyer-Kalos P, McGurk SR, Cather C, Saade S, Robinson DG, Schooler NR, Rosenheck RA, 
Kane JM. The NAVIGATE Program for First-Episode Psychosis: Rationale, Overview, and Description 
of Psychosocial Components. Psychiatric Services. 2015 Jul;66(7):680-690.  
34. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of 
the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, 
National Implementation Research Network; 2005.  

 
 
Version #5: 26-Jun-2019 
30
35. Fixsen DL, Blasé KA, Van Dyke MK. Mobilizing Communities for Implementing Evidence-Based 
Youth Violence Prevention Programming: A Commentary. American Journal of Community Psychology. 
2011 Sep;48(1-2):133-137.  
36. Arora S, Kalishman S, Dion D, et al. Partnering urban academic medical centers and rural primary care 
clinicians to provide complex chronic disease care. Health Aff (Millwood). 2011;30(6):1176-1184.  
37. Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus infection by primary 
care providers. N Engl J Med. 2011;364(23):2199-2207. 
38. United States Congress. Public Law 114-270-Dec. 14, 2016. Accessed from the United States Congress 
website: https://www.congress.gov/114/plaws/publ270/PLAW-114publ270.pdf.  
39. Sockalingam S, Arena A, Serhal E, Mohri L, Alloo J, Crawford A. Building Provincial Mental Health 
Capacity in Primary Care: An Evaluation of a Project ECHO Mental Health Program. Academic 
Psychiatry. 2017 Jun 7. [Epub ahead of print]  
40. Henderson J, Browline E, Rosenkranz S, Chaim G, Beitchman J. Integrated Knowledge Translation and 
Grant Development: Addressing the Research Practice Gap through Stakeholder-informed Research. 
Journal of the Canadian Academy of Child and Adolescent Psychiatry. 2013 Nov;22(4):268-274.  
41. Chaim G, Henderson J. (2013). Youth Services System Review – Report. CAMH. Retrieved from: 
http://eenet.ca/sites/default/files/Youth-Services-System-Review-Report-Part-1-FINAL3.pdf  
42. Chaim G, Henderson J. (2014). Youth Service System Review: Phase 2 Report. CAMH. Retrieved from: 
http://eenet.ca/sites/default/files/YSSR_Phase2_Report_July28.pdf  
43. Henderson J, Chaim G. (2014). National Youth Screening Project Phase Two Report. CAMH. Retrieved 
from: http://eenet.ca/sites/default/files/NYSP_Phase2Report_June23.pdf  
44. Henderson JL, Cheung A, Cleverley K, Chaim G, Moretti ME, de Oliveira C, Hawke LD, Willan AR, 
O’Brien D, Heffernan O, Herzog T, Courey L, McDonald H, Grant E, Szatmari P. Integrated 
Collaborative Care Teams to Enhance Service Delivery to Youth with Mental Health and Substance Use 
Challenges: Protocol for a Pragmatic Randomised Controlled Trial. BMJ Open. 2017 Feb 
6;7(2):e014080.  
45. Zaheer J, Berkhout S, Kurdyak P, Foussias G, Soklaridis S, Voineskos A. Client and Family Perspectives 
on Standardized, Manualized, Coordinated Care in First Episode Psychosis: A Qualitative Mixed-
Methods Study. In Preparation.  
46. Ferrari M, Flora N, Anderson KK, et al. The African, Caribbean and European (ACE) Pathways to Care 
Study: A Qualitative Exploration of Similarities and Differences Between African-origin, Caribbean-
Origin and European-Origin Groups in Pathways to Care for Psychosis. BMJ Open. 2015;5(1):e006562.  
47. Lester H, Khan N, Jones P, et al. Service Users’ Views of Moving on from Early Intervention Services 
for Psychosis: A longitudinal Qualitative Study in Primary Care. Br J Gen Pract. 2012;62(596):183-190.  
48. Hui CLM, Lo MCL, Chan EHC, et al. Perception Towards Relapse and its Predictors in Psychosis 
Patients: A Qualitative Study. Early Interv Psychiatry. 2016:1-7.  
49. Connell M, Schweitzer R, King R. Recovery From First-Episode Psychosis and Recovering Self: A 
Qualitative Study. Psychiatr Rehabil J. 2015;38(4):359-364.  
50. Nilsen L, Norheim I, Frich JC, Friis S, Røssberg JI. Challenges for group leaders working with families 
dealing with early psychosis: A qualitative study. BMC Psychiatry. 2015;15(1):1-8.  
51. Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in 
Observational Studies. Multivariate Behav. Res. 2011;46(3):399-424  
52. de Oliveira C, Cheng J, Rehm J, Kurdyak P. The Economic Burden of Chronic Psychotic Disorders in 
Ontario. Journal of Mental Health Policy and Economics. 2016 Dec 1;19(4):181-192.  
53. Heinrichs DW, Hanlon TE, Carpenter WT, Jr. The Quality of Life Scale: an instrument for rating the 
schizophrenic deficit syndrome. Schizophrenia Bulletin. 1984;10(3):388-398.  

 
 
Version #5: 26-Jun-2019 
31
54. Leifker FR, Patterson TL, Heaton RK, Harvey PD. Validating Measures of Real-World Outcome: The 
Results of the VALERO expert survey and RAND panel. Schizophrenia Bulletin. 2011 Mar;37(2):334-
343.  
55. Üstün, T. B., & World Health Organization. (2010). Measuring health and disability: Manual for WHO 
Disability Assessment Schedule WHODAS 2.0. Geneva: World Health Organization.  
56. First MB, Williams JBW, Karg RS, Spitzer RL: Structured Clinical Interview for DSM-5—Research 
Version (SCID-5 for DSM-5, Research Version; SCID-5-RV, Version 1.0.0). Arlington, VA, American 
Psychiatric Association, 2015  
57. Zarate CA, Weinstock L, Cukor P, Morabito C, Leah L, Burns C, Baer L. Applicability of Telemedicine 
for Assessing Patients with Schizophrenia: Acceptance and Reliability. Journal of Clinical Psychiatry. 
1997 Jan;58(1):22-25.  
58. Durlak JA, DuPre EP. Implementation Matters: A Review of Research on the Influence of 
Implementation on Program Outcomes and the Factors Affecting Implementation. American Journal of 
Community Psychology. 2008 Jun;41(3-4):327-350.  
59. Meyers DC, Durlak JA, Wandersman A. The Quality Implementation Framework: A Synthesis of 
Critical Steps in the Implementation Process. American Journal of Community Psychology. 2012 
Dec;50(3-4):462-480.  
60. Fortney J, Enderle M, McDougall S, Clothier J, Altman L, Curran G. Implementation Outcomes of 
Evidence-Based Quality Improvement for Depression in VA Community Based Outpatient Clinics. 
Implementation Science. 2012 Apr 11;7:30.  
61. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation Research in 
Mental Health Services: An Emerging Science with Conceptual, Methodological, and Training 
Challenges. Administration and Policy in Mental Health. 2009 Jan;36(1):24-34.  
62. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering Implementation of 
Health Services Research Findings Into Practice: A Consolidated Framework for advancing 
Implementation Science. Implementation Science. 2009 Aug 7;4:50.  
63. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ, What Does it Mean to 
“Employ” the RE-AIM Model? Evaluation & the Health Professions. 2013 Mar;36(1):44-66.  
64. Damschroder LJ, Lowery JC. Evaluation of a Large-Scale Weight Management Program Using the 
Consolidated Framework for Implementation Research (CFIR). Implementation Science. 2013 May 
10;8:51.  
65. Damschroder LJ, Moin T, Datta Sk, Reardon CM, Steinle N, Weinreb J, Billington CJ, Maciejewski ML, 
Yancy WS, Hughes M, Makki F, Richardson CR. Implementation and Evaluation of the VA DPP 
Clinical Demonstration: Protocol for a Multi-Site Non-Randomized Hybrid Effectiveness-
Implementation Type III Trial. Implementation Science. 2015 May 21;10:68.  
66. Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S: The Impact of Project ECHO on Participant 
and Patient Outcomes: A Systematic Review. Academic Medicine. 2016 Oct;91(10):1439-1461.  
67. Moore DE Jr, Green JS, Gallis HA. Achieving Desired Results and Improved Outcomes: Integrating 
Planning and Assessment Throughout Learning Activities. Journal of Continuing Education in the 
Health Professions. 2009 Winter;29(1):1-15.  
68. Scott VC, Kenworthy T, Godly-Reynolds E, Bastien G, Scaccia J, McMickens C, Rachel S, Cooper S, 
Wrenn G, Wandersman A. The Readiness for Integrated Care Questionnaire (RICQ): An instrument to 
assess readiness to integrate behavioral health and primary care. American Journal of Orthopsychiatry. 
2017;87(5):520-530. 
69. Kurdyak P, Lin E, Green D, Vigod S. Validation of a Population-Based Algorithm to Detect Chronic 
Psychotic Illness. Canadian Journal of Psychiatry. 2015 Aug;60(8):362-368.  

 
 
Version #5: 26-Jun-2019 
32
70. Gatov E, Kurdyak P, Sinyor M, Holder L, Schaffer A. Comparison of Vital Statistics Definitions of 
Suicide Against a Coroner Reference Standard: A Population-Based Linkage Study. Canadian Journal of 
Psychiatry. 2017 Jan 1:706743717737033.  
71. Saunders NR, Lebenbaum M, Stukel TA, Lu H, Urquia ML, Kurdyak P, Guttmann A. Suicide and Self-
Harm Trends in Recent Immigrant Youth in Ontario, 1996-2012: A Population-Based Longitudinal 
Cohort Study. MBJ Open. 2017 Sep1;7(9):e014863.  
72. Steele LS, Glazier RH, Lin E, Evans M. Using Administrative Data to Measure Ambulatory Mental 
Health Service Provision in Primary Care. Medical Care. 2004 Oct;42(10):960-965.  
73. Wodchis WP, Austin PC, Henry DA. A 3-Year Study of High-Cost Users of Health Care. CMAJ. 2016 
Feb 16;188(3):182-188.  
74. Doyle R, Turner N, Fanning F, Brennan D, Renwick L, Lawlor E, Clarke M. First-Episode Psychosis and 
Disengagement from Treatment: A Systematic Review. Psychiatric Services. 2014 May 1;65(5):603-611.  
75. Herman DB, Susser ES, Jandorf L, Lavelle J, Bromet EJ. Homelessness Among Individuals with 
Psychotic Disorders Hospitalized for the First Time: Findings from the Suffolk County Mental Health 
Project. American Journal of Psychiatry. 1998 Jan;155(1):109-113.  
76. Anderson KK, Flora N, Ferrari M, Tuck M, Archie S, Kidd S, Tang T, Kirmayer LJ, McKenzie K; Ace 
Project Team. Pathways to First-Episode Care for Psychosis in African-Carribbean-, and European-
Origin Groups in Ontario. Canadian Journal of Psychiatry. 2015 May;60(5):223-231.  
77. Anderson KK, McKenzie JK, Kurdyak P. Examining the Impact of Migrant Status on Ethnic Differences 
in Mental Health Service Use Preceding a First Diagnosis of Schizophrenia. Social Psychiatry and 
Psychiatric Epidemiology. 2017 Aug;52(8):949-961.  
78. Nagendra A, Schooler NR, Kane JM, Robinson DG, Mueser KT, Estroff SE, Addington J, Marcy P, 
Penn DL. Demographic, Psychosocial, Clinical, and Neurocognitive Baseline Characteristics of Black 
Americans in the RAISE-ETP study. Schizophrenia Research. 2017 Jul 11. pii: S0920-9964(17)30376-6. 
Epub ahead of print.  
79. Whitbeck LB, Sittner Hartshorn KJ, Crawford DM, Walls ML, Gentzler KC, Hoyt DR. Mental and 
Substance Use Disorders from Early Adolescence to Young Adulthood among Indigenous Young 
People: Final Diagnostic Results from an 8-Year Panel Study. Social Psychiatry and Psychiatric 
Epidemiology. 2014 Jun;449(6):961-973.  
80. Soleimani A, Rosychuk RJ, Newton AS. Predicting Time to Emergency Department Re-Visits and 
Impatient Hospitalization Among Adolescents Who Visited an Emergency Department for Psychotic 
Symptoms: A Retrospective Cohort Study. BMC Psychiatry. 2016 Nov 9;16(1):385.  
81. Bolton SL, Sareen J. Sexual Orientation and its Relation to Mental Disorders and Suicide Attempts: 
Findings from a Nationally Representative Sample. Canadian Journal of Psychiatry. 2011 Jan;56(1):35-
43.  
82. Hart, Roger A. (1992). Children's Participation: From tokenism to citizenship, Innocenti Essay no. 4, 
International Child Development Centre, Florence.  
83. Pereira, N. (2007). Ready... Set... Engage! Building Effective Youth/Adult Partnerships for a Stronger 
Child and Youth Mental Health System. Toronto: Children’s Mental Health Ontario & Ottawa: The 
Provincial Centre of Excellence for Child and Youth Mental Health at CHEO.  
84. Zeldin S, Christens BD, Powers JL. The Psychology and Practice of Youth-Adult Partnership: Bridging 
Generations for Youth Development and Community Change. American Journal of Community 
Psychology. 2013 Jun;51(3-4):385-397.  
85. Abelson J. (2015) Patient Engagement and Canada’s SPOR Initiative. McMaster University. Retrieved 
from: http://ossu.ca/wp-content/uploads/OSSU-Patient-Engagement-Resource-Document-May2015.pdf  
86. CIHR’s 
Citizen 
Engagement 
Handbook. 
(2012). 
CIHR. 
Retrieved 
from: 
http://www.cihr-
irsc.gc.ca/e/documents/ce_handbook_e.pdf  

 
 
Version #5: 26-Jun-2019 
33
87. INVOLVE (2012) Briefing notes for researchers: involving the public in NHS, public health and social 
care research. INVOLVE, Eastleigh.  
88. PCORI 
Engagement 
Rubric 
(2014) 
Retrieved 
from: 
https://www.pcori.org/sites/default/files/Engagement-Rubric.pdf  
89. Strategy for Patient-Oriented Research: Patient Engagement Framework. (2014). CIHR. Retrieved from: 
http://www.cihr-irsc.gc.ca/e/documents/spor_framework-en.pdf  
90. Tambuyzer E, Pieters G, Van Audenhove C. Patient Involvement in Mental Health Care: One Size Does 
Not Fit All. Health Expectations. 2014 Feb;17(1):138-150.  
91. Marshall MN. Sampling for Qualitative Research. Family Practice.1996 Dec;13(6):522-525.  
92. Boivin A, Abelson J, Gauvin FP. Patient and Public Engagement Evaluation Toolkit. Centre of 
Excellence on Partnership Patients and the Public.  
93. Abelson J, Li K, Wilson G, Shields K, Schneider C, Boesveld S. Supporting Quality Public and Patient 
Engagement in Health System Organizations: Development and Usability Testing of the Public and 
Patient Engagement Evaluation Tool. Health Expectations. 2016 Aug;19(4):817-827.  
94. Forsythe LP, Ellis LE, Edmundson L, Sabharwal R, Rein A, Konopka K, Frank L. Patient and 
Stakeholder Engagement in the PCORI Pilot Projects: Description and Lessons Learned. Journal of 
General Internal Medicine. 2016 Jan;31(1):13-21.  
95. Moberg, DP. Screening for Alcohol and Other Drug Problems using the Adolescent Alcohol and Drug 
Involvement Scale (AADIS). Center for Health Policy and Program Evaluation: University of Wisconsin 
Madison. 2005. 
96. Rosen A, Trauer T, Hadzi-Pavlovic D, Parker G. Development of a brief form of the Life Skills Profile: 
the LSP-20. Aust N Z J Psychiatry. 2001 Oct; 35(5): 677-83. 
97. Tobon JI, Reid GJ, Goffin RD. Continuity of Care in Children's Mental Health: development of a 
measure. Adm Policy Ment Health. 2014;41(5):668-686 
98. Brunk, M., Koch, J. R., & McCall, B. (2000). Report on parent satisfaction with services at community 
services boards. Richmond, VA: Virginia Department of Mental Health, Mental Retardation, and 
Substance Abuse Services. 
99. McGuire-Snieckus R, McCabe R, Catty J, Hansson L, Priebe S. A new scale to assess the therapeutic 
relationship in community mental health care: STAR. Psychol Med. 2007;37(1):85-95. 
100. 
O’Connell, M., Tondora, J., Croog, G., Evans, A., & Davidson, L. (2005). From rhetoric to routine: 
Assessing perceptions of recovery-oriented practices in a state mental health and addiction 
system. Psychiatric Rehabilitation Journal, 28(4), 378-386