UH3 Pragmatic Trial of Population-based Programs to Prevent Suicide Attempt
Phase: NA
Conditions: Depression, Suicidal Ideation
Sponsor: Kaiser Permanente

  
  
CTOBB Protocol Template [DRAFT] 
 
 
CLINICAL STUDY PROTOCOL  
 
 
 
PRAGMATIC TRIAL OF POPULATION-BASED PROGRAMS TO PREVENT 
SUICIDE ATTEMPT 
  
 
Version Number: 13 
Version Date: 9/23/2016 
 
 
Principal Investigator:  Gregory Simon MD MPH, Group Health Research Institute 
_____________________________ 
 
  
  
 
Supported by: 
 
The National Institute of Mental Health 
 
UH3 MH007755-02  
 
 
   
  
 
 
 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 2 of 33 
Grant Number: UH3 MH007755-02 
Protocol Number: n/a 
 
Name of Coordinating Center [CC]: Group Health Research Institute 
Name of Data center [DCC]: Group Health Research Institute 
 
Principal Investigator [PI] & Clinical Site [list all]: 
PI Name 
Site #  Location 
Phone 
E-mail 
Gregory Simon 
1 
Group Health 
Cooperative 
206-287-2979 
Simon.g@ghc.org 
Arne Beck 
2 
Kaiser Permanente 
Colorado 
303-614-1326 
Arne.Beck@kp.org 
Rebecca 
Rossom 
3 
HealthPartners 
952-967-5006 
Rebecca.C.Rossom@ 
HealthPartners.com 
Gregory Clarke 
4 
Kaiser Permanente 
Northwest 
503-335-6673 
Greg.Clarke@kpchr.org 
 
 
Study Phase and Target enrollment:  
 
Phase of clinical investigation: N/A (effectiveness trial of system-level intervention). 
   
Target enrollment:  
19,500 Patients       
 
Project Uses Ionizing Radiation:    No  
 Yes (attach RSC/RDRC documentation) 
  
 
 Medically-indicated only 
  
 
 Research-related only 
  
 
 Both 
 
IND/IDE 
 
 
 
 No   
Yes (attach FDA documentation) 
  
Drug/Device: _____________________ 
Specify the assigned name and/or #, when applicable, the code number of the 
investigational drug(s). 
 
 
IND #_____________________ 
  
Drug/Device supplied by: ____________________________ 
 
Regulatory Approvals: 
 
Multi-institutional Project 
 No   
 Yes  
 
 
 
 
 
Group Health Cooperative      
Date of IRB approval ______________(attach IRB documentation) 
 
 
 
Kaiser Permanente Colorado      
Date of IRB approval ______________(attach IRB documentation) 
 
 
 
Health Partners      
Date of IRB approval ______________(attach IRB documentation) 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 3 of 33 
 
Kaiser Permanente Northwest      
Date of IRB approval ______________(attach IRB documentation) 
 
 
NIMH Data and Safety Monitoring Board  
 No   
 Yes  
 
Samples are being stored  
 
 
 No   
 Yes  
 
      
 
 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 4 of 33 
Précis: 
 
Objective - This pragmatic clinical trial will evaluate the effectiveness of two population-
based outreach programs for preventing suicide attempts among patients identified as at-
risk for suicide.   
 
Study population - The study will be conducted at Group Health (the lead site) and at 
HealthPartners, KP Colorado, KP Northwest.  Electronic health records will be used to 
identify outpatients aged 18 or older who respond to a routinely administered PHQ 
depression questionnaire by reporting thoughts of death or self-harm “most of the days” 
or “nearly every day.”   
 
Design - Eligible patients will be automatically assigned to continue in usual care (1/3) or 
to usual care plus either of the two intervention programs (1/3 each).  Those assigned to 
usual care will not be contacted at all by study staff and will continue to receive usual 
care from treating primary care and mental health providers.  Both intervention programs 
involve outreach by Epic secure messaging (with optional telephone outreach for those 
not reading messages).  The Care Management intervention includes routine outreach to 
assess ongoing risk of suicide attempt, and care management to monitor and facilitate 
ongoing engagement in outpatient follow-up.  The Care Manager will coordinate care 
with treating Group Health providers (ongoing usual care) using Epic Staff Messaging (or 
telephone contacts if necessary).  The Skills Training intervention uses an online skills 
training program to support patients in developing and using self-management skills for 
emotion regulation and crisis management.  A Coach will monitor each participant’s use 
of the program and send periodic messages (using Epic secure messaging) to encourage 
and support use of the program and practice of program skills.  Each intervention 
continues for up to one year.   
 
Outcome measures - In all three groups (Usual Care, Care Management, and Skills 
Training) computerized health system records will be used to identify medically-treated 
suicide attempts over 18 months following randomization.  Procedures for invitation, 
consent, and intervention delivery are described in more detail below. 
 
 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 5 of 33 
Table of Contents 
 
Introduction 
6 
Study Objective 
6 
Subjects 
6 
Design and Methods 
7 
Storage of Data 
14 
Risks and Discomforts 
14 
Safety Monitoring in Intervention Programs 
15 
Outcome Measures 
16 
Statistical Analysis 
19 
Human Subjects Protection 
21 
Anticipated Benefit 
21 
Classification of Risk 
22 
Consent Documents and Process 
23 
Data and Safety Monitoring 
24 
Quality Assurance 
26 
Adverse Event Reporting 
26 
Alternatives to Participation 
27 
Confidentiality 
28 
Conflicts of Interest 
28 
Technology and Data Sharing 
29 
Research and Travel Compensation 
30 
Governance Plan 
30 
Key References 
32 
List of Attachments/Appendices 
32 
 
List of Abbreviations 
 
• GHRI – Group Health Research Institute 
• KPCO – Kaiser Permanente Colorado 
• KPNW – Kaiser Permanente Northwest 
• PHQ9 – Patient Health Questionnaire 9 
  
 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 6 of 33 
Introduction/Background/Significance 
 
Suicide ranks 10th among all causes of mortality in the US, accounting for over 38,000 
deaths in 2010.  Non-fatal suicide attempts result in 600,000 emergency room visits and 
nearly 200,000 hospitalizations each year.  Recent developments have opened new 
opportunities to develop and evaluate population-based selective prevention programs for 
suicidal behavior.  First, increasing use of standard depression severity measures and 
recording of results in electronic medical records will allow timely and efficient 
identification of people at risk for suicidal behavior.  Second, efficient and scalable 
interventions (both structured risk assessment / care management programs and low-
intensity emotion regulation skills training) have shown promise for reducing risk of 
suicide attempt in at-risk populations.  Third, the NIMH-funded Mental Health Research 
Network has established an infrastructure to adequately evaluate population-based 
prevention. 
 
1. Study objective 
 
Evaluate the effectiveness of two population-based outreach programs for reducing risk 
of suicide attempt among patients who report frequent thoughts of death or self-harm on 
PHQ9 questionnaires completed during routine care. 
 
2. Subjects: 
 
 a.  Description of study populations  - At the four participating health systems, 
electronic health records will be used to identify outpatients aged 18 or older who 
respond to a routinely administered PHQ depression questionnaire by reporting thoughts 
of death or self-harm “most of the days” or “nearly every day.”  Following a modified 
Zelen design, those identified will be automatically assigned to continue in usual care OR 
offered one of the two outreach interventions while continuing in usual care.  Analyses 
will compare risk of subsequent suicide attempt according to initial assignment, 
regardless of participation in intervention programs or other treatment received. 
 
b. Inclusion criteria – We propose to automatically enroll all health system patients 
who: 
• Complete PHQ  questionnaire and respond to item 9 (regarding thoughts of death 
or self-harm) “more than half the days” or “nearly every day” 
• Are currently enrolled in the health system 
• Are aged 18 or older 
• Have used health system online messaging services in the prior 12 months 
 
c. Exclusion criteria – We propose to exclude those who: 
• Have a recorded diagnosis of dementia or developmental delay 
• Have already been enrolled by virtue of a previous PHQ questionnaire 
 
 
 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 7 of 33 
3. Study Design and Methods: 
 
a. Study overview  
 
We propose to conduct a multi-site pragmatic clinical trial to evaluate two population-
based programs to prevent suicide attempts among patients who report suicidal ideation 
during outpatient visits.  While the two proposed prevention programs differ in specific 
content and mode of delivery, both emphasize systematic outreach and follow-up to 
reduce ongoing risk (in contrast to the traditional emphasis on acute or emergent 
management).  Each intervention is intended to supplement usual care by primary care 
and mental health providers.  Patients receiving continued usual care will serve as the 
comparison or control group.  Key aspects of the proposed pragmatic trial include: 
• Potential participants will be automatically identified from records of PHQ depression 
questionnaires routinely administered during outpatient primary care and mental 
health visits. 
• All those eligible will be randomly assigned to continue in usual care or to receive 
usual care plus one of the two prevention programs (using a modified Zelen design). 
• A DBT skills training intervention will include systematic outreach, an online 
program to develop emotion regulation skills, and a structured coaching program to 
support engagement and adherence. 
• A risk assessment and care management intervention will include systematic 
outreach, structured assessment of risk of suicide attempt, and ongoing care 
management to facilitate engagement in specified risk-based care pathways. 
• For both prevention programs, intervention delivery and quality control will be 
supported by specific tools integrated into the Epic electronic medical records 
systems already in use at all sites. 
• The primary outcome, diagnosis of probable suicide attempt during the 18 months 
following randomization, will be assessed using electronic medical records and 
insurance claims data. 
• Secondary analyses will examine intervention effects on use of mental health services 
and on alternative definitions of suicide attempt, as well as test for possible biased 
ascertainment. 
• Analyses will examine the consequences of original assignment to intervention 
conditions, regardless of participation in intervention programs. 
 
A diagram illustrating the overall study design is attached as an Appendix. 
 
b. 
Recruitment 
 
Each week, a the study programmer/analyst at each site will use electronic medical 
records data to identify adult outpatients who completed a PHQ depression questionnaire 
in the previous week and reported thoughts of death or self-harm “most of the days” or 
“nearly every day.” 
 
Those identified will be automatically assigned to continued Usual Care or to Usual Care 
plus one of two intervention programs (Care Management or Skills Training). 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 8 of 33 
 
For those assigned to the Care Management group, the Care Manager will send an 
invitation message including essential elements of informed consent (see Appendix b for 
invitation message with informed consent language).  Those who actively consent will 
continue in the Care Management program, those who actively decline will not be 
contacted again, and those who neither consent nor decline may be invited up to two 
more times over the next two months.   
 
For those assigned to the Skills Training group, the Coach will send an invitation 
message including essential elements of informed consent (see Appendix c for invitation 
message with informed consent language).  Those who actively consent will continue in 
the Skills Training program, those who actively decline will not be contacted again, and 
those who neither consent nor decline may be invited up to two more times over the next 
two months.   
 
c. Screening  
 
As described above, participants will be identified automatically from health system 
electronic medical records.  There will be no study contact with participants to assess 
eligibility or any inclusion or exclusion criteria.  Those identified will be automatically 
assigned to one of the three study groups (Risk Assessment intervention, Skills Training 
intervention, or continued usual care). 
 
d. 
Intervention Procedures 
 
CARE MANAGEMENT INTERVENTION PROGRAM 
Intervention Overview – The aim of this program is to provide systematic 
monitoring of suicide risk linked to risk-based care pathways.  A structured care 
management program (delivered by online messaging and/or telephone) will include 
motivational interventions to promote engagement in recommended treatment 
pathways and coordination of care with responsible providers.  The program is a 
supplement to (not a substitute) for usual medical and mental health care.  The design 
of the program draws from the research team’s extensive experience with systematic 
outreach, motivational enhancement, and care management to promote engagement in 
and adherence to mental health treatment.  Outreach messages also include content 
from “caring letter” interventions previously shown to reduce suicidal behavior. 
Organizational Structure – Services will be delivered by intervention clinicians 
based in each health system, using tools embedded in each system’s Epic EMR 
system.  GHRI investigators will be responsible for training and supervision of 
intervention staff at all sites.  Clinical liaisons at KPCO, KPNW and HealthPartners 
will assist with adaptation of intervention procedures and coordination with local 
clinicians and leaders.   
Care Management Clinician Qualifications, Training, and Supervision – Care 
managers will be masters-prepared mental health clinicians, recruited (whenever 
possible) from the workforce of participating health care systems.  Each care manager 
will receive standard training required of mental health clinicians in her/his health 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 9 of 33 
care system (use of EMR systems, records privacy, relevant regulatory compliance, 
etc).  Project-specific training for delivery of this preventive intervention will include 
8 hours of initial training (two 4-hour sessions) followed by two 2-hour booster 
sessions by videoconference (all organized by Evette Ludman).  This low-intensity 
training is consistent with the principles of pragmatic trials and is intended to 
maximize the chances of eventual dissemination and implementation.  Dr. Ludman 
has extensive experience with implementation of care management programs using 
similar low-intensity training.  During the intervention period, care managers will 
participate in quality control meetings to monitor caseload-level indicators of 
successful intervention delivery (described below).   
Outreach Contacts – A series of structured outreach contacts will be scheduled 
throughout the 12-month intervention period, with the interval between contacts 
determined by risk level at last contact, engagement in care, and interim PHQ scores 
recorded in the medical record.  The algorithm for contact intervals (implemented and 
evaluated in pilot testing) calls for intervals between outreach contacts ranging from 
as brief as one week (for high-risk participants not yet engaged in care) to three 
months (for low-risk participants engaged in care and showing resolution of suicidal 
ideation on subsequent PHQ questionnaires).  Initial contact attempts will be made 
through online messaging with telephone follow-up for those not reading or 
responding to online messages.   
Assessment Tools – Each outreach contact will begin with a structured assessment of 
risk using an adapted version of the Columbia Suicide Severity Rating Scale.  Online 
administration of an adapted CSSR was successfully implemented in the pilot phase. 
Risk-Based Care Pathways – A specific algorithm will link risk levels (as 
determined by the C-SSR) to specific pathways for follow-up care – following the 
care pathways developed and successfully implemented in the Henry Ford program 
and adapted based on feedback from health system clinical leaders.  Recommended 
follow-up options will range from immediate emergency department assessment for 
highest-risk participants, decreasing to as-needed outpatient follow-up for those with 
complete resolution of suicidal ideation. 
Motivational Interventions – We anticipate that simple recommendations for 
follow-up care will not be sufficient to prompt engagement in care – especially 
among people with suicidal ideation and significant depression.  As in our previous 
care management interventions, outreach messages will include structured 
motivational enhancement content (motivational assessment, algorithm-based 
motivational enhancement messages) suitable for secure message or telephone.   
Integration with Usual Care – Following each contact, care managers will 
communicate assessment findings and recommended follow-up care to treating 
providers using provider-to-provider messaging within the Epic EMR.  Messages will 
be clearly labeled as informational only (“FYI”) or requests for specific action.  
Direct telephone communication will be used for communication of urgent needs.  
Care managers will also communicate directly with primary care and mental health 
nursing staff and reception/appointing staff (using EMR-based messaging or 
telephone) to facilitate recommended follow-up care.  The purpose of the intervention 
is to prompt appropriate follow-up treatment by responsible outpatient providers.  

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 10 of 33 
Care managers will serve as treatment facilitators rather than direct treatment 
providers. 
Management of Clinical Emergencies - For emergent needs, all health systems have 
capacity for in-person emergency room assessment, 24-hour access to on-call mental 
health providers, and mobile crisis teams.  Care management algorithms will include 
specific recommendations for use of these services. 
Caseload and Workflow – A study site enrolling 5000 participants would offer this 
intervention to approximately 1700 participants, or approximately 12 per week during 
the anticipated 33-month enrollment period.  While the number of contacts per 
participant will vary widely (depending on risk level and engagement in care), our 
experience in the pilot phase suggests an average of approximately 5 contacts per 
participant over the 12-month period.  Consequently, enrollment of 12 patients per 
week would generate a steady-state workload of 60 care management contacts per 
week.  Based on extensive experience with telephone and online messaging care 
management programs, we estimate that approximately 24 hours of care manager 
time per week (or 0.60 FTE) will be required to complete those 60 contacts (including 
time for record-keeping, administrative tasks, and meetings).   
Embedded EMR Tools – Care managers will take advantage of each health system’s 
electronic medical records system for communicating with participants via secure 
online patient-provider messaging, real-time monitoring of engagement in treatment, 
communication with treating clinicians by provider-to-provider messaging, and 
documenting all care management contacts.  In addition to increasing efficiency, 
these embedded communication tools create a comprehensive electronic record of all 
intervention activity.  Care managers will also take advantage of the Epic EMR 
capacity for care automation and workflow management using other specific tools: 
• Risk assessment questionnaires embedded in Epic patient-provider messaging tools 
• Customizable “smart phrases” allow care managers to generate algorithm-driven 
messages to both patients and clinicians, automatically personalizing messages with 
patient identifiers and allowing additional personalization by the care manager. 
• Customized patient lists support contact and workflow management by organizing 
information regarding contact dates, risk assessment results, expected date and type 
of next provider contact, provider contacts completed, provider contacts scheduled, 
and prescriptions filled or refilled.  These lists identify participants overdue for care 
manager outreach and those failing to engage in recommended care pathways.  
Outreach contacts can be immediately “launched” from this EMR interface.   
Each of these tools is a standard feature of the Epic EMR, so complex programming 
will not be necessary to generate project-specific tools for workflow management and 
quality control. 
Quality Monitoring – Day-to-day clinical and administrative supervision of care 
managers will be the responsibility of investigators and delivery system colleagues at 
each site.   Care managers from all sites will also participate in a bi-weekly project-
wide quality control meeting led by Drs. Ludman and Simon.  This meeting will use 
the EMR tools described above to review timeliness of intervention service delivery 
and adherence to care management protocols across sites.  Site investigators will meet 
twice per month to review specific indicators of intervention implementation (see 
sections 14 and 15 below). 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 11 of 33 
 
 
SKILLS TRAINING INTERVENTION PROGRAM 
Intervention Components – The aim of this program is to efficiently deliver an 
online intervention to develop specific emotion regulation skills derived from 
Dialectical Behavior Therapy.  Skills training will be delivered through an interactive, 
self-contained online program but will be supported by outreach and coaching (via 
online messaging and telephone) to provide support and promote engagement in the 
online program.  The program is a supplement to (not a substitute for) usual medical 
and mental health care.  Program content draws from the research team’s extensive 
experience with structured outreach programs, online psycho-educational programs, 
and structured motivational interventions.   
Organizational Structure – As described above regarding the care management 
intervention, coaching will be delivered by distinct staff at each study site supported 
by centralized training and supervision.   
Online Skills Training Program – Content will focus on four core skills found to 
mediate the clinical benefits of dialectical behavior therapy: mindfulness, mindfulness 
of current emotion, paced breathing, and opposite action.  Following our experience 
with development of previous interactive programs to develop self-management 
skills, each specific skill topic (e.g. using opposite action as a strategy to manage 
strong negative emotions) will be presented in a separate module – with each module 
including text and graphic descriptions, video demonstrations, online exercises, and 
tools to generate personalized printed homework assignments.  The program will 
include four modules designed for completion over a period of six to twelve weeks.  
Each module also includes video testimonials and encouragement from people with 
lived experience of suicidal ideation and suicidal behavior. 
Coaching Support – Previous research regarding online interventions for depression 
and anxiety disorders demonstrates that online interventions have efficacy similar to 
traditional in-person therapies – but only for those who engage in and adhere to the 
online treatment.  Personalized outreach to promote engagement and adherence are 
critical to the success of online interventions.  Compared to a therapist role, this 
coaching role is both less intensive (fewer, briefer contacts) and narrower (focused on 
motivating program use and reinforcing use of program tools).  A series of structured 
outreach contacts will be scheduled over the first 6 months of the intervention period, 
with the number of and intervals between contacts determined by level of engagement 
in the online program.  Initial contact attempts will be made through online 
messaging (for participants using online messaging) or telephone (for those not 
reading or responding to online messaging).  In addition to specific motivational 
content (described below), each contact will include expressions of support based on 
previous “caring letter” programs.  Coaches supporting the online intervention will be 
bachelors- or masters-prepared mental health clinicians.  To preserve integrity of the 
two intervention programs, no individual clinician will perform both coaching and 
care management (described above) functions.  Each coach will receive standard 
training required of mental health clinicians in her/his health care system (use of 
EMR systems, records privacy, relevant regulatory compliance, etc).  Project-specific 
training for delivery of this preventive intervention will include 8 hours of initial 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 12 of 33 
training (two 4-hour sessions) followed by two 2-hour booster sessions by 
videoconference (by Evette Ludman and Ursula Whiteside).  The training will focus 
on program objectives, use of program procedures and tools, and principles of 
motivation enhancement interventions.  This low-intensity training program is 
consistent with the principles of pragmatic trials and is intended to maximize the 
chances of eventual dissemination and implementation.  Our recent trials have 
successfully implemented similar care management programs using similar low-
intensity training.  Coaches will participate in biweekly quality control meetings to 
monitor caseload-level indicators of intervention delivery (described below). 
Motivational Interventions – Coaching protocols will include scripted messages for 
motivational assessment and motivational enhancement based on those used 
successfully in our previous outreach and coaching programs.  These messages will 
focus on specific behavioral goals (initial enrollment in online program, completion 
of specific online modules, practice of emotion regulation skills).  Messages will be 
suitable for delivery either by telephone or online messaging. 
Integration with Usual Care – As described above, coaches will communicate with 
treating mental health and/or general medical providers following each participant’s 
enrollment in the online program.  These standardized messages will briefly describe 
the program content, invite treating clinicians to reinforce use of programs skills, and 
invite treating clinicians to contact the care manager for additional information.  
Because the coaching program does not include specific recommendations regarding 
follow-up treatment, messages from coaches will not include recommendations for 
providers.  If, however, coaching contacts discover clear need for evaluation or 
intervention by mental health or general medical providers, coaches will 
communicate that need to participants and providers and will facilitate necessary 
follow-up care.   
Management of Clinical Emergencies – The coaching program does not include 
specific assessment of suicidal ideation or suicidal behavior.  Nevertheless, it is 
possible that coaching contacts may discover need for urgent or emergent 
intervention.  For emergent needs, all health systems have capacity for in-person 
emergency department assessment, 24-hour consultation with on-call mental health 
providers, and mobile crisis teams.  Coaching algorithms will include specific 
recommendations for use of these services. 
Caseload and Workflow – As with the risk assessment and care management 
program described above, a study site enrolling 5000 participants would offer this 
program to approximately 12 new participants each week.  While the number of 
contacts per participant will vary (depending on engagement in the online program), 
we anticipate an average of 6 total contacts.  Consequently, enrollment of 12 patients 
per week would generate a steady-state workload of 72 care management contacts per 
week.  Based on extensive experience with telephone and online messaging care 
management programs, we estimate that approximately 24 hours of care manager 
time per week (or 0.60 FTE) will be required to complete those 72 contacts (including 
time for record-keeping, administrative tasks, and meetings). 
Technology Platform – The skills training intervention has been implemented using 
the DatStat Illume content management system, a standard commercial content 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 13 of 33 
management system for delivery of tailored online surveys and educational 
interventions. 
Quality Monitoring - Day-to-day clinical and administrative supervision of coaches 
will be the responsibility of investigators and delivery system colleagues at each site.  
Coaches from all sites will also participate in a twice-monthly quality monitoring 
meeting led by Drs. Ludman and/or Whiteside.  This meeting will use Epic EMR 
tools and reports generated by DatStat Illume to monitor intervention participation 
and adherence to coaching protocols across sites.  Site investigators will meet twice 
per month to review specific indicators of intervention implementation (see sections 
14 and 15 below). 
 
TREATMENT AS USUAL 
Participants assigned to continue treatment as usual will not be contacted by study 
staff.  No additional services will be offered, but no services normally available will 
be restricted or withheld.  Treating mental health and primary care providers will be 
aware of participants’ thoughts of death or self-harm, since results of PHQ9 
questionnaires were entered by providers during in-person or telephone encounters.  
Follow-up care (including referral to outpatient mental health specialty care or 
inpatient mental health care) will be at the judgment of the treating primary care or 
mental health providers.  Study staff do not have permission to view records of usual 
care participants, and usual care will not be monitored by study staff.   
 
 
e. Coordination of Care / End of Participation 
 
The intervention programs are intended as supplements to (not replacements for) ongoing 
care by treating mental health and primary care clinicians.  In both intervention programs, 
intervention clinicians (care managers or coaches) will coordinate care with other treating 
clinicians throughout the 12-month intervention period.  This coordination will typically 
occur through electronic provider-to-provider messaging in the electronic medical record, 
but telephone contacts may be necessary for more urgent concerns.  For each participant, 
an initial message to treating clinicians will describe the purpose, content, and procedures 
of the intervention program.  In the Care Management intervention, each cycle of 
outreach for risk assessment and care management will include a summary message to 
treating providers regarding current risk level and follow-up plan.  As necessary 
(according to each participant’s risk level and engagement in care), the Care Manager 
may send additional messages to treating providers to arrange appropriate follow-up care.  
In the Skills Training intervention, the Coach will send periodic messages to treating 
providers regarding each participant’s progress in the online program – with the number 
and timing of those messages varying according to each participant’s pattern of progress.   
 
In both programs, a standard process will notify study participants and treating providers 
regarding the end of intervention services.  That process will include: 
• A message to treating clinicians regarding the end of the intervention period, 
summarizing the participant’s involvement in the program and the plan for 
follow-up care 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 14 of 33 
• A message to the participant regarding the end of intervention services (e.g. 
outreach contacts, coaching messages) and plans for follow-up care 
• Documentation in the electronic medical record 
 
4. Storage of data  
 
At each site, electronic health records will be used to identify participants (described 
above) and assess outcomes (described below).  The study programmer/analyst at each site 
will maintain a list of participant identifiers linked to study identifiers.  These lists will be 
stored in password-protected directories in each health system’s HIPAA/HITECH 
compliant computer networks.  Identifiable information will only be available to study staff 
with clear need to know (programmer/analysts and intervention clinicians).  Identifying 
information will not be shared across sites.  Analytic datasets will be completely de-
identified (including appropriate procedures to prevent backwards-identification).  All 
identifying information will be discarded once final analytic files are created. 
 
Information used for delivery of the Care Management intervention will be stored within 
the electronic health records of each healthcare system and will be protected by the same 
technical safeguards and policies/procedures that protect the privacy of participants’ other 
medical record information. 
 
The online Skills Training program will not receive or store any identifiable information.  
Within that program, each participant will be identified only by a study identifying number. 
 
 
5. Risks and discomforts 
 
• Risks associated with participant identification and recruitment – As described above, 
participants will be identified from health system electronic medical records.  Use of 
records for this purpose may create a risk of violating patient/participant confidentiality.  
Procedures for protection of participants’ health information are described in #5 above. 
 
• Risk associated with Care Management intervention program - The Care Management 
program includes less-intensive versions of intervention components that have been 
previously found effective in high-risk populations.  Intervention recommendations are 
based on best practice and are intended to be beneficial.  It is possible, however, that 
some participants may find specific aspects of the intervention to be unhelpful or 
upsetting.  This possibility will be described in invitation and consent messages.  
Participants will be free to decline any portion of either program and will be free to 
withdraw from either program at any time. 
 
• Risk associated with Skills Training intervention program - The Skills Training program 
includes less-intensive versions of intervention components that have been previously 
found effective in high-risk populations.  Skills training modules and coaching messages 
are based on best practice and are intended to be beneficial.  It is possible, however, that 
some participants may find specific aspects of the intervention to be unhelpful or 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 15 of 33 
upsetting.  This possibility will be described in invitation and consent messages.  
Participants will be free to decline any portion of either program and will be free to 
withdraw from either program at any time. 
 
• Risks associated with use of health records for ascertainment of outcomes - As described 
above, study outcomes (fatal and non-fatal suicide attempts) will be identified from 
electronic medical records and insurance claims data.  Use of records for this purpose 
may create a risk of violating patient/participant confidentiality.  Procedures for 
protection of participants’ health information are described in #5 above. 
 
6. Participant safety monitoring within intervention programs 
 
Participants assigned to the Usual Care control group will not be contacted, and no real-
time information will be available regarding suicide risk or treatment received.  
Consequently, no real-time monitoring of participant safety is possible. 
 
Within the two intervention groups, specific procedures for safety monitoring will vary 
according to the nature and timeliness of data available to intervention staff. 
 
In the Care Management intervention, participants will be asked to complete periodic 
standardized assessments of suicide risk, and these assessments may reveal high risk of 
self-harm (Columbia Suicide Severity Rating Scale score of 4 or 5) in responses to either 
an online assessment or a telephone assessment with a Care Manager.  In addition, 
participants may spontaneously reveal indicators of risk during online or telephone 
communication with Care Managers.  If indication of high risk is received via online 
communication, the Care Manager will receive this information no later than the next 
working day, depending on when the assessment is completed.  Immediately on receipt of 
this information, the care manager will initiate an outreach contact (either by online 
message or telephone, depending on the participant’s previous communication behavior 
and preferences).  The goal of this outreach contact is to arrange appropriate follow-up 
care, including (if clinically indicated) urgent assessment in the emergency department or 
by a local 24-hour mobile crisis team.  If indication of high risk is received during a live 
telephone call, the Care Manager will immediately work with the participant to arrange 
appropriate follow-up care, including (if clinically indicated) urgent assessment in the 
emergency department or by a local 24-hour mobile crisis team.  In each of these high-
risk cases, Care Managers will document outreach contacts, recommended follow-up 
care, and appropriate resolution.  All Care Managers will be experienced mental health 
providers with appropriate training and licensure for assessment of suicide risk or other 
urgent mental health need.  Care Managers are not, however, direct providers of mental 
health or general medical care for study participants.  The clinical role of Care Managers 
is limited to outreach, structured assessment, and facilitation of appropriate follow-up 
care through standard recommendations made to participants and those participants’ 
treating clinicians.  At each site, clinical supervisors will be available to Care Managers 
for as-needed clinical consultation.   
 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 16 of 33 
The Skills Training intervention does not include any specific assessment of suicide risk 
or risk of self-harm.  Nevertheless, participants may spontaneously reveal indicators of 
risk during online or telephone communication with Coaches.  If indication of high risk is 
received via online communication, the Coach will receive this information no later than 
the next working day.  Immediately on receipt of this information, the care manager will 
initiate an outreach contact (either by online message or telephone, depending on the 
participant’s previous communication behavior and preferences).  The goal of this 
outreach contact is to arrange appropriate follow-up care, including (if clinically 
indicated) urgent assessment in the emergency department or by a local 24-hour mobile 
crisis team.  If indication of high risk is received during a live telephone call, the Coach 
will immediately work with the participant to arrange appropriate follow-up care, 
including (if clinically indicated) urgent assessment in the emergency department or by a 
local 24-hour mobile crisis team.  In each of these high-risk cases, Coach will document 
outreach contacts, recommended follow-up care, and appropriate resolution.  All Coaches 
will be experienced mental health providers with appropriate training and licensure for 
assessment of suicide risk or other urgent mental health need. Coaches are not, however, 
direct providers of mental health or general medical care for study participants.  The 
clinical role of Coaches is limited to outreach and encouragement to use the online skills 
training program through standard recommendations made to participants.  At each site, 
clinical supervisors will be available to Coaches for as-needed clinical consultation. 
 
At each site, local clinical supervisors will review all instances of outreach to high-risk 
cases within 2 working days to assure compliance with study procedures and clinical 
appropriateness of Coaches’ or Care Managers’ assessments and recommendations.  As 
described in section 14 below, reports of all outreach to high risk cases at all sites will be 
reviewed by an independent medical monitor bi-monthly, as well as monthly by the 
intervention implementation team (including clinical and health system representatives 
from all sites) .  Study staff responses to high-risk cases will be reported three times each 
year to the DSMB. 
 
As described above, the likelihood of detection of high-risk cases will be much greater in 
those assigned to Care Management than in those assigned to Skills Training.  And there 
will be no detection of high-risk cases in the Usual Care group as study staff will have no 
contact with these participants.  Consequently, data regarding study response to high-risk 
cases cannot be used for any valid comparison across treatment groups.  
 
In addition to this real-time safety monitoring within the two intervention programs, 
health system records will be used to systematically identify suicide attempts and suicide 
deaths in all three treatment groups.  These data will be reported to DSMB as described 
in section 15 below. 
 
7. Outcome measures 
• Primary outcome measures 
The primary trial outcome will be fatal or non-fatal suicide attempt up to 18 months 
following randomization, ascertained from computerized medical records, computerized 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 17 of 33 
records of insurance claims, and death certificate data.  Three specific criteria will be 
used to identify medically treated nonfatal suicide attempts. 
• Outpatient or inpatient encounter with ICD9 diagnosis of definite self-inflicted injury 
(E950-E958)  
• Outpatient or inpatient encounter with ICD9 diagnosis of possible self-inflicted injury 
(E950-E958 
• Emergency department or inpatient encounter with ICD diagnoses of poisoning (960-
989) or open wound (870-897) associated with diagnosis of suicidal ideation (V62.84) 
Our pilot work has validated the use of these criteria at the three study sites.  As 
discussed above, enrollment will be limited to those for whom EMR and insurance data 
will be available.  In addition suicide deaths not identified by the above will be identified 
by death certificate cause-of-death codes of definite self-inflicted injury (X60 to X84) or 
possible self-inflicted injury (Y10 through Y34). 
 
Health system records will also be used to identify censoring variables: health system 
disenrollment and death from causes other than suicide. 
 
Consistent with the principles of pragmatic effectiveness trials, outcomes will be assessed 
for all randomized participants, including those who decline intervention services or drop 
out of intervention services.  As described in section 10 below, analyses will compare 
risk of suicide attempt according to initial treatment assignment, regardless of level of 
participation in intervention services. 
 
• Secondary outcome measures 
Specific secondary analyses are planned to: 
• Describe intervention effects on care process (both use of specific intervention 
services and traditional mental health services) 
• Examine intervention effects using alternative outcome definitions (broader and 
narrower criteria for defining suicide attempts) 
• Estimated incremental cost and incremental net benefit 
• Test for possible biased outcome ascertainment due to intervention effects on care-
seeking behavior 
Intervention effects on care processes: 
• Intervention uptake and adherence – Data generated within both intervention 
programs will be used to assess specific measures of uptake and adherence 
described in section 14 below.  For the Skills Training Intervention, additional 
analyses will describe proportions initiating and completing each module of the 
skills training program as well as use of specific components (video 
demonstrations, interactive exercises, printable homework) within each module.  
These findings will inform any subsequent revisions to both programs and to 
estimate resources needed for any future implementation.   
• Use of specific health services – Increasing use of follow-up care is an explicit 
goal of the risk assessment and care management intervention.  While the skills 
training intervention will not specifically attempt to increase service use, it is 
possible that skills training might either increase or decrease care-seeking.  

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 18 of 33 
Therefore, we propose to compare each of the intervention groups to the usual care 
group regarding these specific categories of health services utilization: 
o Mental health specialty outpatient visits 
o Emergency department visits 
o Primary care visits associated with any mental health diagnosis (ICD9 diagnosis 
290 to 316) 
o Hospitalizations associated with any mental health diagnosis (ICD9 diagnosis 290 
to 316) 
Based on experience with similar analyses, we anticipate skewed distributions for 
each of these utilization measures, so appropriate transformations or non-parametric 
analyses (e.g. poisson regression, comparison of ranks) will be required.   
Intervention effects on alternative outcome definitions: The effect of treatment on 
secondary time-to-event outcomes will be estimated using proportional hazards models, 
paralleling the analysis plan described for the primary outcome. 
• Broader categories of injury/poisoning diagnoses – As discussed above, failure 
to record any cause-of-injury code (E-code) or mis-coding of self-inflicted injuries 
as accidental may lead to under-ascertainment of suicide attempts from EMR or 
claims data.  While our pilot data support the use of E-codes at the three study sites, 
we must acknowledge that coding practice could change during the study period 1.  
Consequently, we propose to examine between-group differences in risk of two 
broader categories of injury/poisoning diagnoses that do not depend on E-codes: 
o One alternative analysis will examine all diagnoses of open wound (ICD9 870 
thru 897) or poisoning (ICD9 960 thru 989), regardless of cause-of-injury code2.  
These specific codes were selected because preliminary research across MHRN 
sites (described above) found that these type-of-injury codes accounted for over 
90% of encounters where cause-of-injury code indicated self-inflicted injury. 
o An additional analysis will examine all the broader category of all diagnoses of 
injury or poisoning (ICD9 800 thru 999), regardless of cause-of-injury code. 
• Restriction to suicide attempts with greater potential lethality – Given the 
possibility of biased outcome ascertainment due to differential care-seeking 
prompted by exposure to the intervention (see below), an additional secondary 
analysis will compare groups after limiting to suicide attempts resulting in death or 
leading to medical or psychiatric hospitalization (presuming that differential care-
seeking would have less influence on more serious suicide attempts). 
Incremental cost and incremental net benefit – Following methods used in several of our 
previous effectiveness trials, health system accounting and claims records will be used to 
calculate costs of all health services provided or paid for by the health system, and study 
personnel records will be used to estimate actual costs of intervention program delivery.  
These data will be used to estimate incremental cost of each intervention program (cost 
for intervention group minus cost for usual care control group) considering the narrower 
category of mental health treatment costs (intervention program costs, mental health 
specialty care costs, costs of general medical visits or hospitalizations with mental health 
diagnoses, and costs of psychotropic medications) as well as the broader category of total 
costs (intervention program costs plus total health services costs).  If either intervention 
program demonstrates a reduction in risk of suicide attempt, then these data will be used 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 19 of 33 
to estimate incremental net benefit (probability of net benefit for different valuations an 
additional suicide attempt avoided). 
Testing for Biased Outcome Ascertainment – It is possible that either of the intervention 
programs might affect participants’ threshold for care-seeking regarding self-injury or 
suicide attempt.  For example, care management and risk assessment contacts might 
result in participants assigned to that condition presenting to emergency departments 
following overdose or self-injury episodes that would have otherwise gone undiagnosed 
and unrecognized.  Such an effect might bias our ascertainment of outcomes from 
electronic records and thereby bias our comparison of either intervention with usual care.  
Consequently, we propose to examine severity of identified suicidal behavior as indicated 
by need for inpatient treatment and means of attempt (firearms/jumping/hanging vs other 
means), a proxy for potential lethality.  If exposure to the intervention lowered care-
seeking thresholds, we would expect to see a lower proportion of more serious suicide 
attempts (leading to hospitalization or by more potentially lethal means) among suicide 
attempts identified in the intervention groups. 
 
8. Statistical Analysis 
 
• Analysis of data/ study outcomes 
Participants assigned to each of the two intervention programs will be separately compared to 
those assigned to continued usual care.  Primary analyses will examine risk of suicide 
attempt for up to 18 months after randomization.  Estimated intervention effects will be 
based on original group assignment (an “as randomized” analysis), regardless of subsequent 
treatment or level of participation in either intervention program. Criteria for defining suicide 
attempts and pilot data supporting the validity of those criteria are described in section 8 
above.  We plan to use proportional hazards models to describe time until first suicide 
attempt following randomization and to compare the risk of suicide attempt or completion 
between groups.   Individuals will be censored at time of health system disenrollment, death 
from cause other than suicide, or administratively, at 18 months following randomization.  
Selection of an 18-month limit for outcome ascertainment is based on our observation that 
the association between score on item 9 of the PHQ (used to determine eligibility and assess 
baseline risk) and risk of subsequent suicide attempts declines to a modest level after 18 
months.  Proportional hazards models will adjust for initial response to PHQ item 9 (2 vs. 3), 
study site, sex, age group (18-29, 30-64, 65+), race/ethnicity (Non-Hispanic White, Black, 
Asian American, Hispanic, other), and visit type at which the qualifying PHQ measure was 
completed (primary care vs. mental health specialty). Previous research and our preliminary 
data indicate that risk of suicide attempt can be expected to vary by these characteristics, and 
adjustment for these factors in survival models will account for any imbalance in these 
factors across randomization groups. 
 
Note regarding intervention uptake or adherence:  Consistent with the principles of pragmatic 
trials, this effectiveness trial intends to address a policy question regarding effectiveness: 
Should health systems implement population-based outreach programs to reduce risk of 
suicide attempt?  In such a trial, analyses must be based on initial assignment or intent-to-
treat, regardless of any participant’s receipt of any intervention services.  Among participants 
assigned to intervention conditions, those who actively engage in all intervention services, 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 20 of 33 
those who never actively engage in interventions, and those who drop out of intervention 
services must still be considered as assigned to the intervention condition.  Intervention drop-
out or non-adherence is not considered a threat to validity but a valid indicator of real-world 
effectiveness (i.e. effectiveness = efficacy times reach).  As discussed in section 16 below, 
low rates of intervention uptake or high rates of drop-out would be expected to reduce 
effectiveness.  DSMB monitoring should evaluate whether rates of intervention uptake or 
drop-out indicate that effectiveness or benefit of either intervention program is not plausible. 
 
• Power analysis  
Availability of detailed data from study sites regarding patients meeting proposed eligibility 
criteria allows us to base sample size estimates on simulations using observed data.  This 
approach, which is similar to the bootstrap, enabled us to estimate power in a population with 
event rates, censoring distributions, and covariate patterns expected in the actual study 
sample.  Proposed eligibility criteria identified 4,643 patients with item 9 scores equal to 3 
and another 5,661 with item 9 scores equal to 2 in 2012.  Records identified 670 suicide 
attempts over an 18 month follow-up period.  We first estimated the baseline hazards and 
covariate effects using a proportional hazards model to describe events in these complete 
datasets, including patient level covariates: item 9 PHQ score (2 or 3), site (HP,GH,KPCO, 
KPNW), age (18-29,30-64,65+), sex, race/ethnicity (Black American, Asian American, 
Hispanic, other), and visit type (primary care vs. specialty). Next, we simulated trial data 
under 8 scenarios (intervention and control sample sizes equal to 5000, 6000, 7000, and 
8000; and hazard ratios equal to 0.75 and 1.0), simulating 1000 trials for each scenario. To 
generate these data, we randomly selected patients from the complete dataset, with 
replacement, into intervention and control groups of specified sizes. When a patient with an 
observed event was selected (so that there was no censoring time), we randomly selected a 
censoring time from patients with no event who had the same covariate distribution (i.e., “hot 
deck” imputation). Next, we simulated events for each patient in the simulated trial using an 
inverse-probability look-up using the cumulative hazard based on the estimated baseline 
hazards combined with observed covariates and estimated covariate effects, observed 
censoring times, and simulated treatment group and proposed treatment effects.  For each 
simulated dataset, we estimated both adjusted and unadjusted treatment effects using a 
proportional hazards model. The estimated power and alpha level (false positive rate) are 
based on the proportion of times an intervention effect is detected if the true hazard ratio is 
0.75 (the smallest effect we would hope to detect) or 1.0 (no true intervention effect).  As 
shown in the table below, a sample of approximately 6500 per group (19,500) should allow 
at least 80% power to detect a hazard ratio of 0.75 or smaller with a false positive rate of 5%. 
 
Number 
per group 
Hazard Ratio 
Event rate 
in usual 
care 
Event rate in 
intervention 
Power 
unadjusted 
Power 
adjusted for 
covariates 
5000 
0.75 
0.032 
0.024 
0.685 
0.686 
6000 
0.75 
0.033 
0.024 
0.764 
0.772 
7000 
0.75 
0.033 
0.024 
0.818 
0.821 
8000 
0.75 
0.033 
0.024 
0.868 
0.869 
5000 
1.00 
0.032 
0.032 
0.059 
0.057 
6000 
1.00 
0.032 
0.033 
0.040 
0.042 
7000 
1.00 
0.033 
0.033 
0.056 
0.052 
8000 
1.00 
0.032 
0.032 
0.041 
0.038 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 21 of 33 
 
 
9. Human Subjects Protection 
• Subject selection – As described above, potential participants will be identified 
automatically from health system records of PHQ9 depression scores.  All eligible 
participants will be included without respect to sex, race, and ethnicity.  Children under 
18 will not be included, because the proposed interventions are not appropriate for 
children under 18.  Those aged 18 and over will be included without respect to age.   
• Justification for exclusion of children – The proposed interventions are not clinically 
appropriate for children under age 18. 
• Justification for exclusion of other vulnerable subjects - Potential participants who 
appear to lack capacity to consent (because of diagnosis of dementia or developmental 
delay) will be excluded.   
• Qualifications of investigators 
 
Gregory Simon MD MPH will serve as overall Principal Investigator for the project and as 
Lead Investigator for the Group Health site.  Dr. Simon is a psychiatrist and mental health 
services researcher with extensive experience in the design, implementation, and evaluation 
of organized care programs for people with mental health conditions.  Dr. Simon will assist 
Dr. Ludman (below) in the training and monitoring of intervention program clinicians at all 
sites, will participate in supervision of intervention clinicians at the Group Health site, and 
will serve as primary clinical liaison to Group Health mental health and primary care 
leadership regarding study procedures and activities. 
 
Evette Ludman PhD will serve as co-investigator at the Group Health site.  Dr. Ludman is a 
clinical psychologist with extensive experience in the design, implementation, and 
evaluation of organized care programs for people with mental health conditions.  She has 
specific expertise in the training and supervision of clinicians providing telehealth 
interventions in multi-site studies.  Dr. Ludman will lead the training and monitoring of 
intervention program clinicians at all sites and will lead clinical supervision of intervention 
clinicians at the Group Health site. 
 
Rebecca Rossom MD will serve as lead investigator for the HealthPartners site.  Dr. 
Rossom is a psychiatrist and mental health researcher with expertise in the implementation 
and evaluation of organized care programs for people with mood disorders, especially those 
based in primary care.  She practices psychiatry in the HealthPartners Medical Group.  Dr. 
Rossom will lead clinical supervision of intervention clinicians at the HealthPartners site 
and will serve as primary clinical liaison to HealthPartners mental health and primary care 
leadership regarding study procedures and activities. 
 
Arne Beck PhD will serve as lead investigator for the KP Colorado site.  Dr. Beck is a 
mental health services researcher with extensive experience in the implementation and 
evaluation of organized care programs for depression.  Dr. Beck will serve as the primary 
liaison to the KP Colorado mental health and primary care leadership regarding study 
procedures and activities.  Because Dr. Beck is not a practicing clinician, David Price MD 
will lead clinical supervision of intervention clinicians at the KP Colorado site. 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 22 of 33 
Gregory Clarke, PhD will serve as lead investigator for the KP Northwest site.  Dr. Clarke 
is a mental health services researcher with extensive experience in the implementation and 
evaluation of organized care programs for depression.  Dr. Clarke will serve as the primary 
liaison to the KP Northwest mental health and primary care leadership regarding study 
procedures and activities.  Because Dr. Clarke is not a practicing clinician, TBD, MD will 
lead clinical supervision of intervention clinicians at the KP Northwest site. 
 
 
10. Anticipated Benefit 
 
• Direct benefit - The Care Management and Skills Training programs are reduced-
intensity versions of interventions previously proven effective in high-risk populations.  
We hope that these programs will have direct benefit to participants, but that benefit is 
not proven.  Consequently, no benefit will be promised, and participants will be clearly 
informed that benefits of the programs are not proven. 
 
• Indirect benefits – The proposed research would be the first randomized evaluation of 
population-based secondary or selective prevention programs to reduce risk of suicide 
attempt.  The programs to be tested were designed to maximize potential for rapid 
dissemination (minimizing cost and utilizing existing tools embedded in commonly used 
electronic medical records systems).  In addition, data collected during the trial should 
yield important insights regarding the effectiveness of various outreach strategies and the 
performance of risk assessment measures. 
 
 
11. Classification of risk 
 
• All participants are, by definition, at increased risk of suicide attempt.  But the research 
does not involve more than minimal risk. 
 
For the Usual Care group – Each participant will receive the same care s/he would have 
received if the study had not occurred.  The only potential risk to participants is the initial 
use of records data to identify the study sample and the later use of records data to 
identify suicide attempts. 
 
For the Care Management and Skills Training intervention groups – Each of the two 
interventions represents a lower-intensity version of an intervention proven effective for 
patients at high risk of suicide attempt.  Each is based on best-practice recommendations 
for outreach and follow-up.  In both interventions, each participant will be repeatedly 
advised that intervention participation is voluntary.  At any time, participants may decline 
further intervention contact, and all such requests will be immediately and permanently 
respected. 
 
During our pilot phase, investigators formally consulted with the NIH Collaboratory 
Regulatory and Ethics core and leadership of the Office for Human Research Protection 
of DHHS regarding risk determination and received notification that proposed procedures 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 23 of 33 
(including pre-consent randomization to intervention groups and alteration of consent for 
intervention participation) did not create more than minimal risk. 
 
 
• Overall risk and benefit consideration – The minimal risks to participants (described 
above) are reasonable in relation to the significant potential indirect benefit of the 
proposed research. 
 
12. Consent documents and process 
 
• Designation of those obtaining consent – N/A (see below) 
 
• Consent procedures  
Following a modified Zelen design, participants will be assigned prior to any contact, 
and those assigned to one of the active intervention conditions will be offered 
intervention services.  Each participant offered intervention services will be free to 
accept or decline any element of the intervention program (see Appendix b for 
consent language in the Care Management intervention invitation and Appendix c for 
the consent language for the Skills Training intervention invitation).  Outcomes 
(medically treated suicide attempt over up to 18 months following randomization) 
will be assessed using electronic health system record.  Outcomes data will be 
collected (anonymously) for all randomized participants, regardless of level of 
intervention participation.  Outcomes will be analyzed according to original treatment 
assignment, regardless of level of intervention participation.  We believe waiver of 
consent for random assignment to the intervention or usual care groups is 
scientifically necessary and ethically justified. 
Regarding scientific necessity:  For both of the low-intensity interventions being 
studied, acceptability to participants and level of continued participation are essential 
components of real-world effectiveness.  If we limited enrollment to those who 
volunteer to receive prevention interventions, findings regarding intervention 
acceptability or effectiveness would have questionable validity and limited 
generalizability.  In other words, we cannot study effectiveness of outreach only 
among those who volunteer to receive it. 
Regarding ethical justification:  The “Common Rule” for protection of research 
participants (45 CFR 46.116d) lists specific requirements for waiver of the usual 
requirement for individual informed consent: 
o The research involves no more than minimal risk to the subjects – For participants 
assigned to one of the active intervention conditions, we do not believe that any 
intervention activities expose participants to greater than minimal risk. Each 
program is based on best available evidence regarding prevention of suicide 
attempt. In addition, each participant will be free to engage – or not engage – to the 
degree s/he finds the program to be helpful. Participants offered any of the active 
interventions will be free to receive any other service or treatment without 
restriction.   For participants assigned to the usual care control group, treatment will 
be identical to what would have been delivered had the study not occurred.  While 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 24 of 33 
those eligible for the trial are (by definition) at risk, assignment to usual care does 
not increase risk.   
o The waiver or alteration will not adversely affect the rights or welfare of the 
subjects – All study participants (in the usual care group or either of the active 
intervention groups) will be free to receive any treatment or services normally 
available. No treatment or service will be restricted or withheld. 
o The research could not practicably be carried out without the waiver – As 
discussed above, limiting the trial to those who actively consent to receive outreach 
would yield an invalid test of effectiveness.   
o Whenever appropriate, the subjects will be provided with additional pertinent 
information after participation – In each intervention, outreach communications 
will clearly identify the program as research and clearly state that participation is 
voluntary.  Participants assigned to the usual care group will not be contacted. For 
this group, after-the-fact notification that the study occurred would offer no 
additional protection, and attempting to contact participants would increase risk of 
violating confidentiality.   
We considered and rejected the option of randomizing providers or clinics to 
intervention or usual care conditions.  The proposed interventions are applied at the 
level of the individual patient, so cross-over or spill-over of intervention effects 
within clinics or providers should not occur.  Because patients seen in specialty 
mental health care often see multiple providers (psychiatrist and psychotherapist), 
provider-level randomization could introduce cross-over or contamination.    
 
13. Data and Safety monitoring 
 
• Local Data and Safety monitoring – As described in Section 6 above, clinical 
supervisors at each site will monitor timeliness and appropriateness of Coaches’ and Care 
Managers’ outreach and clinical interventions for all high-risk cases (i.e. patients with 
Columbia risk scores >=4 OR patients spontaneously reporting high risk of self-harm to 
intervention staff).  Each such case will be reported to the local clinical supervisor within 
two working days.  In addition, the Intervention Implementation team (including 
investigators and health system representatives from each site) will review tabular reports 
of responses to high-risk cases at each monthly meeting. 
 
• Independent Medical Monitor- An independent medical monitor or safety officer will 
provide ongoing supervision of each site’s adherence to intervention protocols, safety 
monitoring procedures, and good clinical practice.  Specific duties of the safety officer 
will include: 
• Bi-monthly review of all study responses to high-risk cases 
• Bi-monthly review of any protocol violations 
• Participation in preparation of DSMB reports, including review of counts and rates of 
systematically ascertained suicide attempts and suicide deaths (other study staff 
will not have access to these reports) 
 
• DSMB monitoring – The NIMH DSMB will review study progress three times yearly, 
with each review to consider: 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 25 of 33 
• Progress in recruitment - Reports to the DSMB will detail overall recruitment progress, 
progress at each site, and recruitment of identified racial/ethnic minority groups. 
• Quality/fidelity of intervention delivery – For participants assigned to the Care 
Management intervention, reports to the DSMB will include: 
• % declining intervention services at initial invitation 
• % not responding to up to three intervention invitations  
• % withdrawing from intervention services after enrollment 
• % more than 30 days overdue for most recent outreach contact 
For participants assigned to the Skills Training intervention, reports to the DSMB will 
include: 
• % declining intervention services at initial invitation 
• % not responding to up to three intervention invitations 
• % withdrawing from intervention services after enrollment 
• %  more than 30 days overdue for most recent intervention contact 
• Quality/fidelity of safety monitoring procedures – As described in section 6 above, 
reports to the DSMB will include tabular descriptions of timing and outcome of each 
Care Manager or Coach outreach contact for high-risk cases (i.e. patients with Columbia 
risk scores >=4 OR patients spontaneously reporting high risk of self-harm to 
intervention staff).  These reports will include: 
• Risk indicator (Columbia Suicide Severity Rating Scale or spontaneous report) 
• Source of risk indicator (online message or telephone) 
• Time from receipt of risk indicator to outreach attempt 
• Study staff response 
• Final resolution 
The DSMB will use these data to assess quality/fidelity of safety procedures and (as 
indicated) recommend changes to those procedures. 
• Evidence of harm (increased risk of suicide attempt) attributable to intervention – As 
described in section 15 below, reports to the DSMB will include counts and rates for 
suicide attempts and suicide deaths for participants assigned to all three intervention 
groups (based on assignment, regardless of actual intervention participation).  These 
interim reports will not include data for participants who actively decline intervention 
services.  The waiver of consent allowing use of these data will only permit access to 
those data at one time at the end of the trial. 
 
• Criteria for stopping the study or suspending enrollment or procedures  
• Insufficient progress in recruitment - If recruitment progress reports indicate 
unsatisfactory progress, the DSMB may choose to suspend overall recruitment or suspend 
recruitment at a specific study site. 
• Inadequate intervention engagement or uptake – If reports indicate that uptake of either 
intervention is so low that significant effectiveness or benefit is not plausible (i.e. high 
rate of declining intervention or low rate of engagement/initiation), then the DSMB may 
choose to suspend delivery of one or both intervention conditions at one or more study 
sites. 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 26 of 33 
• Inadequate quality/fidelity of intervention delivery – If intervention quality reports 
indicate significant deficiencies, the DSMB may choose to suspend delivery of one or 
both intervention conditions at one or more study sites. 
• Inadequate quality/fidelity of safety monitoring procedures - If reports regarding safety 
procedures for high-risk cases indicate significant deficiencies, the DSMB may choose to 
suspend delivery of one or both intervention conditions at one or more study sites. 
• Preliminary evidence of harm from either intervention (when compared to usual care) – If 
reports regarding suicide attempts and suicide deaths indicate the possibility of harm 
(increased risk of suicide attempt or death) for either intervention condition, the DSMB 
may request more formal interim analyses or may choose to either suspend trial 
recruitment or end randomization to one of the intervention conditions. 
 
14.   Quality Assurance 
 
• Monitoring intervention quality and fidelity – At each study site, clinical supervisors 
will provide direct supervision for Coaches and Care Managers through weekly 
supervision meetings, including review of reports regarding all high-risk cases.  Across 
all sites, monthly implementation team meetings (including investigators and health 
system leaders from each site) will review reports of quality metrics for each intervention 
program (described in section 14 above). 
 
• Monitoring compliance with high-risk outreach procedures – As described in sections 
6 and 13 above, all instances of outreach to high risk cases (participants with Columbia 
risk scores of 4 or 5 and those spontaneously reporting high risk) will be reviewed by site 
clinical supervisors within two working days and by the independent medical monitor 
within 15 days.  Summary reports of these events reviewed monthly by the cross-site 
intervention implementation committee. 
 
• Monitoring quality of outcome data – The Mental Health Research Network (in which 
all 3 study sites participate) conducts regular quality assessment regarding health system 
data for key mental health diagnosis and treatment indicators (including rates of suicide 
attempt and suicide death). 
 
15. Adverse event and unanticipated problem reporting 
 
Availability of Data Regarding Expected Adverse Events – In contrast with adverse event 
data collected and reported by traditional clinical trials, data regarding adverse events in 
this pragmatic trial will be limited in detail and delayed in time. Data regarding diagnoses 
of probable or possible suicide attempts (see section 7 above) and suicide deaths will be 
periodically extracted from health system administrative databases regarding encounter 
diagnoses and insurance claims.  Given that these data are collected under a waiver of 
consent and waiver of HIPAA authorization, investigators are only allowed to extract and 
store the minimum information necessary to address study hypotheses.  Consequently 
these data will be limited in detail.  Because insurance claims data may be submitted up 
to three months following a health care encounter, data regarding suicide attempt 
diagnoses from insurance claims may be delayed by up to three months.  Because suicide 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 27 of 33 
deaths are most often identified by linkage with state death certificate data, data regarding 
suicide deaths may be delayed by several months. 
 
Definition and Reporting of Serious Adverse Events (SAEs)– All suicide attempts 
(regardless of potential lethality or perceptions of “suicidal intent”) will be considered 
serious adverse events.  Only SAEs identified through the health system administrative 
databases will be reviewed by the independent Medical Monitor and reported to the 
DSMB. The only data available from the data extraction regarding each of these events 
will include: group assignment, time since randomization, and 3-level severity rating 
(fatal attempt, nonfatal attempt resulting in hospitalization, nonfatal attempt not resulting 
in hospitalization).  Over the course of the study, these data will be available for all 
randomized participants, regardless of group assignment or level of participation in either 
intervention program – excluding those who actively decline participation in the 
intervention programs.  Reports to the DSMB will include a tabular description of the 
number, rates, types, and timing of events in each of the three treatment groups.  These 
interim reports will not include data regarding participants who actively decline 
intervention services.  Because outcome data for that group are obtained under waiver of 
consent, access to those data is limited a single data extraction at the end of the study 
period. 
 
Definition and Reporting of Other Adverse Events – Potential adverse events not rated as 
serious (e.g. inadvertent violation of privacy, participant report of distress or upset due to 
intervention procedures) will be recorded by intervention clinicians and reported to the 
DSMB in tabular form.  Given known differential ascertainment across treatment groups 
(more frequently in the Care Management intervention group, less frequently in the Skills 
Training intervention group, not at all in the Usual Care group) any comparison of event 
rates or event characteristics across treatment groups would be irretrievably biased and 
uninterpretable.  
 
Reporting Responses to High-Risk Cases – As described in sections 6 and 13 above, 
reports to the DSMB will include tabular descriptions of intervention clinician (Care 
Manager or Coach) response to each high-risk case, including: 
• Initial risk indicator (scheduled Columbia Suicide Severity Rating Scale or 
spontaneous report to study staff) 
• Source of risk indicator (online message or telephone) 
• Time from receipt of risk indicator to outreach attempt 
• Study staff response 
• Final resolution (e.g. urgent visit scheduled and attended, patient declined 
suggestion of urgent visit, etc.) 
 
As described in sections 13 and 14 above, the DSMB will use these data to assess 
quality/fidelity of safety procedures and (as indicated) recommend changes to those 
procedures.  As explained in section 6 above, these high risk scenarios will be identified 
more frequently in the Care Management intervention group, less frequently in the Skills 
Training intervention group, and not at all in the Usual Care group.  Consequently, any 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 28 of 33 
comparison of event rates or characteristics across treatment groups would be 
irretrievably biased and uninterpretable. 
 
Reporting Unanticipated Problems – In addition to adverse events described above, any 
other unanticipated problems (e.g. significant deviations from the study protocol) will be 
reported to the independent Medical Monitor and responsible IRBs within 7 days and to 
the DSMB within 15 days.  
 
 
 
16. Alternatives to participation  
The Care Management and Skills Training interventions are intended as supplements to 
(not replacements for) ongoing care by treating mental health and primary care providers.  
Each participant offered either of the interventions will be free to (and will be encouraged 
to) continue to receive any care s/he finds helpful.  No treatment normally available will 
be withdrawn or withheld. 
 
Participants assigned to usual care will not be contacted, and treating providers will not 
be notified.  Each participant will receive the care s/he would have received if the study 
had not occurred.  No treatment normally available will be withdrawn or withheld. 
 
For all participants, health system records will be used to anonymously extract data 
regarding medically treated suicide attempts over up to 18 months following 
randomization. 
 
17.  Confidentiality 
 
• For research data and investigator medical records (repeated from #5 above)  
 
At each site, electronic health records will be used to identify participants (described 
above) and assess outcomes (described below).  The study programmer/analyst at each site 
will maintain a list of participant identifiers linked to study identifiers.  These lists will be 
stored in password-protected directories in each health system’s HIPAA/HITECH 
compliant computer networks.  Identifiable information will only be available to study staff 
with clear need to know (programmer/analysts and intervention clinicians).  Identifying 
information will not be shared across sites.  Analytic datasets will be completely de-
identified (including appropriate procedures to prevent backwards-identification).  All 
identifying information will be discarded once final analytic files are created. 
 
Information used for delivery of the Care Management intervention will be stored within 
the electronic health records of each healthcare system and will be protected by the same 
technical safeguards and policies/procedures that protect the privacy of participants’ other 
medical record information. 
 
The online Skills Training program will not receive or store any identifiable information.  
Within that program, each participant will be identified only by a study identifying number. 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 29 of 33 
 
Consistent with the NIH Collaboratory data-sharing policy, an appropriately de-identified 
analytic dataset will be made available to any interested investigators no later than the 
acceptance for publication of the manuscript describing primary trial results.  Summary 
results will also be posted on clinicaltrials.gov. 
 
20. Conflict of Interest 
 
a.   Distribution of NIH Guidelines  
 
NIH guidelines on conflict of interest have been distributed to all investigators.   
 
b.   Conflict of interest  
 
Investigators have no conflicts of interest or relevant financial interests.   
 
c.  Role of a commercial company or sponsor – N/A 
 
 
21. Technology Transfer and Data Sharing 
 
Consistent with the NIH Collaboratory resource sharing policy, all intellectual property 
or technology resources developed by this project (e.g. computer code, intervention 
algorithms) will be placed in the public domain and will be freely shared with any 
investigators or health systems following completion of the trial. 
 
We anticipate that the Collaboratory Coordinating Center may create a common process and 
platform for sharing of data created by all Collaboratory UH2/UH3 projects.  If no such process is 
established, then we propose the following process for this specific project: 
• 
Data will be distributed through the public website of the Mental Health Research Network 
(described above) 
• 
Those downloading specific datasets will be asked to briefly describe the intended use of data 
(in order to document dissemination efforts and inform future dissemination activities) 
• 
We will request (but we cannot strictly require) that users acknowledge the role of NIH 
funding in any publications or presentations using project data 
 
We describe below plans for distributing data regarding specific outcomes (and plans for 
protecting privacy of potentially identifiable information). 
• 
Datasets for primary analyses will be freely shared with any interested researcher or member 
of the public using an unsupervised public data archive model.  Data will be available no later 
than publication data of primary study findings.  This person-level dataset will include the 
following variables: 
o Time to suicide attempt or death 
o Time to censoring for disenrollment or death from other cause 
o Age group 
o Sex 
o Race/ethnicity 
o Site of care (mental health specialty, primary care, other) 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 30 of 33 
o Treatment history (current or past depression or other mental health treatment) 
o Year of enrollment 
o Intervention assignment (Skills Training, Risk Assessment/Care Management, Usual 
Care) 
We do not propose to include study site (health system) in this publicly available dataset 
given concerns by participating health systems that such data could be used for inappropriate 
comparisons of suicide attempt rates across health systems.  We acknowledge the importance 
of examining variation in intervention effects across health systems, and datasets including 
health system identifiers will be available on request, following a supervised data archive, 
subject to specific agreements regarding use and re-disclosure. 
• 
The above-described public-use datasets will also contain time-to-event variables for 
proposed broader definitions of suicide attempt (psychotropic drug poisoning, open wound or 
poisoning, any injury or poisoning).   
• 
We do not plan to release any data regarding suicide death.  First, the anticipated number of 
suicide deaths will be too small to support any meaningful conclusions regarding intervention 
effects.  Second, data regarding dates and cause of death may be available through other 
sources (e.g. state vital records), allowing potential re-identification of study participants. 
• 
Person-level datasets regarding use and costs of specific health services could also allow 
backward-identification of study participants.  But study data regarding these outcomes may 
have scientific or public health value to other investigators.  Consequently, we will make 
such datasets available at the time study findings in this area are published.  But those 
datasets will be available only upon request and will be subject to formal agreements 
regarding protection of privacy and re-disclosure (i.e. supervised data archive model). 
 
All datasets for public distribution will completely satisfy HIPAA standards for de-identification.  
Consequently no data use or data transfer agreements will be required. 
 
22. Research and Travel Compensation  
 
No compensation or reimbursement will be offered. 
 
23. Governance Plan 
 
Trial Steering Committee – The Steering Committee will supervise all scientific and 
administrative aspects of the proposed trial.  Members will include the overall lead 
investigator (Simon), investigators from each study site (Beck, Rossom, Ludman) and 
one or more NIMH program representatives.  The Committee will meet (by 
teleconference) monthly across all project years to review:  implementation of study 
procedures, progress of recruitment, delivery of interventions, quality of outcome data, 
and data analysis.  This team of investigators has an extensive history of successful 
collaboration through the NIMH-funded Mental Health Research Network, including 
recent completion of a 4-site effectiveness trial of behavioral activation psychotherapy 
for depression in pregnancy. 
 
Group Health Research Institute Management Team – The GHRI project managers 
(Richards and Kirlin) and overall lead investigator (Simon) will be responsible for 
administrative operations including: 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 31 of 33 
• Financial management, including monitoring of budgets and spend rates at all 
sites 
• Regulatory compliance, including human subjects and privacy protection 
• Communication, including scheduling and reporting of other committee and team 
meetings 
 
Intervention Implementation Committee – Continuing the work of the Healthcare System 
Engagement Committee in the UH2 phase, the Intervention Implementation Committee 
will supervise quality and fidelity of intervention implementation across all study sites.  
Members will include mental health delivery system representatives from each study site, 
lead investigators from each study site, and the overall lead investigator.  The committee 
will meet (by teleconference) monthly during months 1 through 24 and bimonthly 
thereafter.  Members of this committee will also serve as liaisons to mental health, 
primary care, and informatics leaders in each study site. 
 
DBT Skills Implementation Team – Drs. Ludman and Whiteside will meet (by 
teleconference) regularly with coaches from each site.  Meetings will occur biweekly in 
months 3 through 18 and monthly thereafter.  This group will be the primary venue for 
administrative and clinical supervision of coaches supporting the online DBT skills 
training program.  As described in the Progress Report / Transition Request, this meeting 
will include structured review of specific indicators of the quality and fidelity of 
intervention implementation as well as coach-initiated review of specific cases.  Meeting 
organization will follow the format used successfully by Drs. Simon and Ludman in 
several previous trials of coaching and care management interventions delivered by 
telephone and online messaging. 
 
Risk Assessment/Care Management Implementation Team - Dr. Ludman and Ms. King 
will meet (by teleconference) regularly with care managers from each site.  Meetings will 
occur biweekly in months 3 through 18 and monthly thereafter.  This group will be the 
primary venue for administrative and clinical supervision of care managers supporting 
the Risk Assessment / Care Management intervention.  As described in the Progress 
Report / Transition Request, this meeting will include structured review of specific 
indicators of the quality and fidelity of intervention implementation as well as care 
manager-initiated review of specific cases.  Meeting organization will follow the format 
used successfully by Drs. Simon and Ludman in several previous trials of coaching and 
care management interventions delivered by telephone and online messaging. 
 
Data Quality and Analysis Team – This group, including Drs. Simon, Shortreed, and 
Penfold, will be responsible for ongoing monitoring of study data quality during months 
1 thru 42.  The team will meet quarterly during this period to monitor: 
• Rates of use and patterns of response for PHQ depression questionnaires in 
participating health systems 
• Background rates of diagnoses of suicide attempts in participating health systems 
• Changes in coding practices that might affect ascertainment of suicide attempts 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 32 of 33 
This group will also lead data analysis during months 42 through 48 – when meetings 
will be merged with those of the Steering Committee. 
 
IRB Oversight – The Group Health Cooperative IRB will have primary responsibility for 
initial and continuing review of all study procedures and study execution.  The IRB will 
review and approve all clinical protocols and invitation/consent messages (included as 
attachments to this DSMB protocol).  Any unanticipated problems will be reported to this 
IRB within 7 days.  Any unexpected adverse events (as described above) will be reported 
at each annual review.  Annual reviews will evaluate progress toward recruitment goals, 
successful delivery of intervention programs, and adherence to safety monitoring 
procedures.   
 
The trial will report to the NIMH Data and Safety Monitoring Board rather than to a 
locally-constituted DSMB.   The NIMH DSMB will conduct: 
• Initial review of all study procedures and materials 
• Detailed annual review of study progress (including review of recruitment 
progress and adverse events) 
• Additional twice-yearly abbreviated review of study progress (including review of 
adherence to safety monitoring procedures as described above) 
• Reviews of unanticipated problems that will be reported to the DSMB within 15 
days. 
Matrix Relationships – The overall lead investigator (Simon) and Steering Committee 
will make regular reports on trial progress to: 
• NIMH Program  and Operations Staff 
• The Healthcare Systems Collaboratory Pragmatic Trials Steering Committee 
• The Mental Health Research Network Steering Committee 
 
 
24. Key References 
 
References regarding Zelen design: 
Zelen M. A new design for randomized clinical trials. N Engl J Med. 1979;300(22):1242-
5. 
Zelen M. Randomized consent designs for clinical trials: an update. Stat Med. 
1990;9(6):645-56. 
Adamson J, Cockayne S, Puffer S, Torgerson DJ. Review of randomised trials using the 
post-randomised consent (Zelen's) design. Contemp Clin Trials. 2006;27(4):305-19. 
Ellenberg SS. Informed consent: protection or obstacle? Some emerging issues. Control 
Clin Trials. 1997;18(6):628-36; discussion 61-6. 
 
References regarding use of Zelen design in studies of suicide prevention programs: 

 
Version Number 13 
 
Version Date – 11/12/14 
 
 
 
 
 
 
 
Page 33 of 33 
Carter GL, Clover K, Whyte IM, Dawson AH, D'Este C. Postcards from the EDge: 24-
month outcomes of a randomised controlled trial for hospital-treated self-poisoning. 
Br J Psychiatry. 2007;191:548-53. 
Hatcher S, Coupe N, Durie M, Elder H, Tapsell R, Wikiriwhi K, Parag V. Te Ira Tangata: 
a Zelen randomised controlled trial of a treatment package including problem solving 
therapy compared to treatment as usual in Maori who present to hospital after self 
harm. Trials. 2011;12:117. PMCID: PMC3103449. 
Hatcher S, Sharon C, House A, Collings S, Parag V, Collins N. The ACCESS study a 
Zelen randomised controlled trial of a treatment package including problem solving 
therapy compared to treatment as usual in people who present to hospital after self-
harm: study protocol for a randomised controlled trial. Trials. 2011;12:135. PMCID: 
PMC3117717. 
 
 
25. Attachments/Appendices 
 
a. Study Diagram 
b. Invitation/Consent language for Risk Assessment Intervention 
c. Invitation/Consent language for Skills Training Intervention 
d. Detailed protocol for Risk Assessment Intervention 
e. Detailed protocol for Skills Training Intervention