An Exposure-Based Implementation Strategy to Decrease Clinician Anxiety About Delivering Suicide Prevention Evidence-Based Practices
Phase: NA
Conditions: Implementation Science
Sponsor: University of Pennsylvania

An Exposure-Based Implementation Strategy to Decrease Clinician Anxiety Around 
Suicide Prevention 
NCT: NCT05172609 
Document Date: 2/5/2024
City of Philadelphia IRB study protocol document for participants recruited outside of the City of 
Philadelphia mental health system 

 
Version Date: 2/5/2024 
 
 
Page 2 of 20 
Social and Behavioral Sciences Human Research Protocol 
 
 
PRINCIPAL INVESTIGATOR:  
Emily Becker-Haimes, PhD 
Assistant Professor of Psychology in Psychiatry 
Perelman School of Medicine, University of Pennsylvania 
3535 Market Street, Room 3007, Philadelphia, PA 19104 
Tele: 215-573-5614   
PROTOCOL TITLE: An Exposure-Based Implementation Strategy to Decrease Clinician Anxiety about Delivering 
Suicide Prevention Evidence-Based Practices 
 
INTRODUCTION AND PURPOSE:  
Despite the availability of effective screening and interventions for suicide prevention, the rate of suicide in the US 
has increased over the past several decades. A potential lever to reduce suicides is to increase the use of evidence-
based suicide screening, assessment, and interventions (SSAIs) in community settings by targeting known barriers 
to effective implementation of SSAIs. Suicide is inherently an affectively charged topic and clinicians often 
experience anxiety and low self-efficacy in detecting and intervening with those at risk. Clinician anxiety and low 
self-efficacy related to SSAI use are thus candidate targets for implementation strategy design to optimize SSAI 
uptake in diverse mental health settings. This study aims to design a novel, exposure-based implementation 
strategy (EBIS) directly targeting clinician anxiety and low self-efficacy for SSAI use with patients at risk for suicide 
in community settings.  
 
This is one of three exploratory projects through the NIMH-funded P50 Penn Innovation in Suicide Prevention 
Implementation Research Center (INSPIRE) Center which brings together psychology, implementation science, 
health economics, machine learning, health information technology, psychiatry, and participatory research experts 
to apply innovative interdisciplinary research to tackle suicide prevention. 
 
OBJECTIVES: 
This exploratory project brings together an interdisciplinary team to leverage decades of research on behavior 
change from exposure theory to design and pilot-test an EBIS to target clinician-level anxiety about SSAI use to 
improve SSAI implementation for two gold-standard SSAIs: the Columbia Suicide Severity Rating Scale (CSSRS; 
Posner et al., 2011) and the Safety Planning Intervention (SPI: Stanley & Brown, 2008).  
 
In partnership with a team of clinicians, setting leaders, implementation scientists, and suicide experts we propose 
to: 
 
Aim 1:  Develop an EBIS to reduce clinician anxiety and bolster self-efficacy to deliver SSAI. The EBIS will leverage 
exposure-based therapies to manage core clinician fears and anxiety related to suicide in high-risk patient 
encounters. Using participatory design methods, design and adaptation will be conducted collaboratively 
with key stakeholders. 
 
Aim 2: Iteratively refine the EBIS in pilot field-testing using rapid cycle prototyping. We will engage 20 clinicians 
who will receive pilot iterations of EBIS; qualitative feedback will inform further refinement. 
 
Aim 3:  Test acceptability, feasibility, and preliminary effectiveness of the EBIS compared to Implementation as 
Usual (IAU) to target SSAI implementation mechanisms and implementation outcomes in a pilot 
randomized clinical trial with 40 clinicians. EBIS acceptability and feasibility will be measured via surveys 
and interviews. Exploratory analyses will examine impact of both conditions on putative implementation 
mechanisms (clinician anxiety, self-efficacy) and outcomes (SSAI adoption, fidelity). We will triangulate 
implementation outcomes via surveys, chart-stimulated recall, and standardized role plays coded by 
objective observers. Qualitative interviews with clinicians will further explore implementation outcomes and 
identify which changes to the EBIS are needed before confirmatory trials. 
 
Note that our objective in the Aim 3 trial is to test whether the EBIS approach to clinician training results in better clinician self-
efficacy and increased use of and fidelity to the gold standard SSAIs of interest in this study (the CSSRS and SPI), which are 
currently already the recommended practice in many City of Philadelphia agencies. We are not testing any patient level 

 
Version Date: 2/5/2024 
 
 
Page 3 of 20 
outcomes in this exploratory study nor are we recruiting or interacting with any patients in this study. 
 
BACKGROUND: 
Designing and testing scalable implementation strategies to increase use of evidence-based practices (EBPs) for 
suicide prevention are critical next steps for reducing suicide rates, which have risen since the turn of the century,18 
with particularly notable increases for disadvantaged populations in recent years.21 Evidence-based suicide 
screening,22 assessment, and interventions (SSAIs) for individuals at risk for suicide24,43,44 can reduce suicidal 
behavior,45 yet they are underused in routine practice,46 particularly in publicly-funded settings that serve 
disadvantaged populations.47,48 There is an urgent need for research focused on testing and refining implementation 
strategies, including optimizing clinician training, to improve quality of care for those at-risk for suicide.49  
 
Implementation strategy design should target mutable implementation mechanisms50 to optimize the likelihood of 
clinician behavior change. The implementation science literature purports that implementation strategies addressing 
specific facilitators and barriers to EBP delivery will be more successful than general strategies.50–52 Tailoring 
strategies requires targeting the specific mechanisms that affect implementation success. To date, implementation 
strategies for delivery of suicide prevention EBPs have been largely atheoretical.53 Thus, an important next step in 
practice-based suicide research is development of theoretically informed implementation strategies to increase 
uptake of suicide prevention EBPs.  
 
Clinician anxiety and low self-efficacy related to SSAI use are candidate target barriers to EBP uptake. Clinician 
anxiety about working with patients at-risk for suicide is well-documented.1,2 In our team’s formative qualitative work 
in this area, while some clinicians perceived themselves to have anxiety that is adaptive and helpful in motivating 
them to critically assess risk, nearly one-third reported moderate to severe anxiety related to using SSAIs.10,42 
Patient death by suicide is the greatest fear therapists report from a long list of possible adverse outcomes.54 This 
anxiety can manifest in several ways. First, clinicians report experiencing anticipatory anxiety about screening 
patients for suicide risk (e.g., fear of patients endorsing suicidal risk and not knowing how to intervene, not having 
enough time to intervene, or concern that asking about suicide will increase risk).7,8,55 In such cases, clinicians may 
avoid asking about suicide altogether or ask in inappropriate ways that limit the likelihood of identifying a patient at 
risk for suicide (e.g. “you are not having thoughts about suicide, are you?”).8 Second, clinicians may fear 
misclassifying a patient’s risk level (e.g., incorrectly deeming a patient safe to go home), fearing that the patient will 
attempt suicide or that they will be liable for making the wrong decision. This may lead to over-referring patients to 
emergency rooms or crisis centers or unnecessary hospitalization,17 even though there is limited evidence that 
inpatient hospitalization reduces suicide risk,17 and concerns about hospitalization’s iatrogenic effects.56  
 
High anxiety about SSAIs is also caused by low self-efficacy, or lack of confidence in one’s ability to effectively 
intervene to mitigate suicide risk.55,24 Higher self-efficacy to intervene with patients at risk for suicide is an important 
independent predictor of SSAI use7 and has emerged in the broader literature as a key driver of EBP implementation 
and sustainment.57 Anxiety and self-efficacy are related but distinct constructs.58,59 Theoretical models of 
maladaptive anxiety posit that low self-efficacy can increase anticipatory anxiety and, thus, the likelihood of 
maladaptive avoidance.60 Figure 1 illustrates our conceptual model of maladaptive anxiety applied to SSAI use. To 
our knowledge, no efforts have specifically targeted clinician anxiety about SSAI use other than offering training,55 
which the implementation literature has established is a necessary, but insufficient, implementation strategy to 
change behavior.61,62 
 
Exposure therapy for clinicians to reduce maladaptive anxiety and bolster clinician self-efficacy related to SSAI use 
is a compelling, but untested, approach. Exposure therapy is the key mechanism through which effective 
psychosocial treatment reduces anxiety.28,34,35  Exposure therapy is grounded in learning theories and cognitive 
behavioral models. Maladaptive anxiety comprises unhelpful cognitions (e.g., threat overestimation) and avoidance 
behaviors related to feared stimuli; anxiety is maintained and worsened through continued avoidance of feared 
stimuli,30 which also contributes to low self-efficacy to engage with these stimuli60,63 (see Figure 1). Exposure breaks 
the cycle of avoidance by supporting individuals to gradually confront and increase their tolerance of feared stimuli.31 
Importantly, exposure is well-tolerated, has few side effects,64 and a robust literature supports the utility of brief, 
one-session exposure-based treatments to target specific fears65–67 (in this case, fears about treating suicidal 
patients). Exposure-based strategies to address clinician anxiety and improve implementation have shown success 
for increasing uptake of exposure therapy itself – another anxiety-provoking EBP for clinicians33 – but have not been 
tested for SSAI. 

 
Version Date: 2/5/2024 
 
 
Page 4 of 20 
 
In sum, clinician anxiety and low self-efficacy are key implementation barriers to SSAI use but have not been 
implementation strategy targets. We will target these mechanisms by developing and pilot-testing an exposure-
based implementation strategy (EBIS) for implementing the following evidence-based SSAIs: the Columbia Suicide 
Severity Rating Scale (CSSRS) screener and assessment tool22 and the Safety Planning Intervention (SPI).24 Data 
will inform an R01 to test EBIS at scale. 
 
CHARACTERISTICS OF THE STUDY POPULATION: 
 
1. Target Population and Accrual: 
 
Aim 1: Aim 1 is a preparatory aim and does not involve human subjects research. 
 
Aim 2: Aim 2 participants will be community clinicians (n = 20) recruited from any local community mental health 
clinics in the Philadelphia region. 
 
Aim 3: Aim 3 participants are community clinicians (n = 40) recruited from 3-5 community mental health clinics in 
the City of Philadelphia. 
 
2. Key Inclusion Criteria: 
 
Aim 1: N/A  
 
Aims 2 and 3: Participants must be: 
1) Practicing mental health clinicians in the City of Philadelphia who provide direct mental health services to a 
treatment-seeking population. 
**This may include mental health clinicians who are both licensed or unlicensed (i.e., intern or trainees 
seeking licensure or practicing under another provider’s license). If the participant is a practicing clinician at 
a clinic from which we recruit, they will be eligible for the study. Potential licensures may include but are not 
limited to:   
a. Licensed Family and Marriage Therapist (LMFT) 
b. Licensed Clinical Social Worker (LCSW) 
c. Licensed Social Worker (LSW) 
d. Licensed Professional Counselor (LPC) 
e. Licensed Mental Health Counselor (LMHC)  
f. Licensed Clinical Professional Counselor (LCPC) 
g. Licensed Professional Clinical Counselor of Mental Health (LPCC)  
h. Licensed Clinical Mental Health Counselor (LCMHC) 
i. 
Licensed Mental Health Practitioner (LMHP) 
j. 
Clinical Psychologist (Licensed PsyD or PhD) 
2) Proficient in the English language  
3) Have access to a computer with internet connectivity  
 
3. Key Exclusion Criteria: 
 
Aim 1: N/A 
 
Aims 2 and 3: Participants will be excluded if they do not see any mental health patients that are at risk for suicide 
(e.g., they screen out all high-risk patients for their individual practice). Participants will not be excluded based on 
sex, demographics, and or past experience.  
 
4. Subject Recruitment and Screening: 
 
Aim 1: N/A  
 
Aim 2: Community clinicians for Aim 2 activities (n = 20) will be recruited from local community mental health 
clinics across the Philadelphia region. Our close partnership with the City of Philadelphia Department of 

 
Version Date: 2/5/2024 
 
 
Page 5 of 20 
Behavioral Health and the guidance of our Advisory Board (formed under Aim 1 activities) will facilitate 
recruitment. In our experience, local clinicians are eager for training opportunities, and we will further incentivize 
participation by offering continuing education credits for participating clinicians. To recruit clinicians, the study 
team will develop email scripts and flyers to disseminate to local agency partners who will in turn share these 
recruitment materials with their clinicians on staff. Clinics will be eligible if they have clinicians who provide direct 
mental health services to a mental health treatment-seeking population. We will only connect with clinicians 
working in these institutions if we receive clinic leadership approval. We will not contact any clinicians if we do not 
have approval from clinic leadership or the clinician does not express interest in participating. These materials will 
instruct potential participants to contact the study coordinator to confirm study eligibility, at which point the 
coordinator will present a study overview and proceed with study consent if the clinician is interested. Additional 
recruitment methods may include senior investigators on the project (Oquendo, Becker-Haimes, Jager-Hyman) 
making brief presentations (in person or virtually) at local agency meetings, during which details of the project and 
requirements of participation will be discussed. At these meetings, clinicians will complete a paper or electronic 
“consent to contact” via REDCap with their preferred contact information. Study staff will then follow up with 
interested clinicians with more information and to review all elements of informed consent. This approach ensures 
that clinicians do not feel pressured to make a decision to participate. Recruitment materials and scripts specific 
to the procedures in Aim 2 are included for the IRB’s review. 
 
Aim 3: We will recruit clinicians for the pilot trial (n = 40) from 3-5 of our local partner community agencies with 
whom we have close relationships. Recruited agencies will be identified in collaboration with our Advisory Board 
and our partners within the City of Philadelphia Department of Behavioral Health. Clinics will be eligible if they have 
clinicians who provide direct mental health services to a mental health treatment-seeking population. We will only 
connect with clinicians working in these institutions if we receive clinic approval. We will not contact any clinicians 
if we do not have approval from clinic leadership or the clinician does not express interest in participating. To recruit 
clinicians, the study team will develop email scripts and flyers to disseminate to local agency partners who will in 
turn share these recruitment materials with their clinicians on staff. If it is preferred by the agency, the study team 
will collaborate with leadership in these agencies to identify a time for Dr. Becker-Haimes or Dr. Oquendo or another 
senior member of the study team to attend a clinic team meeting (either virtually or in person) to present an overview 
of the study to interested clinicians. At this meeting, clinicians will complete a paper or electronic “consent to contact” 
via REDCap with their preferred contact information. Following the same procedures described in Aim 2 above, 
study staff will then follow up with interested clinicians to provide more information and to review all elements of 
informed consent.  
Recruitment materials and scripts specific to the procedures in Aim 3 are included for the IRB’s review. 
 
5. Early Withdrawal of Subjects: 
 
There is little risk associated with this study, and there is no safety concern associated with participation that would 
lead to participants being withdrawn from the study by the investigators. Importantly, exposure-based treatments 
upon which the EBIS implementation activities are based are generally well-tolerated with few, if any, side effects 
(see Risks below). As outlined in the informed consent document, there will be no consequences for any providers 
or other personnel who indicate interest and then do not enroll in the study. All participants will be free to withdraw 
their participation at any time by contacting study staff to inform them that they no longer wish to participate in the 
study.  
 
6. Vulnerable Populations:  
 
We will work directly with clinicians. Vulnerable populations will not be recruited. 
 
7. Populations vulnerable to undue influence or coercion: 
 
N/A 
 
STUDY DESIGN: 
 
This study brings together an interdisciplinary team with expertise in suicide prevention, implementation science, 
and exposure therapy to design a novel, exposure-based implementation strategy (EBIS) directly targeting clinician 
anxiety and low self-efficacy for SSAI use with patients at risk for suicide.  

 
Version Date: 2/5/2024 
 
 
Page 6 of 20 
Study activities will take place over approximately three years. The estimated length of individual clinician 
participation in Aim 2 is 3-4 weeks. The estimated length of individual clinician participation in Aim 3 is 4-6 months. 
The proposed start date for all study activities is 8/1/22. See below for proposed timeline: 
      
 
 
METHODS: 
 
Overview 
Aim 1: Develop an implementation strategy grounded in exposure theory (EBIS) to reduce clinician anxiety and 
bolster clinician self-efficacy to deliver SSAIs. We will develop and refine EBIS via presentation and workshopping 
with a stakeholder Advisory Board over multiple meetings, using a modified process of co-design.40,75 The outcome 
of Aim 1 will be a drafted manual to guide EBIS administration that will be further refined in Aim 2. 
A conceptual prototype for EBIS has been developed through our preliminary work and is described below.  Final 
materials will be developed based on Aim 1 output and submitted for IRB approval. 
Description of EBIS prototype. EBIS will be informed by the latest science in exposure theory,31 borrowing 
heavily from brief exposure-based treatments that can be delivered in a single session.66,67,72 We anticipate 
that EBIS will occur in four phases that map on to standard exposure therapy practice for patients with 
anxiety disorders: psychoeducation, assessment, practice, and relapse prevention; however, the final 
version will depend on the outcomes of Aim 1. Table 1 presents each anticipated component; each is also 
described below.  
Psychoeducation. In exposure therapy, psychoeducation is a critical step toward obtaining 
buy-in to engage in exposure practice. Typically, psychoeducation includes information about both 
adaptive and maladaptive anxiety. We will provide brief information about the role of anxiety in SSAI 
delivery, explain the “cycle of avoidance” that can occur when one experiences anxiety related to 
patient suicide risk, and present clinical implications (Figure 1).  
Assessment/Hierarchy-Building. We will use the “post-it method” in a group format for 
assessment and hierarchy-building. Clinicians will be presented with 8-12 flashcards that delineate 
various fears about interacting with a suicidal patient (e.g., “patient endorses passive suicidal 
ideation”) and fears related to outcomes of intervention (e.g., “patient completes SPI but attempts 
suicide after leaving clinic,”). Clinicians then rank-order each fear with respect to how anxiety 
provoking it is, and for each scenario, rate their anticipated level of anxiety using the Subjective Units 
of Distress Scale (SUDS) ranging from 0 (no anxiety) to 10 (maximal anxiety). We will select each 
clinician’s three most-feared scenarios for exposure practice.  
Study Timeline (By Project Quarter) 
Activity 
Project Period Quarter 
Y1 
Y2 
Y3 
1 2 3 4 1 2 3 4 1 2 3 4 
Preparation (secure IRB, hire staff, finalize data collection procedures) 
 
 
 
 
 
 
 
 
 
 
 
 
Aim 1  
 
 
 
 
 
 
 
 
 
 
 
 
Initial adaptation and refinement of EBIS w/ Advisory board  
 
 
 
 
 
 
 
 
 
 
 
 
Aim 2 
 
 
 
 
 
 
 
 
 
 
 
 
Iterative refinement via field test through rapid cycle prototyping (n=20 clinicians) 
 
 
 
 
 
 
 
 
 
 
 
 
     Cumulative Enrollment Benchmark:  5 clinicians 
 
 
 
 
 
 
 
 
 
 
 
 
     Cumulative Enrollment Benchmark: 20 clinicians 
 
 
 
 
 
 
 
 
 
 
 
 
Aim 3 
 
 
 
 
 
 
 
 
 
 
 
 
Pilot randomized trial (n = 40 clinicians) 
 
 
 
 
 
 
 
 
 
 
 
 
     Cumulative Enrollment Benchmark: 10 clinicians 
 
 
 
 
 
 
 
 
 
 
 
 
     Cumulative Enrollment Benchmark: 20 clinicians 
 
 
 
 
 
 
 
 
 
 
 
 
     Cumulative Enrollment Benchmark: 30 clinicians 
 
 
 
 
 
 
 
 
 
 
 
 
     Cumulative Enrollment Benchmark: 40 clinicians 
 
 
 
 
 
 
 
 
 
 
 
 
Interview transcription and coding 
 
 
 
 
 
 
 
 
 
 
 
 
Data analysis 
 
 
 
 
 
 
 
 
 
 
 
 
Manuscript preparation, dissemination of findings, R01 submission 
 
 
 
 
 
 
 
 
 
 
 
 

 
Version Date: 2/5/2024 
 
 
Page 7 of 20 
Guided Practice (In Vivo & Imaginal Exposure63). Clinicians will engage in either in vivo 
practice or imaginal exposure with a trained actor (research assistant, RA) based on feared 
scenarios selected in the step above. Consistent with a standard exposure model,73 RAs will be 
trained to guide clinicians to (1) identify core fears and anticipated anxiety via SUDS, (2) engage in 
targeted practice to violate assumptions of core fears and track SUDS changes, (3) engage in 
targeted cognitive debriefing to enhance coping self-efficacy. RAs will be guided by an “exposure 
checklist” that Project Co-PI Dr. Becker-Haimes developed and supervised closely by Dr. Becker-
Haimes during administration.74 A final adapted copy created in Aim 1 activities will be submitted to 
the IRB for review before use in Aim 2 or Aim 3 activities.  
Relapse Prevention/Application to Clinical Practice. Clinicians will be guided to 
summarize their experience and set intentions for managing anxiety they may experience in practice. 
Follow-up consultation will also include targeted discussion of clinician anxiety and avoidance to 
facilitate maintenance of gains.  
 
 
 
 
 
 
 
 
 
 
 
 
 
Aim 2: Iteratively refine EBIS in a pilot field test using rapid cycle prototyping. The primary goal of Aim 2 is to identify 
feasibility challenges to inform refinements to EBIS needed prior to formal testing, using rapid cycle prototyping. 
Secondary goals are to collect qualitative feedback from clinicians on the extent to which EBIS successfully engages 
target mechanisms of clinician anxiety and self-efficacy. The outcome of Aim 2 will be a refined manual to guide 
EBIS in the Aim 3 clinical trial. 
 
Aim 3: Test acceptability, feasibility, and preliminary effectiveness of EBIS to target implementation mechanisms 
and outcomes relative to implementation as usual (IAU) via a pilot randomized clinical trial with 40 community 
clinicians. Our primary Aim 3 dependent variables are EBIS acceptability and feasibility, measured through 
questionnaires, interviews, and recruitment and retention statistics. Secondary outcomes are preliminary 
effectiveness of EBIS on implementation outcomes (SSAI adoption and fidelity), and engagement of target 
implementation mechanisms (clinician anxiety and self-efficacy related to SSAI use), assessed via mixed methods 
(questionnaires, chart-stimulated recall, observer-coded role plays, and interviews). We also will gather data on 
other possible implementation mechanisms (e.g., SSAI knowledge, attitudes, intentions, norms, and organizational 
factors). Qualitative interviews with clinicians will probe for contextual implementation barriers to SSAI use not 
addressed by EBIS to inform augmentation prior to fully powered evaluation. 
 
1. Study Instruments: 
 
Aim 1: N/A  
 
Aim 2:  Data will consist of responses to semi-structured interviews and completion of a brief survey about 
clinician background and perceptions related to work with patients at risk for suicide. Qualitative data will be 
integrated to iteratively adapt and refine the EBIS implementation strategy in preparation for the Aim 3 trial. All 
data will be used exclusively for research purposes. 
- 
The qualitative interview guide and Aim 2 brief survey battery was previously reviewed and approved by 
the IRB 
 
Aim 3: Clinical trial data will consist of clinician responses to self-report questionnaires, chart-stimulated recall, 
semi-structured interviews, and fidelity role-plays with research staff. In addition, recruitment and retention 
Table 1. Anticipated EBIS Components and Rationale  
EBIS Component  
Rationale 
1. Psychoeducation 
Obtain clinician buy-in, explain how anxiety can interfere with suicide EBP 
delivery 
2. Assessment/ 
Hierarchy Building  
Identify tailored exposure practice targets that best match an individual 
clinician’s fears 
3. Exposure Practice 
Provide exposure to feared outcomes to foster clinician self-efficacy in 
managing high-risk patients, facilitate clinician practice managing anxiety 
in high-risk encounters.  
4.Relapse/ Prevention 
Application to Clinical 
Practice  
Transition “learning” about one’s ability to manage high-risk encounters to 
clinical practice; continue to use an exposure frame to support 
implementation  

 
Version Date: 2/5/2024 
 
 
Page 8 of 20 
statistics will capture feasibility metrics. No identifiable data about patient level information will be collected. All 
data will be used exclusively for research purposes and will be administered at the following time points: T1 = 
baseline/pretraining; T2 = post-training; T3 = 2-week follow-up; T4 = 12-week follow-up. Specific measures that 
will be administered are described below. Table 2 shows the anticipated assessment battery for the Aim 3 clinical 
trial. All measures will be administered to all clinicians enrolled in the trial, regardless of assigned condition.  
 
Primary Outcomes: Acceptability & Feasibility 
- 
Acceptability of Intervention Measure: A 4-item, psychometrically-validated measure that indexes the extent 
to which stakeholders believe an implementation strategy (in this case, IAU or EBIS) is acceptable. This will 
be administered at Time 2 and Time 4.  
o This measure was previously reviewed and approved by the IRB. 
 
- 
Feasibility of Intervention Measure. A 4-item, psychometrically-validated measure that indexes the extent to 
which stakeholders perceive an implementation strategy (in this case, IAU or EBIS) is feasible. This will be 
administered at Time 2 and Time 4. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Qualitative Interview. This interview will consist of three sections. Section 1 will query about the acceptability 
of the implementation condition to which they were assigned (EBIS or IAU) and any barriers that arose in 
engaging with any component of the study. Section 2 will focus on how the implementation condition 
engaged target implementation mechanisms. The final section will be guided by the Consolidated 
Framework for Implementation Research to query about barriers to SSAI use, with the primary goal of 
identifying contextual determinants of use not adequately addressed by EBIS. This will be administered at 
Time 4. 
o This measure was previously reviewed and approved by the IRB. 
 
Secondary Outcomes: Target Implementation Mechanisms  
- 
Subjective Units of Distress Scale (SUDS). The SUDS is a one-item, 10-point rating scale of perceived 
distress, commonly used to guide exposure therapy. This will be administered at Time 1- 4. 
o This measure was previously reviewed and approved by the IRB 
 
- 
Suicidal Patient Comfort Survey. A brief, 5-item measure assessing clinician anxiety about interacting with 
and treating patients with suicidality. This will be administered at Time 1- 4. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Self-Efficacy Questionnaire. Established question stems from behavioral measuring clinicians’ perceptions 
of themselves as having the skills and abilities to perform a task (in this case, SSAI use) on a 7-point scale. 
This will be administered at Time 1- 4. 
o This measure was previously reviewed and approved by the IRB. 
 
Secondary Outcomes: Implementation Outcomes  
- 
Chart Stimulated Recall (CSR). An established technique for examining clinician decision-making and 
clinical processes beyond what can be determined from chart review or self-report alone. A trained research 
team member will review the clinician’s caseload with them for the past clinic week and ask them brief 
questions (no more than 5 minutes) related to the clinician’s suicide-related practices (e.g., “Did you conduct 
a screen for suicide risk? How did you screen for risk? If risk was present, was a full CSSRS/SPI 
administered?). This will be administered at Time 1 and 3-4. No identifiable patient information is collected 
during the CSR and the study staff will not ever view the chart directly. Clinicians are instructed to refrain 
from sharing any identifiable patient details during the CSR administration. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Brief Role Plays. Participants will receive a vignette and prepare for a 45-60-minute role play, during which 
they will be asked to complete an SPI with a patient who was determined to be at-risk for suicide following 
CSSRS administration with a trained actor. Role plays will be audio-recorded and coded for competence 
with the Safety Planning Intervention Rating Scale (SPIRS). This will be administered at Time 3 and Time 
4. 

 
Version Date: 2/5/2024 
 
 
Page 9 of 20 
o An example of the procedures and coding scheme for the role play are included for the IRB’s review. 
 
Secondary Outcomes: Additional Mechanisms of Interest  
- 
Self-Perceived Knowledge & Attitudes About Suicide Scale. Selected subscales (Knowledge, Attitudes) of 
the Short Survey on Knowledge, Self-Confidence, and Attitudes Towards Suicidal Behavior will assess a 
clinician’s perceived knowledge about suicide and how to intervene. This will be administered at Time 1- 4. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Clinician Attitudes Toward Safety Planning. A 12-item questionnaire assessing clinician attitudes and beliefs 
about the use of safety planning interventions in clinical practice with suicidal patients. This will be 
administered at Time 1- 4. 
o This measure was previously reviewed and approved by the IRB. 
  
- 
SSAI Intentions and Determinants of Intention. Established question stems from behavioral science using 
items on a 7-point scale asking how willing and how likely one is to use SSAIs as well as attitudes and 
perceived norms of SSAI use. This will be administered at Time 1- 4. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Clinician Demographics. Brief questionnaire about clinician demographic (e.g., age, sex) and professional 
characteristics (e.g., years of experience, theoretical orientation, caseload descriptions), work setting, and 
a question about the clinician’s person experience with suicide (e.g., “Have you ever had a client die by 
suicide while under your care?”). This will be administered at Time 1. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Anxiety Sensitivity Index (ASI-3). Questionnaire about the clinician’s physical, cognitive, or social concerns 
about anxiety. This will be administered at Time 1. 
o This measure was previously reviewed and approved by the IRB. 
 
- 
Organizational Innovation Specific Capacity for Suicide Prevention Evidence-based practices (OISCSE). 
Questionnaire about the organizational policies, procedures, and practices within the clinician’s organization 
to observe potential factors that could influence use of SSAI’s. This measure has 14-items on a Likert scale 
plus a yes or no question to query any additional factors about the clinician’s organization not mentioned in 
the first 14-items. If the clinician responds positively, they are asked an additional open response question. 
This measure was adapted from a previously developed measure to assess policies and procedures needed 
to support exposure therapy delivery, guided by responses to qualitative interviews completed in Aim 2 of 
this study and items from the Zero Suicide Organizational Self-Study in collaboration with suicide prevention 
and implementation science experts.  
o A copy of this measure is included for the IRB’s review. 
 
 
Table 2. Assessment Schedule for Aim 3  
Outcome  
Measure 
T1 T2 T3 T4 
Primary Outcomes  
EBIS Acceptability  
Acceptability of Intervention Measure; Qualitative Interview (T4 only) 
 
X 
 
X 
EBIS Feasibility   
Feasibility of Intervention Measure; Recruitment/Retention statistics 
 
X 
 
X 
Secondary Outcomes  
 
Target Implementation Mechanisms 
 
Clinician Anxiety 
Subjective Units of Distress Scale;71 Suicidal Patient Comfort Survey55; 
Anxiety Sensitivity Index (ASI-3) (Time 1 only) 
X 
X 
X 
X 
Clinician Self-efficacy  
Established question stems81 
X 
X 
X 
X 
Implementation Outcomes 
SSAI use (CSSRS, SPI) 
Chart Stimulated Recall82 
X 
 
X 
X 
SPI Fidelity  
Role Play coded w/ SPIRS 
 
 
X 
X 
Additional Implementation Mechanisms of Interest  
SSAI Knowledge  
Self-Perceived Knowledge About Suicide Scale83 
X 
X 
 
X 
SSAI Attitudes 
Attitudes Toward Suicide Prevention;84 Clinician Attitudes Toward Safety X 
X 
X 
X 

 
Version Date: 2/5/2024 
 
 
Page 10 of 20 
 
 
 
 
 
 
 
 
 
2. Group Modifications: 
 
Aims 1 and 2: N/A 
 
Aim 3: After consenting to participate, clinicians will be randomly assigned to one of the two implementation 
conditions; the measurement battery will be identical for clinicians in each condition, with the exception that 
qualitative interviews for clinicians randomized to the EBIS condition will include several additional questions 
about their specific experience with EBIS.  
 
3. Method for Assigning Subjects to Groups: 
 
Aim 1: N/A 
 
Aim 2: No participants will be randomized as a part of Aim 2 activities.  
 
Aim 3:  Clinician participants will be randomized 1:1 to receive either Implementation as Usual (IAU) or IAU+EBIS. 
Randomization will take place after the informed consent process and prior to beginning either training condition.  
  
4. Administration of Surveys and/or Process:  
 
Aim 1: N/A  
 
Aim 2: To accomplish Aim 2, we will deliver the draft version of EBIS as outlined in the procedures below. Following 
receipt of the draft version of EBIS, clinicians will complete brief 45-60 minute qualitative interviews with trained 
study staff. Qualitative interviews will take place via phone, videoconference, or in person, depending on participant 
preference.  
 
Aim 3: During the clinical trial, all measures will be administered by trained research staff in accordance with the 
assessment schedule outlined above. Questionnaires at each time point will be administered either in paper format 
or electronically using REDCap, depending on participant preference (See Study Instruments above). Qualitative 
interviews will take place via phone, videoconference, or in person, depending on participant preference. 
 
5. Data Management: 
 
Drs. Maria Oquendo (PI) and Becker-Haimes (co-PI) will oversee all data cleaning and management with support 
from co-Investigator Dr. Jager-Hyman.  
 
All primary data, including audio-recordings, will be stored using Research Electronic Data Capture (REDCap), a 
HIPAA-compliant web-based survey platform. The REDCap system provides secure, web-based applications that 
are flexible enough to be used for a variety of types of research, provide an intuitive interface for users to enter data 
and have real time validation rules (with automated data type and range checks) at the time of entry. These systems 
offer easy data manipulation with audit trails for reporting, monitoring and querying patient records, and an 
automated export mechanism to common statistical packages (SPSS, SAS, Stata, R/S-Plus). REDCap servers are 
housed in a local data center at the CRCU at the Perelman School of Medicine and all web-based information 
transmission is encrypted. REDCap was developed specifically around HIPAA-Security guidelines. REDCap has 
been disseminated for use locally at other institutions and currently supports 240+ academic/non-profit consortium 
partners on six continents and over 26,000 research end-users (www.project-redcap.org). Any physical copies of 
the surveys, field notes, and interview data collected in Aims 2 and 3 will be kept in a locked filing cabinet with 
Planning85 
SSAI Intentions 
Established question stems 81 
X 
X 
X 
X 
SSAI Norms 
Established question stems 81 
X 
X 
X 
X 
Demographics  
Demographics survey  
X 
 
 
 
Organizational Factors 
Organizational Innovation Specific Capacity for Suicide Prevention 
Evidence-based practices (OISCSE) 
 
X 
 
 
T1=baseline. T2=post-training. T3=2-week. T4=12-week follow-up. 

 
Version Date: 2/5/2024 
 
 
Page 11 of 20 
access by research staff only. All participants will be assigned an ID number. The file linking participants’ ID numbers 
and identifiable information will be stored in a separate REDCap database with access by study staff only. All 
consent forms with identifiers will be stored in a separate locked filing cabinet or obtained electronically and 
maintained in a separate REDCap database.  
 
All personnel will complete required training before being granted access to any identifying information. This 
includes training on confidentiality through the Collaborative IRB Training Initiative (CITI) course. All personnel will 
also sign confidentiality statements. These confidentiality statements will specify the procedures for reporting 
unintentional breaches in confidentiality to the PIs. The confidentiality statements also specify that violations of 
participants’ confidentiality, either unintentional or deliberate, may result in termination of hire. The PI will conduct 
training with all research personnel regarding data, limits of confidentiality, maintaining confidentiality, and proper 
study procedures. Participants will be notified of the above procedures during informed consent. No identified data 
will be disclosed to personnel not listed on the study protocol.  
 
7. Subject Follow-up: 
 
The study investigators have a robust record of conducting community-partnered research to enroll clinicians in 
research trials. We attribute the success of these studies to: 1) taking a true partnered approach to research by 
including key stakeholders in research design at all stages of the research process, 2) the training of staff to build 
rapport with participants, and 3) local travel to participants to facilitate their completion and/or use of phone 
interviews to facilitate the gathering of qualitative data. The senior investigative team (Oquendo, Becker-Haimes, 
Jager-Hyman) will meet weekly with study staff to monitor project enrollment and retention so that any problems 
can be identified early and remediated. The project has clear benchmarks for recruitment (see Study Timeline), so 
any deviations from projected benchmarks will be readily apparent. We will track data on clinician enrollment and 
reasons for non-enrollment and study dropout. These data will help us identify specific barriers to participation that 
might be addressed through revised recruitment procedures. In addition, the investigative team has a robust record 
of conducting community partnered research to enroll clinicians in research trials with similar demands of 
participants. We attribute this success to strong community partnerships, rigorous training of all study staff in 
building rapport with participants and working closely with participants to facilitate data collection and completion. 
To facilitate retention of clinicians across the multiple assessment time points in Aim 3, we will additionally employ 
the following strategies that have been successful in our prior studies: (1) We will spread out clinician payments for 
research activities so that clinicians are compensated after each assessment to increase motivation to complete 
each activity, (2) We will collaborate closely with clinician participants to schedule standardized role-plays and 
interviews at hours most convenient for them, including after business hours and on the weekends, and (3) We will 
collaborate with our Advisory Board to identify additional strategies for keeping clinician participants engaged and 
motivated throughout the study period. Furthermore, the City of Philadelphia Department of Behavioral Health has 
expressed support and enthusiasm for this work (see Letter of Support). 
 
STUDY PROCEDURES: 
 
1. Detailed Description: 
 
Aim 1: Develop an implementation strategy grounded in exposure theory (EBIS) to reduce clinician anxiety and 
bolster clinician self-efficacy to deliver SSAIs. 
 
We will develop and refine EBIS via presentation and workshopping with a stakeholder Advisory Board over 
multiple meetings, using a modified process of co-design.40,75 An initial prototype for EBIS has been 
developed through our preliminary work, as described above. The Advisory Board will be assembled 
leveraging established community partnerships cultivated by Co-PI Becker-Haimes. Select faculty from the 
larger P50 Penn INSPIRE Center also will provide content expertise to the Advisory Board. The Board will 
be kept under 6, an ideal size for generating ideas.76 If an advisory board member discontinues participation, 
we will work to identify another board member for feedback on the development process. 
Drs. Oquendo and Becker-Haimes will co-facilitate all Advisory Board meetings. In the first meeting, we will 
present initial prototypes for each EBIS component and engage in group discussion about how best to refine 
them to maximize acceptability and feasibility while also maximizing ability to target clinician anxiety and 
self-efficacy about SSAIs. Questions posed to the Board for EBIS components will be general (e.g., “Tell us 

 
Version Date: 2/5/2024 
 
 
Page 12 of 20 
your thoughts on the psychoeducation component?” “What exposure tasks do you think will be most 
important?”) and specific (e.g., “How can we make the psychoeducation component more engaging?” “How 
can we optimize imaginal exposure to a feared outcome of being sued?”). We will query the Board about 
their perceptions as to how EBIS can be designed to be maximally acceptable and feasible for diverse 
settings. Drs. Oquendo and Becker-Haimes will integrate all feedback into a revised prototype that will be 
presented to the Advisory Board in a second meeting. We will repeat this until saturation is reached 
(anticipated max. 4 meetings) and a final prototyped version of EBIS is identified for field-testing in Aim 2.  
As noted previously, all Aim 1 activities are considered preparatory to human subjects research. All 
members of the Advisory Board are considered consulting members of the research team, not participants.  
 
Aim 2: Iteratively refine EBIS in a pilot field test using rapid cycle prototyping.  
 
We will use rapid cycle prototyping to iteratively refine EBIS based on feedback from up to 20 clinicians to 
optimize its acceptability and feasibility and identify early implementation challenges. Recruited clinician 
participants will receive approximately 3-4 hours of training in the CSSRS and SPI and the drafted EBIS 
prototype (exact sequencing of didactic and EBIS-based activities will be determined in Aim 1) either virtually 
or at a location on the University of Pennsylvania’s campus. We will be in communication with clinic 
leadership during the study. As part of the communication, we will ask the leadership at each participating 
clinic if the study activities may be done on work time. Based on this response, we will communicate to all 
participants at each respective clinic site if the study activities are permitted to take place during work hours. 
To avoid selecting for a specific type of therapist, we will consider designing trainings to take place in smaller 
increments of time that are maximally feasible to be completed in the work setting (e.g., over the lunch hour); 
we will also clarify to leadership that study training activities with accompanying CEUs will satisfy licensure 
requirements for suicide training. Bachelor level RAs, shown to be effective in administering exposure, will 
be guided to administer EBIS by Co-PI Dr. Becker-Haimes, an exposure therapy expert.  RAs will receive 
close supervision from Dr. Becker-Haimes.  
In this prototyping phase, EBIS will be administered either individually to clinicians or in small groups of 2-4 
clinicians. The syllabus that will guide EBIS administration is included for the IRB’s review. All trainings and 
consultation sessions will be video recorded and transcribed using HIPAA compliant software. During 
administration, study staff will take field notes on implementation challenges. Clinicians will complete a brief 
demographic questionnaire before and after completing the training. After receiving EBIS, clinicians will 
complete brief qualitative interviews (45-60 minutes) about their experience with EBIS either virtually or at a 
private location (e.g., the University of Pennsylvania campus or your office location). Interviews will be audio-
recorded and transcribed. The first part of the interview will query clinicians on acceptability and feasibility 
of EBIS. The second part will comprise semi-structured questions about perceptions of how EBIS influenced 
anxiety and self-efficacy to deliver the CSSRS and SPI. The last part will solicit suggestions for 
improvements to optimize EBIS. Clinicians will also receive continuing education credits for training 
activities. We anticipate it will take clinicians approximately 4-5 hours to complete study procedures (up to 
4 hours of training and 1 hour for the interview). 
We will make iterative refinements after each EBIS administration in response to observed implementation 
challenges and clinician feedback until saturation is reached (anticipated maximum: 20 clinicians). When 
clarity is needed on whether certain adaptations are needed, we will return to our Advisory Board for 
guidance. 
The finalized manual that will guide EBIS administration in Aim 3 was submitted for the IRB’s review.  
 
Aim 3: Test acceptability, feasibility, and preliminary effectiveness of EBIS to target implementation mechanisms 
and outcomes relative to implementation as usual (IAU) via a pilot randomized trial. 
 
After consenting, clinicians will be randomized to one of two implementation arms: IAU (n=20), or EBIS 
(n=20), which comprises all IAU activities plus EBIS as refined in Aim 2. If a clinician drops out of the study 
prior to attending their initial workshop (which is when official randomization will occur), we will make 
efforts to replace that individual to obtain a final sample of 40 clinicians randomized.  
 
IAU Condition. Gold-standard IAU for SSAIs typically comprises pre-implementation 

 
Version Date: 2/5/2024 
 
 
Page 13 of 20 
preparation, didactic training, knowledge tests, experiential role plays, ongoing expert consultation, 
and providing certification status to clinicians who attain established benchmarks. Pre-
implementation preparation will include provision of materials (e.g., SPI manual,78 instructions,79 and 
forms). Didactic training will occur in two parts: (1) CSSRS screening and assessment, and (2) SPI 
use. Part one will consist of materials we previously developed based on community clinician 
feedback. Part two will follow established SPI guidelines, including didactic training about SPI 
rationale and evidence base and experiential practice. IAU also will include supports for electronic 
health record integration (e.g., previously developed templates). After training, clinicians will receive 
six, one-hour sessions of expert consultation over the course of 12 weeks to discuss implementation 
barriers and receive more role play practice.  
EBIS Condition. Clinicians randomized to EBIS will receive all IAU elements outlined above 
plus the exposure-based intervention described above and refined in Aims 1 and 2 (see Table 1).  
In both arms, clinicians will complete study measures as outlined in Table 2 above at four time points: 
baseline/pre-training (T1), post-training (T2), two-week follow-up (T3), and twelve-week follow-up (T4). A 
12-week follow-up was selected based on base rates of suicidal ideation in outpatient settings to maximize 
likelihood that clinicians will encounter at least one patient at risk for suicide during the trial and to align with 
the traditional length of many evidence-based treatment protocols.77 Training will either take place virtually 
or at a location on the University of Pennsylvania’s campus. All trainings and consultation will be video 
recorded using HIPAA compliant software (in both implementation arms). Surveys will be completed online 
or on paper, depending on participant preference. Interviews will take place either virtually or at a private 
location that is convenient for the clinician (e.g., the University of Pennsylvania’s campus or your office 
location). All interviews will be audio-recorded and transcribed. We will be in communication with clinic 
leadership during the study. As part of the communication, we will ask the leadership at each participating 
clinic if the study activities may be done on work time. Based on this response, we will communicate to all 
participants at each respective clinic site if the study activities are permitted to take place during work hours. 
To avoid selecting for a specific type of therapist, we will consider designing trainings to take place in smaller 
increments of time that are maximally feasible to be completed in the work setting (e.g., over the lunch hour); 
we will also clarify to leadership that study training activities with accompanying CEUs will satisfy licensure 
requirements for suicide training. 
 
The time allotted for the training in Aim 3 will be standardized across conditions, but the content of the 
training will differ. Exact hours and time the training takes to complete may vary slightly depending on the 
number of clinicians in each training session, but the content of the training and the trainer will be the same 
for each session. As described throughout the protocol, the control training will we are using gold standard 
implementation as usual (IAU) training, role plays, and IAU-focused consultations. The intervention or EBIS 
condition would receive additional EBIS constructs in training (i.e., specific psychoeducation about how 
clinician anxiety is normal and to be anticipated in suicide prevention work), exposure-focused role plays 
(vs. traditional role plays in the IAU condition) and consultations (IAU = traditional consultation; EBIS = 
emotion-focused consultation). The training for both conditions will be led by Dr. Shari Jager-Hyman, an 
expert in suicide prevention EBP implementation, and Jesslyn Jamison, a post-doctoral candidate, who will 
be trained and supervised by Dr. Jager-Hyman. 
 
Participants will complete baseline surveys (30 min) and measures of SSAI use via chart-stimulated recall 
(CSR) (15 min) at T1 and then receive either IAU or EBIS. T2 assessment will comprise surveys (30 min) 
and will take place immediately after training in both arms. At T3 and T4 (two- and twelve-week follow-ups), 
clinicians will complete brief questionnaires (30 min), a CSR to measure SSAI use (15 min), and a role play 
to index SPI fidelity (45-60 min). At T4, clinicians will also complete a qualitative interview (30 min). We 
anticipate it will take clinicians approximately 17 hours to complete study procedures over 12 weeks (6 hours 
for training, 6 hours for expert consultation, and ~5 hours for research assessments across T1-T4).  
 
Note: As Aim 3 activities meet the NIH definition of a clinical trial, Aim 3 activities are registered 
on ClinicalTrials.gov as a condition of award (NCT05172609).  
 
2. Data Collection: 
Not applicable. 
 

 
Version Date: 2/5/2024 
 
 
Page 14 of 20 
3. Genetic Testing: 
Not Applicable 
 
4. Use of Deception: 
No deception will be used in this project.  
 
5. Statistical Analysis: 
 
Quantitative analyses This pilot feasibility trial is not intended to be adequately powered to detect significant 
effects.94,95 Findings will provide key preliminary data to support the feasibility of EBIS and study procedures for a 
fully powered R01 (e.g., frequency of suicidal ideation over the 8-week follow-up period). Qualitative analysis of 
interview transcripts and mixed methods analysis will supplement quantitative analyses.  
 
Preliminary Analysis. Data screening and missing data analysis will be conducted in accordance with best practice 
recommendations.96 Prior to conducting main analyses, we will examine the psychometric properties of all scales 
used to assess constructs of interest (e.g., coefficient alpha) to confirm adequate performance. We will conduct 
analyses of baseline variables as a randomization check to ensure comparable baseline characteristics for clinicians 
in EBIS and IAU. Any differences will be controlled for in subsequent analyses.  
 
Hypothesis 1: EBIS will be an acceptable and feasible implementation strategy. We will calculate descriptive 
statistics on the AIM and FIM measures and compare scores between EBIS and IAU. We will also evaluate EBIS 
feasibility by calculating the proportion of clinicians randomized to EBIS who complete all exposure tasks.  
 
Exploratory Hypotheses 1 and 2: EBIS will engage target implementation mechanisms of clinician anxiety and self-
efficacy better than IAU; clinicians randomized to EBIS will show improved SSAI adoption and fidelity relative to 
clinicians randomized to IAU. We will examine the effect of condition over time on target mechanisms (anxiety and 
self-efficacy) and implementation outcomes (mean CSSRS screening frequency across encounters, proportions of 
appropriate encounters in which clinicians conduct follow-up assessment with the CSSRS and SPI use, as 
measured by CSR, and average SPI fidelity scores on role plays) using repeated measures analysis of covariance 
(ANCOVA), controlling for organization (given the small sample size and that we will recruit from only a small 
number of organizations for this pilot, we do not anticipate the need for multilevel analysis).  
 
Qualitative and Mixed-Method analyses. Qualitative analysis of interviews will complement quantitative data to 
better understand: (1) the efficacy of EBIS to reduce clinician anxiety and bolster self-efficacy, (2) processes by 
which EBIS does or does not faciliate SSAI use, and (3) contextual factors in clinicians’ settings that are not 
addressed by EBIS. Interviews will be transcribed and analyzed via qualitative software, guided by an integrated 
approach97 which uses an inductive process of iterative coding to identify recurrent themes, categories, and 
relationships. A structured codebook will be developed. We will code for a priori attributes of interest (i.e., the extent 
to which EBIS engages our target mechanisms of anxiety and self-efficacy) and also use modified grounded 
theory,98 which provides a systematic and rigorous approach to identifying codes and themes (e.g., to identify 
additional barriers that arise to SSAI use). Using the qualitative software program, Dr. Becker-Haimes and a 
postdoctoral fellow will separately code three transcripts and compare their application of the coding scheme to 
assess its reliability and robustness. Any disagreements will be resolved through discussion and the codebook 
refined and applied to all transcripts. Code reliability will be monitored via bi-weekly meetings to ensure κ ≥ .85. 
After coding, we will read through all codes to examine themes and produce memos of examples and commentary. 
Mixed methods will analyze themes as a function of quantitative data to identify patterns of responding, following 
NIH guidelines for best practices in mixed methods.99 We will first use findings from quantitative data to identify 
patterns in qualitative data by entering quantitative findings into analysis software as participant attributes. 
Quantitative attributes will categorize and compare important themes among subgroups. For example, we may 
categorize clinicians into two groups: those with high and low SSAI use on CSR data. Then, if clinician anxiety 
emerges as a theme from interviews, we can query instances when anxiety is discussed among clinicians with low 
and high SSAI use; this will allow us to identify patterns and make interpretations across groups based on 
quantitative data. Dr. Becker-Haimes is experienced in mixed-methods implementation research42,100 and will 
oversee these analyses. 
 
RISK/BENEFIT ASSESSMENT: 

 
Version Date: 2/5/2024 
 
 
Page 15 of 20 
 
1. Risks:  
 
Potential harm to participants is minimal as we are testing two ways of training clinicians in gold-standard suicide 
prevention practices, not testing the use of a new suicide prevention practice. There are no known physical, 
financial, or legal risks to participating in the study. However, there is a risk of loss of confidentiality in all research. 
We will minimize this risk by using secure, encrypted servers to host all data and conduct the analysis. We will 
implement multiple, redundant protective measures to guarantee the privacy and security of the participant data. 
All investigators and study staff with direct access to the identifiable data will be required to undergo annual 
responsible conduct of research, cybersecurity, and HIPAA certification in accordance with University of 
Pennsylvania regulations. Data will be stored, managed, and analyzed on a secure, encrypted server behind the 
University of Pennsylvania Health System (UPHS) firewall. All study personnel that will use this data are listed on 
the IRB application and have completed training in HIPAA standards and the CITI human subjects research. All 
data analyses will be performed using participants’ identification numbers rather than names to minimize the circle 
of study team members who know the identity of the research participants. In addition, risk of loss of confidentiality 
will be minimized by storing completed paper surveys and copies of the signed informed consent forms (separate 
from one another) in locked file cabinets in locked offices accessible only to trained study staff.  
 
We will minimize the risk of coercion by making clear in the consent process that participation in the study is truly 
voluntary and will not affect participant employment in any way. Our experience conducting similar research 
protocols is that therapists who are not interested in participating in the research decline freely. 
Managers/leadership will not be present when study staff describes the study to potential participants, enabling 
participants to choose not to participate without having to do so in the presence of leadership. In addition, we will 
provide participants with ample time to ask questions or have family/friends ask questions about their participation. 
To ensure no participants (potential or actively participating) feel coercion to participate in any of the study aims, we 
will work closely with our IRB, Advisory Board, and stakeholders to ensure that our study is not coercive. In our previous 
work, we have ensured that all data provided by clinicians are confidential and never shared with peers, supervisors, 
clinic leadership, or the payer in an individual or identified manner. We will continue to take this approach in the 
proposed work.  
 
Self-reported questionnaires, role-plays, CSRs, and semi-structured interviews could lead participants to feel 
temporarily uncomfortable. Importantly, exposure-based treatments upon which the EBIS implementation activities 
are based, are generally well-tolerated with few to no side effects. To minimize discomfort, we will follow established 
procedures used in multiple studies executed in the proposed settings by the study investigators. Specifically, a 
licensed clinical psychologist or physician from the research team will be available to speak with any participants 
who feel unduly distressed and will make appropriate referrals. Participants are made aware of their ability to 
discontinue participation at anytime without penalty.  
 
Although we will not be asking participants about their own suicidal thoughts or behaviors, if a participant 
spontaneously discloses suicidal thoughts, plans, or behaviors to a member of the research team, we will follow the 
High Risk Management Protocol used routinely in the Penn Center for the Prevention of Suicide (CPS) IRB-
approved research studies. Co-Investigator Shari Jager-Hyman, Ph.D. from CPS will train all study staff in these 
procedures.  
 
In the event that a clinician mentions child abuse/neglect or a patient’s imminent suicidality/homicidality during 
qualitative interviews, or if suicidality/homicidality is otherwise observed while study staff are embedded in the 
clinical settings, we will follow established protocols from similar studies carried out by our team. Specifically, a 
clinically-trained member of the research team will consult with the patient’s treatment team to ensure that any 
information about child abuse/neglect or intent to harm self or others has been reported to the appropriate parties, 
as required by law. A licensed clinical psychologist or physician with extensive experience assessing risk (Drs. 
Oquendo, Becker-Haimes, or Jager-Hyman) will be available for consultation at all times that researchers are 
conducting study assessments.  
 
2. Benefits: 

 
Version Date: 2/5/2024 
 
 
Page 16 of 20 
 
There are no direct benefits to participants; however, participants may find satisfaction in sharing their ideas and 
experiences with research staff. Clinician participants may indirectly benefit from participation by receiving free 
training and consultation on the use of suicide prevention screening, assessment and the SPI with their patients. 
This may contribute to clinicians providing more effective care for treatment-seeking individuals and/or increase 
their understanding of key concepts or strategies involved in psychosocial treatment. The benefit to the clinical 
community is that the results may ultimately help improve the quality of care. The ratio of risks to benefit is positive, 
considering that we do not think that the study poses any risks and the scientific yield from the study could be greatly 
beneficial.  
 
3. Subject Privacy: 
 
Privacy will be given utmost consideration and is highly valued in the proposed research. Research staff will 
exercise caution in ensuring privacy within the constraints of the research setting (clinicians’ work environment). 
For participants in Aims 2 and 3, research staff will ensure that all study questionnaires and interviews and are 
administered in private places where no one can overhear the content. Research staff will ensure that participants 
are comfortable proceeding with the interview in their environment prior to beginning.  
 
4. Subject Confidentiality: 
 
How will confidentiality of data be maintained?  Check all that apply. 
 
X Paper-based records will be kept in a secure location and only be accessible to personnel involved in 
the study. 
X Computer-based files will only be made available to personnel involved in the study through the use of 
access privileges and passwords. 
X Prior to access to any study-related information, personnel will be required to sign statements agreeing 
to protect the security and confidentiality of identifiable information. 
X Whenever feasible, identifiers will be removed from study-related information.  
 A Certificate of Confidentiality will be obtained, because the research could place the subject at risk of 
criminal or civil liability or cause damage to the subject’s financial standing, employability, or liability. 
 A waiver of documentation of consent is being requested, because the only link between the subject 
and the study would be the consent document and the primary risk is a breach of confidentiality.  (This is 
not an option for FDA-regulated research.)  
X Precautions are in place to ensure the data is secure by using passwords and encryption, because the 
research involves web-based surveys.   
X Audio and/or video recordings will be transcribed and then destroyed to eliminate audible identification 
of subjects.  
 Other (specify):     
 
The only data will be collected from mental health clinicians interacting with the strategies being refined and tested. 
No identifying patient information will be collected. All project staff with direct access to the identifiable data will be 
required to undergo annual responsible conduct of research, cybersecurity, and HIPAA certification in accordance 
with University of Pennsylvania regulations. When results of the research are presented at scientific meetings or 
published, no identifying information will be included. Only authorized persons from Penn and their associated 
Institutional Review Board will have access to review research records. Below we outline how we plan to ensure 
confidentiality specific for all study related activities.  
 
To ensure confidentiality across all aims, we will be using the following methods for protection of data living on Center 
for Mental Health (CMH) servers. All data will be de-identified and subject identity will be masked using numeric codes 
and password-protected master lists and will not be linked in any way to the participant’s names. Identifiable data will 
be entered and stored into password-protected files on password-protected drives on the CMH encrypted server or on 
REDCap, and files for analysis will only be identified with subject numbers, and will not contain any identifying 

 
Version Date: 2/5/2024 
 
 
Page 17 of 20 
information. Given that only the study research team will have access to the identities of the participants, and all 
appropriate safeguards are in place, it is unlikely there will be a breach of confidentiality.  
 
We will only store and use the data at the following locations: 1) Password-protected PCs in locked offices at Penn; 2) 
The desktop computer that will access the data will be part of the CMH domain, with Windows Active Directory group 
policies in place to distribute and enforce best-practice desktop security policies. Security and software updates are 
pushed through the system in a timely manner using desktop management tools (Tivoli/BigFix and Windows Automated 
Update Service).; and 3) A server at the University of Pennsylvania within the facilities managed room in the data 
center. 
 
Physical copies of any data will be encrypted while at rest and will be held in a locked cabinet within the CMH office. 
Additionally, all data hosted on the server will be accessed over an encrypted (Value-Added Network) VAN connection. 
Individually identifiable or deducible data will not be transmitted by unsecured telecommunications. Any data transfer 
will be sent via secured communications, which include secure Penn Medicine email servers or SecureShare, a web-
based application for secure, encrypted file exchange available to Penn faculty, staff, and students. Further, the data 
will not be physically moved or transmitted in any way from the CMH servers or secured desktop(s) without written 
approval.   
 
Brief qualitative interviews and role plays will be audio-recorded during Aims 2 and 3. All trainings and consultation 
sessions will be video recorded. These audio and video recordings will be directly uploaded into REDCap and 
subsequently transcribed and de-identified using HIPAA compliant vendors. All transcripts will be double checked 
to ensure that no identifying information remains.  This means if a participant mentions someone’s name or specific 
location (e.g., clinic, street name) in the recording, the identifying information will be removed and replaced with a 
description of the information (e.g., [NAME], [LOCATION]). The recordings will be destroyed after the transcription 
process is completed. The de-identified transcripts will be stored in password-protected files in the CMH encrypted 
server or on REDCap, and the deidentified data will be loaded into qualitative software for data management and 
analysis.  
 
During the clinical trial phase of the study (Aim 3), quantitative surveys may also collected using pen-and-paper 
hard copy, depending on participant preference; study staff will enter any pen-and-paper data directly into REDCap. 
All study participants will be given an ID number for identification purposes. A document with participants’ names 
and identification numbers will be electronically stored in a separate REDCap file. Only the research team will have 
access to this file. All data will be protected at the file-level and operating system level. All data for analysis will 
have personal identifiers removed and the data will be aggregated through qualitative analysis. Field notes will be 
taken and immediately encrypted. Any paper materials related to the study will be stored in a locked file cabinet in 
a locked office at CMH. Only research staff will have access to this locked cabinet.  
 
Individually identifiable or deducible data will not be transmitted by unsecured telecommunications., Any data 
transfer will be sent via secured communications, which include secure Penn Medicine email servers, or SecureShare, 
a web-based application for secure, encrypted file exchange available to Penn faculty, staff, and students. The data 
will ultimately be transferred to CMH encrypted servers and all precautions above will be taken.  
 
The master list linking ID numbers and names will be kept on password protected computer files accessible only to the 
study investigators. Tracking information will be housed in a locked file cabinet within a locked office at Penn and this 
master list will only be used for the coordination of data collection and participant payment. All research staff will sign 
confidentiality statements and be trained in the protection of human subjects by the PIs and through the Collaborative 
IRB Training Initiative (CITI) course. Participants will be notified that the alternative to participation is to not participate, 
they can withdraw their participation at any time without penalty, and their decision to participate will not affect services 
or employment. 
 
All data, including the master list linking identifiers to the ID number and notes will be retained for the length 
required by federal guidelines. After this time period has lapsed (three years from the date of the final Federal 
Financial Report), the master list linking identifiers to ID numbers and notes will be destroyed. 
 
5. Protected Health Information 
 

 
Version Date: 2/5/2024 
 
 
Page 18 of 20 
The following PHI will be collected 
• 
Name 
• 
Address 
• 
Electronic mail addresses 
• 
Telephone numbers  
• 
Biometric identifiers, including finger and voice prints 
 
Name, address, electronic mail address, and telephone numbers will be collected for the purpose of contacting 
participants about research activities and compensation only. Additionally, PHI (voice prints) will be collected via 
audio recording and will be maintained on REDCap, a HIPAA-compliant server. 
 
6. Compensation:  
 
Aim 1: N/A, no human subjects activities.  
 
Aim 2: Clinicians will be compensated $75 for all Aim 2 activities (engaging with the EBIS prototype and completing 
follow up interviews and surveys) and will receive continuing education credits for training received in the CSSRS 
and the SPI.  
  
Aim 3:  Clinicians will receive up to $250 for completing study assessments: $50 compensation for completing 
study activities at T1, $25 for T2, and $75 for T3 and $100 for T4 study activities. Clinicians also will receive 
continuing education credits for training received in the CSSRS and the SPI.  
  
7. Data and Safety Monitoring: 
 
The NIMH-funded P50 Penn INSPIRE Center has convened a 3-member DSMB from distinguished faculty outside 
of Penn and the INSPIRE advisory board. The DSMB includes identified experts by the INSPIRE MPIs and consists 
of an independent group of experts charged with reviewing study data for data quality and integrity, adherence to 
the protocol, participant safety, study conduct and progress, and making determinations regarding study 
continuations, modifications, and suspensions/terminations. DSMB members are independent from any 
professional or financial conflict of interest with the research project or study investigators. The DSMB will meet at 
least once for the duration of the project. 
 
As noted above, this study is considered minimal risk to participants, as it consists primarily of observations and 
quality improvement practices. There is no direct interaction with patient populations. Clinician participation in the 
study does not create or increase the risk of completed suicide for the patients with whom they work. However, it 
is possible that study staff may learn that a patient is at high risk of suicide and that appropriate safeguards have 
not been enacted to ensure that patient’s safety. In the event this occurs, we will follow this risk management 
protocols described above (see Risks) that we have applied successfully in other studies.  
 
8. Investigator’s Risk/Benefit Assessment: 
 
There are no direct risks to participation. Some strategies may interface with patients, but no new interventions will 
be introduced, as the goal is to increase the use of established SSAIs supported by the City of Philadelphia within 
routine clinical practice. Providers will maintain complete autonomy with respect to patient care. 
 
The benefit to the clinical community is that the results may ultimately help enhance the implementation of suicide 
prevention practices. The ratio of risks to benefit is positive, considering that we do not think that the study poses 
any risks and the scientific yield from the study could be greatly beneficial. Participants will be adequately 
compensated for their participation throughout all aspects of the project. 
 
INFORMED CONSENT: 
 
1. Consent Process: 
 

 
Version Date: 2/5/2024 
 
 
Page 19 of 20 
Aim 1 does not involve collection of research data and, therefore, there is not a consent process for this phase. 
 
Aim 2. Prior to beginning data collection for Aim 2, members of the research team will provide a verbal 
description of the study to clinicians. Clinicians will then be asked to electronically sign consent forms in REDCap 
(using REDCap’s established e-consent signature option) that thoroughly describe the procedures to be followed 
in the study and the type of assessments involved. Clinicians will be informed that all information they provide will 
be kept confidential (unless it is determined that it must be reported as required by law), within the study 
personnel. No information gathered as part of this research will be shared with agency executive directors or 
supervisors (unless it is determined that information must be reported as required by law). Clinicians will also be 
informed that participation is voluntary and there is no penalty for declining to participate or withdrawing from the 
study at any point. We anticipate all clinicians will be competent to provide informed consent.  
 
Participants will be provided with copies of signed consent forms while original copies of the signed consent forms 
will be stored electronically in REDCap or kept in locked files at Penn, which no one will have access to other than 
the research team.  
 
Aim 3. We will obtain agency leadership approval to recruit clinicians from organizations before consenting any 
participants. Prior to participation, Dr. Oquendo, Becker-Haimes or another member of the research team will 
verbally provide a full description of procedures to be followed and the type of assessments involved to all clinicians. 
Participants will also be informed at this point that all information they provide will be fully kept confidential allowable 
by law. No identifiable information gathered as part of this research will be shared with a recruited clinicians’ agency 
executive director, supervisor, or local regulatory department. Participants will be asked to electronically sign a 
consent form through REDCap that thoroughly describes the procedures to be followed in the study and the type 
of assessments involved. There will be no consequences for individuals who elect not to enroll in the study. 
Clinicians will also be informed that they can withdraw from the study at time of their choosing point without penalty. 
Informed consent will be completed by study staff by phone or in person. We anticipate all clinicians will be 
competent to provide informed consent.  
 
Given that we are recruiting practicing mental health clinicians, we do not anticipate recruiting any participants under 
the age of 18; thus, assent procedures will not be necessary.  
  
2. Waiver of Informed Consent: 
 
We are not requesting exemption or alteration of written informed consent documentation 
 
RESOURCES NECESSARY FOR HUMAN RESEARCH PROTECTION: 
Study investigators include MD and PhD-level researchers with expertise in suicide prevention (Oquendo, Jager-
Hyman), exposure therapy (Becker-Haimes), and implementation science (Becker-Haimes, Jager-Hyman).  A 
postdoctoral fellow and clinical research coordinator will be thoroughly trained in the study protocol and will facilitate 
all data collection. Given that study activities constitute a clinical trial, all study staff will complete Good Clinical 
Practice (GCP) training. Regular meetings among study team members will ensure that research staff are 
adequately informed of the protocol and their research-related duties. We are confident that the size of the research 
team will be adequate to carry out the proposed project.  
 
For Aim 1, we additionally will bring together an Advisory Board of clinicians, administrators, suicide prevention 
experts, implementation scientists, and exposure theorists to refine the EBIS implementation strategy. Dr. Becker-
Haimes has close relationships with local community mental health clinics which will facilitate recruitment of 
Advisory Board members.  
 
The Department of Psychiatry at the University of Pennsylvania’s Perelman School of Medicine will provide the 
research infrastructure, physical accommodations, and resources necessary to ensure the project’s success. The 
study team members have dedicated office space with password protected computer systems. Researchers will 
have the necessary word processing and statistical software for the project. 
 
Acronym Description 

 
Version Date: 2/5/2024 
 
 
Page 20 of 20 
SSAIs 
suicide screening, assessment, 
and interventions 
EBIS 
exposure-based implementation 
strategy 
CSSRS 
Columbia Suicide Severity Rating 
Scale 
SPI 
Safety Planning Intervention  
IAU 
Implementation as Usual 
EBPs 
evidence-based practices 
RA 
research assistant 
CSR 
Chart Stimulated Recall 
SPIRS 
Safety Planning Intervention Rating 
Scale 
ASI-3 
Anxiety Sensitivity Index 
PI 
Principal Investigator 
co-PI 
Co-Principal Investigator 
REDCap 
Research Electronic Data Capture  
CRCU 
Clinical Research Computing Unit 
CITI 
Collaborative IRB Training Initiative 
UPHS 
University of Pennsylvania Health 
System 
CPS 
Center for the Prevention of 
Suicide 
CMH 
Center for Mental Health 
PCs 
Personal Computers 
VAN 
Value-Added Network 
PHI 
Protected Health Information 
INSPIRE 
Penn Interdisciplinary Network for 
Scientists Promoting Inclusion, 
Retention, and Equity 
DSMB 
Data and Safety Monitoring Board 
GCP 
Good Clinical Practice