The BH-Works Suicide Prevention Program for Sexual and Gender Minority Youth

Study Overview
=================
Brief Summary
-----------------
Suicide is the second leading cause of death for 15-to-24-year-olds in the United States. Compared to their heterosexual and cisgender peers, sexual and gender minority (SGM) adolescents report significantly higher rates of suicidal ideation and suicide attempts. Unfortunately, many barriers complicate the implementation of suicide prevention in SGM communities. SGM youth often report feeling unwelcome and misunderstood in traditional behavioral health service organizations. Consequently, treatment attendance and retention remain low. Instead, this population generally seeks mental health services in community organizations for lesbian, gay, bisexual, transgender, and queer (LGBTQ) youth. Unfortunately, these organizations are often unprepared for this clinical challenge. The Behavioral Health-Works (BH-Works) suicide risk management system may offer a potential solution to this problem. BH-Works is an evidence-based, comprehensive youth suicide prevention program. It offers support for policy development, staff training, suicide and behavioral health screening, technology-assisted safety planning, an electronic patient referral system, real-time data analytics for program monitoring, and a learning collaborative structure to support sustainability. All functions are supported on a web-based software platform that facilitates cross-system communication, implementation, adoption, and expansion. In this project, the investigators will adapt this program for LGBTQ organizations and test feasibility, acceptability and preliminary effectiveness. This project builds upon robust partnerships with two diverse LGBTQ organizations in Philadelphia, Pennsylvania and rural Southwest, Virginia) and their respective behavioral health (BH) partnering sites. To facilitate BH-Works adaptation for SGM adolescents, the investigators will employ the Enhancing Engagement trajectory from Lau's cultural adaptation framework. To pilot the program within LGBTQ organizations and their partners, the investigators will use an Effectiveness-Implementation Hybrid Type 2 design with a historical comparison group. Informed by the Consolidated Framework for Implementation Research, the investigators will also pilot test a sequenced implementation strategy. This strategy focuses on promoting engagement, building partnerships, and creating sustainability. In Years 1 and 2, the investigators will collect treatment as usual data, and work with their partners to adapt BH-Works policy, content, practices, and workflow. The investigators will also train staff/providers in suicide risk management, family engagement and affirmative care. In Years 3 and 4, the investigators will test the adapted SGM BH-Works Program and examine several essential program targets (training impact, partnership development, software usability) and outcomes (successful referral, program satisfaction, caregiver involvement, suicide identification). The proposed research responds to the growing national need to identify and refer vulnerable youth at risk for suicide. A future R01 will examine SGM BH-Works program effectiveness with partnering LGBTQ and BH sites across the nation.

Detailed Description
-----------------
Suicide is the second leading cause of death for 15-to-24-year-olds in the United States (U.S.). Yet, only 14% of youth with suicidal ideation and 22% of those who make a suicide attempt, report receiving mental health services. The circumstances that sexual and gender minority (SGM) youth face are particularly alarming. Compared to their heterosexual and cisgender peers, SGM adolescents report far higher rates of suicidal ideation and suicide attempts. Consequently, adoption of effective suicide prevention programs, that increase identification and referral in organizations serving this population, are sorely needed. Unfortunately, many barriers complicate the implementation of suicide prevention for SGM communities. SGM youth often report feeling unwelcome and misunderstood in traditional behavioral health service organizations. Consequently, treatment attendance and retention remain low. Instead, this population generally seeks mental health services in community organizations for lesbian, gay, bisexual, transgender, and queer (LGBTQ) youth. Unfortunately, these organizations are often unprepared for this clinical challenge. Specifically, they lack a) training in risk assessment, b) standardized screening tools, and c) access to behavioral health (BH) services that staff trust. In addition, staff in LGBTQ organizations express concern that many BH providers lack the SGM-sensitivity needed to work with this high risk, vulnerable population. Given these challenges, suicide prevention for SGM youth requires a multi-faceted program aimed to improve resources within these organizations and relationships between service systems. A potential solution to this challenge is the Behavioral Health-Works (BH-Works) suicide risk management system (Diamond et al., 2021). Similar to the identify, treat and refer structure of screening, brief intervention, and referral to treatment (SBIRT) for substance use, BH-Works includes support for policy development, staff training, suicide and behavioral health screening, technology-assisted safety planning, an electronic patient referral system, real-time data management for program monitoring, and a learning collaborative structure to support sustainability. All functions are supported on a web-based platform that facilitates cross-system communication, implementation, adoption, and expansion. BH-Works has been used in both clinical and non-clinical settings. In this project, the investigators will adapt BH-Works for SGM adolescents presenting in LGBTQ organizations and use data from the web-based screening and EMR systems to measure targets and outcomes. The investigators will employ the Enhancing Engagement trajectory, from Lau's (2006) cultural adaptation framework for this purpose. Lau recommends that adaptation of evidence-based treatments (EBTs) is necessary when contextual processes (e.g. discrimination, caregiver support, mistrust of health systems) contribute to unique vulnerabilities in specific populations, particularly those living in contexts where fewer specialized services exist. This project builds upon partnerships with two LGBTQ organizations in Philadelphia, and rural Southwest, Virginia) and their respective behavioral health (BH) partners. The investigators will use an Effectiveness-Implementation Hybrid Type 2 design, with a historical comparison group, to test the feasibility, acceptability, and preliminary effectiveness of BH-Works within the LGBTQ organizations and their BH partners. Informed by the Consolidated Framework for Implementation Research (CFIR), the investigators will pilot test a sequenced implementation strategy. This strategy focuses on building partnerships and involves a) promoting engagement, b) strengthening relationships, and c) creating sustainability. In Year 1, the investigators will collect treatment as usual data, and work with stakeholders to adapt BH-Works policy, content, practices, and workflow. The investigators will also train staff in suicide risk management, family engagement, and affirmative care. In Years 2 and 3, the investigators will test the adapted SGM BH-Works Program and examine several essential program targets and outcomes, which are outlined in the aims. Three aims focus on engagement, adaptation, and feasibility/acceptability of SGM BH-Works. Aim #1: Engage LGBTQ organization staff and partnering behavioral health providers. This aim focuses on: a) engaging a stakeholder advisory group, and b) initiating the implementation strategy. Aim #2: Adapt and pilot the BH-Works Program for LGBTQ organizations and partnering behavioral health sites. The adapted BH-Works Program will be implemented into LGBTQ organizations' workflow for a one-month open trial. Qualitative and quantitative data will be collected to evaluate initial feasibility and acceptability as well as to explore barriers and facilitators to usability in urban and rural organizations. The manual will undergo revisions. Aim #3: Test the feasibility, acceptability, and preliminary effectiveness of the SGM BH-Works Program compared to a historical control group. This quasi-experimental design will test the relationships between targets (training impact, partnership development, software usability) and outcomes (successful referral, program satisfaction, caregiver involvement, suicide identification). The proposed research responds to the growing national need to identify and refer vulnerable youth at risk for suicide. Timeline: Control group data will be collected from month 5-18, while the investigators do manual and program adaptation (Aim #1). In months 19-24, the investigators will conduct the Zero Suicide evaluation, conduct trainings with LGBTQ organizations and BH site staff, pilot the program for a month, and gather initial feedback (focus groups) on the program to make final adaptations to the manual. Training consultations with staff will proceed bi-monthly through month 41 (Aim #2). In months 25-41, the investigators will run the program and collect satisfaction (consumers), feasibility, acceptability, and preliminary effectiveness data (Aim #3). In months 42-44, the investigators will do closing focus groups with staff, administrators, caregivers, and patients at all sites. Following this, the investigators will write up manuscripts and an R01 to test the SGM BH-Works program on a larger scale. Research aims for this three-year project will be completed with the addition of a fourth no cost extension year. Procedure: Procedures are broken down by aim. Aim #1: Engage LGBTQ organization staff and partnering behavioral health providers. This aim focuses on: a) engaging a stakeholder advisory group, and b) initiating the implementation strategy. Administrative stakeholder participants have already agreed to participate in this research and serve as Co-Is on the project. LGBTQ staff/behavioral health provider participants will be recruited by leadership to participate in this project. The investigators expect to include 4-8 staff/providers/administrators at each site. Consenting processes will occur immediately before initial evaluation activities and the first trainings begins. Each agency reports having at least 4 to 6 intake workers and all will be trained in the program. These staff members will complete assessments at the beginning of the study and then five times over the course of years 2-4. They will participate in a final interview after the one month pilot period and at the end of the study. Engage advisory board and workgroup. A local and national advisory board will serve as project collaborators. This group includes academics, educators, administrators, practicing professionals, and community members who are committed to SGM health (see letters of support and commitment). The principal investigator's partners at will assist in identifying SGM youth and their caregivers to serve on the board. The investigators' collaboration with an organizing body for LGBTQ community centers in the world, will have a central role in steering this project. A smaller workgroup will consist of project investigators, leadership from partnering sites, and an implementation consultant. The advisory board meets every two-three months throughout the project. Aim #2: Adapt and pilot the BH-Works Program for LGBTQ organizations and partnering behavioral health sites. The adapted BH-Works Program will be implemented into LGBTQ organizations' workflow for a one-month open trial. Qualitative and quantitative data will be collected to evaluate initial feasibility and acceptability as well as to explore barriers and facilitators to usability in urban and rural organizations. The manual will undergo revisions. Adaptation process. The investigators will employ Lau's (2006) framework for the cultural adaptations of evidence-based treatments (EBTs). Lau recommends that adaptation of EBTs is necessary when contextual processes (e.g. discrimination, caregiver support, mistrust of health systems) contribute to unique vulnerabilities in specific populations, such as SGM youth (particularly those living in rural contexts where fewer specialized LGBTQ services exist). In this project, the investigators focus on Lau's Enhancing Engagement trajectory of adaptation work. As such, the workgroup will focus on generating BH-Works program adaptations that will increase social validity, a potential target for increasing engagement. The role of the workgroup is essential in the adaptation process. The investigators expect this group will increase the social validity of the program by helping us adapt the screening language to be more affirmative, better manage matters of pronoun use, and address concerns about discrimination in standard operating procedures. Adolescent, emerging adult, and caregiver participation in the pilot process mirrors what is described below in Aim #3. Following the Aim #2 pilot of the BH-Works program, organization staff, patients and caregivers will be invited to participate in a focus group to discuss features of the BH-Works program that they find appealing and unappealing, as well as suggestions for improvement. Responses will be consolidated across group and location type (urban vs. rural). Aim #3: Test the feasibility, acceptability, and preliminary effectiveness of the SGM BH-Works Program compared to a historical control group. This quasi-experimental design will test the relationships between targets (training impact, partnership development, software usability) and outcomes (successful referral, program satisfaction, caregiver involvement, suicide identification). The proposed research responds to the growing national need to identify and refer vulnerable youth at risk for suicide. Treatment as usual data will be extracted as de-identified medical records data at the LGBTQ organizations. Data will be collected on the number of patients who were a) assessed for suicide, b) identified as at risk for suicide, and c) referred for behavioral health services. As part of standard care procedures, staff currently conduct a one week follow up call on any patient referred for services, asking if they attended and about their experience. To facilitate comparison with the intervention group, the investigators will ask LGTBQ staff members to include 4-item Acceptability of Intervention Measure on their experience with a) the referral process, and b) their first behavioral health appointment (see measures section). Treatment as usual data for the control group will be extracted as de-identified records data at the LGBTQ organizations in Years 1 and 2. Data will be collected on the number of patients who were a) assessed for suicide, b) identified as at risk for suicide, and c) referred for behavioral health services. To facilitate comparison with the intervention group, the investigators will ask LGTBQ staff members to include 4-item Acceptability of Intervention Measure (AIM) on their experience with a) the referral process, and b) their first behavioral health appointment. Once the BH-Works program is implemented in Years 2 and 3, adolescent consent will begin at the point of screening. The BH-Works screening tool will be included in the standard of care procedures. However, the principal investigator's Institution Review Board (IRB) has approved a brief consent at the beginning of the screen asking permission to use de-identified screening data for research; 90% of patients agree to participate. At the end of the BHS, participants complete a brief satisfaction measure about their experience with the screening tool. This is included as part of the screening process for ongoing quality improvement (QI) purposes. In investigators' past studies, if patients endorse any level of current suicidal ideation, a consent form is automatically presented at the end of the screening asking permission to follow up with the adolescent in one week and one month to see if services were recommended and obtained. However, in this study, the LGBTQ agencies already do a standard of care follow up call to see if patients went to services. As such, they will ask if the research team can call to follow up with them about participating in an interview about seeking services. Consent for participation in this follow-up assessment will occur in the first part of the meeting. In both PA and VA, youth, ages 14 and older, can seek their own mental health services without parental consent. At partnering LGBTQ organizations, many adolescents do not want their parents involved. These youth can still participate in this project without involving their parent, even though caregiver engagement will be encouraged. Caregivers will be recruited in a similar fashion as adolescent patients. After receiving family engagement training, it is expected that staff will be able to engage approximately 50-60% patients' caregivers in the referral process. If caregivers have been engaged, staff will ask if the research team can contact them about participating in a follow up research assessment about seeking help. As with patient participants, caregivers will be contacted to set up a 60-minute meeting a week after their child was screened and referred. Caregivers will provide consent at the beginning of the meeting. The investigators will not exclude caregivers from participation if their adolescent chooses not to participate (and vice versa). It is likely these individual participants can offer important perspectives on screening and referral processes. Hypotheses and Analytic Approach: For Hypothesis #1: All stakeholder partners will approve the adapted manual. This hypothesis will be tested by evaluating the number of stakeholders who approved the manual on the advisory board. For Hypothesis #2: LGBTQ staff and behavioral health providers will report increased preparedness, likelihood of skill use, and self-efficacy; Hypothesis #3a: LGBTQ staff will report increased confidence referring SGM patients to partnering behavioral health providers; Hypothesis #3b: Behavioral health providers will report increased confidence working with SGM patient referrals; Hypothesis #4: LGBTQ staff and behavioral health providers will report increased software usability. To test these four hypotheses, the investigators will use a mixed-effects linear model to estimate the pre-training and post-training changes in staff training impact outcomes. Because of the repeated measurements of the dependent variable, the investigators will treat the multiple observations as nested within individuals. We will also cluster the standard errors to account for any differences between the two study sites. An identical approach will be used for analysis of other outcomes such as comparing the trajectories of the outcomes over the consultation time period (three times points of data). For Hypothesis # 5: The SGM BH-Works program will increase the percentage of identification of SGM adolescents (14-17) and emerging adults (18-19) at-risk for suicide, compared to a historical comparison group; Hypothesis #6: The SGM BH-Works program will increase percentages of successful referrals to behavioral health providers, compared to a historical comparison group, to determine whether participating in the SGM BH-Works program changes these percentages, the investigators will compare the differences in proportions on the outcome variables (proportion identified as at-risk; proportion of successful referrals) based on the phase of the study (i.e., historical control vs. intervention), using a two-sample test of proportions. The investigators will report the z-statistics. The investigators will also compare differences in primary outcomes across sites. If significant differences are found between program sites, then the proportional tests will be conducted separately for both sites. Hypothesis #7: All participant groups will report satisfaction with SGM BH-Works program elements. The investigators will test this by running descriptive statistics on the AIM for consumer groups. Program satisfaction for staff/providers will be measured using AIM at three time points. Change in program satisfaction over time will be estimated using mixed models. Standard errors will be clustered to account for differences between the two study sites. Interview data will be transcribed verbatim and imported into qualitative data analysis software. Interview transcripts and staff/provider free responses will be analyzed by two coders using theoretical thematic analysis procedures as outlined by Braun and Clarke (2006). The coding will be guided by the primary research question: What are patient and caregiver perceptions of the SGM BH-Works program? Hypothesis #8: The SGM BH-Works program will increase the percent of caregivers involved in their child's referral process, compared to a historical comparison group. To explore this mechanism of change, the investigators will compare the percent of caregivers involved in their child's referral process to a historical comparison group, using the two-sample proportional test.

Official Title
-----------------
The BH-Works Suicide Prevention Program for Sexual and Gender Minority Youth

Conditions
-----------------
Suicide, Engagement, Patient

Intervention / Treatment
-----------------
* Behavioral: The Behavioral Health-Works Suicide Prevention Program for Sexual and Gender Minority Youth


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: youth must be seeking services at the participating LGBTQ organizations. youth must endorse current suicidal ideation on the BHS. Exclusion Criteria: youth lack enough English-language proficiency to complete the BH-Works program youth are not capable of understanding the requirements for study participation

Ages Eligible for Study
-----------------
Minimum Age: 14 Years
Maximum Age: 19 Years

Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
No

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Prevention
Intervention Model: Single Group Assignment
Interventional Model Description: Behavioral health-Works (BH-Works) is a systems-level, multicomponent suicide prevention program designed for youth populations. The program includes two primary components, comprehensive web-based behavioral health screening (BHS), and an electronic referral system. This project will adapt and import this program into lesbian, gay, bisexual, transgender, and queer (LGBTQ) organizations. This project builds upon robust partnerships with two diverse LGBTQ organizations in Philadelphia, Pennsylvania and in rural Roanoke, Virginia and their behavioral health partners.
Masking: None (Open Label)


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Experimental: SGM BH-Works Implementation<br>For this phase of the study, the adapted version of the BH-works program (SGM BH-Works) will be implemented into LGBTQ+ Community Organizations. The BH-Works program offers screening, training, and referral coordination. | Behavioral: The Behavioral Health-Works Suicide Prevention Program for Sexual and Gender Minority Youth<br>* BH-Works is a web-based, comprehensive program for suicide prevention. The BH-Works program is a systems-level intervention that provides tools and resources to make organization adoption more feasible.<br>* Other names: SGM BH-Works;|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Successful Referral (Youth attends a first behavioral health session with a behavioral health provider at the recommendation of LGBTQ organization staff) | Youth and medical records indicate that the youth has attended a first behavioral health appointment at the behavioral health site that LGBTQ organization staff referred them to. This data will be recorded dichotomously (no= 0; yes=1). | Self and staff report (medical records) collected within one week to one month after participant enrolls in study and completes the behavioral health screen. | 
| Acceptability of Intervention Measure (AIM) | Using this 4-item measure, youth will report on their perceptions of the acceptability (satisfaction) for the screening and referral process occurring with LGBTQ community staff as well as for their first appointment with the behavioral health provider to whom they were referred. | Self report collected within one week to one month after participant enrolls in study and completes the behavioral health screen. | 
| Caregiver Involvement (Caregiver was observed by LGBTQ organization staff as being involved in the screening and referral process with their youth) | LGBTQ organization staff will indicate whether a caregiver has been involved in the screening and referral process taking place at LGBTQ organizations. This data will be recorded dichotomously (no= 0; yes=1). | Staff report collected within one week to one month after participant enrolls in study and completes the behavioral health screen. | 
| Behavioral Health Screen Suicide Subscale | The suicide subscale of the behavioral health screen consists of four items asking youth about suicide ideation and behavior. Item responses followed a three-point scale: no reported symptoms, moderate symptoms, or severe symptoms. Higher scores indicate higher risk for suicide. | Collected at time of study enrollment. | 

Secondary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Gatekeeper Behavior Scale | Impact of training on staff/providers will be measured using the investigator's modified version of the Gatekeeper Behavior Scale. This scale was developed and validated on a sample of over 8,000 educators. It consists of 11 items making up three subscales. The investigators have created version for pre-training, post-training and follow-up evaluations for the safety planning and family engagement elements of the suicide risk management training curriculum. The investigators have used these adapted versions extensively with over 1,000 providers from over 10 trainings. The adapted scale has demonstrated strong psychometrics. For this proposed study, the investigators will also adapt this scale to measure the impact the affirmative care training curriculum. Modifications of this tool will be reviewed with our advisory board in Year 1. | Completed pre- and post- training during month 20, and during the experimental phase at months 24, 32, and 40. | 
| Partnership Development (Questionnaire) | Partnership development will be measured by a brief questionnaire (to be developed in collaboration with our advisory board). For LGBTQ staff, the investigators will ask questions about partnerships with behavioral health partners (e.g., to what extent do you feel confident behavioral health partners can work with LGBTQ youth?). | Completed pre- and post- training during month 20, and during the experimental phase at months 24, 32, and 40. | 
| Software Usability | The Software Usability Measurement Interview will be used to measure usability for staff/providers. It has 50 Likert-scale items and takes about 5 minutes. It addresses a standard set of usability factors consisting of: Affect, Control, Helpfulness, Learnability, and Efficiency. The quantitative goal is for each factor to achieve a score of at least 80% of the maximum possible score. Progress over the course of the pilot period meeting these goals will be tracked by the successive usability scores. | Completed pre- and post- training during month 20, and during the experimental phase at months 24, 32, and 40. | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
Sexual and Gender Minority Youth, Suicide Prevention Program, Effectiveness-Implementation Hybrid