Clinical trial NCT06515236: Strategic Treatment and Assessment for Youth (STAY)
Official title: Strategic Treatment Assessment With Youth (STAY): A Measurement-based Care Approach to Promote Treatment Retention Among Racial and Ethnic Minoritized Youth With Depression or Suicide Risk
Study type: INTERVENTIONAL
Overall status: ENROLLING_BY_INVITATION
Phase: NA
Conditions: Depression in Adolescence
Suicide Prevention
Lead sponsor: Yale University
Sponsor class: OTHER
Collaborators: name: Teachers College, Columbia University
name: National Institute of Mental Health (NIMH)
First posted: 2024-07-23
Enrollment: 140
Sex: ALL
Minimum age (years): 12
Standard ages: CHILD
ADULT
OLDER_ADULT
Accepts healthy volunteers: no
Allocation: RANDOMIZED
Intervention model: PARALLEL
Intervention model description: Strategic Treatment Assessment for Youth (STAY) is a theoretically-driven, culturally-tailored measurement-based care (MBC) approach for racial and ethnic minoritized youth with depressive symptoms and/or suicidal thoughts and behaviors.
Primary purpose: TREATMENT

Brief summary: This multi-method, multi-phase pilot trial is designed to explore initial implementation and effectiveness of a theoretically-driven, culturally-tailored, measurement-based care (MBC) approach, Strategic Treatment Assessment for Youth (STAY), for racial and ethnic minoritized (REM) youth with depressive symptoms or suicidal thoughts and behaviors (STB). STAY offers an innovative, culturally-tailored approach to retain REM youth with depressive symptoms with or without STB in mental health treatment. In this R34 study, the investigators will first refine the STAY clinical protocol and implementation plan, develop STAY instrumentation (STAY, Aim 1), and then pilot test the effectiveness of STAY in three community-based mental health clinics (Aim 2). Aim 2 focuses on testing the protocol through a pilot Hybrid Type 2 trial by comparing clinicians randomly assigned to STAY (intervention condition) as compared to those assigned to the MBC As Usual (active comparison condition). Results from this pilot trial will inform optimal study procedures, measures, and site selection for a subsequent, fully-powered Hybrid Type 2 trial to examine STAY implementation, effectiveness, and scalability.

Detailed description: This multi-method, multi-phase pilot trial is designed to explore initial implementation and effectiveness of a theoretically-driven, culturally-tailored, measurement-based care (MBC) approach, Strategic Treatment Assessment for Youth (STAY), for racial and ethnic minoritized (REM) youth with depressive symptoms or suicidal thoughts and behaviors (STB). STAY offers an innovative, culturally-tailored approach to retain REM youth with depressive symptoms with or without STB in mental health treatment. In this R34 study, the investigators will first refine the STAY clinical protocol and implementation plan, develop STAY instrumentation (STAY, Aim 1), and then pilot test the effectiveness of STAY in three community-based mental health clinics (Aim 2). Aim 2 focuses on testing the protocol through a pilot Hybrid Type 2 trial by comparing clinicians randomly assigned to STAY (intervention condition) as compared to those assigned to the MBC As Usual (active comparison condition). Results from this pilot trial will inform optimal study procedures, measures, and site selection for a subsequent, fully-powered Hybrid Type 2 trial to examine STAY implementation, effectiveness, and scalability.

In Aim 1 of the research plan, the investigators will refine the STAY protocol and intervention plan based on the study team's preliminary development of the STAY model, protocol, and implementation plan. The STAY protocol includes a clinician manual (including clinician guide for each STAY component, step-by-step instructions, sample scripts, clinical vignettes) and assessment measures. The STAY implementation plan includes the STAY training, goals and objectives, responsibilities, and timelines. The investigators will employ user-centered design (UCD) methods guided by a Discover, Design, Build, and Test (DDBT) Development process to revise the STAY protocol and implementation plan to ensure STAY is aligned with the needs of deployment contexts and end users. A sample size of N=12 users (i.e., clinicians and clinical administrators) divided into three cohorts of four participants each will be used as per recommendations for user-centered design (UCD) testing for complex interventions. Aligned with best practices, sampling will balance both novice and more experienced users. Participants will interact with the STAY protocol and implementation plan prototypes and engage in usability tests, including cognitive walk-throughs and lab-based user testing. Participants will also compete the 10-item Intervention Usability Scale. Iterative data analysis and prototyping between each cohort will be used to rapidly refine the STAY protocol and implementation plan to optimize usability. Then, the STAY Observational Coding System (SOCS) will be developed to assess its feasibility and scalability for future trials, as well as a 10-item STAY Knowledge, Skills and Attitudes (KSAT) Measure to tap core knowledge domains and implementation mechanisms to be assessed in the Aim 2 trial. At the end of Aim 1, the study team will have refined a STAY protocol and implementation plan. The investigators will also have an observational coding system to assess STAY fidelity and a brief, pragmatic measure to assess knowledge, attitudes and practices of clinicians trained in STAY.

In Aim 2 of the research plan, the investigators will conduct a pilot effectiveness-implementation Hybrid Type 2 trial of STAY vs MBC As Usual. Using a randomized design, 20 clinicians at three public mental health sites will be stratified by site and randomly assigned to STAY (intervention condition) or to MBC As Usual (active control condition). Clinicians randomized to STAY will receive the STAY protocol, and MBC As Usual clinicians will receive standard MBC educational materials used in the study team's other MBC studies. Clinicians in both conditions will attend a one-day training and monthly, 1-hour, post-training consultation calls for 6 months. Clinicians in both conditions will be asked to implement MBC with REM youth and their families who meet eligibility criteria and facilitate study referrals. All clinicians will complete measures at baseline (pre-training) and post-training and post-implementation (6-month post-training follow-up). Half of the clinicians in each condition will be randomly selected to participate in the SOCS. Youth/caregiver dyads will be recruited on a rolling basis due to enrollment patterns at the clinical sites, which is why the investigators are planning 20 months of data collection to achieve 4-month follow-up measures from all youth. Youth and caregivers will complete measures at baseline (within 30 days of their first contact with the site), 2-months (8 weeks) and 4-months (16 weeks) following the baseline measure. Youth measures capture implementation outcomes (e.g., clinicians' cultural humility and cultural comfort), engagement mechanisms (e.g., treatment relevance and acceptability, therapeutic alliance), service outcomes (e.g., attitudinal engagement), and youth symptom outcomes (e.g., depression symptoms, suicidal ideation and behavior). Caregiver measures capture implementation outcomes (e.g., clinicians' cultural comfort), engagement mechanisms (e.g., treatment relevance and acceptability, therapeutic alliance), and service outcomes (e.g., attitudinal engagement). Youth and caregivers will also be asked to consent to secondary data collection by the study team to obtain medical records information for their attendance, discharge reason and fidelity of MBC. Clinician measures capture implementation outcomes (e.g., feasibility, acceptability, appropriateness).

The investigators will assess STAY implementation outcomes of fidelity, feasibility, acceptability, and appropriateness (primary outcomes). Further, the investigators will explore whether STAY, as compared to MBC As Usual, will result in (secondary outcomes): (1) Greater therapeutic alliance and treatment relevance and acceptability (engagement mechanisms); (2) Greater session attendance, attitudinal engagement and treatment completion (service outcomes); and (3) Greater reduction in youth depressive symptoms, suicidal ideation and behavior (treatment outcomes).

Appropriateness of Methods Follow up intervals are informed by typical treatment length and retention measures in prior research. Sample size was determined based on the resources and time frame constraints of 3 years; participating clinicians and families will be recruited in year 1, quarter 4 to finish all data collection in year 2. Also, individual clinician patterns in MBC administration and feedback can feasibly be examined with this small sample size, as has been useful in other implementation pilot studies.

Knowledge gained from this pilot will be used to refine the research strategy for a fully powered, subsequent R01 with optimal procedures. Piloting STAY with procedures that would be exactly mirrored in a future, multi-site R01 provides numerous learning opportunities about the feasibility and parameters of measures, study recruitment and data collection procedures (e.g., collecting youth- and caregiver-reported data over the phone), clinician engagement in consultation calls, record review procedures to track fidelity, missing data patterns, and range and pattern of session attendance for both the STAY and MBC As Usual groups. Also, site variation in this trial is expected to provide information about how different sites handle STAY implementation, which the investigators can further investigate as site-level variation in a fully-powered R01 trial. Outcomes of this trial could inform potential additions to the R01 design, such as 1) enhancements to STAY to more comprehensively address logistical barriers, 2) testing STAY among REM adolescents with conditions other than depression, and/or 3) a fully-powered Type 2 Hybrid trial to detect clinician effects, moderators, and effectiveness outcomes.

Eligibility criteria: \*Note: This study includes data collection from mental health clinicians, youth, and parents/caregivers. The age limits for each participant group vary and are specified below.

Inclusion Criteria for Aim 2 Adolescent client participants:

* Age 12 to 18
* Identify as a racial or ethnic minoritized (i.e., Black or African American; American Indian or Alaska Native; Asian American, Native Hawaiian or Other Pacific Islander; Hispanic or Latinx) youth
* Have at least moderate depression symptoms based on moderate range of a validated depression assessment collected at intake by the participating agency such as the Center for Epidemiological Studies Depression Scale for Children (CES-DC) or Hamilton Depression Index with or without Parent- or adolescent- reported or healthcare provider- documented thoughts and/or behaviors related to self-harm or suicide in the three (3) months prior to first contact with participating agency
* Assigned or triaged to work with one of the clinician participants included in the study (below)
* Made first contact (e.g., initial phone call, attended intake appointment) with a participating agency less than or equal to one (1) month prior to the recruitment date (but ideally within (1) week of intake appointment)
* English- or Spanish-speaking
* Legal parent or guardian available to provide informed consent
* Able to provide informed assent

Inclusion Criteria for Aim 2 Parent/Guardian participants:

* Legal guardian of one of the adolescent client participants included in the study (above)
* English-speaking
* Able to provide informed consent

Inclusion Criteria for Aim 2 Clinician participants:

* Provides community-based mental health treatment to youth
* Holds a professional license or certification in their state to provide mental health treatment OR are supervised by a clinician with professional license or certification in their state to provide mental health treatment
* Are a clinician identified for recruitment because they are part of a participating mental health clinic/site
* Age 18 or older
* English- or Spanish-speaking
* Able to provide informed consent

Individuals who do not meet all inclusion criteria are, by definition, excluded. There are no other exclusion criteria.

Primary outcomes:
measure: Patient Health Questionnaire-9 Item Version
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Columbia-Suicide Severity Rating Scale (C-SSRS)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Session Attendance (Behavioral Treatment Engagement)
timeFrame: 4-Month Follow-Up
measure: Patient Activation Measure (Attitudinal Treatment Engagement)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Treatment Retention (Treatment Completion/Continuation)
timeFrame: 4-Month Follow-Up
measure: Treatment Evaluation Inventory - Short Form
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Working Alliance Inventory - Short Revised
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR) Self-Rated Level 1 Cross-Cutting Symptom Measure Child Age 11-17
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR) Parent/Guardian-Rated Level 1 Cross-Cutting Symptom Measure-Child Age 6-17
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Revised Child Anxiety and Depression Scale (RCADS-25)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Secondary outcomes:
measure: Strategic Treatment Assessment for Youth (STAY) Observational Coding System
timeFrame: 6 Months Post-Training
measure: Strategic Treatment Assessment for Youth (STAY) Feasibility
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Strategic Treatment Assessment for Youth (STAY) Appropriateness
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Strategic Treatment Assessment for Youth (STAY) Acceptability
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Strategic Treatment Assessment for Youth (STAY) Knowledge, Skills and Attitudes
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Current Assessment Practice Evaluation (CAPE)
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Attitudes toward Standardized Assessment Scale - Monitoring and Feedback
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Multicultural Counseling Knowledge and Awareness Scale
timeFrame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
measure: Cultural Humility Scale (CHS)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Implementation Climate Scale (ICS)
timeFrame: Baseline, 6 Months Post-Training
measure: Treatment Outcome Expectations Scale (TOES)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Treatment Process Expectations Index (TPEI)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Therapist Cultural Comfort Scale (TCCS)
timeFrame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
measure: Intervention Usability Scale (IUS)
timeFrame: 6 Months Post-Training
Other outcomes:
Locations:
city: New Haven
country: United States
facility: Yale University
state: Connecticut
Study officials:
affiliation: Yale University
name: Elizabeth Connors, PhD
role: PRINCIPAL_INVESTIGATOR
References:
citation: Ingoldsby EM. Review of Interventions to Improve Family Engagement and Retention in Parent and Child Mental Health Programs. J Child Fam Stud. 2010 Oct 1;19(5):629-645. doi: 10.1007/s10826-009-9350-2.
citation: Lyon AR, Dopp AR, Brewer SK, Kientz JA, Munson SA. Designing the Future of Children's Mental Health Services. Adm Policy Ment Health. 2020 Sep;47(5):735-751. doi: 10.1007/s10488-020-01038-x.
citation: Lyon AR, Koerner K. User-Centered Design for Psychosocial Intervention Development and Implementation. Clin Psychol (New York). 2016 Jun;23(2):180-200. doi: 10.1111/cpsp.12154. Epub 2016 Jun 17.
citation: Lyon AR, Munson SA, Renn BN, Atkins DC, Pullmann MD, Friedman E, Arean PA. Use of Human-Centered Design to Improve Implementation of Evidence-Based Psychotherapies in Low-Resource Communities: Protocol for Studies Applying a Framework to Assess Usability . JMIR Res Protoc. 2019 Oct 9;8(10):e14990. doi: 10.2196/14990.
citation: Turner AM, Reeder B, Ramey J. Scenarios, personas and user stories: user-centered evidence-based design representations of communicable disease investigations. J Biomed Inform. 2013 Aug;46(4):575-84. doi: 10.1016/j.jbi.2013.04.006. Epub 2013 Apr 22.
citation: Yeager DS, Romero C, Paunesku D, Hulleman CS, Schneider B, Hinojosa C, Lee HY, O'Brien J, Flint K, Roberts A, Trott J, Greene D, Walton GM, Dweck CS. Using Design Thinking to Improve Psychological Interventions: The Case of the Growth Mindset During the Transition to High School. J Educ Psychol. 2016 Apr;108(3):374-391. doi: 10.1037/edu0000098.
citation: Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606-13. doi: 10.1046/j.1525-1497.2001.016009606.x.
citation: Bauer AM, Baldwin SA, Anguera JA, Arean PA, Atkins DC. Comparing Approaches to Mobile Depression Assessment for Measurement-Based Care: Prospective Study. J Med Internet Res. 2018 Jun 19;20(6):e10001. doi: 10.2196/10001.
citation: Fortney JC, Unutzer J, Wrenn G, Pyne JM, Smith GR, Schoenbaum M, Harbin HT. A Tipping Point for Measurement-Based Care. Focus (Am Psychiatr Publ). 2018 Jul;16(3):341-350. doi: 10.1176/appi.focus.16303. Epub 2018 Jul 16.
citation: Guo T, Xiang YT, Xiao L, Hu CQ, Chiu HF, Ungvari GS, Correll CU, Lai KY, Feng L, Geng Y, Feng Y, Wang G. Measurement-Based Care Versus Standard Care for Major Depression: A Randomized Controlled Trial With Blind Raters. Am J Psychiatry. 2015 Oct;172(10):1004-13. doi: 10.1176/appi.ajp.2015.14050652. Epub 2015 Aug 28.
citation: Glazer K, Rootes-Murdy K, Van Wert M, Mondimore F, Zandi P. The utility of PHQ-9 and CGI-S in measurement-based care for predicting suicidal ideation and behaviors. J Affect Disord. 2020 Apr 1;266:766-771. doi: 10.1016/j.jad.2018.05.054. Epub 2018 Jun 15.
citation: Richardson TM, He H, Podgorski C, Tu X, Conwell Y. Screening depression aging services clients. Am J Geriatr Psychiatry. 2010 Dec;18(12):1116-23. doi: 10.1097/JGP.0b013e3181dd1c26.
citation: Gipson PY, Agarwala P, Opperman KJ, Horwitz A, King CA. Columbia-suicide severity rating scale: predictive validity with adolescent psychiatric emergency patients. Pediatr Emerg Care. 2015 Feb;31(2):88-94. doi: 10.1097/PEC.0000000000000225.
citation: Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin GA, Greenhill L, Shen S, Mann JJ. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011 Dec;168(12):1266-77. doi: 10.1176/appi.ajp.2011.10111704.
citation: Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004 Aug;39(4 Pt 1):1005-26. doi: 10.1111/j.1475-6773.2004.00269.x.
citation: Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and testing of a short form of the patient activation measure. Health Serv Res. 2005 Dec;40(6 Pt 1):1918-30. doi: 10.1111/j.1475-6773.2005.00438.x.
citation: Miller GE, Prinz RJ. Engagement of families in treatment for childhood conduct problems. Behavior Therapy. 2003; 34(4): 517-534.
citation: Kazdin AE. Acceptability of alternative treatments for deviant child behavior. J Appl Behav Anal. 1980 Summer;13(2):259-73. doi: 10.1901/jaba.1980.13-259.
citation: Spirrison, CL, Noland, KA, Savoie, LB. Factor structure of the Treatment Evaluation Inventory: Implications for measurement of treatment acceptability. J Psychopathol Behav Assess. 1992; 14: 65-79.
citation: Carter, SL. Review of Recent Treatment Acceptability Research. Education and Training in Developmental Disabilities. 2007; 42(3): 301-316.
citation: Hatcher, RL, Gillaspy, JA. Development and validation of a revised short version of the working alliance inventory. Psychotherapy Research. 2006; 16(1): 12-25.
citation: Munder T, Wilmers F, Leonhart R, Linster HW, Barth J. Working Alliance Inventory-Short Revised (WAI-SR): psychometric properties in outpatients and inpatients. Clin Psychol Psychother. 2010 May-Jun;17(3):231-9. doi: 10.1002/cpp.658.
citation: Ebesutani C, Chorpita BF, Higa-McMillan CK, Nakamura BJ, Regan J, Lynch RE. A psychometric analysis of the Revised Child Anxiety and Depression Scales--parent version in a school sample. J Abnorm Child Psychol. 2011 Feb;39(2):173-85. doi: 10.1007/s10802-010-9460-8.
citation: Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017 Aug 29;12(1):108. doi: 10.1186/s13012-017-0635-3.
citation: Stumpf RE, Higa-McMillan CK, Chorpita BF. Implementation of evidence-based services for youth: assessing provider knowledge. Behav Modif. 2009 Jan;33(1):48-65. doi: 10.1177/0145445508322625. Epub 2008 Aug 22.
citation: Lawson GM, Moore TM, Okamura KH, Becker-Haimes EM, Beidas RS. Knowledge of Evidence-Based Services Questionnaire: Development and Validation of a Short Form. Adm Policy Ment Health. 2020 Jul;47(4):581-596. doi: 10.1007/s10488-020-01020-7.
citation: Jensen-Doss A, Haimes EMB, Smith AM, Lyon AR, Lewis CC, Stanick CF, Hawley KM. Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice. Adm Policy Ment Health. 2018 Jan;45(1):48-61. doi: 10.1007/s10488-016-0763-0.
citation: Lyon AR, Pullmann MD, Whitaker K, Ludwig K, Wasse JK, McCauley E. A Digital Feedback System to Support Implementation of Measurement-Based Care by School-Based Mental Health Clinicians. J Clin Child Adolesc Psychol. 2019;48(sup1):S168-S179. doi: 10.1080/15374416.2017.1280808. Epub 2017 Mar 2.
citation: Connors EH, Douglas S, Jensen-Doss A, Landes SJ, Lewis CC, McLeod BD, Stanick C, Lyon AR. What Gets Measured Gets Done: How Mental Health Agencies can Leverage Measurement-Based Care for Better Patient Care, Clinician Supports, and Organizational Goals. Adm Policy Ment Health. 2021 Mar;48(2):250-265. doi: 10.1007/s10488-020-01063-w.
citation: Hook JN, Davis DE, Owen J, Worthington EL, Utsey SO. Cultural humility: measuring openness to culturally diverse clients. J Couns Psychol. 2013 Jul;60(3):353-366. doi: 10.1037/a0032595. Epub 2013 May 6.
citation: Ponterotto, JG, Potere, JC. The Multicultural Counseling Knowledge and Awareness Scale (MCKAS): Validity, Reliability, and User Guidelines. In D. B. Pope-Davis, H. L. K. Coleman, W. M. Liu, & R. L. Toporek (Eds.), Handbook of multicultural competencies: In counseling & psychology. Sage Publications, Inc. 2003: 137-153.
citation: Constantine, MG, Ladany, N. Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology. 2000; 47(2): 155-164.
citation: Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implement Sci. 2014 Oct 23;9:157. doi: 10.1186/s13012-014-0157-1.
citation: Dew-Reeves SE, Athay MM. Validation and use of the youth and caregiver Treatment Outcome Expectations Scale (TOES) to assess the relationships between expectations, pretreatment characteristics, and outcomes. Adm Policy Ment Health. 2012 Mar;39(1-2):90-103. doi: 10.1007/s10488-012-0406-z.
citation: Riemer M, Athay MM, Bickman L, Breda C, Kelley SD, Vides de Andrade AR. The Peabody Treatment Progress Battery: history and methods for developing a comprehensive measurement battery for youth mental health. Adm Policy Ment Health. 2012 Mar;39(1-2):3-12. doi: 10.1007/s10488-012-0404-1.
citation: Perez-Rojas AE, Bartholomew TT, Lockard AJ, Gonzalez JM. Development and initial validation of the Therapist Cultural Comfort Scale. J Couns Psychol. 2019 Oct;66(5):534-549. doi: 10.1037/cou0000344. Epub 2019 Apr 18.
citation: Bartholomew TT, Perez-Rojas AE, Lockard AJ, Joy EE, Robbins KA, Kang E, Maldonado-Aguiniga S. Therapists' cultural comfort and clients' distress: An initial exploration. Psychotherapy (Chic). 2021 Jun;58(2):275-281. doi: 10.1037/pst0000331. Epub 2020 Nov 19.
citation: Lyon AR, Pullmann MD, Jacobson J, Osterhage K, Achkar MA, Renn BN, Munson SA, Arean PA. Assessing the Usability of Complex Psychosocial Interventions: The Intervention Usability Scale. Implement Res Pract. 2021 Jan-Dec;2:2633489520987828. doi: 10.1177/2633489520987828. Epub 2021 Feb 8.