IMPACT Center Signature Project 1, "Novel Methods for Implementing Measurement-Based Care for Youth in Low-resource Environments", is Embedded Within IMPACT Center P50 Entitled: Optimizing Evidence-Based Practice Implementation for Clinical Impact: the IMPACT Center.
Phase: NA
Conditions: Mental Health
Sponsor: Kaiser Permanente

 
 
 
 
 
 
 
 
 
 
 
 
 
 
Title: Novel Methods for Implementing Measurement-Based Care for Youth in Low-resource 
Environments 
NCT number: NCT05644756 
Date of this protocol: 4/4/2024 
 
 

KPiIRB Protocol Submission Form 
 
1. Protocol  
 
 
Protocol Title: Novel methods for Implementing Measurement-Based care for youth in Low-
resource Environments (NIMBLE) 
Principal Investigator:  Rita Mangione-Smith, MD, MPH 
Scientific Lead/Co-Investigator: Ruben Martinez, PhD, Brown University 
Version Date:    4/4/2024 
Funding Source/ Type: NIMH 
Grant Number (if applicable): 5P50MH126219-02 
This study is one of the Signature Projects embedded within the IMPACT Center P50 entitled: 
Optimizing Evidence-Based Practice Implementation for Clinical Impact: the IMPACT Center 
 
2. Objectives 
 
a. Purpose 
Measurement Based Care (MBC) is the systematic, routine evaluation of symptoms to inform care 
decisions. Especially in youth, MBC increases the rate of symptom improvement, detects clients 
who would otherwise deteriorate, and alerts therapists to non-responders. Despite its 
demonstrated utility, MBC is rarely implemented with fidelity; fewer than 15% of therapists report 
using MBC per recommendations. Previous efforts to support MBC implementation have yielded 
suboptimal outcomes because Community Mental Health Clinic (CMHC) leaders are challenged to 
identify and prioritize barriers and select strategies to overcome them. New methods are needed to 
optimize MBC implementation and treatment quality to improve youth mental health outcomes in 
community settings. The purpose of this project (referred to as NIMBLE) is to pilot test and refine 
IMPACT Center methods in six diverse CMHC clinics across Washington state (three of six serve 
primarily Latinx populations) to optimize MBC implementation.   
 
b. Primary Objective/Aim 
 
Aim 1: Co-create plans for optimizing Measurement Based Care (MBC) 
implementation for youth with comorbidities. 
We will test IMPACT methods to identify and prioritize barriers of MBC implementation, and to select 
and match strategies to prioritized barriers.  IMPACT methods include (a) rapid evidence reviews 
to uncover empirical data regarding MBC barriers (therapists only); (b) rapid ethnography to 
identify and describe local barriers and situate them in specific organizational, social, and task 
contexts (therapists & clinic staff only); and (c) design kits (e.g., kits with disposable camera, 
journals) to allow therapists and youth to capture and reflect on aspects of their context that are 
salient for MBC. These three methods will result in a list of barriers that will be rated by therapists 
and other staff who agree to join a clinic specific implementation team. Subsequently, we will use 
facilitated group processes (barrier prioritization and causal pathway diagramming) to develop 
plans to match strategies to the top three barriers and clarify what needs to be in place to optimize 

MBC implementation. We will check back with clinic-specific implementation teams biweekly to 
track strategy deployment for six months; after 3 months if MBC hasn’t significantly improved, 
clinic teams will move onto deploying their other strategies.  
 
c. Secondary Objectives/Aims 
AIM 2: Compare MBC fidelity post IMPACT Center methods deployment versus 
historical controls. 
We will evaluate at the clinic level whether IMPACT methods were effective in optimizing MBC 
fidelity. To do this, we will employ two methods: 1) an anonymous self-report survey pre-
implementation, 3 months and again 6 months post implementation strategies, and 2) a historical 
control design comparing MBC fidelity pre- and post- IMPACT methods (e.g., design kits). Pre-
IMPACT method data will be captured from a pre-existing, routinely gathered, de-identified 
database from the Washington State CBT+ initiative (further described in Section 6 – Setting). The 
database will represent MBC fidelity 2 years prior to IMPACT center methods. For post-IMPACT 
method data, at least 5 therapists at each clinic will be asked to report MBC fidelity data to the 
Washington State CBT+ Initiative Dashboard for up to 5 cases. Therapists at each clinic are familiar 
with the Dashboard, as its use is part of their routine training. Data recorded related to MBC fidelity 
consists of therapist’s use of the three main components of MBC – (1) Collect, (2) Share, and (3) 
Act (described further in Scientific Background and Study Procedure sections). 
3. Background Information and Rationale 
 
a. Scientific Background 
MBC is effective in improving mental health outcomes for youth with comorbidities but only 
if implemented with fidelity. MBC is the systematic evaluation of youth outcomes (like 
symptoms) before or during each clinical encounter to inform care. When relying only on 
clinical judgement, therapists detect few cases of client deterioration (only 21.4%). When 
implemented with fidelity, MBC consistently yields medium effect-sized improvements 
over usual care and can be especially beneficial for guiding treatment focus with youth who 
have comorbidities. For youth in community mental health clinics, where comorbidities are 
the rule rather than exception, MBC holds powerful promise for increasing the quality of 
any psychological intervention by facilitating communication and focusing treatment. 
We operationalize MBC’s fidelity elements in accordance with Barber & Resnick’s 
transtheoretical model called Collect, Share, Act. Collect refers to the routine 
administration of symptom measures before therapy sessions. Share refers to therapist 
and client score review. Act refers to collaborative reevaluation of the treatment plan. Each 
element is necessary to maximizing the promise and clinical utility of MBC.   
Bickman and colleagues studied MBC implementation at two clinics. Higher 
implementation with fidelity corresponded to better youth outcomes at one clinic, despite 
facing more discrete barriers. The clinic with poor outcomes did not address key barriers 

(e.g., leadership buy-in), suggesting that lack of barrier prioritization undermined MBC 
fidelity. Similarly, Lewis found that clinic-led tailoring improved one of three elements of 
MBC fidelity (measure administration). However, score review and reevaluation of the 
treatment plan remained suboptimal. Without rigorous methods to support the practice 
community in optimizing all MBC elements, or treatment quality will remain low, and youth 
will not experience mental health benefits.  
This study is the first signature project of the IMPACT center in partnership with the 
Washington State CBT+ initiative. The current project uses new methods, including rapid 
evidence synthesis, rapid ethnography, and design kits to identify locally contextualized 
barriers to clinic-wide use of MBC, as well as causal pathway diagramming (CPD) a method 
for selecting strategies and matching them to barriers that will help clinics optimize their 
use of MBC (Aim 1).) We will investigate whether IMPACT methods improve MBC fidelity 
through a historical control design (Aim 2). 
b. Preliminary Data 
None. 
4. Study Design 
 
The study design is a prospective implementation study that includes a retrospective database 
review arm. We will pilot IMPACT’s methods in six clinics in partnership with the CBT+ initiative. We 
will approach clinics that are diverse in patient makeup (e.g., approaching clinics that see primarily 
Latinx youth), rural/urban status, size, and length of time implementing EBPs. There will be 2 
cohorts of 2 clinics each, staggered by 6 months. Total duration of the study for participating clinics 
is 24 months. 

The study design is represented in picture form in Figure 1 below. Figure 1 is broken down by aim 
and site visit. In the diagram, Aim 1, activity 1 is coded as A1a, Aim 1, activity 2 is coded as A1b, and 
so forth.  
 
Figure 1 – Map of the NIMBLE study 
 
Note on in-person/hybrid/virtual study activities: All study activities may be done in-person or 
virtually, depending on clinic preference and staff availability. For any virtual activity, we will use 
the KP Microsoft Teams platform or the clinic’s preferred HIPAA-compliant teleconferencing 
platform. 
 
Aim 1 (A1): Co-create Plans for Optimizing MBC Implementation for Youth with 
Comorbidities 
To identify barriers to using MBC with fidelity, IMPACT will pilot 3 new, complementary methods: 
rapid evidence reviews (RER), rapid ethnographic assessment (REA), and design kits and further 
pilot causal pathway diagramming.  
 
A1a: MBC training (Site visit 1) 
The title of the training is: Collect, Share, Act – making measurement-based care work for 
your youth mental health practice. 
Description of the training: Measurement-based care (MBC) is a promising addition to any 
psychological treatment for any mental health problem. This one-time training offers an 

overview of MBC: what it is, why it works, the primary components, typical challenges, and 
strategies for use with minoritized youth with complex psychosocial presentations.  
A1b: Build engagement and identify context-specific barriers through rapid 
ethnographic assessment (Site visit 1) 
Understanding implementation context is critical to the success of implementation efforts, 
as it allows you to better match strategies to barriers. Several methods exist to understand 
the implementation context, but their use brings up a myriad of conceptual and 
philosophical difficulties. In this study, we use REA methods over 2-day virtual or in-person 
site visits at each clinic.  
A1c: Identify barriers using design kits (Site visit 1) 
To complement REA and collect real-time data from therapists and youth, at the end of site 
visits, we will leave (or mail out) design probe materials including disposable cameras, 
journals, pens and prompts to use to take pictures, make diary entries, draw maps, and 
record images.  
A1d: Barrier prioritization (Site visit 2) 
To prioritize barriers to using MBC with fidelity, IMPACT will pilot a new group-facilitated 
activity with our clinic staff participants. The activity will occur in-person (Site Visit 2) but 
could be delivered virtually if needed. Identified barriers (50+ expected) will be prioritized 
using a participant-engaged method. We will engage 5-8 therapists and/or staff in the 
prioritization activity for a 2-hour facilitated group activity.  
A1e: Co-generate implementation plans/Causal Pathway Diagramming (Site visit 3) 
Following the prioritization of barriers, causal pathways diagrams (CPDs) will be used to 
compare and select implementation strategies best suited to address the barriers of using 
MBC with fidelity. The same implementation team members will be engaged in this 2-hour 
in-person or virtual activity as in Site Visit 2. 
A1f: Focus groups to improve IMPACT methods (Site visit 4) 
We will engage our partners in a semi-structured focus group to capture their experience 
engaging in all IMPACT Center methods in the 3 site visits to inform further toolkit 
refinement. This will occur in person or virtually. 
Aim 2 (A2): Compare MBC fidelity post IMPACT Center methods deployment versus 
historical controls (no site visit) 

The purpose of IMPACT methods is to improve clinician’s fidelity using MBC. Fidelity will be 
conceptualized as therapist’s use of the three main components of MBC – (1) Collect, (2) 
Share, and (3) Act. We will assess fidelity in two ways. First, we will use a brief anonymous 
self-report survey before implementing strategies and 6 months after. Previously, we had 
planned to use data from the Washington State CBT+ Initiative Dashboard, but we have had 
difficulty recruiting clinicians to engage in this task. For our second method of fidelity 
measurement we will ask clinics to send a dataset from their electronic health record for at 
least 6 months before IMPACT methods were deployed, and to 6-months after the active 
implementation period. The dataset will not contain any PHI. We will compare historical 
data at 6- and -18 months post site visits. . 
5. Study Population 
 
a. Number of Subjects/Participants 
Aim 1: Co-create plans for optimizing Measurement Based Care (MBC) implementation for 
youth with comorbidities.  
Therapists/Clinic Staff: 
Rapid Ethnographic Assessment Interviews – 120 participants  
Design Kit – 32 participants (subset of the 120 interviewed)  
Barrier Prioritization – 24 participants (subset of 120 interviewed) 
Causal Pathway Diagramming – 20 participants (subset of 120 interviewed) 
Focus Groups to improve IMPACT methods– 20 participants (subset of 120 
interviewed) 
Youth: 
Design Kit - 32 participants 
AIM 2: Compare MBC fidelity post IMPACT Center methods deployment versus historical 
controls (not associated with a site visit) 
NIMBLE will compare data at a clinic level prior to MBC training and at 6- and 18-months post site 
visit 1.  
 
Our approach is to evaluate, at a clinic level, use of MBC before and after IMPACT method 
deployment. Specifically, we focus on whether therapists engaged in the main components 
of MBC: (1) Collect, or how often measures were provided to patients (frequency of 
administration), (2) Share, or how often therapists discuss outcome measures with their 
patients, and (3) Act, or whether therapists and patients used outcome measure data to 
change their direction in therapy (reevaluation using data). 

We will use two different methods to assess use of MBC. The first is a short anonymous self-report 
survey filled out before implementing new strategies to address barriers to MBC use, and a second 
time 6 months after implementation. The second method is to ask for 5 therapists to enter data 
from 5 cases related to MBC use into the Washington State CBT+ Dashboard. Data will be 
aggregated at the clinic level, as we are investigating whether IMPACT methods impact clinic-wide 
changes. We are not studying whether any individual person's behavior changes as a result of 
IMPACT methods. 
 
Clinic staff/Therapists (N=30) across 4 CMHCs in Washington state. 
Inclusion: Must be aged 21+ and be working at a CMHC that is participating in the WA State CBT 
(Cognitive Behavioral Therapy) + Initiative. 
Exclusion: Must be willing to participate in study activities 
 
Youth participants: NA, there are no Youth Participants in this aim 
b. Inclusion and Exclusion Criteria 
Aim 1: Co-create plans for optimizing Measurement Based Care (MBC) 
implementation for youth with comorbidities.  
Clinic staff/Therapists (N=78) across 4 CMHCs in Washington state. 
Inclusion: Must be aged 21+ and be working at a CMHC that is participating in the WA State CBT 
(Cognitive Behavioral Therapy) + Initiative. 
Exclusion: Must be willing to participate in study activities. 
 
Youth (N=22) 
Inclusion: aged 13-17 years, receiving treatment at CMHC participating in the WA State CBT 
(Cognitive Behavioral Therapy) + Initiative.  
Exclusions: youth who are suicidal, psychotic, not cognitively able to safely participate in MBC, and 
not able to read and write in English. 
 
Aim 2: Compare MBC fidelity post IMPACT Center methods deployment versus historical 
controls. 
Clinic staff/Therapists (N=30; subset of 78 clinic staff/therapists in Aim 1) across 4 CMHCs in 
Washington state. 
Inclusion: Must be aged 21+ and be working at a CMHC that is participating in the WA State CBT 
(Cognitive Behavioral Therapy) + Initiative. 
Exclusion: Must be willing to participate in study activities 
 
Youth: NA, there are no Youth Participants in this aim 
c. Vulnerable Populations 
 
The only vulnerable population we will include is Children 13-17 years of age since the study 
focuses on improving mental health outcomes for this population. Children must have the ability to 
read and write in English, given the nature of the research methods.  
 
d. Potentially Vulnerable Populations 
N/A 

6. Setting 
 
This study was originally being done completely through KPWHRI. Dr. Martinez, the scientific lead 
and site PI changed institutions to Brown University and continued his role as scientific lead. At 
KPWHRI, Rita Mangione-Smith is the new site PI. All study activities will be done at KPWHRI and 
Brown University. Dr. Martinez will be engaged in all study activities (recruitment, providing study 
information sheet, interviewing, communicating with clinics, etc.). An IAA is under preparation 
between Brown University and KPiIRB.   
 
The research will be conducted in Community Mental Health Clinics in Richland and Yakima, 
Washington, as well as other clinics in the Puget Sound area of Washington state, with all clinics 
being part of the Washington State CBT+ Initiative. The Washington State CBT+ initiative (called 
CBT+) provides an ideal backdrop for conducting this work. CBT+ is an academic-community 
partnership funded by the state’s Division of Behavioral Health and Recovery. Over nearly 15 years, 
the initiative has provided clinician training and some organizational support for delivering EBPs for 
children and adolescents in public mental health, with high practice community collaboration and 
leadership.  
7. Recruitment & Consent 
 
a. Recruitment & Screening 
 
Recruitment will happen in Washington State only, but Dr. Martinez will be involved with 
recruitment of clinics and participants while at Brown University. All study activities will be 
done on a KP-provided computer. See section 9f for more information. 
Aim 1: Co-create plans for optimizing Measurement Based Care (MBC) implementation 
for youth with comorbidities.  
Clinic Staff/Therapists:  
We will host a 2-hour kick-off meeting (in person or virtual) at all 4 clinics where study activities will 
be 
described, and clinic staff/therapists will have an opportunity to ask questions. We will provide 
NIMBLE 
Study Team contact information for any person who would like to ask questions individually.  
 
A1b: Rapid Ethnographic Assessment (REA) observations and interviews 
At the end of the workshop the co-investigators will describe the other components in the study 
and 
give or email clinic staff and therapists an information sheet detailing the opportunity to volunteer 
as 
participants in interviews and observations. Clinic staff/therapists who do not want to participate 
will 
not be included in the REA observation and will not be approached for a 1-time interview.  
 
A1c: Design Kit activity 

Therapists: We will provide a physical or emailed clinic staff information sheet detailing the design  
kit activity and ask for volunteer therapists at each clinic to participate. 
 
Youth identification: Front desk at clinics will share a recruitment card attached with youth clients 
(aged 
13-17) being seen at each clinic, and youth will call the study team if interested in participating. The 
recruitment card will contain study team contact information so that interested youth can call the 
study team to get more information and have a chance to ask questions about what study activities 
include. Once all questions have been answered and the participant has decided to participate, 
the assent process will be conducted over the phone with NIMBLE study investigator Dr. Ruben 
Martinez or NIMBLE study project manager Carolyn Bain. If the youth does not have the Youth 
Design Kit information sheet with them, we would ask for their email address and email a message 
with a vague subject title to them so we could look at it in real time as we discuss the design kit 
activity. Since youth will call us to participate, we do not feel there is undue influence or coercion 
from receiving the recruitment card at the therapist office. The recruitment card and information 
sheet can be taken home to share with parents if the youth wish to share their participation with 
their parents. 
 
A1d: Barrier Prioritization, A1e: Causal Pathway Diagramming, and A1f: Focus groups 
The clinic staff information sheet explains each of these additional activities. 
At the first site visit we will ask clinic staff to provide their email addresses to us and tell us 
if they may be interested in participating in one, two or all three of these activities. One 
month before each activity, we will send clinic staff a reminder email with the information 
sheet, asking them to contact us if they are willing to participate. We will provide the 
interview guides/agendas to IRB for each activity with a modification prior to carrying out 
these activities.  
 
AIM 2: Compare MBC fidelity post IMPACT Center methods deployment versus historical 
controls 
 
Clinic Staff/Therapists: At the end of each Aim 1 interview with clinic staff, we will ask them 
whether 
they would be willing to participate in data collection efforts for Aim 2. We will provide the clinic 
staff 
information sheet. Therapists will already be familiar with this process as they do this as part of 
standard clinic training.  
 
Youth identification: NA, no youth activities 
 
b. Informed Consent Process  
Consent and Waiver Table  
Population 
Study 
Activity/Activities 
Consent/Waiver Type 
 
 
NIMBLE participants  
Identification Study 
Enrollment 
☐Signed Consent Form 
☒ Waiver of Signed Consent (oral 
consent, information statement) 

☐ Waiver of Consent 
 
 
Clinic Staff and Therapist Consent Process 
The NIMBLE team at KPWHRI and Brown University will begin the consent process on the initial visit 
to each CMHC. The first activity in the visit is a training workshop which focuses on the importance 
of MBC. This workshop is desired by each of the CHMC and leadership at each CHMC have 
deemed this a required workshop for all staff providing clinical services to youth. Of note, even 
though the MBC workshop is mandatory for therapists, the rest of the study activities are not. The 
study team will elicit questions and ensure that all staff and therapists understand what 
participation entails. To minimize the possibility of coercion or undue influence, the study team will 
ask those interested in learning more to remain in the room, while those not interested will be 
asked to leave, at which point the study team will complete the oral consenting process there. Over 
the course of the study, the team will revisit the activities with participants and make sure they still 
are comfortable participating. For in-person site visits to ensure that staff and therapists are 
comfortable with our team’s presence in the clinic during clinic observation activities we will 
implement the following procedures: 
 
1. Announce that the meeting/huddle is being observed at the start of the meeting. Explain 
what type of information will be noted as part of the observation (i.e., workflows, 
relationships, direct anonymized quotes, etc.). Give people the opportunity to “opt-out” at 
the earliest possible time in the meeting/huddle. 
2. Putting a sign on the door or in the room, or both, explaining that this meeting is being 
observed as part of an ethnographic study. Notes are being taken about MBC topics. If you 
would prefer to not be included in the notes, please contact study team member (Abby 
Matson or Carolyn Bain) (will provide photos of team members). This ensures people that 
came in late are aware they are part of a study and know how to opt out. 
3. Remind folks at the end of the meeting that this session was observed, and notes made 
about MBC specific topics, and how to opt out. 
c. Justification: Waiver of Informed Consent  
N/A 
 
d. Justification: Waiver of Signed Informed Consent  
The research involves no more than minimal risk to participants and involves no procedures for 
which written consent is normally required outside of the research context.    
 
For youth, there is no in-person contact planned with the study team; it is only by phone. 
 
e. Limited English Proficiency Participants  
N/A 
 
f. Assent of Children and Parent Permission 
Youth will assent for participation in this research. 
 

Youth clients receiving care at participating CMHCs will receive a recruitment information card and 
information sheet from the front desk of the clinic upon check-in, with a phone number to call if 
they are interested in participating. 
 
We are requesting a waiver of parental consent because per WA State law, youth aged 13+ are 
permitted to access mental health services without parental consent or knowledge. Given this 
legal framework, and the study focus on youth mental health within the Community Mental Health 
Clinics, we will have youth assent process. 
 
g. Adults Unable to Consent/Decisionally Impaired  
N/A 
h. HIPAA Privacy Rule Authorization  
N/A 
i. Justification: Waiver or Alteration of HIPAA Privacy Rule Authorization  
N/A 
8. Study Procedures 
a. Study Design 
Aim 1 (A1): Co-create plans for optimizing Measurement Based Care (MBC) implementation 
for youth with comorbidities.  
Site visit 1 (2 days) 
A1a MBC Training. We will kick off the site visit with a 2-hour orientation/training about 
Measurement Based Care for which therapists can receive CEUs. At the end of the training, 
we will ask them to complete IMPACT’s center-wide 18-item Implementation Climate Scale 
over REDCap (ImplementationClimateScale_NIMBLE REDCap.pdf submitted in this 
package. 
A1b. Rapid Ethnographic Assessment (REA). REA plan was developed in conjunction with two 
REA experts (Gray and Palazzo). REA will be conducted over 2-day site visits (in-person or virtual) at 
each clinic, following the kickoff training, by Martinez (project co-lead), Bain (project manager), 
and research specialist (Matson). For in-person visits, this team will conduct ethnographic 
observations on both days, using primarily unobtrusive techniques on Day 1, documenting 
observations of activities (e.g., clinician documentation), interactions (e.g., client check in), and 
events (e.g., staff meeting) using written and audio-recorded field notes. The team will try to engage 
all clinic staff in ethnographic interviews with questions in 4 categories. Grand tour: Could you 
describe a typical day at this clinic? Mini tour: Could you describe a typical therapy session? Task: 
Could you show me how you use MBC in your work and describe what you are doing? Example: 
Could you give me examples of “good” and "poor" use of elements of MBC? No identifying 
information will be collected, and all staff/therapists will be identified by a STUDY ID. 
We will interview staff for responses to the “example” question described above. We expect 
up to 30 interviews per clinic: 30–45 minutes for therapists, 15 minutes for staff. We will 
synthesize learnings from observations, interviews, and surveys in a debriefing session, 
using notes to generate a list of unique barriers. 
Day 2 (if in person) will be focused observations of therapists shadowed by team members to 
document, track, and note the duration and location of activities, guided by barriers identified 

in the evidence review and Day 1 learnings. We will conduct semi-structured, focused 
interviews virtually or in-person based on Day 1 learnings, with therapists (therapists, 
supervisors, psychiatrists) and staff (up to 30). Day 2 interviews will be longer (15-30 minutes) 
and audio recorded if done in person.  
If study activities are done virtually, there will be no site observation, but the study team will 
conduct interviews. In this case, videos will be video recorded and transcribed using the 
Microsoft Teams platform. 
A1c. Design Kits. To complement REA and collect real-time data from therapists and youth, 
at the end of site visits, we will leave (or mail) design kit materials including disposable 
cameras, journals, pens and prompts to use to describe using MBC by taking pictures, make 
diary entries, draw maps, and record images. We will ask therapists (5-8 per clinic) to 
participate. We will strive to recruit at least one youth from each participating therapist to 
also complete design kits.  
We will recruit 5-8 youth clients to also complete design kits. This is the only youth-related 
activity in the study. Participants (therapists and youth) will have one week to document 
experiences before returning the toolkits to the study team in self-addressed, postage-paid 
stamped envelopes. Within a week of receiving the data, team members will conduct follow-
up 15-minute phone interviews, inviting participants to interpret and comment on their 
responses. Team members will create memos documenting barriers and describing their 
salience, meaning, and importance to the participant. 
A1d. Barrier Prioritization.  
We will engage 5-8 therapists and/or staff in the prioritization activity for a 2-hour facilitated 
group activity. The activity will occur in-person (Site Visit 2) but could be delivered virtually 
if needed. Barriers identified through the rapid evidence review, rapid ethnographic 
assessment, and design kits (50+ expected) will be prioritized using a participant-engaged 
method. We will begin by showing clinic staff/therapist participants a list of identified 
barriers grouped by level (e.g., individual therapist, clinic-level, supervisor-level) and ask 
them to rank their 20 most important barriers. After, we will ask them to rate how 
important, long-lasting, and pervasive those barriers are in their use of MBC. These 
meetings will not be recorded. 
A1e. Causal Pathway Diagramming. For each clinic, we will develop Causal Pathway 
Diagrams CPDs for the top 3 prioritized barriers. The goal is to assess how well each 
implementation strategy is matched to a prioritized barrier, based on its mechanism of 
action, and to clarify the causal chain of events that must take place to achieve MBC 
fidelity. Our research team will develop a CPD for the top barrier, to use as an example, in 
advance of meeting with the same group of 5-8 therapists and staff in each clinic who 
participated in the barrier prioritization activity (Site Visit 2). We will present the CPD, 
introduce clinic staff participants to the 5-step construction process, discuss how the CPD 
aligns with their experiences, and modify it as needed. We will then collaboratively guide 

clinic staff participants in constructing CPDs for the other barriers using the 5-step process 
described below. Plain language questions will be used to guide stakeholders through the 
process of developing CPDs. These meetings will be recorded using the Microsoft Teams 
platform if done virtually. 
A1f. Focus Groups to Improve IMPACT methods. We will engage our partners in a semi-
structured focus group to capture their experience engaging in all IMPACT Center methods 
in the 3 site visits to inform further toolkit refinement. A facilitator not previously engaged in 
any of the activities will conduct a 90-minute focus group to solicit reactions to all aspects 
of IMPACT’s methods, including the steps and personnel requirements of each method, 
and their output (e.g., prioritized barriers; utility of the information captured in the CPDs). 
We will also be collecting implementation outcome measures, Acceptability, 
appropriateness, feasibility using established measures. The focus group data will 
supplement observer notes taken throughout the site visits.  
To gather quantitative data related to IMPACT methods, up to N = 30 workshop participants 
will also be asked to complete Weiner et al.’s (2017) acceptability of intervention measure 
(AIM), feasibility of intervention measure; (FIM), and intervention appropriateness measure 
(IAM). The focus of these measures will be on assessing the acceptability, feasibility, and 
appropriateness of IMPACT center methods (e.g., design kits). These measures are 
uploaded in the IRBNet submission. 
AIM 2 (A2): Compare MBC fidelity post IMPACT Center methods deployment versus historical 
controls. To examine MBC fidelity in Aim 2, we use routine session process data entered by 
therapists as part of their participation in the Washington CBT+ Initiative.  In addition, we will 
request that therapists fill out a short anonymous self-reported survey on outcome use before the 
active implementation period, at 3 months and again at 6 months (REDCap survey: MBC Outcome 
Questionnaire). 
 
Dr. Shannon Dorsey is a co-director of the Washington State CBT+ Initiative and the MPI of 
the IMPACT center. The dashboard with de-identified cases from which we are collecting 
data from is maintained as part of the CBT+ Initiative, and the CBT+ Initiative is the primary 
“laboratory” for IMPACT center methods. When data from CBT+ aligns with IMPACT 
aims, data can be curated by their team and sent to IMPACT NIMBLE co-investigator Ruben 
Martinez, via secure file transfer per CBT+ operational guidelines and procedures. Data will 
be shared back with clinics and the CBT+ initiative only in aggregate form for the purposes 
of quality/continuous improvement. CBT+ has routinely used data from the dashboard for 
quality improvement over the past 10 years. 
Historical data has already been collected and is available to the study team as part of 
their involvement in the CBT+ initiative and ongoing practice improvement work. As part of 

their clinic training, each therapist in these clinics participates in a CBT+ training. For the 
training, they track MBC data for up to 5 cases. To log the data for these 5 cases, therapists 
log into an account set up for them by an administrator and enter the data under a 
pseudonym. They can choose any code (e.g., “Marty McFly”) to identify their patient so long 
is it does not contain the name of their patient or any other identifying information. Thus, all 
client-level data are de-identified before they are entered into the system. While 
administrators have access to website data that contains therapist names and other 
information, that information is not and will not be accessible to this study team.  
For post-IMPACT comparison data, therapists providing data will fall into two categories. The first 
category are therapists who have not gone through the CBT+ training yet. These therapists will 
follow the same procedures as those who entered historical data. The second category are 
therapists who have already completed their CBT+ training. We will ask the second category 
therapists to track data for 5-8 new cases. They will go through the same procedure as the other 
therapists with one exception. Rather than creating codes to identify patients themselves, our 
study team will have each therapist select a paper list of up to 8 identifying codes (e.g., AA1, 
AA2…AA8) to use to identify their patients. We will not keep any document that links any specific 
codes to any therapist. In this way, we will be able to isolate data identified by those codes, and 
identify that they came from a therapist within a specific clinic, but we will not be able to identify 
therapists individually. 
 
Data Collection Instruments (these documents have all been uploaded to IRBNet: 
Clinic Site Visit interviews: Supervisor-level interview guide, Management-level interview guide, 
Clinician-level interview guide, Admin Support Interview guide, AIM FIM IAM measures, and 
Implementation Climate Scale. 
NIMBLE Youth Design Kits: Youth Design Kit Instructions, Youth Activity Cards, Follow-up design kit 
interview. 
NIMBLE Clinician Design Kit: Clinician Design Kit Instructions, Clinician Activity Cards, Follow-up 
design kit interview. 
Implementation outcome scale (AIM FIM IAM measures.docx) – Package 1 
Implementation Climate Scale: ImplementationClimateScale_NIMBLE REDCap.pdf – Package 1 
and electronic/REDCap version in current package 
MBC Fidelity Outcome self-report scale: MBCOutcomeQuestionnaire_NIMBLE(2).pdf – Package 6 
 
 
b. Data Analysis  
 

Aim 1 – The purpose of all data collected in aim 1 is to identify barriers to the use of 
measurement-based care. All rapid ethnography and design kit data will be used to this 
end. 
Rapid Ethnography data analysis will occur in four main steps. 
Step 1: Data Condensing. All data sources will be compiled as text and entered a 
Microsoft Excel database. We do not anticipate any names will appear in these data, but if 
names do appear they will be deleted. Data triangulation will consist of word clouds and 
heat maps.  
Step 2: Data Reduction. At least two team members will read all text data gathered 
in the project.  Two team members will iteratively code data into themes.  
Step 3: Data Interpretation. We will categorize coded data with the explicit purpose 
of (1) identifying barriers (what is getting in the way of MBC use), (2) identifying facilitators 
(are there established strengths we can leverage to strengthen MBC use), and (3) 
identifying themes that may aid in development of implementation strategies (do clinic 
staff participants identify possible solutions for overcoming barriers). 
Step 4: Data Representation. Ultimately, these data will be used to generate a list 
of clinic-specific barriers categorized as in Lewis et al., 2019. Barriers identified through 
this process may be represented in several ways (e.g., box display, matrix, network) 
depending on the content and form of codes and categories.  
Design Kit data analysis will occur in five main steps, with only Step 1 differing from 
REA. 
 
Step 1: Data description and transcription. Visual data (photos) will be described 
by two members of the study team. While we do not anticipate any identifying information 
in photographs, we will first inspect and shred any photograph that includes any possible 
identifying information. Second, two members of the study team will jointly create visual 
descriptions of photos. Journal information will be transcribed into a Microsoft form. All 
names or other identifying information will be removed. 
 
Steps 2-5: The study team will undergo the same steps as rapid ethnography to 
identify a list of barriers: data condensing, data reduction, data interpretation, and data 
representation. 
Aim 2 – Analytic Plan 
Analyses will be the same for each fidelity outcome tool. We will use 3-level generalized mixed 
effects models to account for the multilevel nature of the data (fidelity score nested within 
therapist nested within clinic Significance of model fit and individual coefficients will be 

determined via deviation tests (likelihood ratio, Akaike Information Criterion, & Bayes Information 
Criterion).  
 
Power. Assuming parameters outlined above, therapist intraclass correlations ranging from .01 to 
.30, and level-2 covariates correlating with outcomes at .15, we have power of .80 to detect a 
minimum effect size of d = .36 - .48  
 
Data from each clinic will be collected and combined into a primary dataset after the 12th month of 
their involvement, or when enough therapists have completed entering their 5 training cases into 
the CBT+ dashboard. Data cleaning and primary analyses will begin once all data from all 
participating clinics are collected and compiled into one dataset. It is expected that analyses will 
take between 3 - 6 months after data are gathered from the final clinic to complete. The primary 
outcomes paper will be written between 4 months - 1 year after data are analyzed. 
 
c. Sharing of Results with Participants 
 
Throughout the study, Study Team Members will work with participating clinic staff, sharing 
feedback as aggregate data reflecting what we have learned and ask for clinic staff to reflect on 
how current clinic state matches data interpretation by study staff. This is important for surfacing 
any factors that need to be addressed during the implementation process to optimize MBC.  
 
There are no study results to provide to individual participants. Youth will not receive any results. 
 
d. Data and/or Specimen Banking 
N/A 
9. Privacy, Confidentiality and Data Security 
 
a. Privacy during Study Activities 
The data collected for this study: REA interviews, Youth and Therapist Design Kits, Barrier 
Prioritization, Causal Pathway Diagramming, and Focus Groups, does not contain personal 
information, it is information related to how the participant experiences or implements 
Measurement Based Care at the CMHC participating in this study. However, we will protect 
confidentiality by using Study ID instead of names. Electronic files collected by design kit study 
participants (if they choose to use this format instead of or in addition to our disposable cameras) 
will be transferred to the research team at KPWHRI via SFT. All files will be stored on the KPWHRI G-
drive or Sharepoint site with access only granted to research staff granting access. All computers 
at KPWHRI are password protected and employees are trained in data safety and sign 
confidentiality agreements annually. 
 
 
b. Storage of data and/or specimens. 
 
All data will be stored on password-protected computers on the study G-drive accessible only to 
IMPACT Center Study Staff or on the KPWA REDCap platform. Physical documents will be stored at 
MPE in locked filing cabinets only accessible to study staff. All materials will be de-identified, using 
a unique study ID that does not require a crosswalk because we only need to know which 

documents correspond with each other. Because this is de-identified data, it will be kept 
indefinitely. Postage paid self-addressed sealed envelopes will be sent back by USPS. All audio and 
digital recordings of rapid ethnography interviews will be kept on the G-drive or or Sharepoint site 
with access only granted to research staff on the IRB.  
 
c. Collection of data from participants electronically 
 
If Clinic staff participants prefer, we will communicate via email to convey basic study information 
such 
as to confirm interview times and send information for how to join meetings via Zoom/Teams or to 
answer questions about completing Design Kit Activities. No PHI is being collected as a part of this 
study.  
In addition, we will set up a secure file transfer so that the participants can upload any potential 
electronic data collected while completing Design Kit activities to the study team. Participants are 
instructed not to collect any personally identifying information in their electronic data, so this data 
will 
not contain personal information or PHI. The Implementation Climate Scale 
(ImplementationClimateScale_NIMBLE REDCap.pdf) and NIMBLE Outcome Questionnaire 
(MBCOutcomeQuestionnaire_NIMBLE(2).pdf) do not collect any PHI and will be distributed via 
the REDCap survey distribution tool, which is secure. 
 
Youth will not be contacted by Zoom/Teams; youth will only be contacted by telephone. 
 
d. Disclosure of PHI to Collaborators 
 
We are not collecting PHI from collaborators. We will give participating therapists a list of 8 
pseudonyms to enter youth data under in the Washington State CBT+ Initiative dashboard. When 
we access the data, we will be able to limit what we gather to the pseudonyms provided. This will 
allow for us to not link youth to therapists. 
 
e. Funder required data sharing 
 
The IMPACT Center, funded by NIMH will share 2 measures which are being collected across all 
studies: 
1. Implementation Outcomes Measure 
a. Appropriateness 
b. Feasibility 
c. Acceptability 
2. Causal Pathway Diagrams  
 
None of the measures listed above contain PHI, patient data, clinical data, individual data, or 
any identifiable data. 
 
f. Data and specimen sharing (data commitments) 
N/A 
 

Dr. Martinez will access any sensitive study information and perform study activities through a KP 
laptop, provided to him after he became an "unpaid member of the workforce." He will access data 
through the GlobalProtect VPN. No DTAs are needed at this time. 
 
g. Data provided from external organizations or other KP regions 
N/A 
10. Provisions to Monitor Data to Ensure the Safety of Participants 
N/A 
11. Risks and Benefits  
a. Risks to Participants 
 
While we will hold conversations about workflow at the clinics, we will not be discussing individual 
level performance, nor reporting any individual outcomes. There is minimal risk that an employee 
would suffer any adverse outcome based on the data we are collecting and sharing. 
 
The only risk for clinic staff/therapists is the possibility that their Design Kit could be intercepted by 
someone not involved in the project, however, this risk is minimized because we are not collecting 
any PHI or personal information in the Design Kit process.  
 
One possible risk for youth is that their parent finds their design kit journal, however, there is no 
detailed information about their mental health or treatment plan, it would likely look like a school 
project. The youth could feel uncomfortable or upset that their parent had read their journal. 
The only possible risk with youth is that their de-identified design kit will be intercepted in the mail, 
which is highly unlikely given that they will be provided with a pre-paid, sealable envelope. While 
we do have to mail the Design Kits to the youth, the materials in the Design Kit are identified by 
Study ID only and the return envelope does not include participant name or address. 
 
b. Potential Benefits to Participants 
 
There is no direct benefit to participants in this study. They may have indirect benefits of feeling like 
they have contributed to science. 
12. Costs to Participants 
Participants will not be responsible for any costs. 
13. Compensation to Participants  
For the design probe/kit activity participants will be compensated for their time and effort; 
therapists will receive a $100 gift certificate and youth will receive a $50 gift certificate. 
 
Therapists who participate in the MBC workshops will receive continuing education units (CEU). 
Clinic staff and therapists will receive $25 gift certificate for participating in clinic visit interviews. 
Clinic staff and therapists will receive a $100 gift certificate for participating in focus groups. 

14. Drugs or Devices 
 
a. Drug Study:   N/A 
b. Device Study:  N/A 
15. Qualifications and Human Subjects Training Assurance 
PI attestation: Study investigators are responsible for assuring that all study team members have 
required HIPAA and human subjects training. KP key personnel are required to link CITI training 
records to their IRBNet profile. Check the following to confirm:  
 
☒ All study staff (at KP or at other sites if relying on the KPiIRB for IRB review) who have access to 
identifiable data, or who interact with participants (e.g., programmers, research specialists, survey 
interviewers, interventionists, etc.) have up-to-date HIPAA and human subjects training records. KP 
training requirements are established by the regional research centers or research compliance 
functions. 
 
☒All study activities will be conducted by individuals with appropriate training, education, 
including licensure and language competency. If this cannot be confirmed, explain:  
 
16. Additional Approvals/Ancillary Reviews 
N/A