Standardized Versus Tailored Implementation of Measurement Based Care for Depression
Phase: NA
Conditions: Depression
Sponsor: Indiana University

 
 
 
 
 
 
 
 
 
 
Standardized Versus Tailored Implementation of Measurement-Based Care for Depression 
Protocol Number: 1R01MH103310-01A1 
National Clinical Trial (NCT) Number: NCT02266134 
 
Principal Investigator: Cara C. Lewis, PhD 
Affiliation: Kaiser Permanente 
Sponsor: Indiana University 
 
Funded by: National Institute of Mental Health (NIMH) 
IRB Approval Status: Approved 
7 July 2016 
 
 
 
 
 

A. SPECIFIC AIMS 
Depression remains among the top ten chronic illnesses, costing $83 billion annually (Greenberg, et al., 
2003). Numerous efficacious psychotherapies exist (e.g., Cognitive Behavioral Therapy; Butler et al., 2006), but 
are relatively unavailable in community mental health settings. Implementation of these complex psychothera- 
pies is resource-intensive (Clark, 2011) and sustainment is rare (Stirman, et al., 2012). Measurement-based care 
(MBC; i.e., routine measurement of client symptoms on which care decisions can be made) is a highly accessible 
evidence-based framework (Lambert et al., 2003; Scott & Lewis, in press) that enhances usual care psychother- 
apy by increasing the number of treatment responders and decreasing rates of deterioration (Bickman et al., 
2011; Whipple, et al., 2003). Simply providing clinicians (i.e., mental health counselors) with symptom scores 
improves outcomes. We contend that MBC can be the minimal intervention needed for change (Glasgow et al., 
2013). Unfortunately, fewer than 20% of community clinicians use MBC (deBeurs et al., 2010) due to perceived 
barriers such as time and feasibility (Garland, Kruse, & Aarons, 2003). Tailored implementation (in which strat- 
egies are designed to target determinants of practice revealed through careful assessment; Wensign, Bosch, & 
Grol, 2010) is touted as the optimal approach (Grol & Grimshaw, 2003). Moreover, it is possible that tailoring 
MBC protocols to the site may optimize sustainment (Chambers, Glasgow, & Stange, 2013). However, no stud- 
ies, to our knowledge, have directly compared standardized and tailored approaches, revealing a critical gap in 
the implementation literature and the potential for high impact results. 
We propose a randomized trial of standardized versus tailored MBC implementation in Centerstone, one 
of the nation’s largest not-for-profit community mental health centers. This proposal builds on a 2-year academic- 
community partnership in which the long-term goal is to introduce MBC across 100+ sites. This hybrid type 2 
design will assess both implementation (clinician-level; MBC fidelity) and intervention (client-level; depression 
severity) outcomes. We have the unique opportunity to evaluate MBC implementation, as Centerstone will have 
recently introduced a new electronic health record (EHR) system. The implementation protocol is informed by 
our 2-site pilot study in which MBC achieved 67% penetration. The Patient Health Questionnaire (PHQ-9; 
Kroenke, 2001) will be embedded in the EHR using tablet computer client data capture. Both conditions will use 
a blended implementation protocol of the best available evidence-based strategies to ensure equal access to 
resources, including (a) a needs assessment, (b) formation of an implementation team, (c) MBC use guidelines, 
(d) training, and (e) triweekly group consultation. Mendel et al.’s (2008) Framework for Dissemination will guide 
evaluation (needs assessment, implementation, and outcome evaluation). This framework will also be used to 
tailor training and consultation content to target barriers identified in the needs assessment (contextual factors, 
e.g., attitudes); the standardized condition content is set a priori. In the tailored condition, the implementation 
team will establish the MBC use guideline (e.g., monthly administration), whereas the standardized condition 
guideline will require routine administration prior to each session with a depressed client (Lambert et al., 2003). 
This design will test the Dynamic Sustainability Framework (Chambers, et al., 2013), as sites in the tailored 
condition will adapt MBC guidelines to fit their context. MBC fidelity (range 0-3; client completed=0/1; clinician 
reviewed=0/1, scores discussed=0/1) and reasons for deviations (e.g., lack of time) will be monitored via EHR 
enhancements; clients will also report on clinician discussion of scores. 
Phase I: Pre-Implementation (Months 0-8). The PHQ-9 and fidelity measures will be embedded in the 
EHR and the PHQ-9 linked to tablet computers for client completion to promote feasibility. Phase 2: Randomized 
Implementation Trial: Dynamic Waitlist Control (Months 8-38; Sites: N=12; Clinicians: N=150; Clients: N=500; 
mixed within- & between-subjects design). MBC implementation will occur across 4 cohorts (2-4 sites) at early 
and later stages (each with 5-months active implementation, 10-months sustainment) to reduce confounds (tim- 
ing) and increase feasibility (phased training). Both conditions will enact the 5 strategies within the active imple- 
mentation period. We will use rapid ethnography (month 1) to synthesize the needs assessment data and de- 
velop the tailored content. Phase 3: Characterization of MBC Fidelity (Months 38-48). MBC fidelity will be coded, 
clinician and client fidelity reports will be triangulated, and sustainment will be evaluated. 
Specific Aims. The hybrid design yields one co-primary aim (outcomes at the clinician- and client-levels) 
and two secondary exploratory aims in the context of a pragmatic trial (Thorpe et al., 2009). 
Aim 1: To compare the effect of standardized versus tailored MBC implementation on clinician-level (1a) and 
client-level (1b) outcomes. We hypothesize that tailored implementation will outperform standardized in terms of 
(H1a) MBC fidelity, and (H1b) reducing client depression severity. 
Aim 2: To identify contextual mediators of MBC fidelity. We hypothesize (H2) that contextual mediators (struc- 
ture, norms, etc.) will be leveraged in the tailored condition, but serve as barriers in the standardized condition. 
 
 
1 

Aim 3: To explore the impact of MBC fidelity on client outcomes (Chambers et al. 2013). We hypothesize (H3) 
that adapted MBC protocols (tailored condition) will outperform weekly administration of PHQ-9s (standardized 
condition) with respect to clinically significant change in depression severity from intake to week 12. 
B1. SIGNIFICANCE 
The long-term goal of this research project is to provide generalizable and practical recommen- 
dations about implementation approaches that promote Measurement-Based Care (MBC) use and ef- 
fectiveness in community mental health centers, a goal developed in partnership with stakeholders at 
Centerstone. Previous attempts to integrate MBC into real world settings have focused on the devel- 
opment of standalone feedback systems (Koerner, Dailey, Lipp, Connor, & Sharma, 2012; Sapyta, 
Riemer, & Bickman, 2005). However, to our knowledge, no studies have investigated the strategies 
necessary to integrate MBC into community mental plementation support or (b) taking into account 
stakeholder perceptions and needs when building the implementation approach. This proposal reflects 
the ideas generated by a 2-year academic-community partnership between the proposal PI (Lewis), 
Centerstone Research Institute Director (Co-I, Ayer) and regional clinic directors whom constitute our 
advisory board (Drs. Harrison, Hardy, Pardue, Moran, Marshall; Letters of Support). 
The proposed implementation of MBC by community mental health clinicians who treat depres- 
sion is significant because despite decades of research revealing numerous efficacious treatments for 
depression, this disease remains among the top ten chronic diseases in the nation (Greenberg et al., 
2003). Moreover, MBC is an evidence-based framework that has established effectiveness, broad 
reach, and multifaceted utility for enhancing usual care. MBC presents a simple framework in which 
care is based on the results of symptom measurement. A meta-analysis indicated that MBC is particu- 
larly effective in improving outcomes when depressed clients are not demonstrating progress 
(Shimokawa, Lambert, & Smart, 2010) and by reducing client deterioration (Lambert, et al., 2003) with 
medium effect size improvements over usual care (Bickman et al 2011; Whipple, et al., 2003). MBC 
has greater reach potential than complex Evidence-Based Practices (EBP) that include multiple theory- 
specific components targeted at single disorders; MBC is a simple standalone practice framework that 
increases the effectiveness of diverse usual care offerings for clients with multiple problems. MBC may 
be the “minimum intervention needed for change” (MINC; Glasgow et al., 2013). The reach of MBC is 
particularly strong because its relevance is transtheoretical (relevant for use by clinicians regardless of 
background) and transdiagnostic (effective in enhancing usual care for numerous disorders) (Scott & 
Lewis, in press). MBC’s utility is multifaceted and aligns with the Affordable Care Act by focusing on 
monitoring outcomes. MBC presents a systematic approach for selecting and adapting interventions 
(Trivedi & Daly, 2007); it flags clients who are not improving; and, it highlights treatment targets (Lam- 
bert et al., 2005). NIMH’s nationwide public health clinical trial, the STAR*D (Sequenced Treatment 
Alternatives to Relieve Depression), demonstrated MBC’s utility for guiding both medication and psy- 
chotherapeutic interventions (Trivedi et al., 2006). MBC also has established utility for promoting care 
coordination across disciplines (Unutzer et al., 2002). MBC provides the basis for evaluating subse- 
quent EBP implementation efforts through a foundation of progress monitoring, which leverages the 
soon to be ubiquitous electronic health record technologies. 
By design, MBC has great potential for implementation success, yet barriers such as attitudes 
(perceptions that standardized measures have limited utility) and feasibility (perceptions that measures 
take too much time) exist (Garland et al., 2003), and the gap between documented MBC effectiveness 
and use in practice remains. Research to date has largely focused on evaluating standardized ap- 
proaches to implementation, despite a recent Cochrane meta-analysis highlighting the potential impact 
of tailoring implementations (Baker et al., 2009). There is a crucial need for process-focused implemen- 
tation research. Hence our proposed comparison of implementation conditions (i.e., standardized ver- 
sus tailored) is both clinically and scientifically significant because it will (a) reveal whether standardized 
or tailored approaches to implementation optimize MBC fidelity, (b) enhance our understanding of both 
theory and processes for standardized versus tailored implementation; and, (c) illuminate predictors, 
moderators, and mediators of successful implementation. Our proposed implementation-effectiveness 
hybrid design will yield important insights regarding MBC clinician level implementation outcomes. It 
will simultaneously shed light on the effect of MBC on adult mental healthcare for depression in 

community settings when adapted by stakeholders to fit their context. Our research will answer the how 
of implementation by evaluating the effect of contextual factors on the process, an effort that directly 
aligns with NIMH’s Strategic Plan 4.1 to “Improve understanding of the factors that affect…the means 
by which newly discovered effective mental health interventions are disseminated and implemented.” 
B2. INNOVATION. This proposal is innovative in at least three ways. 
1) Minimal Intervention Needed for Change (MINC; Glasgow et al., 2013). Our focus on test- 
ing strategies to implement MBC in community mental health is innovative because this simple MBC 
framework may be the minimal intervention needed for significantly reducing the burden of depression 
on society. For the treatment of depression, we opted to focus on MBC rather than a complex, theoret- 
ically-driven EBP like Cognitive Behavioral Therapy not only because of the MBC implementation gap, 
but also because the simplicity and accessibility of the MBC framework will reduce the number of im- 
plementation barriers. Moreover, MBC has been isolated as a core component of many of EBPs. There- 
fore, identifying effective implementation strategies for MBC would build the case for a phased or staged 
approach to full package EBP implementation to determine whether later EBP implementations en- 
hance outcomes beyond improvements observed with MBC. 
2) Leveraging Low Cost Technology. We will use tablet computers linked to an online MBC 
platform that is embedded within the electronic health record system. The use of technology will de- 
crease the time burden of MBC and will enhance the utility it brings to clinicians, clients, the organiza- 
tion, and research (Powell et al., 2005). The tablets’ high-tech features have the potential to promote 
client engagement in treatment and may help to emphasize the importance of symptom monitoring. 
Linking the tablet data collection with the electronic health record will also allow us to build useful, 
innovative features such as symptom trajectory graphs, alerts when suicidality is endorsed, and 
ideas/suggestions for treatment targets. With electronic health record prevalence increasing and tablet 
computer costs decreasing, this approach presents a generalizable and cost-effective method for en- 
gaging in systematic outcome monitoring that maximizes therapeutic benefit. 
3) Tailored Implementation. The majority of existing implementation research has focused 
either on descriptive studies that explore barriers and facilitators (determinants of practice) or on 
comparisons of a priori selected implementation 
strategies that generally neglect contextual tailoring of 
the interventions. A qualitative analysis of 22 
implementation studies revealed that few focused on 
matching strategies to deteminants of practice (Bosch, 
van der Weijden, Wensing, & Grol, 2007). A critical 
research agenda has emerged seeking to identify, “how 
Table 1. Implementation Conditions 
Standardized 
Tailored 
Manualized implementa- 
tion content 
Content adapted by implemen- 
tation team to address barriers 
Developed a priori 
Responsive to local needs 
Standardized MBC guide- 
MBC administration set by site 
and why implementation processes are effective” (Proctor, Powell, Baumann, et al., 2012) by 
experimentally evaluating implementations that are tailored to the context. Table 1 summarizes the 
conceptual differences between standardized and tailored conditions in this proposal. 
Tailored Implementation and Quality Improvement approaches overlap with respect to their ulti- 
mate goal of improving client care through changing provider behavior using context-specific strategies. 
However, Tailored Implementation and Quality Improvement differ in three key ways. One, Quality Im- 
provement focuses primarily on change as it occurs outside of the intervention whereas Tailored Im- 
plementation allows for changes to the intervention to fit the context (Chambers, Glasgow, & Stange, 
2013). Recent findings support the need to adapt evidence-based practices during the implementation 
process (Aarons, Miller, Green et al., 2014; Stirman, et al., 2013), but no studies have directly compared 
this approach to standardized EBP implementation. Two, Quality Improvement engages the Plan-Do- 
Study-Act as a means for identifying necessary change strategies driven by data in a cyclical fashion, 
whereas Tailored Implementation begins with a needs assessment to identify context-specific determi- 
nants of practice and then adapts implementation content to match determinants of practice. Three, 
implementation is focused on the goal of integrating an EBP, in this case MBC, into real world settings 
(Eccles, et al., 2009) whereas Quality Improvement is a general organizational management approach 
3 

to improving care not necessarily focused on EBP integration (Baker, 2006). This proposal reflects a 
movement in the field (e.g., Dynamic Adaptation Process; Aarons, et al., 2012) to consider planned 
adaptations of the EBP and systems/organizations to promote EBP integration and sustainment. 
C. APPROACH 
Overview. Given the underwhelming use of MBC in community mental health settings coupled 
with the demand for performance outcome assessment (Affordable Care Act), this randomized imple- 
mentation trial aims to compare the effectiveness of a standardized versus tailored approach to imple- 
mentation of MBC with co-primary outcomes at the clinician (MBC fidelity) and client (depression se- 
verity) levels. We will randomize 12 sites of a large community mental health center (Centerstone) to 
early or later stage standardized or tailored implementation enrolling 150 clinicians and 500 depressed 
clients in a pragmatic trial (Thorpe et al., 2009). The main clinician level outcome is MBC fidelity, defined 
as (a) client completion of the PHQ-9, (b) clinician review of scores in the electronic health record, and 
(c) discussion of scores in session. Phase 1 (months 0-8) will interface the most widely used and vali- 
dated depressive symptom severity measure (Patient Health Questionnaire-9 item; PHQ-9; Kroenke, 
2001) with the electronic health record; this measure is the main client level outcome. Phase 2 (months 
8-38) constitutes the active implementation (5 months) and sustainment (10 months) phase. In both 
conditions, clients will have the option to complete the PHQ-9 on tablets in the waiting room prior to 
session, which will feed scores to clinicians via the electronic health record. Sites randomized to the 
standardized condition will begin with a baseline mixed methods needs assessment (for the purposes 
of putative mediator data collection) and receive the guideline that MBC is to be used in each session 
with a depressed client. The standardized condition will include manualized training and triweekly group 
consultation with experts to promote MBC fidelity and optimize its clinical utility. An implementation 
team (including a site administrator, opinion leader, MBC champion, and research personnel) will con- 
vene prior to each triweekly group consultation to review progress and troubleshoot problems. Sites 
randomized to the tailored condition will also begin with a needs assessment to identify contextual 
factors that may serve as barriers to the implementation (guided by the Framework for Dissemination; 
Mendel et al., 2008; see C7). Training and triweekly group consultation will be tailored to address these 
barriers (e.g., clinician attitudes toward MBC). The implementation team (same composition as in 
standardized) will define a site-specific guideline for MBC use (e.g., monthly) and convene prior to each 
triweekly group consultation to troubleshoot MBC fidelity barriers. At the start of the active implemen- 
tation across both conditions, depressed clients of participating clinicians will be enrolled to compare 
the effect of standardized versus tailored MBC implementation on client outcomes. Phase 3 (months 
38-48) will characterize MBC fidelity using electronic health record data capture. The evaluation ap- 
proach is guided by the Framework for Dissemination (Mendel et al., 2008) and includes a baseline 
needs assessment, implementation/process evaluation (5 months into implementation), and out- 
come/impact evaluation (15 months into implementation). This design will allow for a direct test of the 
assumption that voltage drop in treatment outcomes occurs in the context of adapting the intervention 
to fit the context (Chambers et al., 2014), while simultaneously exploring the effect of standardized and 
tailored approaches to MBC implementation. 
C1. Research Team Expertise, Roles, and Plan for Collaborating 
The PI is strategically guided by two tiers of influence with the first tier (Kroenke & Ayer) including key 
leadership to help manage practical aspects of a clinical trial and the second tier (Mendel, Simon, Marti, 
& Rutkowski) including essential content and methodological expertise. Kroenke has notable experi- 
ence with NIH-funded R01-level effectiveness trials that include depressed patients. Kroenke will men- 
tor PI, Lewis, in the management of a scientific trial. Ayer will provide mentorship in community trials 
given his experience as site PI for federally funded trials at Centerstone (STAR*D & CATIE); Ayer will 
oversee participant enrollment and data collection. Lewis will meet with Kroenke and Ayer weekly dur- 
ing the start up phase and twice monthly thereafter. The second tier brings expertise in implementation 
science and community-based participatory research (Mendel) and measurement-based care and com- 
munity-based research (Simon); both experts will convene with Lewis at least monthly. Marti and 
Rutkowski bring quantitative and mixed methods expertise, respectively, with meeting frequency 

approximately monthly, but more frequent when expertise is needed (e.g., Rutkowski weekly during 
active implementation). Quarterly research team meetings will be held with all members. 
C2. Centerstone Readiness and Representativeness 
As a large behavioral health center with 100+ sites across IN and TN, Centerstone employs approxi- 
mately 400 clinicians and annually provides services to over 70,000 individuals and families. Center- 
stone will soon introduce a new electronic health record system through a contract with NetSmart, a 
leading provider of technology solutions for health and human services, with advanced capabilities 
through their electronic clinical expert technology (Morrison Letter of Support). Given these forthcoming 
changes and their on-site research institute (Director & Co-I, Ayer), we 
have an unprecedented opportunity to evaluate an MBC implementation. 
Centerstone clinicians and clients are highly representative of the broader 
population; therefore we anticipate that the results of this project will gen- 
eralize to other community mental health centers. Table 2 depicts de- 
mographics for Centerstone clinicians. Consistent with typical community 
mental health center clinicians in the US, the majority of Centerstone cli- 
nicians are Caucasian females with Masters level training. Centerstone 
clientele represent the broader population of adults seeking mental health 
services in community settings. Each year, roughly 36,000 Centerstone clients meet criteria for a pri- 
mary nonpsychotic depressive disorder. 
C3. Overview of Measurement-Based Care 
MBC is the systematic monitoring of client outcomes, using standardized measures, to inform treat- 
ment. Recent efforts to implement MBC in community mental health settings have primarily focused on 
the use of measurement feedback systems and not on the implementation process (e.g., Bickman, 
2008). These systems are not readily available due to high costs, nor are they easily integrated within 
the electronic health record to interface with existing documentation requirements. To address these 
limitations, our team will introduce the MBC as an evidence-based intervention framework (Scott & 
Lewis, in press) and capitalize on the ways MBC has been used effectively in medicine (Löwe et al., 
2004) and in the UK’s Improving Access to Psychological Therapies program (Clark et al., 2009). Cen- 
terstone clinicians (N=165) revealed that fewer than 24% use MBC with depressed clients (Lewis, Scott, 
Marti, & Ayer, under review), which is likely an overestimate given the self-report nature of the data 
(Martino et al., 2009). To improve this rate of MBC use, we will integrate MBC capacities within the 
electronic health record as well as support and evaluate the implementation process. 
C4. PHQ-9 Relevance, Specificity, Sensitivity 
We will use the Patient Health Questionnaire depression scale (PHQ-9; Kroenke, 2001) as the primary 
depression outcome measure in this proposal and as the core component of MBC. The PHQ-9 consists 
of 9 items that map directly onto the symptoms of a major depressive episode (DSM-IV TR; American 
Psychiatric Association, 2000), as well as 1 item pertaining to impairment. The PHQ-9 is one of the 
best-validated depression measures used in >1000 research studies (Kroenke, Spitzer, Williams, & 
Lowe, 2010). The PHQ-9 has depressive severity cutoff scores, is sensitive to change (Löwe, Kroenke, 
Herzog, & Gräfe, 2004), and is useful for weekly administration as an indicator of treatment effective- 
ness (Kroenke & Spitzer, 2002); a five-point change reflects clinically significant reduction in symptom 
severity (Kroenke & Spitzer, 2002). Three diagnostic meta-analyses and a recent review have con- 
firmed the good sensitivity and specificity of the PHQ-9 in making a major depressive disorder diagnosis 
(Kroenke, Spitzer, Williams, & Lowe, 2010). Clinicians in our pilot study appreciated the brevity of the 
measure; clients stated that they thought the PHQ-9 (a) was relevant, (b) would facilitate suicidality 
endorsement, and (c) would help them to understand their symptoms. 
C5. Leveraging Technology: Interfacing Tablet Data Collection with the Electronic Health Rec- 
ord NetSmart (Morrison Letter of Support) has agreed to support the integration of the PHQ-9 into the 
electronic health record system and to work with our team to develop advanced features of the MBC 
5 
Table 2. Clinician Demographics 
Gender: Female 
83.4% 
Age 
43 yrs 
Education: Masters 
96.0% 
Time at Centerstone 
6.04 yrs 
Licensed: Yes 
11% 
Weekly Caseload 
45 
Productivity Requirement 
1200 hrs/yr 
Met Productivity: yr 11/12 
62.0% 
Supervisor : Clinician 
1:3 

interface. Total score graphical displays showing session-by-session progress will be available to clini- 
cians within the electronic health record and to clients on their personal health record via the tablet. 
Clinicians and clients will be able to view individual item-level information, with suicidality highlighted 
for careful review. Clinically significant change (5-point reduction), and lack thereof, will be flagged from 
session-to-session. The PHQ-9 will be available to clients on tablet kiosks in the waiting area for com- 
pletion prior to session. In our pilot, clinicians indicated the importance of tablets as they: (1) felt clinician 
administration of the PHQ-9 was unnecessarily time-consuming, and (2) wanted to encourage clients 
to take ownership of symptom monitoring given the potential therapeutic benefits (Eisen, Dickey, & 
Sederer, 2000). Clients in our focus group unanimously requested private completion of the PHQ-9, 
indicating that this would promote comfort in sharing suicidality. Tablets are a cost-effective option that 
has been successfully used in similar protocols (Crits-Christoph et al., 2012). The technological and 
clinical innovations developed for this proposal present a generalizable protocol (documented in a man- 
ual) for widespread implementation given the proliferation of electronic health records. These techno- 
logical enhancements will be made in both conditions during Phase I (months 0-8; see C11. Overview 
of Study Design). 
C6. MBC Fidelity 
In this proposal, fidelity to MBC is a three-level categorical variable defined by (a) client completion of 
the PHQ-9 (No=0, Yes=1), (b) clinician review of scores in the electronic health record (No=0, Yes=1), 
and (c) discussion of scores in session (No=0, Yes=1). The former two (PHQ-9 completion and clinician 
review of scores) will be automatically captured through the electronic health record. Clinicians will also 
check a box in the electronic health record progress note to indicate if they discussed scores in session, 
as this practice optimizes the effectiveness of MBC (Eisen, Dickey, & Sederer, 2000). Given the known 
limitations to clinician self-report, this latter fidelity criterion will also be assessed with client self-report 
via an automated telephone survey immediately after their session. In their first month of implementa- 
tion, all clinicians (regardless of condition) will be asked to indicate factors they perceived to have lim- 
ited MBC use. Through a drop-down 
menu embedded in the electronic 
health record progress note, clini- 
cians will select contextual factors 
based on the literature (Dowrick et al., 
2009; Garland, Kruse, & Aarons, 
2003; Valenstein et al., 2009) and cli- 
nician-identified factors (from our pilot 
data); See Appendix for complete list 
of options (e.g. lack of time, client re- 
fused, clinician forgot). 
C7. The Evaluation Process Figure 
Figure 1. Framework for Dissemination 
 
 
1 depicts the Framework for Dissemination, which outlines the diffusion and evaluation processes for 
this study (Mendel, Meredith, Shoenbaum, Sherbourne, & Wells, 2008). Contextual factors (Box 1) 
theorized to influence the stages of diffusion (Box 2) are identified and both individual and organiza- 
tional outcomes are considered (Box 3). This framework is based upon the best available research and 
includes a 3-phase evaluation process (see bottom row) with a community partnership emphasis di- 
rectly suited to guide this proposal. This model will allow us to evaluate the standardized implementa- 
tion protocol and to identify contextual factors that may limit or facilitate its effectiveness. This model 
will also be used to guide the tailored approach to implementation (Mendel; Letter of Support). 
 C8. Complementary Testable Model. Chambers, Glasgow, and Stange (2013) recently put forth the 
Dynamic Sustainability Framework to promote testing of falsifiable hypotheses that program drift from 
“fidelity” of evidence-based practices leads to a voltage drop in implementation outcomes as compared 
to effect sizes observed in efficacy trials. Chambers et al. (2013) contend that an alternative hypothesis 
is important to consider: intentional adaptations to EBP implementation that are informed by community 

stakeholders and account for relevant multi-level contextual factors may optimize sustainment and in- 
tended outcomes. This proposal intends to test this model. Specifically, the standardized condition re- 
quires (via a guideline) that MBC be implemented weekly prior to each psychotherapy session accord- 
ing to its documented efficacy (e.g., Bickman et al., 2008; Whipple et al., 2003). Conversely, the tailored 
condition will allow the implementation team at each site to adapt this guideline, taking into account 
their specific context (e.g., monthly MBC implementation). 
C9. Pilot Studies 
Two Centerstone sites participated in our pilot study to assess the feasibility of the implementation and 
evaluation process and to inform the implementation protocols for this proposal (C10. Multifaceted Pro- 
tocol). 
Baseline Needs Assessment. The mixed methods needs assessment included baseline in- 
person clinician (N=10, 1.5 hours) and client (N=6, 1 hour) focus groups and clinician (N=24) self-report 
survey assessments to evaluate the contextual factors of diffusion (Box 1, Figure 1). This needs as- 
sessment procedure proved successful for data collection and resulted in 100% clinician participation. 
At baseline, the Attitudes towards Standardized Assessment scale (ASA, scale ratings 1-5; Jensen- 
Doss & Hawley, 2010) indicated that clinicians at both sites had generally neutral attitudes about MBC 
practicality (M=3.03, SD=0.54 and M=3.14, SD=0.42, respectively; Cohen’s d=0.23), its benefit over 
clinical judgment (M=2.88, SD=0.46 and M=3.40, SD=0.93, respectively; Cohen’s d=0.71), and its psy- 
chometric quality (M=3.37, SD=0.56 and M=3.89, SD=0.53, respectively; Cohen’s d=0.95), with Site A 
demonstrating moderately lower scores across the subscales. Sites also differed on their intention to 
use MBC; Site B had more positive views of social influence to use MBC (t=2.48, p=.042) and MBC 
self-efficacy (t=3.49, p=.010). 
Qualitative analyses of clinician focus groups (2 sites, N=10) also revealed site differences in 
implementation barriers. Notably, significant site differences were observed in the ratio of negative to 
positive attitudes about MBC (Site A: 44 negative to 17 positive, versus, Site B: 35 negative to 33 
positive). Clinicians at Site A frequently cited that MBC was artificial and was overly evaluative of the 
clinician’s abilities. At Site B, clinicians endorsed concerns about how MBC would impact clinical 
productivity, the feasibility of MBC implementation, and the utility of MBC in achieving therapy goals. 
Simultaneously, clinicians at Site B indicated that MBC would be useful for evaluation and diagnosis 
and could be used to identify lack of progress. Clinic director participants indicated that leveraging 
technology for client completion would be essential. 
Implementation. We then piloted a standardized implementation protocol consisting of (a) em- 
bedding the PHQ-9 into the electronic health record (requiring the clinician to verbally assess clients 
as tablets were not available), (b) a site-specific guideline for PHQ-9 use, and (c) a brief training. The 
4-hour training content was based on the work of Persons, Hong, and Koerner (2012); topics included: 
MBC as a foundational framework; MBC clinical utility; the research evidence for MBC and the PHQ-9; 
the MBC protocol; research and IT supports to enact the protocol; introduction to the electronic health 
record interface; and steps for working with lack of progress. These strategies reflect a minimum set of 
discrete evidence-based implementation strategies that could be blended to facilitate implementation. 
Pre-training Intention to Use MBC scores were highly correlated with the frequency of subsequent PHQ- 
9 administration (r=.67, p=.036). Across sites, 67% MBC penetration was achieved (number of clini- 
cians implementing MBC/number of participating clinicians). Site A achieved 44% penetration while 
Site B achieved 80% penetration. One limiting factor at Site A was technology problems as the new 
electronic health record had not yet been introduced and the PHQ-9 did not consistently show up for 
clinicians. Despite these relatively high penetration scores at the clinician level, this minimal standard- 
ized implementation protocol and site-specified MBC guideline resulted in 13.4% (Site A) and 38.6% 
(Site B) use of the PHQ-9 (by dividing number of administrations by total possible number of admin- 
istrations) over the course of one year follow up. Unfortunately, we were unable to assess the differen- 
tial effect of MBC implementation on client outcomes to determine whether adapted guidelines limited 
MBC’s utility. These findings support the proposal in three ways: (a) they emphasize the utility of a 
7 

mixed methods needs assessment in identifying site differences in barriers, (b) they highlight the im- 
portance of tailoring strategies to barriers as this minimal standardized protocol led to superior out- 
comes in Site B where fewer barriers were present, and, (c) they underscore the potential importance 
of incorporating technologically advanced solutions for client PHQ-9 completion. 
C10. Multifaceted Standardized and Tailored Implementation Conditions 
Overview of the Blended Implementation Protocol. Both conditions (standardized and tai- 
lored) will employ the same blended protocol of implementation strategies to remove time and resource 
confounds (see Table 3). The guiding model (Framework for Dissemination; Figure 1; Mendel et al., 
2008), the best available literature on related efforts to promote MBC implementation (e.g., Evans & 
Hser, 2004; Harding et al., 2011; Lambert et al., 2005; Morris & Trivedi, 2011; Persons, Hong, & 
Koerner, 2012; Teruya, Hardy, Hser, & Evans, 2006), personal communication regarding program eval- 
uation of a loosely tailored MBC implementation effort at Group Health clinics (Steinfeld, February 11, 
2014), the MBC intervention framework, the partnership goals, and the pilot study defined the blended 
protocol of implementation strategies (Powell, McMillen, Proctor, et al., 2012). It was determined that 
embedding the PHQ-9 into the electronic health record was not sufficient but that using tablet comput- 
ers would be essential to promote feasibility of implementation. The proposed local needs assessment 
is necessary to assess site-specific contextual factors and to identify opinion leaders and champions. 
In our pilot, we did not involve clinicians on the implementation team, but literature suggests this may 
be critical to achieving sustainment (Teruya, Hardy, Hser, & Evans, 2006) and it is consistent with our 
partnership goals. As such, each site will form an implementation team consisting of the PI, Co-I (Ayer), 
the site administrator, a clinician identified as an opinion leader (via self-report; Childers, 1986), and a 
self-nominated MBC champion who will meet for at least 30 minutes prior to each triweekly consultation 
session. Training and consultation with experts are essential strategies for promoting clinician behavior 
change and fidelity to the intervention, particularly in the case of evidence-based psychosocial inter- 
ventions (Herschell et al., 2010). A guideline will be set to specify the frequency of expected PHQ-9 
administration with depressed clients. The order of strategies will proceed as follows within each site’s 
5-month active implementation period: (a) embed PHQ-9 in electronic health record; (b) conduct needs 
assessment; (c) form implementation team; (d) set guideline; (e) offer initial training; (d) conduct tri- 
weekly group consultation meetings (see C11. Overview of Study Design). In the 10 months post active 
imple- 
menta- 
tion, the 
imple- 
menta- 
tion 
team 
will 
be 
encour- 
aged to 
continue 
to meet 
and 
Table 3. Standardized versus Tailored Protocol and Focus 
Contextual Factor 
Strategies 
Standardized Focus 
Tailored Focus 
Resources 
PHQ-9 in EHR w/ Tablets 
Client Completion on Tablets 
Client Completion on Tablets 
Networks & Linkages 
Form Implementation Team 
Monitor MBC Fidelity 
To Identify Barriers 
Policies and Incentives 
Guideline 
Each Session w/ Client 
Determined by Site 
Norms & Attitudes 
Training 
Standardized Protocol 
Targets Barriers 
Structure & Process 
Progress Note Modifications 
For Clinician Score Review 
For Clinician Score Review 
Media & Change Agents 
Consultation with Experts 
MBC Fidelity 
Targets Barriers 
consult with clinicians to promote sustainment. Table 3 depicts the protocol and unique focus of the 
implementation strategies across conditions. A similar blended protocol of implementation strategies 
led to successful MBC implementation using the PHQ-9 with 90% completion with over 30,000 clients 
each quarter at Group Health (personal communication, Steinfeld, February 11, 2014). 
Standardized Condition. The standardized condition includes all aforementioned strategies in 
the order listed above. The needs assessment will be conducted similar to our pilot study in that it is for 
data collection purposes only (i.e., no tailoring to identified barriers). The implementation team meetings 
will focus on monitoring MBC fidelity per the guideline. Depressed clients of clinicians in the standard- 
ized condition will be asked to complete the PHQ-9 prior to each session on a tablet in the waiting room 

(guideline). PHQ-9 data will then be fed to the electronic health record for clinician review. Training will 
be offered to all enrolled clinicians using standardized material (described in C9. Pilot Studies). Con- 
sultation will focus on incorporating clinician review and discussion of PHQ-9 scores in session and 
providing tips on targeting lack of progress. 
Tailored Implementation Condition. Tailored Implementation refers to the responsive applica- 
tion of implementation strategies and content matched to determinants of practice (i.e., barriers) iden- 
tified via a needs assessment. The same strategies outlined in the standardized condition will be em- 
ployed in the tailored condition, but with content tailored to the site. The needs assessment will reveal 
contextual factors that may serve as barriers to the implementation process (focus groups will be ana- 
lyzed using rapid ethnography methods; C13b. Qualitative and Mixed Methods Data Analysis Plan). An 
implementation team will convene to define the site-specific guidelines for PHQ-9 completion. For in- 
stance, it may be that Site X decides that monthly PHQ-9 administration is optimal with respect to 
feasibility and clinical utility. Conversely, Site Y may prefer to have clients complete the PHQ-9 every 
other session. Depressed clients of clinicians in the tailored condition will complete the PHQ-9 in the 
waiting room on a tablet computer prior to their appointment with the enrolled clinician based on the 
site-specific administration guideline. PHQ-9 data will then be fed to the electronic health record for 
clinician review. Training will be offered to all enrolled clinicians using tailored materials that will target 
the identified barriers from the needs assessment. For instance, if clinicians at a particular site perceive 
the PHQ-9 to be irrelevant to clients, training content will incorporate client perspectives on its utility. If 
clinicians indicate that lack of time is a barrier, then training will incorporate experimentation to stream- 
line review and discussion of scores. The implementation team will focus on identifying remaining bar- 
riers for discussion in the triweekly group consultation session. Similar to the standardized condition, 
consultation will also focus on incorporating clinician review and discussion of PHQ-9 scores in session 
and providing tips on targeting lack of progress. 
C11. Overview of Study Design, Timeline, Sample, and Data Collection 
The two years of planning and pilot 
studies in partnership with Center- 
stone have informed the implementa- 
tion protocols as well as its overall re- 
search design and data collection 
procedures. We propose an effec- 
tiveness-implementation hybrid de- 
sign type 2 to allow for simultaneous 
investigation of clinician- and client- 
level outcomes. This design will be a 
dynamic waitlist randomized prag- 
Figure 2. Design Overview 
 
matic trial comparing the effect of a standardized versus a tailored approach to MBC implementation 
(Figure 2 & Table 4). 
Phase 1: Start-up/Pre-Implementation. Months 0-8 will be characterized by preparatory work 
for the randomized trial. Little needs to be done to create the research infrastructure in the participating 
clinics given the existing role of Centerstone Research Institute directed by the proposal Co-I (Ayer). 
We will prepare the tablet computers for client data collection. We will work with Netsmart (Morrison 
Letter of Support) to link the electronic health record to the PHQ-9 and create the optimal interface for 
clinicians within the progress note. We will enroll the identified sites (N=12), match sites based on size 
and urban/rural status, and randomize to either early or later stage implementation and to condition 
(standardized or tailored) (Brown et al., 2006). There will be four cohorts each with 5 months of active 
implementation and 10 months of sustainment monitoring. We will modify the progress note for clini- 
cians to document fidelity (PHQ-9 discussed in session) and reasons for deviation (C6. MBC Fidelity). 
We will pilot the progress note modifications with our advisory board of clinic directors to maximize the 
information gleaned and to minimize the burden. Because of our use of technology, we will create IT 
9 

Frequently Asked Questions and instructions, which will be embedded within Centerstone’s internal 
news. We will also interface study research assistants with clinic staff by attending regular staff meet- 
ings at each site. Finally, we will refine measures using established protocols (e.g., Norms; Francis et 
al., 2004). 
Phase 2: Randomized Trial: Implementation & Sustainment. Implementation will take place 
across 12 sites in months 8-38 using the dynamic waitlist control design (Brown et al., 2006; C15. 
Justification and Feasibility). Specifically, matched sites (based on size and urban/rural status) will be 
randomized to either early or later stage implementation in 4 cohorts (2-4 sites per cohort spaced 5 
months apart), with half the sites randomized to the standardized and half to the tailored condition; see 
Figure 2 for a concise depiction of the implementation timing. The timing of this protocol is based on 
the published work of Miller et al. (2012) as well as the successful naturalistic MBC implementation at 
Group Health (personal communication, Steinfeld, February 11, 2014). Beginning at month 8, the ear- 
liest cohort (2 sites) will engage in the baseline mixed methods needs assessment (Assessment Time 
1; AT1). Using purposeful sampling (Palinkas et al., 2013), a subset of clinicians (N=5-8 at each site) 
representing extreme variation (nominated by clinic directors based on their support for or against MBC 
implementation) will participate in a 1.5-hour focus group. This sampling approach is critical to gain a 
wide range of clinician views. Rapid ethnography will then be used to uncover site-specific insights that 
will guide the content of training and consultation in the tailored condition only (with the aid of mixed 
method expert Rutkowski; C13b. Data Analysis Plan). During the needs assessment, all enrolled clini- 
cians will complete the battery of baseline measures (see Table 5, Section II), from which the opinion 
leader (Childers, 1998) and self-nominated MBC champion will be identified and invited to join the 
implementation team. Across conditions, this team will convene triweekly during the 5-month active 
implementation period to ensure implementation strategy fidelity (Proctor et al., 2013) and review MBC 
fidelity. Following the needs assessment, clinicians will participate in a 4-hour MBC training workshop 
(see C9. Pilot Studies for content) and will begin triweekly consultation. To characterize the differences 
in implementation team meetings and consultation between conditions, sessions will be audiorecorded 
and contextual factors logged by graduate research specialist on the MBC Barriers Log (informed by 
the Framework for Dissemination; Mendel et al., 2008; see Appendix for form template). Simultaneously 
a site team member will log meetings (using the same form) and data triangulated to calibrate site team 
members and prep for the exit of research personnel. Implementation/Process Evaluations (Mendel et 
al., 2008; AT2) at the clinician level will occur 5 months after the needs assessment at which point the 
research personnel will be removed. The site implementation teams will be encouraged to continue 
meeting to promote MBC sustainment and the responsibility to log and address MBC barriers will trans- 
fer completely to the site team members. Outcome/Impact Evaluation (Mendel et al., 2008; AT3) at the 
clinician and client level will occur 10 months after the Implementation/Process Evaluation. Focus 
groups will be held with the implementation team at AT3 to review their experience and the site’s pro- 
gress since the research personnel exited the team. All aforementioned steps will be repeated across 
the remaining three cohorts. 
Phase 3: Post Hoc Characterization of MBC Fidelity, Data Analysis, and Manuscript Prep- 
aration. Consistent with the Dynamic Sustainability Framework (Chambers, Glasgow, & Stange, 2013), 
the approach to MBC imple- 
mentation has the potential to 
be adapted by sites in the tai- 
lored condition. That is, early in 
the active implementation pro- 
cess (within the first month), the 
implementation team will gener- 
ate a guideline for MBC imple- 
mentation that is specific to their 
site. Characterizing fidelity for 
this condition will need to reflect 
the guideline established for Note. Co = Cohort (2-4 sites with 1-2 randomized to each condition). 

each site. For instance, if a site set a guideline to administer the PHQ-9 monthly for each client then 
actual administration will need to be confirmed monthly. Data capture from the electronic health record 
will also include whether or not the clinician reviewed the PHQ-9 scores prior to or in session. Finally, 
with respect to coding of MBC review in session the clinician self-report (via the electronic health record) 
and client report (via automated phone survey) will be triangulated. Concurrently, focus group data will 
be formally coded (see C13b. Qualitative and Mixed Methods Data Analysis) for manuscript prepara- 
tion. 
C12. Participants: Recruitment, Retention, and Data Collection Procedures 
C12a. Clinicians 
Recruitment. Participants will be recruited across 12 sites (Clinicians: N=187; target: N=150) 
(Letter of Support; CEO Guth). The number of sites and clinicians has been determined by a simulation- 
style power analysis to detect a small effect size predicted by previous research (C13. Data Analysis 
Plan). The main sites have been identified, but not enrolled (Letters of Support from regional clinic 
directors). As there are at least 16 sites from which to select (easily accessible from primary site) and 
at least 250 eligible clinicians across these sites, we are confident that we can obtain the targeted 
number of both sites and clinicians. Said differently, even if a site administrator enrolls a site, not all 
clinicians will need to participate, an important factor to avoid any perception of coercion. Given site 
diversity, the large number of eligible sites will maximize the likelihood that results will generalize to the 
broader population of community mental health centers. Sites range in size from 12 to 52 clinicians. 
Within each enrolled site, clinicians will have the option to participate if they (a) are at least 80% full- 
time equivalent (b) provide individual psychotherapy to (c) adults with depression (d) in English. These 
inclusion criteria reflect over 95% of clinicians at the eligible sites. Based on our pilot study, we have 
experienced that even those who are not supportive of MBC have agreed to participate, therefore mak- 
ing it unlikely that self-selection to participate would limit the generalizability of our sample. During 
Phase 1 (months 5-8; Table 4), clinician enrollment will begin across all sites and site randomization 
will occur as site level randomization improves project feasibility and reduces contamination confounds. 
This decision to randomize by site was also influenced by literature indicating the impact of organiza- 
tional culture and climate on the implementation process (Aarons et al., 2012; Damschroder et al., 
2009), which will be assessed at baseline (AT1). 
Data Collection. See Table 5: Study Measures, Section II for concise descriptions of the clini- 
cian battery. Each contextual factor of diffusion (Mendel et al., 2008) will be assessed via self-report 
(and qualitatively via focus groups). MBC fidelity is the main clinician level implementation outcome. All 
clinician measures will be administered across three time points for each cohort with respect to their 
time since starting implementation: 1) Phase 1: Baseline Needs Assessment (AT1), prior to MBC im- 
plementation; 2) Phase 2: Implementation/Process Evaluation (AT2, 5 months in), following the active 
implementation phase; and, 3) Phase 3: Outcome/Impact Evaluation (AT3, 15 months in) 10 months 
after the research personnel have exited. Barriers to implementation will be collected for the first month 
of implementation via the electronic health record. 
Table 5. Study Measures 
Domain 
Measures & Indicators 
Interval 
I. Client Measures: Effectiveness Outcomes 
Depression 
Severity 
Patient Health Questionnaire-9 (PHQ-9; Kroenke, Spitzer, & Williams, 2001). The PHQ-9 is a 9 item self-report that as- 
sesses depression severity and has demonstrated good internal consistency (α=.85-.89) and sensitivity to change (Löwe, 
Kroenke, Herzog, & Gräfe, 2004). 
All Ses- 
sions 
 II. Clinician Measures: Contextual Factors of the Diffusion Process (Putative Predictors, Moderators, & Mediators) 
Demographics 
Developed by Lewis & Simons (2011) to assess clinician demographic information and training background. 
BL 
Norms 
A measure of subjective norms will be developed (3 items; Francis et al., 2004) in proposal Phase I and used to assess 
normative behavior per the theory of planned behavior measurement development manual. 
BL, AT2, 
AT3 
Attitudes 
Attitudes Towards Standardized Assessment (ASA; Jensen-Doss & Hawley, 2010) is a 22-item self-report that assesses 
attitudes toward use of standardized assessments (e.g., PHQ-9). The ASA has good internal consistency (α=.72-.75) for 
each subscale (Benefit Over Clinical Judgment, Psychometric Quality, Practicality) and good structural validity. 
BL, AT2, 
AT3 
Culture/Climate 
Organizational Culture Scale (Glaser, Zamanou, & Hacker, 1987) is a 31-item self-report that assesses organizational 
culture using a 5-point Likert scale for 6 factors: teamwork, morale, information flow, involvement, supervision, and meet- 
ings. The OCS has demonstrated strong inter-item reliability (α=.98) among the subscales. 
BL, AT2, 
AT3 
11 

Structure/ Pro- 
cess Policies/ 
Incentives 
Infrastructure Survey (Keough, Comtois, Lewis, & Landes, 2013) is a 30-item self-report that assesses the impact of infra- 
structure (e.g., documentation, performance evaluation, productivity requirements) of clinical settings on the implementa- 
tion and sustainment of empirically supported treatments. Psychometrics will be available Fall 2014. 
BL, AT2, 
AT3 
Resources 
Organizational Resources Scale (ORS; Salanova, Agut, & Peiró, 2005). is an 11-item self report that assesses organiza- 
tional resources across subscales of Training, Autonomy, and Technology and has good internal consistency (α=.84-.91). 
BL, AT2, 
AT3 
Networks & 
Linkages 
We will map the network within each clinic and calculate: the number of links connecting network members (density), and 
the number of direct connections to and from the opinion leader (centrality). 
BL, AT2, 
AT3 
Media & 
Change Agents 
Opinion Leadership Scale (OLS; Childers, 1986). The OLS is a 6-item opinion leader self-identification scale that will be 
employed to identify clinician opinion leaders who may serve as change agents. The OLS has demonstrated good inter- 
nal consistency (α=.83). 
BL, AT2, 
AT3 
  III. Clinician Measures: Implementation Outcomes 
MBC Fidelity 
1. If clients complete the PHQ-9, it will be captured in the EHR (0=No, 1=Yes). 2. The EHR will reveal whether clinicians 
reviewed scores (0=No, 1=Yes). 3. Clinicians will self-report if they discussed scores in session (0=No, 1=Yes). Clients 
will respond via text to indicate whether clinician initiated score discussion in session. Reasons for MBC fidelity deviation 
will be captured via clinician report on progress notes. 
BL, AT2, 
AT3 
Note 
BL(AT1) = Baseline(Assessment Time 1), AT2 = Assessment Time 2, AT3 = Assessment Time 3. 
Retention. The total time required for clinician study participation is 15 months; however, the 
first two assessments are obtained within the first 5 months. The average length of employment for a 
clinician at Centerstone is 6 years with an average of 11.72% attrition last year. We anticipate stable 
clinician participation for the granting period. Fortunately, literature suggests that participatory ap- 
proaches to organizational change in practice patterns result in lower clinician attrition (Minkler & Sal- 
vatore, 2012). Even so, we will recruit 1.25 times more clinicians (N=187) to accommodate attrition and 
achieve the target (N=150). Because of the dynamic waitlist design, we will revisit enrollment in Phase 
2, 3 months prior to each site’s planned implementation. Finally, the design can handle attrition given 
the within- and between-subjects analyses and multiple assessment points (3 ATs), as can the anal- 
yses (multilevel modeling with multiple imputation, described below). 
 C12b. Depressed Adult Clients 
Recruitment. Adult clients (N=625; target N=500) seeking treatment for depression at Center- 
stone will be eligible for enrollment in the research study. As this is a pragmatic trial (Thorpe et al., 
2009) client participation criteria include: (a) age 18 and above; (b) depression is one of the primary 
treatment foci based on diagnosis made by clinicians using usual care interview methods to reflect 
major depressive disorder, dysthymic disorder, depressive disorder NOS, adjustment disorder with de- 
pressed mood; (c) significant depressive symptom severity (PHQ-9 total score > 9); (d) receipt of indi- 
vidual psychotherapy; (e) fluency in English; and, (f) new client beginning treatment (g) with an enrolled 
study clinician during the proposed funding period. Exclusion criterion is minimal: an inability to sign the 
consent due to lack of competence or inability to read. Intake clinicians will be trained to identify eligible 
clients and assess if they would be interested in hearing more information about a research study. If 
the client agrees, the clinician will share the client’s contact information with the project coordinator. 
Because Centerstone has a strong research commitment, clinicians and clients are accustomed to a 
research environment where the option to participate in studies is more routine than in many community 
mental health centers. Both Centerstone clinicians and clients have participated in a number of re- 
search studies and recruitment is expected to be very straightforward. However, to supplement this 
approach, Co-I (Ayer, Director of Centerstone Research Institute) will run weekly queries of the Cen- 
terstone database (in accordance with Health and Human Services guidance) to identify eligible clients 
(see Human Subjects for previously used HIPAA compliant procedures) who will be contacted by our 
project coordinator to assess interest in study participation. Once a client has been enrolled, the study 
team will flag the client in the electronic health record to initiate study procedures for the clinician. Given 
the number of clients seeking treatment for depression at Centerstone (> 30,000/yr), we anticipate that 
our client recruitment goal will be feasible. 
Data Collection. The goal of client data collection is to determine if routine measurement (in the 
standardized condition) improves depressive symptom severity and whether adaptations to the MBC 
assessment schedule (tailored condition) have differential effects on depression outcomes. Clients will 
complete the PHQ-9 on a tablet kiosk in the waiting room according to site-generated guidelines; this 
information will be automatically fed into the electronic health record for clinicians to review in session. 
Given the potential for variable PHQ-9 completion in the tailored condition (based on the site-generated 
guideline), we will also have clients enrolled in both conditions complete the PHQ-9 at baseline (imme- 
diately post consent) and at week 12 of treatment (via automated phone survey). The 12-week window 

reflects a commonly used time period in randomized clinical trials during which depressive symptoms 
are expected to remit. We will also supplement data collection with client 5-axis diagnoses and employ- 
ment status captured in intake and progress review reports. 
Retention. At recruitment, clients will complete a consent form and provide options for multiple 
contact methods (e.g. multiple phone numbers, address). Centerstone has consistently achieved 
greater than 75% client retention in federally-funded randomized trials (CATIE, Keefe et al., 2007; 
STAR*D, Insel, 2006). We will replicate previous efforts to maximize retention in the proposed study. 
In order to have sufficient power to detect statistical significance and account for client attrition, we 
intend to recruit 1.25 more client participants (N=625) to assure that we achieve our enrollment goal 
(N=500). 
C13. Data Analysis Plan: C13a. Quantitative Data Analysis Plan 
Data screening. Via frequency distributions and scatterplots, variables will be examined for un- 
usual distributions, out-of-range, and extreme values. We will confirm randomization using chi-square 
tests, t-tests, and the Kruskal-Wallis nonparametric test. We will test assumptions and underlying sta- 
tistical models during model fitting. For generalized linear models and multilevel models, we will exam- 
ine the homogeneity of error, normality of residuals, linearity of relationships between independent and 
dependent variables, and outliers. For multilevel models, we will assess multivariate normality of ran- 
dom effects. 
Missing data. All analyses will adhere to the intention-to-treat approach, which includes all par- 
ticipants assigned to their condition regardless of study completion (Wells, 1999). Multiple imputation 
(MI), an optimal technique for missing data (Graham, 2009), will include all participants and available 
data. For all MI analyses, 20 data sets containing plausible values for missing data will be constructed 
using the Amelia II package for R, which incorporates both cross-sectional and longitudinal information 
in data imputation (Honaker, King, Blackwell, 2012). Each data set will be analyzed separately and 
model parameters combined for inferential tests (Rubin, 1987). The missing at random assumption will 
be assessed using a sensitivity analysis in which a series of pattern-mixture models that adjust param- 
eter estimates for missing data patterns (Hedeker & Gibbons, 1997), will be compared with basic mod- 
els to identify any potentially non-ignorable missing data patterns. 
Aim 1: Hypotheses: Tailored implementation will outperform standardized in terms of (H1a) 
fidelity. MBC clinician fidelity was evaluated with two variables: (1) whether the PHQ-9 was recorded in 
the EHR, and (2) among encounters for which the PHQ-9 was recorded, whether it was discussed. 
Both outcomes were modeled using generalized linear mixed models with a logit link function for a 
binary distribution using the lme4 package. All models included random intercepts for clinicians and 
patients. Models for both outcomes were constructed in an identical sequence. Models comparisons in 
the model-building sequence described below were evaluated using the Bayesian Information Criterion 
(BIC); the model with the better BIC was selected for presentation. Several unconditional growth mod- 
els (i.e., models with only parameters for time) were fit (i.e., linear, quadratic, log-transformed number 
of months). After establishing the unconditional growth model, additional fixed effects were added in 
the following sequence: (1) group main effect and (2) group x time interaction. Finally, new versus 
existing patient status was investigated as a main effect using a dummy variable for new patient status 
and as a patient status x group interaction. 
Aim 2: Hypotheses: Contextual mediators will be leveraged in the tailored condition, but 
serve as barriers in the standardized condition. Mediation models were fit to evaluate which contextual 
factors (based on clinician-completed surveys, Table 5; and, clinician selection of drop-down menu 
reasons for MBC deviations captured in the electronic health record) mediate the impact of the imple- 
mentation condition on both clinician- and client-level outcomes. The two critical effects underlying me- 
diation were evaluated in separate models: treatment condition predicts the mediator (path a) and the 
mediator predicts the outcome (path b). Specifically, mediators were assessed at the conclusion of the 
implementation window and the mediators were used to predict PHQ9 completion in the sustainment 
window. The a path was assessed using linear mixed models with Satterthwaithe degrees of freedom 
13 

implemented using the R lmerTest package with a random intercept for clinic. The b path was assessed 
using generalized linear mixed models with a logit link function for a binary distribution using the lme4 
package. Aim 3 Hypotheses: Adapted MBC protocols (i.e., tailored condition) will outperform routine 
weekly administration of PHQ-9s (i.e., standardized condition) with respect to clinically significant 
change in depression severity from intake to week 12. Depression severity was analyzed using linear 
mixed models with Satterthwaithe degrees of freedom implemented using the R lmerTest package. 
Multiple imputation was used to estimate missing values across 20 datasets in which 22.8% of patients 
did not complete the 12-week interview. We first assessed whether there was a change in PHQ-9 be- 
tween intake and week 12 in a mixed model containing patient status as a covariate and random inter- 
cepts for patient and clinic. Next, implementation group was added to the model. In addition, number 
of sessions, number of sessions that the PHQ-9 was recorded, and number of sessions that the PHQ- 
9 was discussed were added to the model. We also examined the interactions between new patient 
status and the focal variables. Power calculations. A power analysis for the multilevel models for Aim 
1 was assessed with Monte Carlo studies in which power is the proportion of significant effects (2-tailed 
α, p < .05) for parameters of interest observed over repeated analyses of simulated data (Muthén & 
Muthén, 2002; Thoemmes, Reiser, & MacKinnon, 2010) using MPlus (version 7). For each model, 
10,000 data sets were simulated and analyzed. Data in the Monte Carlo studies were simulated with 
the goal of identifying the smallest detectable effect size with power > 80. Dropout was simulated to 
reflect an increase of 5% missing data per wave. Repeated measurements were nested within partici- 
pants (ICC = .50) and participants were nested in sites (ICC = .05). Average effect sizes for Aim 1 
analyses were computed using an approximation of Cohen’s d for growth models (Feingold, 2009). 
Using this metric, we are sufficiently powered to detect effect sizes as small as d = .46 for the clinician 
models (N = 150) and effect sizes as small as d = .30 for the client models (N = 500). For Aim 2, we 
conducted power analyses for the hypothesized mediation effects with the same assumptions de- 
scribed above but with the use of effective samples size estimates (an ICC corrected sample size) 
based on a design effect adjustment (Bickel, 2007) in order to use standard effect size metrics. The κ2 
effect size for mediation (.01, .09, and .25 represent small, medium, and large κ2 respectively) was 
computed (Preacher & Kelly, 2011). We are sufficiently powered to detect effect sizes as small as κ2 = 
.16 for the clinician models and effect sizes as small as κ2 = .10 for the client models. For Aim 3 anal- 
yses, we are sufficiently powered to detect effect sizes as small as r = .27 for clinicians and as small 
as r = .21 for clients. The planned sample sizes are consistently sufficient for detecting medium effect 
sizes for clinician outcomes and small effect sizes for client outcomes, with the power to treat site as a 
random effect based 6 sites per condition (Atkins & Baldwin, 2013). 
C13b. Qualitative and Mixed Method Data Analysis Plan 
Overview. Mixed methods will be used to integrate findings from Aim 2 using a quantitative + 
qualitative structure (wherein both types of data are collected simultaneously) to achieve the function 
of data expansion for the purposes of evaluation and elaboration (Palinkas et al., 2011). We will use a 
connecting process (whereby the datasets build upon one another) and work closely with our mixed 
methods expert (Rutkowski). Rapid ethnography (Millen, 2000) will be used to synthesize the needs 
assessment data only in the tailored condition to characterize participant experiences. Focus groups 
will be analyzed separately to characterize participant responses within each site, across conditions. 
Graduate student researcher (Scott), who was trained in qualitative inquiry methods during the pilot 
studies, will work with the postdoctoral fellow to train research assistants to identify and code analyza- 
ble units of meaning in the focus group transcripts. An iterative approach to coding will resolve disa- 
greements through research team discussion. Inductive analyses based on emergent themes rooted 
in grounded theory will be conducted (using QSR N-Vivo software). Codes will also be assigned based 
on contextual factors using Mendel et al.’s (2008) Framework of Dissemination. The final list of con- 
sensus codes will include themes established a priori and through emergent themes analysis. 
Using the U.S. NIH guidelines for mixed methods best practices (Creswell, et al., 2011), we will 
connect the quantitative and qualitative datasets in QSR N-Vivo to allow for case-specific pattern iden- 
tification and hypothesis testing. For example, we will enter clinician-specific MBC fidelity data 

(categorized as “none”, “low”, “moderate”, “high”) and query each qualitative theme for matched clini- 
cian focus group quotes in order to investigate the influence of contextual factors on level of MBC 
fidelity. Based on pilot study qualitative findings, we anticipate that focus groups will yield both positive 
and negative valenced statements for each of the a priori and emergent themes. We will investigate 
the extent to which differences exist in the valence of contextual themes between conditions. This ap- 
proach will allow us to distinguish factors that might explain the differences in the quantitative findings, 
and notably MBC fidelity. 
C14. Potential Problems & Alternative Strategies.  
Our hypothesis is that tailored implementation will outperform the standardized approach. However, it 
is possible that this hypothesis will not be supported. Findings of this nature would not be undesirable, 
but rather would illustrate that while emerging research suggests the need to contextualize implemen- 
tation interventions (Fixsen, 2005; Wallerstein & Duran, 2010), this customization compromises ulti- 
mate outcomes—a critical realization for the field of implementation science. Furthermore, if we are 
unable to detect mediators of implementation and effectiveness outcomes included in this proposal, it 
may suggest the need to re-evaluate the Framework of Dissemination. Specifically, putative mediators 
of the dissemination and implementation process not included in this model might require empirical 
investigation. Our qualitative analyses will allow for careful examination of unanticipated mediators. At 
the level of the clinician, we anticipate substantial variability in competency and general approach to 
psychotherapy. However, we are not concerned about the effects of this variability on study outcomes 
because (1) randomization should result in equivalent variability across conditions and (2) MBC is con- 
ceptualized as transtheoretically relevant, regardless of the therapist’s orientation or training. 
C15. Justification & Feasibility. 
We selected an effectiveness-implementation hybrid to investigate one co-primary aim that fo- 
cuses on (1) implementation outcomes at the clinician level and (2) effectiveness with respect to client 
outcomes. To test the Dynamic Sustainability Framework (Chambers et al., 2013), we will need to 
compare the clinical effectiveness of the adapted use of MBC (tailored) as compared to the empirically 
supported MBC approach (standardized). Simultaneously, because no studies have attempted to scale 
up MBC in community mental health, we need to evaluate the implementation (clinician-level: fidelity) 
outcomes. The effectiveness-implementation hybrid design type 2 was identified as the optimal and 
innovative design solution for this proposal. 
Within the effectiveness-implementation hybrid design, we will randomize sites to condition using 
a dynamic waitlist controlled approach. Half of the sites will receive a standardized approach to imple- 
mentation and the other half will receive a tailored approach. This design reduces the effect of man- 
agement- or clinician-level readiness or client factors that might otherwise confound the training effects 
allowing for unbiased data (Brown, et al., 2006). This design also affords the opportunity to conduct 
blocking on smaller time units and statistically provides large gains in efficiency (range from 33 to 100% 
gains). Moreover, training efficiency is achieved as not all sites must be trained at once, therefore al- 
lowing the same trainer(s) to train all sites to reduce any possible effect at the level of trainer expertise 
(Herschell, Kolko, Baumann, & Davis, 2010). 
We have already piloted many of the implementation strategies and the evaluation approach 
(C8: Pilot Studies). Moreover, Centerstone has successfully completed >150 studies since 2003; two 
of these studies were NIMH-sponsored (CATIE, Keefe et al., 2007; STAR*D, Insel, 2006), whereas 
others involved wide-scale implementation of EBPs for SAMHSA-sponsored program evaluations. Co- 
I (Ayer) has coordinated many of these multi-site clinical trials at Centerstone. This proposal’s design 
considerations and the established strong collaborative relationship of our investigative team, we have 
no concerns regarding this proposal’s feasibility. 
C 16. Implications and Future Directions 
Few experimental implementation studies have been conducted. Findings from this proposal have the 
potential to: (1) yield a robust theoretical and practical model delineating contextual factors and 
15 

evaluative processes for implementation in an academic-community partnership; (2) demarcate the 
benefit of using standardized versus tailored protocols for implementation; (3) establish a blended pro- 
tocol for MBC implementation with capacity for generalization to community mental health centers na- 
tionwide; and, (4) highlight contextual factors responsible for sustained MBC implementation. With de- 
pression maintaining its place among the nation’s top chronic illnesses, costing billions of dollars annu- 
ally, positive findings from this proposal will present a feasible and effective approach to alleviating 
great societal burden through a minimal intervention needed for change. This study will lay the ground- 
work for subsequent research (by establishing progress monitoring) to enhance usual care for multi- 
problem clinical presentations, should MBC effects plateau and additional interventions be necessary. 

References 
Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evi- 
dence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61-74. 
Aarons, G. A., Glisson, C., Green, P. D., Hoagwood, K., Kelleher, K. J., & Landsverk, J. A. (2012). The organi- 
zational social context of mental health services and clinician attitudes toward evidence-based practice: 
a United States national study. Implementation Science, 7, 56. 
Aarons, G. A., Miller, E. A., Green, A. E., Perrott, J. A., & Bradway, R. (2012). Adaptation happens: a qualita- 
tive case study of implementation of The Incredible Years evidence-based parent training programme in a 
residential substance abuse treatment programme. Journal of Children’s Services, 7(4), 233–245. 
doi:10.1108/17466661211286463 
Ajzen, I. (2002). Constructing a theory of planned behavior questionnaire. Unpublished manuscript. Retrieved 
from http://chuang.epage.au.edu.tw/ezfiles/168/1168/attach/20/pta_41176_7688352_57138.pdf 
American Psychiatric Association (2000). Diagnostic and Statistical Manual of Mental Disorders (4th ed., text 
revision) (DSM-IV-TR) (2nd ed.). Washington: American Psychiatric Association. 
Aquilino, W. S. (1994). Interview mode effects in surveys of drug and alcohol use: A field experiment. The Pub- 
lic Opinion Quarterly, 58(2), 210-240. doi: 10.2307/2749537 
Atkins, D., & Baldwin, S. (2013). Design and analysis challenges with multilevel implementation data. Pre- 
sented at the Seattle Implementation Research Conference, Seattle, WA. 
Baron, R. M., & Kenny, D. A. (1986). The moderator mediator variable distinction in social psychological-re- 
search - conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychol- 
ogy, 51(6), 1173-1182. doi: 10.1037/0022-3514.51.6.1173 
Bickel, R. (2007). Multilevel analysis for applied research: It's just regression! New York: Guilford Publications. 
Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. 
Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114-1119. doi: 
10.1097/CHI.0b013e3181825af8 
Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to 
clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatric Services, 
62(12), 1423-1429. 
Bosch, M., van der Weijden, T., Wensing, M., & Grol, R. (2007). Tailoring quality improvement interventions to 
identified barriers: a multiple case analysis. Journal of Evaluation in Clinical Practice, 13(2), 161-168. 
doi: 10.1111/j.1365-2753.2006.00660.x 
Brown, C. H., Wyman, P. A., Guo, J., & Pena, J. (2006). Dynamic wait-listed designs for randomized trials: new 
designs for prevention of youth suicide. Clinical Trials, 3(3), 259-271. 
Chambers, D., Glasgow, R., & Stange, K. (2013). The dynamic sustainability framework: addressing the 
paradox of sustainment amid ongoing change. Implement Sci, 8(1), 117. 
Chambers, D. A., Pringle, B., & Juliano-Bult, D. (2012). Connecting science and practice in child and adoles- 
cent mental health services research. Administration and Policy in Mental Health and Mental Health 
Services Research, 39(4), 321-326. doi: 10.1007/s10488-011-0399-z 
Childers, T. L. (1986). Assessment of the psychometric properties of an opinion leadership scale. Journal of 
Marketing Research, 23(2), 184-188. doi: 10.2307/3151666 
Chorpita, B. F., Bernstein, A., Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using infor- 
mation resources to structure the decision models in service organizations. Administration and Policy in 
Mental Health and Mental Health Services Research, 35(1-2), 114-123. doi: 10.1007/s10488-007-0151- 
x 
Clark, D. M., Layard, R., Smithies, R., Richards, D. A., Suckling, R., & Wright, B. (2009). Improving access to 
psychological therapy: Initial evaluation of two UK demonstration sites. Behaviour Research and 
 
17 

Therapy, 47(11), 910-920. doi: 10.1016/j.brat.2009.07.010 
Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Clegg Smith, K. (2011). Best practices for mixed methods 
research in the health sciences. Office of Behavioral and Social Sciences Research (OBSSR). 
Crits-Cristoph, P., Ring-Kurtz, S., Hamilton, J.L., Lambert, M.J., Gallop, R., McClure, B . . . Rotrosen, J. (2012). 
A preliminary study of the effects of individual patient-level feedback in outpatient substance abuse 
treatment programs. Journal of Substance Abuse Treatment, 42(3), 301-309. doi: 
10.1016/j.jsat.2011.09.003 
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering 
implementation of health services research findings into practice: a consolidated framework for advanc- 
ing implementation science. Implementation Science, 4, 50. 
Davis, J. R., Rawana, E. P., & Capponi, D. R. (1989). Acceptability of behavioral staff management tech- 
niques. Behavioral Interventions, 4(1), 23-44. 
De Beurs, E., den Hollander‐Gijsman, M. E., Van Rood, Y. R., Van der Wee, N. J. A., Giltay, E. J., Van Noor- 
den, M. S., ... & Zitman, F. G. (2011). Routine outcome monitoring in the Netherlands: practical experiences 
with a web‐based strategy for the assessment of treatment outcome in clinical practice. Clinical psychology & 
psychotherapy, 18(1), 1-12. 
Dowrick, C., Leydon, G. M., McBride, A., Howe, A., Burgess, H., Clarke, P., . . . Kendrick, T. (2009). Patients' 
and doctors' views on depression severity questionnaires incentivised in UK quality and outcomes 
framework: qualitative study. British Medical Journal, 338. 
Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implementation Science, 1(1), 1. 
Eisen, S. V., Dickey, B., & Sederer, L. I. (2000). A self-report symptom and problem rating scale to increase 
inpatients' involvement in treatment. Psychiatric Services, 51(3), 349-353. 
Evans, E., & Hser, Y. (2004). Pilot-testing a statewide outcome monitoring system: Overview of the California 
Treatment Outcome Project (CALTOP). Journal of Psychoactive Drugs, 36(2), 109-114. 
Feingold, A. (2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as 
for classical analysis. Psychological Methods, 14(1), 43-53. doi: 10.1037/a0014699 
Foley, P., Levine, E., Askew, S., Puleo, E., Whiteley, J., Batch, B., … Bennett, G. (2012). Weight gain pre- 
vention among black women in the rural community health center setting: The Shape Program. BMC Public 
Health, 12(1), 305. doi:10.1186/1471-2458-12-305 
First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1997). Structured Clinical Interview for DSM-IV-TR 
axis I disorders: Research Version, Patient Edition. (SCIF-I/P) New York: Biometrics Research, New 
York State Psychiatric Institute. 
Fixsen, D. L. (2005). Implementation research: A synthesis of the literature: National Implementation Research 
Network. 
Francis, J. J., Eccles, M. P., Johnston, M., Walker, A. E., Grimshaw, J. M., Foy, R., . . . Bonetti, D. (2004). Construct- 
ing questionnaires based on the theory of planned behaviour: A Manual for Health Services Researchers. Cen- 
tre of Health Services Research, University of Newcastle upon Tyne Newcastle upon Tyne. 
Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by 
investigating usual mental health care. Administration and Policy in Mental Health and Mental Health 
Services Research, 37(1-2), 15-26. 
Garland, A. F., Kruse, M., & Aarons, G. A. (2003). Clinicians and outcome measurement: What's the use? 
Journal of Behavioral Health Services & Research, 30(4), 393-405. 
Glaser, S. R., Zamanou, S., & Hacker, K. (1987). Measuring and interpreting organizational culture. Manage- 
ment Communication Quarterly, 1(2), 173–198. 
Glasgow, R. E., Fisher, L., Strycker, L. A., Hessler, D., Toobert, D. J., King, D. K., & Jacobs, T. (2013). Minimal 
intervention needed for change: definition, use, and value for improving health and health research. 
Translational Behavioral Medicine, 1-8. 

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 
60, 549-576. 
Greenberg, P. E., Kessler, R. C., Birnbaum, H. G., Leong, S. A., Lowe, S. W., Berglund, P. A., & Corey-Lisle, 
P. K. (2003). The economic burden of depression in the United States: how did it change between 1990 
and 2000?. Journal of clinical psychiatry, 64(12), 1465-1475. 
Hamilton, M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry, 23(1), 
56. 
Harding, K. J., Rush, A. J., Arbuckle, M., Trivedi, M. H., & Pincus, H. A. (2011). Measurement-based care in 
psychiatric practice: a policy framework for implementation. The Journal of clinical psychiatry, 72(8), 
1136-1143. 
Hedeker, D., & Gibbons, R. D. (1997). Application of random-effects pattern-mixture models for missing data in 
longitudinal studies. Psychological Methods, 2(1), 64-78. doi: 10.1037/1082-989x.2.1.64 
Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the imple- 
mentation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology 
Review, 30(4), 448-466. doi: 10.1016/j.cpr.2010.02.005 
Honaker, J., King, G., & Blackwell, M. (2012). Amelia (Version 1.6.1) [Computer program and manual]. Re- 
trieved from http://cran.r-project.org/web/packages/Amelia/index.html. 
Insel, T. R. (2006). Beyond efficacy: the STAR* D trial. American Journal of Psychiatry, 163(1), 5-7. 
Jacobson, N. S., & Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change 
in psychotherapy research. Journal of consulting and clinical psychology, 59(1), 12. 
Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to Evidence-Based Assessment: Clinician 
attitudes toward standardized assessment tools. Journal of Clinical Child and Adolescent Psychology, 
39(6), 885-896. 
Judge, T. A., Thoresen, C. J., Pucik, V., & Welbourne, T. M. (1999). Managerial coping with organizational 
change: A dispositional perspective. Journal of Applied Psychology, 84(1), 107-122. doi: 
10.1037//0021-9010.84.1.107 
Keefe, R. S., Bilder, R. M., Davis, S. M., Harvey, P. D., Palmer, B. W., Gold, J. M., ... & Lieberman, J. A. 
(2007). Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the 
CATIE Trial. Archives of general psychiatry, 64(6), 633-647. 
Keough, M., Comtois, K. A., Lewis, C. C., & Landes, S. J. (2013). The Infrastructure Survey. Retrieved from 
mkeough@u.washington.edu. 
Koerner, K., Dailey, D., Lipp, M., Connor, H., & Sharma, R. (2012). Data Visualization for Psychotherapy Pro- 
gress Tracking. Paper presented at the Association for Computing Machinery Special Interest Group on 
Design of Communication, Seattle, WA. 
Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002). Mediators and moderators of treatment 
effects in randomized clinical trials. Archives of General Psychiatry, 59(10), 877-883. doi: 10.1001/arch- 
psyc.59.10.877 
Kroenke, K., & Spitzer, R. L. (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiat- 
ric Annals, 32(9), 509-515. 
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9 - Validity of a brief depression severity 
measure. Journal of General Internal Medicine, 16(9), 606-613. 
Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010). The patient health questionnaire somatic, 
anxiety, and depressive symptom scales: a systematic review. General Hospital Psychiatry, 32(4), 345- 
359. 
Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to 
19 

psychotherapists on their patients' progress: Clinical results and practice suggestions. Journal of Clini- 
cal Psychology, 61(2), 165-174. 
Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it 
time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology - Science 
and Practice, 10(3), 288-301. 
Lewis, C. C., Scott, K., Marti, N. & Ayer, D. (under review). Clinical pathways for depression: preliminary re- 
sults from a needs assessment and implications for implementation. 
Lewis, C. C., & Simons, A. D. (2011). A pilot study disseminating Cognitive Behavioral Therapy for Depression: 
Therapist factors and perceptions of barriers to implementation. Administration and Policy in Mental 
Health and Mental Health Services Research, 38(4), 324-334. doi: 10.1007/s10488-011-0348-x 
Löwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004). Measuring depression outcome with a brief self-report 
instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9). Journal of Affective Dis- 
orders, 81(1), 61-66. 
MacKinnon, D. P. (2008). Introduction to Statistical Mediation Analysis. New York, NY: Lawrence Erlbaum As- 
sociates. 
Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational en- 
hancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy 
Research, 19(2), 181-193. doi: 10.1080/10503300802688460 
Mendel, P., Meredith, L. S., Schoenbaum, M., Sherbourne, C. D., & Wells, K. B. (2008). Interventions in organ- 
izational and community context: A framework for building evidence on dissemination and implementa- 
tion in health services research. Administration and Policy in Mental Health and Mental Health Services 
Research, 35(1-2), 21-37. 
Millen, D. R. (2000). Rapid ethnography: time deepening strategies for HCI field research. Presented at the 
Designing interactive systems: processes, practices, methods, and techniques, New York, NY. 
Miller, A. L., Krusky, A. M., Franzen, S., Cochran, S., & Zimmerman, M. A. (2012). Partnering to translate evi- 
dence-based programs to community settings: bridging the gap between research and practice. Health 
Promotion Practice, 13(4), 559-566. doi: 10.1177/1524839912438749 
Minkler, M., & Salvatore, A. L. (2012). Participatory approaches for study design and analysis in dissemination 
and implementation research. Dissemination and Implementation Research in Health: Translating Sci- 
ence to Practice, 192. 
Morris, D., & Trivedi, M. (2011). Measurement-based care for unipolar depression. Current Psychiatry Reports, 
13, 446-458. 
Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus User's Guide (7th ed.). Los Angeles: Muthen & Muthen. 
Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed 
method designs in implementation research. Administration and Policy in Mental Health and Mental 
Health Services Research, 38(1), 44-53. 
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2013). Purposeful 
Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. 
Administration and Policy in Mental Health and Mental Health Services Research, 1-12. 
Persons, J. B., Koerner, K., Eidelman, P., Thomas, C., & Liu, H. (2012). We Reduced Barriers and Increased 
Practitioners’ Implementation of Progress Monitoring. [Powerpoint Slides]. 
Powell, J., & Buchan, I. (2005). Electronic health records should support clinical research. Journal of Medical 
Internet Research, 7(1). 
Powell, B.J., McMillen, J.C., Proctor, E.K., Carpenter, C.R., Griffey, R.T., Bunger, A.C., . . . York, J. L. (2012). 
A compilation of strategies for implementing clinical innovations in health and mental health. Medical 
Care Research and Review, 69(2), 123-157. 
Preacher, K.J., & Hayes, A.F. (2008). Asymptotic and resampling strategies for assessing and comparing 
indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879-891. doi: 

10.3758/brm.40.3.879 
Preacher, K.J., & Kelley, K. (2011). Effect size measures for mediation models: Quantitative strategies for com- 
municating indirect effects. Psychological Methods, 16(2), 93-115. doi: 10.1037/a0022658 
Preacher, K.J., Zyphur, M.J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel 
mediation. Psychological Methods, 15(3), 209-233. doi: 10.1037/a0020141 
Proctor, E.K., Powell, B.J., Baumann, A. A., Hamilton, A. M., & Santens, R. L. (2012). Writing implementation 
research grant proposals: ten key ingredients. Implementation Science, 7, 96. doi: 10.1186/1748-5908- 
7-96 
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: recommendations for speci- 
fying and reporting. Implementation Science, 8(1), 139. 
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods 
(Vol. 1): SAGE Publications, Incorporated. 
Rock, D., Combrinck, J., & Groves, A. (2001). Issues associated with the implementation of routine outcome 
measures in public mental health services. Australasian Psychiatry, 9(1), 43-46. 
Rubin, D. B. (1987). Multiple Imputation for nonresponse in surveys. New York, NY: John Wiley & Sons, Inc. 
Salanova, M., Agut, S., & Peiro, J. M. (2005). Linking organizational resources and work engagement to em- 
ployee performance and customer loyalty: The mediation of service climate. Journal of Applied Psy- 
chology, 90(6), 1217-1227. doi: 10.1037/0021-9010.90.6.1217 
Sapyta, J., Riemer, M., & Bickman, L. (2005). Feedback to clinicians: Theory, research, and practice. Journal 
of Clinical Psychology, 61(2), 145-153. 
Scott, K., & Lewis, C. C. (in prep). A qualitative analysis of contextual factors influencing implementation of 
measurement based care. 
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of 
treatment failure: Meta-analytic and mega-analytic review of a Psychotherapy quality assurance sys- 
tem. Journal of Consulting and Clinical Psychology, 78(3), 298-311. doi: 10.1037/a0019247 
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: modeling change and event occur- 
rence: modeling change and event occurrence: Oxford University Press, USA. 
Steckler, A., Goodman, R. M., McLeroy, K. R., Davis, S., & Koch, G. (1992). Measuring the diffusion of innova- 
tive health promotion programs. American Journal of Health Promotion, 6(3), 214-224. 
Sundaravej, T. (2008). Empirical validation of unified theory of acceptance and use of technology model. Jour- 
nal of Global Information Technology Management, 13(1), 5-27. 
Teruya, C., Hardy, M., Hser, Y., & Evans, E. (2006). Implementation of a statewide outcome monitoring sys- 
tem: Lessons learned from substance abuse treatment provider staff. Qualitative Health Research, 
16(3), 337-352. 
Thoemmes, G., Resier, M. R., & MacKinnon, D. P. (2010). Power for complex meditational designs using 
Monte Carlo Methods. Structural Equation Modeling, 17(3), 510-534. 
Thorpe, K. E., Zwarenstein, M., Oxman, A. D., Treweek, S., Furberg, C. D., Altman, D. G., ... & Chalkidou, K. 
(2009). A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial 
de- 
signers. Journal of clinical epidemiology, 62(5), 464-475. 
Trent, L. R. (2007). Development of a Measure of Disseminability (MOD). (M.A.), University of Mississippi. 
Trivedi, M. H., & Daly, E. J. (2007). Measurement-based care for refractory depression: A clinical decision sup- 
port model for clinical research and practice. Drug and Alcohol Dependence, 88(2), S61-S71. 
Trivedi, M. H., Rush, A. J., Wisniewski, S. R., Nierenberg, A. A., Warden, D., Ritz, L., . . . Team, S. S. (2006). 
Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: Im- 
plications for clinical practice. American Journal of Psychiatry, 163(1), 28-40. doi: 
10.1176/appi.ajp.163.1.28 
21 

Valenstein, M., Adler, D. A., Berlant, J., Dixon, L. B., Dulit, R. A., Goldman, B., . . . Sonis, W. A. (2009). Imple- 
menting standardized assessments in clinical care: now's the time. Psychiatric Services, 60(10), 1372- 
1375. doi: 10.1176/appi.ps.60.10.1372 
Wallerstein, N., & Duran, B. (2010). Community-Based Participatory Research contributions to intervention re- 
search: The intersection of science and practice to improve health equity. American Journal of Public 
Health, 100(s1), S40-S46. doi: 10.2105/ajph.2009.184036 
Wells, K. B. (1999). The design of Partners in Care: evaluating the cost-effectiveness of improving care for de- 
pression in primary care. Social Psychiatry and Psychiatric Epidemiology, 34(1), 20-29. doi: 
10.1007/s001270050107 
Whipple, J. L., Lambert, M. J., Vermeersch, D. A., Smart, D. W., Nielsen, S. L., & Hawkins, E. J. (2003). Im- 
proving the effects of psychotherapy: The use of early identification of treatment failure and problem- 
solving strategies in routine practice. Journal of Counseling Psychology, 50(1), 59-68. doi: 
10.1037//0022-0167.50.1.59 
Wiltsey-Stirman, S., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of 
new programs and innovations: a review of the empirical literature and recommendations for future re- 
search. Implementation Science, 7, 17. doi: 10.1186/1748-5908-7-17 
Wiltsey-Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013). Development of a framework and coding 
system for modifications and adaptations of evidence-based interventions. Implementation Science, 8(1), 
65. 

SECTION I: STUDY DESCRIPTION 
SECTION V: SUBJECT POPULATION 
SUMMARY SAFEGUARD STATEMENT 
 
IRB STUDY NUMBER: 
1407578183 
PRINCIPAL INVESTIGATOR: Cara C. Lewis, PhD 
DOCUMENT DATE: 7/7/16 
 
STUDY TITLE: Standardized versus Tailored Implementation of Measurement Based Care for Depression 
 
A. Please describe (in lay terms) the general objective(s) of the proposed research, including research question(s), hypothesis, and a 
short summary of the main interactions/interventions. If appropriate, describe any usual methods, that were considered, but not 
chosen, and why. 
 
Depression remains among the nation’s top 10 chronic illnesses, costing over $80 billion annually. Depression has been called the 
“Common Cold” of mental illness, but one with serious risk of morbidity and mortality. There are now many evidence-based 
practices for the treatment of depression, but unfortunately these practices remain largely unavailable to clients receiving services 
in community mental health centers. Measurement Based Care (MBC) is a relatively simple evidence-based intervention frame- 
work. MBC, by definition, is the practice of using symptom measurement to inform mental health care. Physicians who routinely 
measure the patient’s blood pressure when the treatment target is high blood pressure demonstrate the medical corollary of MBC. 
When MBC is used in the treatment of depressed adults, it has been shown to improve outcomes by identifying clients who are 
not making progress and reducing the likelihood that clients will deteriorate in treatment. However, despite the demonstrated ef- 
fectiveness of MBC, the majority of community mental health counselors (i.e., clinicians) do not regularly assess target problem 
symptoms to guide their work over the course of treatment. To our knowledge, no studies to date have focused on the process of 
implementing MBC in community mental health settings. The long-term goal of this research project is to provide generalizable 
and practical recommendations about implementation approaches that promote MBC use and fidelity in community mental health 
centers. Specifically, this study will test a standardized versus a tailored approach to implementing MBC in one of the nation’s 
largest not-for-profit providers of behavioral health services. Although touted as superior, tailored implementations have rarely 
been compared to standardized approaches. Moreover, recent research has demonstrated an apparent need to adapt evidence- 
based practices to fit the specific context in which they are being implemented, particularly if they are to be sustained. This pro- 
posal reflects a movement in the field of implementation science in which planned adaptations are being tested and compared to 
standardized versions. The proposed research is a three-phase, mixed methods (quantitative/qualitative) study to investigate the 
effect of these two different approaches to MBC implementation on both clinician-level (e.g., MBC fidelity) and client-level (de- 
pression symptom change) outcomes. We will focus on contextual factors (e.g., attitudes, resources, process, etc.) that may influ- 
ence the implementation process with the goal of identifying a generalizable and practical way of bringing MBC to community 
mental health centers treating depressed adults. 
 
 
A. Subject Population. Check all subject population categories below for which there is a reasonable expectation of enrollment into 
this research study: 
 
Children (Complete the Request Form for the Inclusion of Children in Research) 
Cognitively Impaired (Complete the Request Form for the Inclusion of Cognitively Impaired Individuals in Research) 
Economically/Educationally Disadvantaged 
Pregnant Women, Human Fetuses, or Fetal Material (Complete the Request Form for the Inclusion of Pregnant Women, 
Human Fetuses, and Neonates in Research) 
Prisoners (Complete the Request Form for the Inclusion of Prisoners in Research) 
Subjects Outside of U.S. Targeted for Enrollment (Complete the Transnational Research Information Form) 
Veterans or research funded by the VA, utilizing VA effort, property or resources, or enrolling VA patients. (Complete the 
Request Form for VA Research) 
Students. When there is a teacher-student relationship dynamic, complete the following questions: 
 
B. Inclusion/Exclusion. 
 
Clinicians 
Inclusion Criteria. Clinicians will be eligible for the study if they: 1) provide individual psychotherapy; 2) provide psychotherapy 
to adults with depression (based on their self-reported typical and current caseload (i.e., clinicians will learn in the eligibility 
23 

discussion that if they consent to participate in the study then they must have eligible clients to potentially enroll as well); 3) con- 
duct psychotherapy sessions in English. 
Exclusion Criteria. Clinicians will be excluded if they indicate during enrollment that they intend to leave the organization and 
will be unable to complete all study phases. 
 
Clients 
Inclusion Criteria. As this is a pragmatic trial (Thorpe et al., 2009), clients will be eligible for the study if they: 1) are age 18 and 
above; 2) have depression as one of the primary treatment foci: diagnosis will be made by clinicians using usual care interview 
methods and can include any of depressive disorder diagnosis (e.g., major depressive episode, dysthymic disorder); 3) have sig- 
nificant depressive symptom severity (PHQ-9 total score > 9 at intake); 4) will receive individual psychotherapy; 5) are receiving 
psychotherapy treatment with an enrolled clinician during the proposed funding period. 
Exclusion Criteria. Because of the pragmatic nature of this trial, there is only one exclusion criterion for clients. Specifically, 
clients will be excluded if they are unable to provide verbal consent due to severe mental illness or inability to read, for instance. 
 
Administrators 
Inclusion Criteria: Administrators must have some sort of administrative role at the site or as part of Centerstone as a whole. 
However, the position they must have an administrative role within one of the twelve sites that will be participating in project. 
Exclusion Criteria: The only exclusion criteria are that 1) the administrator must be involved with one of the twelve sites partici- 
pating in the project. Those who have an administrative role in one of Centerstone’s other sites will not be eligible to participate. 
2) Those who indicate they intend to leave the organization and will be unable to complete all study phases will not be eligible. 
 
Office Professionals 
Inclusion Criteria: Office professionals must have a role at one of the twelve participating Centerstone sites. 
Exclusion Criteria: The only exclusion criteria are that 1) the office professional must be involved with one of the twelve sites 
participating in the project and 2) those who indicate they intend to leave the organization will be unable to complete all study 
phases will not be eligible. 
 
Staff  
Inclusion Criteria: Staff must provide services to the client at one of the 12 participating Centerstone sites or provide support to 
another staff member providing services to the client. Depending on the site, staff could include but is not limited to nurses, doc- 
tors, psychiatrists, and case managers. 
Exclusion Criteria: The only exclusion criteria are that 1) the staff must be involved with one of the twelve sites participating in 
the project and 2) those who indicate they intend to leave the organization will be unable to complete all study phases will not be 
eligible. 
 
C. Number of Subjects. 
 
Clinicians 
Clinicians will be recruited from 12 Centerstone sites across two states (Indiana and Tennessee). Approximately 187 clinicians 
(average of 16 per site) will be recruited during the latter half of Phase 1 of the study (months 5 to 8) to achieve our target enroll- 
ment of 150 (accounting for clinician turnover) and allow for initial site randomization (to early or late and standardized or tai- 
lored implementation) to occur just prior to Phase 2 (months 8-38). 
 
Clients 
Clients seeking treatment for depression at the 12 enrolled Centerstone sites will be recruited for participation in the proposed 
research during Phase 2 of the study (months 8 to 38; we will stop recruitment at month 35 to allow for the necessary 12 week 
assessment to be completed during Phase 2). Approximately 625 clients will be recruited (given the potential for unplanned attri- 
tion) to achieve our goal enrollment of 500 clients (an average of 42 per site, 3 per clinician) based on power analyses. 
 
Administrators 
Administrators will be recruited from the 12 enrolled Centerstone sties across two states as well as those who have a larger role 
within Centerstone, such as the executive director. About 12 administrators (about 1 per site) will be recruited during the latter 
half of Phase I of the study (months 5 to 8) to participate in individual interviews and then later implementation team meetings. 
 
Office Professionals 
Office professionals will be recruited from the 12 enrolled Centerstone sites across two states. Approximately 12 office profes- 
sionals (about 1 per site) will participate in the implementation team meetings. 
 
Staff 
Staff will be recruited from the 12 enrolled Centerstone sites across two states. Approximately 300 staff members will participate 
by completing small batteries of questionnaires. 

 
 
The study consists of three main phases each is described briefly below. 
Phase 1: Start up/Pre-Implementation (months 0-8) will interface the most widely used and validated depressive symptom 
severity measure (Patient Health Questionnaire-9 item; PHQ-9; Kroenke, 2001) with the electronic health record. Enrollment of clini- 
cian, administrators, office professionals, and client participants will begin. Randomization at the site level will occur to one of the 
two conditions, standardized or tailored implementation of measurement-based care for depression. In addition, sites will be random- 
ized to early and late stage implementation across 4 cohorts (2-4 sites in each cohort), with each cohort having 5 months of active im- 
plementation and 10 months of sustainment monitoring. 
Phase 2: Randomized Trial: Implementation & Sustainment (months 8-38) constitutes the active implementation (5 months) 
of measurement based care and sustainment (10 months) phase. Clinician, administrator, office professional, staff, and client participa- 
tion will occur during this phase. 
Finally, phase 3: Post Hoc Characterization of MBC Fidelity, Analysis and Manuscript Preparation (38-48 months) will con- 
sist of post-hoc characterization of measurement-based care fidelity (MBC fidelity; Appendix B.2), defined as (a) client completion of 
the PHQ-9, (b) clinician review of scores in the electronic health record, and (c) discussion of scores in session. 
 
Clinicians: Across Both Conditions (i.e., Standardized and Tailored) 
Phase 1 (Months 0-8) 
Pre-Implementation. The research team will recruit clinicians for participation via email and site visits. The study research 
assistants assigned to the Centerstone sites will follow-up with interested clinicians over the phone to provide additional information 
or answer any questions s/he may have about participating in the study. 
 
Phase 2 (Months 8 – 38) 
5-Month Active Implementation. Beginning at month eight, clinicians who provide individual psychotherapy to adults with 
depression (based on their self-reported typical and current caseload (i.e., clinicians will learn in the eligibility discussion that if they 
consent to participate in the study then they must have eligible clients to potentially enroll as well) and who conduct psychotherapy 
sessions in English will engage in the baseline mixed methods needs assessment. This assessment will consist of a battery of surveys 
and then, following the completion of these surveys a focus group will take place. Clinicians will be asked to complete the following 
baseline battery of self-reports (Appendix A): Demographic Questionnaire (Lewis & Simons, 2011; Appendix A.1); Progress Moni- 
toring Theory of Planned Behavior (Francis et al., 2004; Appendix A.2); Attitudes toward Standardized Assessment-Monitoring and 
Feedback (Jensen-Doss et al., in press; Appendix A.3); Monitoring and Feedback Attitudes Scale (Jensen-Doss et al., in press; Appen- 
dix A.4); Monitoring As Usual Measure (Lyon et al., 2015; A.5); Evidenced-Based Practices Attitudes Scale (Aarons, 2004; A.6); 
Survey of Organizational Functioning (Broome et al., 2007; A.7), Implementation Climate (Jacobs et al., 2014; A.8); Implementation 
Climate Scale (Earhart et al., 2014; A.9); Barriers and Facilitators to Implementing Survey (Salyers, Rollins, McGuire, & Gearhart, 
2008; A.10); Sociometric Questionnaire (Valente et al., 2007; A.11); Opinion Leadership Scale (Childers, 1986) and self-nomination 
question (A.12); Implementation Leadership Scale (Aarons et al., 2014; A.13); and Perceptions of Adopting Measurement-Based Care 
(Moore & Benbasat, 1991; A.14). All questionnaires will be completed online using assessment software by emailing them a link or 
via hard copy (in-person). These questionnaires will take about 60 minutes to complete and clinicians who complete the baseline as- 
sessment will receive $50 compensation. Using the data from these surveys, a subsample of clinicians at each site, selected using pur- 
poseful sampling for extreme variation (Palinkas et al., 2013), will be asked to participate in a 1.5 hour focus group (Appendix E.1). 
Clinicians who participate in the focus group(s) will receive $25 compensation. Focus groups will be recorded and transcribed with 
data de-identified for the purposes of qualitative coding. Staff who do not provide individual psychotherapy to adults with depression 
in English (for example, this could be clinicians who primarily provide therapy to children, groups, diagnoses other than depression 
and/or conduct therapy in another language; or psychiatrists, nurses, nurse practitioners, case managers, other non-clinical staff) will 
engage in an abbreviated version of the battery of self-report measures which will include: (Appendix A): Demographic Questionnaire 
(Lewis & Simons, 2011; Appendix A.1); Progress Monitoring Theory of Planned Behavior (Francis et al., 2004; Appendix A.2); Atti- 
tudes toward Standardized Assessment-Monitoring and Feedback (Jensen-Doss et al., in press; Appendix A.3); Monitoring and Feed- 
back Attitudes Scale (Jensen-Doss et al., in press; Appendix A.4); Evidenced-Based Practices Attitudes Scale (Aarons, 2004; A.6); 
Sociometric Questionnaire (Valente et al., 2007; A.11); Opinion Leadership Scale (Childers, 1986). All questionnaires will be com- 
pleted online using assessment software by emailing them a link or via hard copy (in-person). The questionnaires will take less than 30 
minutes to complete and staff who complete the abbreviated baseline assessment will receive $20 in compensation per assessment 
point. Note, from this point forward we will refer to this group of staff as the “general staff” to differentiate them from those complet- 
ing the full baseline battery which we will refer to as simply “clinicians.” 
Following the needs assessment, clinicians will be asked to participate in one four-hour training session that will introduce 
either the standardized MBC approach or the tailored MBC procedures. Clinicians will receive Continuing Education Units (i.e., edu- 
cation credit needed for licensure; maximum of four; to be determined) for attending the training. One training will be held at each 
site. The training will be scheduled such that all participating clinicians can attend and clinicians will receive productivity coverage 
credit for the four-hour training. In addition, immediately following the training a small battery of questionnaires will be administered 
to the clinicians to understand their training experience. This battery includes: Acceptability, Feasibility, Appropriateness for Training 
(Lyon, 2011, A.16-3); Post-Training Workshop EVALuation (TCU Institute of Behavioral Research, 2009; A.17); Clinical Teaching 
25 
SECTION VII: STUDY PROCEDURES 

Assessment Instrument (Litzelman, Stratogs, Marriot, & Skeff, 1998; A.18); Counselor Rating Form-Short (Corrigan & Schmidt, 
1983; A.15), Progress Monitoring Theory of Planned Behavior (Francis et al., 2004; Appendix A.2); Attitudes toward Standardized 
Assessment-Monitoring and Feedback (Jensen-Doss et al., in press; Appendix A.3); Monitoring and Feedback Attitudes Scale (Jen- 
sen-Doss et al., in press; Appendix A.4); Perceptions of Adopting Measurement-Based Care (Moore & Benbasat, 1991; A.14) and 
Measure of Effective Attributes of Trainers (Boyd et al, unpublished; A.22). This will be completed by clinicians in-person following 
the training and will take about 20-25 minutes to complete. Clinicians will receive $10 in compensation for completing the post-train- 
ing battery. 
In addition, one opinion leader and one self-nominated MBC champion (the opinion leader could also be the MBC champion, 
but in some cases this will be two different people) will be invited to participate on the implementation team based on the survey re- 
sults from the needs assessment. The implementation team consists of research team members, site administrator, and 2-4 clinician 
opinion leader and champion, a client, and office professional. This team will convene for triweekly meetings during the 5-month ac- 
tive implementation phase and they are encouraged to meet without the researchers present during the 10-month follow-up period to 
facilitate identification of additional barriers (the number of actual meetings and attendance will be documented). Clinicians will be 
informed that consultation and training sessions will be audio-recorded for research team coding purposes, with the focus on capturing 
the barriers that emerged and the function of the discussions. Clinicians will be reminded at the beginning of each session not to use 
client names during consultation sessions to protect client confidentiality. Audio-recorded sessions will be transcribed using the proce- 
dures outlined for the focus groups. No clinician study participant identifying information will be retained in the transcripts and only 
6-digit anonymous identification numbers will be used. The form found in Appendix B (MBC Barriers Coding Form; Appendix B.1) 
will be completed by implementation teams during their triweekly meetings; deidentified MBC Barriers Coding Forms will be ana- 
lyzed for the purposes of research. Additionally, clinicians will have the option to complete the form found in Appendix B (MBC 
Consultation Form) prior to the triweekly meetings to facilitate consultation, the deidentified MBC Consultation Form form will also 
be analyzed for the purposes of research. Finally, a battery of surveys will be administered in-person and completed by those on the 
implementation teams at 5-months and 15-months during an implementation team meeting. These surveys include the Acceptability, 
Feasibility, Appropriateness for Implementation Teams and Conditions (Lyon, 2011; Appendix A.16-1, A.16-2); Mendel Implementa- 
tion Team Survey (A.19), and Edmondson Team Measure (Edmondson et al, 1999; A.20). These surveys will take about 25 minutes to 
complete and staff will receive $15 in compensation per assessment point. 
Clinicians have the option of entering and reviewing the PHQ-9 data with clients in the EHR. When they enter the data they 
should be given the option of answering additional questions about if/why they reviewed scores with client (see Appendix B.2). Due 
to technological issues with the EHR, not all clinicians currently have the option of completing these questions. To ameliorate this 
issue, clinicians at affected sites will have the option of filling out this information on paper each week (see Appendix B.4-1, B.4-2). 
This data will then be collected in person by CRI staff who will transfer if to a secure database. To offset the inconvenience of filling 
out a paper version of the questions and to encourage clinicians to do so, $1 will be included with each paper survey. 
At the end of the 5-month active implementation, clinicians will be asked to complete the same questionnaires in the above- 
listed baseline battery. Similar to the baseline battery of self-reports, these assessments will be completed online or via hard copy, and 
clinicians will receive $50 per assessment battery completed. In addition, a subsample of clinicians at each site similar to at the begin- 
ning of Phase 2 will be asked to participate in a 1.5 hour focus group (Appendix E.1). Clinicians who participate in the focus group(s) 
will receive $25 compensation. A subsample of clinicians at each site will also be invited to participate in individual exit interviews 
regarding their views about and use of the PHQ-9 (Appendix A.21). Clinicians who participate will receive $20 compensation. 
10-month Post-Implementation/Sustainment. Towards the end of the post-implementation/sustainment phase, another focus 
group with a subsample of clinicians from each site selected via purposeful sampling will occur (Appendix E.1). Similar to the focus 
group at the beginning of Phase 2 and end of Phase 2, participants will be compensated $25. The same aforementioned baseline and 5- 
month active implementation assessment battery will be re-administered towards the end of the 10-month post-implementation/sus- 
tainment phase to assess factors influencing sustainment of MBC. Similar to the assessments before, clinicians will receive $50 for 
completing the assessment. 
The job category/title, licensure, supervisor information and demographics will also be obtained from the electronic health 
records at each site to obtain information about clinician experience. This information will be retrieved by a trained Centerstone Re- 
search employee who will be de-identify the information before providing it to the research team. 
 
Clients: Across Both Conditions (i.e., Standardized and Tailored) 
Phase 1 (Months 0-8) 
Pre-Implementation. A research assistant will contact potentially eligible clients via telephone and will perform an initial 10- 
20 minute phone screen to assess eligibility (RCR.3). A telephone version of the Patient Health Questionnaire-9 (PHQ-9; Appendix 
C.2) will be performed during the phone screening with the Centerstone Research Institute research assistant (CRI RA) in both condi- 
tions. This questionnaire should only take about five minutes to complete and clients will receive a $5 incentive for completing the 
PHQ-9 regardless of their study eligibility. The client is deemed eligible if the client indicates that depression is a primary focus of 
treatment, s/he is seeking treatment with an enrolled study clinician, and s/he scores a 9 or higher on the PHQ-9. If the client is eligi- 
ble, s/he will be asked to respond to a Client Demographics Questionnaire (Appendix C.1) over the phone during this initial screen. 
The data obtained from the initial PHQ-9 screen (RCR.3) and demographics questions is to be collected, labeled with a 9-digit anony- 
mous research ID, and kept in a secure database in the secure server at Centerstone. If s/he is eligible and interested in participating, 
s/he will be asked to give verbal consent to participation over the phone during the initial screen conducted by the CRI RA. The CRI 
RA will also offer to call the client back if s/he would like time to review the informed consent statement before agreeing to partici- 
pate. After a client verbally consents to participation or indicates that they would like time to review the consent statement, a 

Welcome Packet, which will, contain the informed consent, authorization for the release of health information for research, and a 
stamped and return-addressed envelope will be sent to the client’s address. The client will a $5 gift card as compensation for PHQ-9 
completion in the initial phone screen. If the client signs and returns the release of health information either via the pre-paid envelope 
or in person at their next session (additional information on the informed consent process can be found below in Section XII: Informed 
Consent Process), s/he is enrolled in the study. At this point the client will receive an additional $5 compensation. If the client con- 
sents to research participation and the authorization for the release of health information for research, their verbal PHQ-9 and de- 
mographics data from the initial phone screen will be utilized as a baseline measure and be transported to IU for storage, given that the 
consent includes that the PHQ-9 completed for the eligibility screener and demographics questions can be used for the purposes of 
research. The 5-Axis diagnosis and employment, status, program activity involvement (i.e., type, start date, end date, care coordinator 
involved, face-to-face contact, location, reasons for discharge), date of first and last service, date of intake session, admission status 
(active/inactive), body mass index (BMI), age at BMI, demographics, medication use (i.e. prescription type, dosage, units, number of 
pills), and blood pressure (Appendix C.3) will also be retrieved from the clinician’s intake report and health records to obtain infor- 
mation about comorbidities and functioning for all enrolled clients. This information will be retrieved by a trained Centerstone Re- 
search employee who will de-identify the information and then provide it to the research team. 
 
Phase 2 (Months 8-38) 
Client’s Treatment (weeks 0-11). Clients in the standardized condition will be asked to complete the hard copy PHQ-9 forms 
in the waiting room prior to every session with their enrolled study clinician, while clients in the tailored condition will complete the 
PHQ-9 according to the measurement-based care guidelines defined by the implementation team at each site. In all cases, clients will 
fill out the hard copy PHQ-9 prior to a session located in waiting room kiosks and completing the survey should only take about five 
to ten minutes. The client will then hand the completed PHQ-9 to his or her clinicians who will then enter it into the electronic health 
record so that the clinicians can review the PHQ-9 scores with the clients during the session. The CRI RA will pull the PHQ-9 data for 
all enrolled clients across the course of their treatment and provide it to the research team using the 9-digit anonymous identification 
number. In addition to the PHQ-9, clients will be asked to respond to a telephone or text message-based survey after each psychother- 
apy session to inquire as to whether their clinician discussed PHQ-9 scores in the session immediately preceding (Yes/No/NA) (Ap- 
pendix C.4). Client can earn $2 for every survey they complete for up to $20. So clients may be able to earn an additional $20 for 
completing these surveys after each session and the amount attained will be mailed with the $10 incentive for completing the week 12 
PHQ-9 (described below). 
Client’s Treatment (week 12). Clients will be reassessed for depression symptom severity via a phone interview (like the ini- 
tial screen) at week 12 of their treatment, regardless of number of treatment sessions attended (Appendix C.2). A postcard reminder 
may be sent to clients before the phone interview as a reminder. The project coordinator and CRI RAs will track client time since en- 
tering treatment, beginning with the intake assessment. Clients will be mailed $20 as an incentive for completing the week 12 assess- 
ment. 
Client’s Treatment (week 13 and beyond). Upon completion of the telephone assessment at week 12, PHQ-9 data collection 
for all enrolled clients will continue to occur according to the site guidelines for the duration of the study funding-period. 
Note. In the unlikely case that an enrolled Centerstone client does not own a telephone, the research assistant will meet the 
client on-site for a hard copy version of the PHQ-9 in instances where a telephone version is supposed to administered (e.g., for deter- 
mining initial eligibility and in week 12). In addition, if a client does not have unlimited text messaging, we will offset the costs for 
the text message surveys by providing the client with $5. 
We will be asking permission to audiorecord client’s therapy sessions for research purposes. These tapes would be reviewed 
and coded by an objective team of researchers trained in confidentiality to understand how measurement-based care is used in the 
community and to improve our understanding of how to help clients using measurement-based care. Only a subsample of client’s au- 
diotaped sessions would be reviewed and coded and these tapes will be randomly selected. 
 
Administrators: Across Both Conditions (i.e., Standardized and Tailored) 
Phase 2 (Months 8 – 38) 
5-Month Active Implementation. Administrator from each site will be asked to participate in a 30 minute individual interview 
either in-person or phone-based at the beginning of Phase 2 (Appendix E.2 and E.3). Administrators who participate in the individual 
interview will receive $20 compensation. Individual interviews will be recorded and transcribed using VoiceBase technologies with 
data de-identified for the purposes of qualitative coding. 
In addition, one administrator per site will be invited to participate on the implementation team at each site. The implementa- 
tion team consists of research team members, site administrator, 2-4 clinician opinion leader and champion, a client, and office profes- 
sional. This team will convene for triweekly meetings during the 5-month active implementation phase and they are encouraged to 
meet without the researchers present during the 10-month follow-up period to facilitate identification of additional barriers (the num- 
ber of actual meetings and attendance will be documented). Administrators will be informed that the meetings will be audio-recorded 
for research team coding purposes, with the focus on capturing the barriers that emerged and the function of the discussions. Adminis- 
trators will be reminded at the beginning of each session not to use client names during consultation sessions to protect client confi- 
dentiality. Audio-recorded sessions will be transcribed using the procedures outlined for the focus groups. No administrator study par- 
ticipant identifying information will be retained in the transcripts and only 6-digit anonymous identification numbers will be used. The 
form found in Appendix B (MBC Barriers Coding Form; Appendix B.1) will be completed by implementation teams during their 
 
27 

SECTION VIII: RISK/BENEFIT RATIO 
triweekly meetings; deidentified MBC Barriers Coding Forms will be analyzed for the purposes of research. Additionally administra- 
tors will complete the battery of self-report measures (the same one that clinicians complete) and receive $50 compensation at the 
baseline, 5-month follow up, and 10-months after the active implementation period. Finally, a battery of surveys will be administered 
in-person and completed by those on the implementation teams at 5-months and 15-months during an implementation team meeting. 
These surveys include the Acceptability, Feasibility, Appropriateness for Implementation Teams and Conditions (Lyon, 2011; Appen- 
dix A.16-1, A.16-2); Mendel Implementation Team Survey (A.19), and Edmondson Team Measure (Edmondson et al, 1999; A.20). 
These surveys will take about 25 minutes to complete and staff on the implementation team will receive $15 in compensation per as- 
sessment point. 
At the end of the active implementation period, the administrator will be invited to participate in the Contamination inter- 
view. This interview will be conducted in person or on the phone and will take about 30 minutes to complete. 
 
Office Professionals: Across Both Conditions (i.e., Standardized and Tailored) 
Phase 2 (Months 8 – 38) 
5-Month Active Implementation. One office professional per site will be invited to participate on the implementation team at 
each site. The implementation team consists research team members, site administrator, 2-4 clinician opinion leaders and champions, a 
client, and office professional. This team will convene for triweekly meetings during the 5-month active implementation phase and 
they are encouraged to meet without the researchers present during the 10-month follow-up period to facilitate identification of addi- 
tional barriers (the number of actual meetings and attendance will be documented). Office professionals will be informed that the con- 
sultation and training sessions will be audio-recorded for research team coding purposes, with the focus on capturing the barriers that 
emerged and the function of the discussions. Office professionals will be reminded at the beginning of each session not to use client 
names during consultation sessions to protect client confidentiality. Audio-recorded sessions will be transcribed using the procedures 
outlined for the focus groups. No office professional study participant identifying information will be retained in the transcripts and 
only 6-digit anonymous identification numbers will be used. The form found in Appendix B (MBC Barriers Coding Form; Appendix 
B.1) will be completed by implementation teams during their triweekly meetings; deidentified MBC Barriers Coding Forms will be 
analyzed for the purposes of research. Finally, a battery of surveys will be administered in-person and completed by those on the im- 
plementation teams at 5-months and 15-months during an implementation team meeting. These surveys include the Acceptability, 
Feasibility, Appropriateness for Implementation Teams and Conditions (Lyon, 2011; Appendix A.16-1, A.16-2); Mendel Implementa- 
tion Team Survey (A.19), and Edmondson Team Measure (Edmondson et al, 1999; A.20). These surveys will take about 25 minutes to 
complete and staff on the implementation team will receive $15 in compensation per assessment point. Additionally, all office profes- 
sionals will be invited to participate in the study by completing parts of the abbreviated baseline battery that are applicable to their 
position at the baseline. The battery will take about 30 minutes to complete and office professionals will be compensated $20 for com- 
pleting the battery at each of the three assessment point. They will complete the battery at the same time points as clinicians: at base- 
line, at the 5-month follow up and 10 months following the active implementation period. 
 
Staff: Across Both Conditions (i.e., Standardized and Tailored) 
Phase 2 (Months 8 – 38) 
 
5-Month Active Implementation. Beginning at month eight, all staff will engage in parts of the abbreviated version of the 
baseline battery of self-reports that are applicable to their position. For example, case managers might not complete the Progress Mon- 
itoring Theory of Planned Behavior Measure (Francis et al., 2004; Appendix A.2) because they do not have the option of doing pro- 
gress monitoring with clients but psychiatrists might have this option and would therefore fill out this measure. All questionnaires will 
be completed online using assessment software by emailing them a link or via hard copy (in-person). The battery will take about 15 
minutes to complete and staff will be compensated $20 for completing the battery at each of the three assessment points. Additionally, 
they will receive $1 when a paper version of the baseline is sent as an added incentive to complete the survey. They will complete the 
battery at the same time points as clinicians: at baseline, at the 5-month follow up, 10 months following the active implementation 
period. 
 
POTENTIAL RISKS 
 
Given the existence and ongoing oversight of Centerstone Research Institute for all research conducted at Centerstone, the inves- 
tigative team has considerable experience conducting research of the proposed nature without problem. The questionnaires, focus 
group, and individual interview approaches proposed in this study are based on established procedures used in a wide range of 
studies without adverse effects. Even so, we acknowledge several potential risks that may occur as a result of clinician, client, site 
administrator, and office professional study participation and have established appropriate procedures for minimizing these risks, 
which include: (a) loss of confidentiality; (b) experience of discomfort answering questions (for clinicians, site administrators, 
and office professionals regarding their work at Centerstone and for clients in reporting their experience with depression in the 
presence of a research assistant), and (c) the potential for clinicians, site administrators, staff and office professionals to perceive 
coercion to participate in the study given that this proposal aligns with the agency’s overall strategic goals. To address the risk of 
loss of confidentiality, a database file connecting clinician, site administrator, staff, and office professional identification numbers 
and names will be located at Indiana University and will be password protected and stored on a locked computer accessible only 
to trained research staff. In addition, due to the potential loss of confidentiality through audiotaped sessions review, clients and 

SECTION IX: PROTECTION PROCEDURES 
clinicians will have the opportunity to request their sessions not be audiotaped and used for research purposes. If clients and clini- 
cians do agree to their audiotapes being used for research purposes, they are assured that the objective coders would remove 
themselves should they recognize a subject’s voice and that all coders are trained to maintain confidentiality of content reviewed 
in the audiotape. To address the risk of clinicians, site administrators, office professionals, staff, and clients experiencing discom- 
fort, research staff will be trained to address distress during focus groups, individual interviews, and assessment procedures. Fi- 
nally, we are engaging explicit procedures to protect against the risk of perceived coercion of participating clinicians, site admin- 
istrators, and office professionals (described in Section IX). 
 
POTENTIAL BENEFITS 
 
Clinicians 
Clinicians who choose to enroll in the study will have the opportunity to receive training in the use of measurement-based care 
(MBC), an evidence-based practice framework. We hope that this training opportunity will enable clinicians to enhance their client 
care by providing them with an additional tool for monitoring client progress throughout treatment and for assessing client treatment 
outcomes. Clinicians will also receive Continuing Education Units (needed for licensure) for attending the training. In addition, 
clinicians may benefit from the enhanced Electronic Health Record system equipped with the PHQ-9 and a clinician-informed 
optimal user interface. 
 
Clients 
Clients who choose to enroll in the study will have the opportunity to receive mental health care from a clinician who has received 
training in the use of MBC, thus providing the potential for the client to receive enhanced treatment as usual. Furthermore, previous 
research has indicated that MBC can result in improved client satisfaction and engagement in treatment (Eisen, Dickey, & Sederer, 
2000). Beyond the potential for enhanced care, clients enrolled in the study will receive treatment for depression as usual at Cen- 
terstone. In sum, we believe that the potential benefits of this study outweigh the potential risks. 
 
Administrators 
Administrators who choose to enroll in the study will have the opportunity to assist in the implementation of measurement-based 
care in his or her site. Their involvement is consistent with a community participatory approach that elevates their voice and values 
in the implementation process. 
 
Office Professionals 
Office professionals who choose to enroll in the study will have the opportunity to assist in the implementation of measurement- 
based care in his or her site. Their involvement is consistent with a community participatory approach that elevates their voice and 
values in the implementation process. 
 
Staff 
Staff who choose to enroll in the study will have the opportunity to make their voices and values heard in the implementation 
process by filling out brief batteries of surveys. 
 
Few experimental implementation studies have been conducted. Findings from this proposal have the potential to: (1) yield a ro- 
bust theoretical and practical model delineating contextual factors and evaluative processes for implementation in an academic- 
community partnership; (2) demarcate the benefit of using standardized versus tailored protocols for implementation; (3) establish 
a blended protocol for MBC implementation with capacity for generalization to community mental health centers nationwide; 
and, (4) highlight contextual factors responsible for sustained MBC implementation. With depression maintaining its place among 
the nation’s top chronic illnesses, costing billions of dollars annually, positive findings from this proposal will present a feasible 
and effective approach to alleviating great societal burden through a minimal intervention needed for change. This study will lay 
the groundwork for subsequent research (by establishing progress monitoring) to enhance usual care for multi-problem clinical 
presentations, should MBC effects plateau and additional interventions be necessary. 
 
Due to the minimal risks for this study protocol and the potential gains for clinicians with respect to professional development and 
potential positive client symptom outcomes, we believe that the benefits of this protocol outweigh the risks. 
 
 
Clinicians  
Given the study procedures presented above, we believe that the study presents minimal risk to clinician participants. For enrolled 
clinicians, all data collection will be completed by trained research staff and all data will be kept in secure locations at either 
 
29 

Centerstone Research Institute or at Indiana University. Data, both quantitative (questionnaires) and qualitative (focus group tran- 
scripts) will be associated with an anonymous 6-digit research identification number for each participant. Data completed via the 
online Qualtrics survey will be stored in secure databases on password-protected computers at Indiana University. All audio-rec- 
orded focus groups and consultation meetings will be transcribed by research staff members trained in confidentiality procedures 
by the PI or via a voice transcription and indexing company, VoiceBase, Inc. All identifiers will be removed at the time of the 
transcription and replaced with the participant code. VoiceBase, Inc. provides a human transcription service for audio recordings. 
This company follows a strict privacy policy that attempts to maintain the utmost confidentiality of all transcriptions. As men- 
tioned, researchers (objective coders) who will review the audiotaped sessions will be trained to maintain confidentiality and 
these audiotapes will be securely locked up and kept in secure locations. 
Another potential risk could be differential treatment by clinical supervisors based on the knowledge of the clinicians’ partic- 
ipation in the study. It is not possible to conceal clinician study participation from supervisors. However, each supervisor will be 
instructed in human subjects research protections and made aware of the seriousness of research participant coercion in any form. 
Further, the supervisors and their managers will be instructed not to use participation in research as criteria for adjustment of 
wages or job position. The clinician participant will be made aware of this risk and given the contact numbers of the compliance 
office at Indiana University to report any unfair treatment based on their participation, or lack of, in research. Moreover, even 
though we plan to randomize at the site level, not all clinicians at each site will need to participate in order for recruitment goals 
to be met. Therefore, clinicians will be explicitly informed that even though their site is enrolled and they may be eligible, there is 
no obligation to participate in the research. 
We anticipate that the only other potential risk related to clinician participation in the study is possible discomfort when 
providing personal opinions to research personnel during focus groups and/or during completion of online questionnaires. To 
address this potential risk, all clinicians will be made aware of the voluntary nature of the study, and told that they may choose to 
end participation at any time. All clinicians will be informed about the confidential nature of the data obtained through participa- 
tion and the procedures put in place to protect each participant. Clinicians will also be informed that participation in the study will 
in no way affect their status as a clinician at Centerstone and their individual data will not be shared with their supervisors. Addi- 
tionally, clinicians will be informed that their qualitative and quantitative data will never be linked to any personal identifying 
information (in discussion with Centerstone, in manuscripts, presentations, etc.) beyond the securely kept code sheet at Indiana 
University and at Centerstone with the CRI RAs used for keeping participant identification numbers consistent over time (to ena- 
ble within-subjects evaluation). 
 
Clients 
We believe that the proposed study presents minimal risk to client participants. Clients who choose to participate in the study 
will be complete their baseline PHQ-9 assessment and week 12 assessment on the phone with the research assistant, trained to 
response to signs of distress, during the consenting phone call. In addition, PHQ-9 assessments completed prior to a session will 
be completed via hard copy forms in the waiting room. The potential risks for clients will be the possibility of discomfort when 
sharing information regarding mental health status with the research staff member, and the potential for a breach of confidential- 
ity. To address these potential risks, all clients will also be made aware of the voluntary nature of the study and that they may also 
choose to end participation at any time. They will also be made aware of the extra provisions for protection of confidentiality af- 
forded them by the NIH approved Certificate of Confidentiality in their informed consent document. Hundreds of studies to date 
have successfully used the PHQ-9 with clients; it is the most widely used depression severity measure. Centerstone has estab- 
lished procedures for handling client distress during a baseline and week 12 assessment. Specifically, research personnel conduct- 
ing baseline and week 12 assessments will be trained to be sensitive to signs of distress (e.g., psychomotor agitation, avoidance of 
responding to questions) and to respond appropriately (i.e., ask the client if they would like to end or continue the assessment). 
Moreover, research personnel conducting assessments will be trained to seek consultation with a licensed mental health profes- 
sional in the event that a client displays distress during the baseline and week 12 assessment. PI, Dr. Lewis, is a licensed clinical 
psychologist and will be on call by cell phone should research personnel need consultation regarding observed client distress. In 
the rare event that the study PI is unavailable, the interviewer will be instructed to consult with one of the clinic directors who are 
all licensed clinical psychologists or with a designated on-call clinician present at each clinic. Baseline and week 12 assessments 
will always be conducted during business hours when an on-call licensed mental health professional is available. Given the simi- 
larity of the PHQ-9 questions to a portion of the typical intake interview at Centerstone, on-call clinicians are well-versed in deal- 
ing with any distress that might arise. In particular, research personnel will be trained to conduct comprehensive suicide risk as- 
sessments given the potential for comorbid depression and suicidal ideation. Specifically, research assistants conducting baseline 
and week 12 assessments will check item 9 of the PHQ-9 during the baseline and week 12 assessment call. If suicidality is en- 
dorsed (anything greater than a 0 on #9 of the PHQ-9 endorsed), research assistants will then ask the client questions on the Co- 
lumbia-Suicide Severity Rating scale (C-SSRS; Research Foundation for Mental Hygiene, Inc., 2008; Appendix D.2). If the client 
then endorses #’s 2-6 on the screener then the researcher will engage in the pre-established comprehensive risk assessment and 
the consultation plan (i.e., contacting the Centerstone crisis line while the client is still on the phone) described above. All re- 
search assistants will be required to complete an Adverse Event Report Form (Appendix D.1) to be submitted to the study PI 
within 24 hours of the incident. This form requires the research assistant to detail the client’s distress, the research assistant’s ac- 
tions, consultation if engaged, and any follow-up plan that was developed. If necessary these forms will be used to follow-up with 
clients, and will be reviewed by the Data Safety Monitoring Board on an ongoing basis in order to assess the overall distress expe- 
rienced by participants throughout the study. The above-listed protocol will be reviewed in Phase 1 (prior to client enrollment) 

with the Data Safety and Monitoring Board, and if needed, it will be revised to reflect the optimal plan for client safety during 
participation in the research protocol. 
Finally, exhaustive efforts to ensure the maintenance of confidentiality will be implemented: all client data collection 
throughout the study will be conducted by trained research personnel, will only be labeled with anonymous 9-digit research iden- 
tification numbers, and will be kept in locked cabinets at both the Centerstone and Indiana University research sites and stored in 
secure, password protected databases on password protected computers accessible only to study research personnel. As men- 
tioned, researchers (objective coders) who will review the audiotaped sessions will be trained to maintain confidentiality and 
these audiotapes will be securely locked up and kept in secure locations. 
 
Administrators 
We believe that the study presents minimal risk to administrator participants. For enrolled administrators, all data collection 
will be completed by trained research staff and all data will be kept in secure locations at either Centerstone Research Institute or 
at Indiana University. Data, both quantitative (questionnaires) and qualitative (individual interview transcripts) will be associated 
with an anonymous 6-digit research identification number for each participant. Data completed via the online Qualtrics survey 
will be stored in secure databases on password-protected computers at Indiana University. All audio-recorded individual inter- 
views and consultation meetings will be transcribed by research staff members trained in confidentiality procedures by the PI or 
via a voice transcription and indexing company, VoiceBase, Inc. All identifiers will be removed at the time of the transcription 
and replaced with the participant code. VoiceBase, Inc. provides a human transcription service for audio recordings. This com- 
pany follows a strict privacy policy that attempts to maintain the utmost confidentiality of all transcriptions. 
Another potential risk could be differential treatment by more superior administrators or supervisors based on the knowledge 
of the administrators’ participation in the study. Although it is not possible to conceal administrator study participation from su- 
pervisors, each supervisor will be instructed in human subjects research protections and made aware of the seriousness of research 
participant coercion in any form. Further, the supervisors will be instructed not to use participation in research as criteria for ad- 
justment of wages or job position. The administrator participant will be made aware of this risk and given the contact numbers of 
the compliance office at Indiana University to report any unfair treatment based on their participation, or lack of, in research. Ad- 
ministrators will be explicitly informed that even though their site is enrolled and they may be eligible, there is no obligation to 
participate in the research. 
We anticipate that the only other potential risk related to administrator participation in the study is possible discomfort when 
providing personal opinions to research personnel during individual interviews and/or during completion of online or hard copy 
questionnaires. To address this potential risk, all administrators will be made aware of the voluntary nature of the study, and told 
that they may choose to end participation at any time. All administrators will be informed about the confidential nature of the data 
obtained through participation and the procedures put in place to protect each participant. Administrators will also be informed 
that participation in the study will in no way affect their status as an administrator at Centerstone and their individual data will not 
be shared with their supervisors. Additionally, administrators will be informed that their qualitative and quantitative data will 
never be linked to any personal identifying information (in discussion with Centerstone, in manuscripts, presentations, etc.) be- 
yond the securely kept code sheet at Indiana University and at Centerstone with the CRI RAs used for keeping participant identi- 
fication numbers consistent over time (to enable within-subjects evaluation). 
 
Office Professionals 
We believe that the study procedures present minimal risk to office professional participants. For enrolled office profession- 
als, all data collection will be completed by trained research staff and all data will be kept in secure locations at either Centerstone 
Research Institute or at Indiana University. Data, both quantitative (questionnaires) and qualitative (consultation transcripts) will 
be associated with a 6-digit anonymous research identification number for each participant. Data completed via the online Qual- 
trics survey will be stored in secure databases on password-protected computers at Indiana University. All audio-recorded consul- 
tation meetings will be transcribed by research staff members trained in confidentiality procedures by the PI or via a voice tran- 
scription and indexing company, VoiceBase, Inc. All identifiers will be removed at the time of the transcription and replaced with 
the participant code. VoiceBase, Inc. provides a human transcription service for audio recordings. This company follows a strict 
privacy policy that attempts to maintain the utmost confidentiality of all transcriptions. 
Another potential risk could be differential treatment by supervisors based on the knowledge of the office professionals’ par- 
ticipation in the study. It is not possible to conceal office professional’s study participation from supervisors. However, each su- 
pervisor will be instructed in human subjects research protections and made aware of the seriousness of research participant coer- 
cion in any form. Further, the supervisors and their managers will be instructed not to use participation in research as criteria for 
adjustment of wages or job position. The office professional participant will be made aware of this risk and given the contact 
numbers of the compliance office at Indiana University to report any unfair treatment based on their participation, or lack of, in 
research. Office professionals will be explicitly informed that even though their site is enrolled and they may be eligible, there is 
no obligation to participate in the research. 
We anticipate that the only other potential risk related to office professional participation in the study is possible discomfort 
when providing personal opinions to research personnel during completion of online or hard copy questionnaires. To address this 
potential risk, all office professionals will be made aware of the voluntary nature of the study, and told that they may choose to 
end participation at any time. All office professionals will be informed about the confidential nature of the data obtained through 
participation and the procedures put in place to protect each participant. Office professionals will also be informed that 
31 

SECTION XI: PAYMENT FOR PARTICIPATION 
participation in the study will in no way affect their status as an office professional at Centerstone and their individual data will 
not be shared with their supervisors. Additionally, office professionals will be informed that their qualitative and quantitative data 
will never be linked to any personal identifying information (in discussion with Centerstone, in manuscripts, presentations, etc.) 
beyond the securely kept code sheet at Indiana University and at Centerstone with the CRI RAs used for keeping participant iden- 
tification numbers consistent over time (to enable within-subjects evaluation). 
 
Staff 
We believe that the study procedures present minimal risk to staff participants. For enrolled staff, all data collection will be 
completed by trained research staff and all data will be kept in secure locations at either Centerstone Research Institute or at Indi- 
ana University. Data will be associated with a 6-digit anonymous research identification number for each participant. Data com- 
pleted via the online Qualtrics survey will be stored in secure databases on password-protected computers at Indiana University. 
Another potential risk could be differential treatment by supervisors based on the knowledge of the staff members’ participa- 
tion in the study. It is not possible to conceal staff members’ study participation from supervisors. However, each supervisor will 
be instructed in human subjects research protections and made aware of the seriousness of research participant coercion in any 
form. Further, the supervisors and their managers will be instructed not to use participation in research as criteria for adjustment 
of wages or job position. The office professional participant will be made aware of this risk and given the contact numbers of the 
compliance office at Indiana University to report any unfair treatment based on their participation, or lack of, in research. staff 
will be explicitly informed that even though their site is enrolled and they may be eligible, there is no obligation to participate in 
the research. 
We anticipate that the only other potential risk related to staff member participation in the study is possible discomfort when 
providing personal opinions to research personnel during completion of online or hard copy questionnaires. To address this poten- 
tial risk, all staff members will be made aware of the voluntary nature of the study, and told that they may choose to end participa- 
tion at any time. All staff members will be informed about the confidential nature of the data obtained through participation and 
the procedures put in place to protect each participant. Staff members will also be informed that participation in the study will in 
no way affect their status as clinical staff at Centerstone and their individual data will not be shared with their supervisors. Addi- 
tionally, staff will be informed that their data will never be linked to any personal identifying information (in discussion with Cen- 
terstone, in manuscripts, presentations, etc.) beyond the securely kept code sheet at Indiana University and at Centerstone with the 
CRI RAs used for keeping participant identification numbers consistent over time (to enable within-subjects evaluation). 
 
 
Clinician, Administrator, Office Professional and Staff confidentiality 
For enrolled clinicians, administrators, office professional, and staff all data collection will be completed by trained research staff 
and all data will be kept in secure locations at either Centerstone Research Institute or at Indiana University. Data, both quantita- 
tive (questionnaires) and qualitative (focus group transcripts, consultation transcripts, and individual interview transcripts) will be 
associated with an anonymous 6-digit research identification number for each participant. Data completed via the online survey 
will be stored in secure databases on password-protected computers at Indiana University. Any hard copies of data (should clini- 
cians, administrators, staff, and/or office professionals complete surveys in person) will be transferred to an electronic database; 
hard copies will be kept in locked cabinets in locked rooms and destroyed at the end of the funding period. 
 
Client confidentiality 
All client data collection throughout the study will be conducted by trained research personnel, will only be labeled with anony- 
mous 9-digit research identification numbers, and will be kept in locked cabinets at both the Centerstone and Indiana University 
research sites and stored in secure, password protected databases on password protected computers accessible only to study re- 
search personnel. 
 
Ensuring security and feasibility of automated telephone surveys. 
Automated telephone surveys will be employed to assess MBC fidelity (using a yes/no/did not attend session option regard- 
ing clinician discussion of the PHQ-9 measure with the client) throughout the study. The automated telephone survey will be set 
to text or call the enrolled study clients in advance of each scheduled psychotherapy session with the enrolled clinician based on a 
code triggered by the Electronic Health Record.. Frequency of use of mobile technologies is increasing for assessment and inter- 
vention, with data suggesting that underserved and disadvantaged populations are typically owners of mobile phones making this 
a feasible data collection method (Foley et al., 2012). Additionally, automated telephone surveys serve as a secure and confiden- 
tial way for clients to respond to the MBC fidelity questions as the client can respond with their touch-tone keypad, thus ensuring 
that question content and responses remain private and confidential. It is unlikely that enrolled Centerstone clients will not own a 
telephone; however, in these rare cases (determined during the initial screening/consenting process) the Centerstone research as- 
sistants will schedule to meet the client on-site for hard copy administration of the PHQ-9. 
 
 
A. Will subjects be paid for participation in the study (e.g. monetary, free services, gifts, course credit, including extra credit)? 
No. Proceed to next section. 
Yes. Complete items 1-3 below. 

1. Explain the payment arrangements (e.g. amount and timing of payment and the proposed method of disbursement), including 
reimbursement of expenses. 
 
Clinicians 
We will compensate clinicians $25 for participation in the 1.5 hour focus group held at the beginning of the active implemen- 
tation, end of the active implementation, and end of Phase 2 of the study. Quantitative questionnaires will be completed at 
baseline, 5-months, and 15-months and clinicians will be provided $50 for each assessment completed in compensation for 
the time required to complete study measures. Those clinicians on the implementation team will complete a battery of quanti- 
tative questionnaires at 5-months and 15-months and will receive $15 in compensation per assessment point. At sites affected 
by technology issues, clinicians will be compensated $1 for each time they answer questions related to PHQ-9 use with cli- 
ents (see Appendix B.4). In addition, at the beginning of Phase 2 of the study, clinicians will be asked to participate in one 
four-hour training session that will introduce either the standardized MBC approach or the tailored MBC procedures. Clini- 
cians will receive Continuing Education Units (i.e., education credit needed for licensure) for attending the training. An addi- 
tional battery of quantitative measures will be completed post-training and clinicians will be provided $10 in compensation. 
Clinicians who participate in the exit interviews at 5 month end of active implementation period will receive $20. 
 
Clients 
All clients who complete the baseline (i.e., initial phone screen) PHQ-9 assessment, regardless of study eligibility, will be 
mailed an incentive of $5 for completing the baseline assessment and another $5 for returning signed study documents. Cli- 
ents who complete the week 12 PHQ-9 assessment will be mailed a $10 incentive for completing it. Client can earn $2 for 
every phone-based survey they complete for up to $20. So clients may be able to earn an additional $20 for completing these 
text messages after each session and this amount attained will be mailed with the $10 incentive for completing the week 12 
PHQ-9 (described below). In addition, if a client does not have unlimited text messaging, we will offset the costs for the text 
message surveys by providing the client with $5. Clients who participate in the three implementation team meetings will re- 
ceive $15 per meeting attended. 
 
Administrators 
All administrators who complete the individual interview will be compensated $20. Those administrators on the implementa- 
tion team who complete a battery of quantitative questionnaires at 5-months and 15-months and will receive $15 in compen- 
sation per assessment point. Administrators will receive $50 compensation for completing the battery at the three time points: 
baseline, at 5-month follow up, and at 10-month follow up. 
 
Office Professionals 
Those office professionals on the implementation team who complete a battery of quantitative questionnaires at 5-months and 
15-months and will receive $15 in compensation per assessment point. All office professionals will be receive $20 compensa- 
tion for completing the abbreviated battery at the three time points: baseline, at 5-month follow up, and at 10-month follow 
up 
 
Staff 
Staff will receive $20 compensation for completing the abbreviated battery at the three time points: baseline, at 5-month fol- 
low up, and at 10 months after active implementation. When a paper version of the baseline is sent, an additional $1 incentive 
will be included. 
 
2. Justify the proposed payment arrangements described in section B. (e.g., how this proposed payment arrangement is not con- 
sidered to be coercive). 
 
Clinicians  
Centerstone clinicians make an average of $23.5/hour; therefore, this compensation amount is appropriate for the time and 
effort required for measure completion and focus group participation. The continuing education credits received for attending 
the MBC training are meant to directly offset the costs of participating. 
 
Clients 
Clients will receive $10 for each completed PHQ-9 assessment across two time points (over the phone at intake, and via 
phone in week 12 of treatment). This payment is meant as an incentive to complete the PHQ-9 assessment. An additional $20 
may be earned by completing the phone based messaging survey after each session, which is meant as incentive to complete 
the text messaging surveys. No compensation/incentive will be provided for client PHQ-9 completion prior to other sessions 
in order to maximize external validity (i.e., mimic the “real world” situation in which MBC would otherwise be expected to 
occur without compensation) and to facilitate sustainment 
 
Administrator 
33 

SECTION XII: INFORMED CONSENT PROCESS 
Administrators will receive $30 for completing an individual interview, $15 per assessment completed as part of participating 
on the implementation teams and $20 for completing the abbreviated battery. We believe these amounts are appropriate for 
the time and effort required to participate in the individual interviews and complete the battery of questionnaires. 
 
Office Professionals 
Office professionals will receive $15 per assessment completed as part of participating on the implementation teams and $20 
per abbreviated battery completed. This payment is believed to be an appropriate amount for the time required to complete 
the battery of questionnaires. 
 
Staff 
Staff will recieve $20 per abbreviated battery completed. This payment is believed to be an appropriate amount for the time 
required to complete the battery of questionnaires. Staff will receive $1 when they receive the paper version of the survey. 
This payment is believe to an appropriate additional incentive to complete the paper version of the survey. 
 
 
3. Explain if there will be any partial payment if the subject withdraws prior to completion of the study (e.g. prorated). Note: 
This payment may be paid at the end of the subject’s participation or at the end of the study. 
 
There will not be partial payment. Clinicians who complete the data collection procedures (i.e., assessments and focus 
groups) will receive the aforementioned compensation or continuing education credits. Similarly, clients who complete the 
PHQ-9 assessment at baseline and/or session 12 will receive the $10 incentive for each assessment completed. Administra- 
tors and Office professionals who complete the data collection procedures outlined above will receive the previously men- 
tioned compensation. 
 
 
   A.   I WILL be obtaining informed consent from all subjects. 
 
1. When (in what timeframe) and where (what setting) will consent take place? Indicate any waiting period between 
informing the subject and obtaining consent. The timeframe and any waiting should ensure the prospective subjects or 
their legally authorized representatives are provided sufficient opportunity to consider whether or not to participate in the 
study. 
 
Consenting Clients 
There is one pathway to identify clients for participation in the study. For full details on the recruitment process, please 
see the HIPAA Recruitment Checklist. There will be one Centerstone Research Institute research assistant (CRI RA) 
responsible for sites located in Tennessee and one CRI RA responsible for sites located in Indiana. Centerstone Research 
Institute has established a system for identifying clients eligible for the various ongoing research projects. Specifically, 
each CRI RA will run queries of the Electronic Health Record system database daily to identify potentially eligible cli- 
ents seeking individual psychotherapy for depression with enrolled study clinicians based on the above listed inclu- 
sion/exclusion criteria (all eligibility information is collected during the intake and stored in the database). The average 
time from intake to the first psychotherapy session at Centerstone is 13 days, which will provide ample time for eligible 
clients to be enrolled in the study prior to the first therapy session. However, to ensure that the potential participant is 
contacted in a timely manner before their first psychotherapy session, the queries will be run daily. Upon identification 
of potential eligibility, the CRI RAs will contact the client via telephone within 24 hours to perform an eligibility 10-20 
minute phone screen. The initial phone screen (RCR.3) is to confirm eligibility for participation, which is based on hav- 
ing a PHQ-9 score greater than 9 and a willingness to participate in the study. The CRI RA will verbally administer the 
PHQ-9 (in its entirety) to confirm project eligibility. If the client is determined to be eligible, the CRI RA will go over 
the informed consent statement (study procedure, risks/benefits, confidentiality, etc.) and ask the client if s/he is willing 
to give verbal consent to participation in the study at that time or if s/he would like to first receive and review the in- 
formed consent statement and the authorization for the release of protected health information. The CRI will stress that 
participation is voluntary and that the client may withdraw consent at any time. Following the phone call, consented par- 
ticipants and those interested in consenting upon review of the informed consent statement will be mailed a Welcome 
Packet, which will contain the informed consent for the client’s records. 
Three days after the Welcome Packet is sent out, CRI RAs will then follow up with clients by telephone so that the client 
may have the opportunity to ask questions about the study. At this time, the CRI RAs may also offer to visit clients in 
person at their local Centerstone site or in their home (only if the RA is comfortable with this option) to discuss remain- 
ing questions about the research project further. Finally, although clinicians are not formally a part of the recruitment 
process, we have created an information sheet for clinicians in case his or her client has questions about the study the 

clinician will be able to provide some background (RCR.8) and a client study brochure (Appendix F.2) that s/he can of- 
fer the client. 
 
Consenting Administrators 
Administrators will first be sent a recruitment email with overall study information and their role in their study (RCR.5). 
Administrators will be consented via two pathways: 1) in person, implicit consent during the initial site visit staff meet- 
ing and 2) email recruitment with implicit consent. The implicit consent procedure is described here: 1) During a staff 
meeting at each site, a member of the research team will provide staff members with a welcome packet and verbally give 
information about the study. Food will be provided to each site administrator in attendance. The welcome packet will 
include a two administrator informed consent forms and the project coordinator’s contact information. The research 
member will indicate that participation in the study is voluntary and will in no way affect their status as administrators at 
Centerstone. Administrators will have the opportunity to fill out the battery of measures thus giving their implicit con- 
sent to participation in person at this initial site visit or if the administrator would like to take additional time to review 
before filling out the battery and therefore consenting, s/he will be given a pre-paid, return-addressed envelope to mail 
the completed battery of measures when s/he is ready OR at the start of the next staff meeting a member of the research 
team will attend the first few minutes to answer questions and collect surveys. The email recruitment/ consent is detailed 
here: For those administrators who could not attend this staff meeting, a recruitment email (RCR.5) will be sent out to 
these administrators by a member of the research team with an e-version of the welcome packet for site administrators 
should they prefer to complete the battery of measures through an electronic survey option. Administrators may give 
their implicit consent to participation by filling out the baseline battery. These administrators will be contacted by CRI 
RAs directly to see if they have any questions or concerns about participating. 
 
Consenting Office Professionals 
Office professionals will be recruited and consented via two avenues: 1) in person, implicit consent during the initial site 
visit and 2) email recruitment with implicit consent. The in person, implicit consent is described here: During a staff 
meeting at each site, a member of the research team will provide staff members with a welcome packet and verbally give 
information about the study. Food will be provided to each site office professional in attendance. The site administrator 
will send out an email (RCR.6) with information about this site visit to the office professionals. The welcome packet will 
include a two office professionals’ informed consent forms, study recruitment letter (RCR.7), and the project coordina- 
tor’s contact information. The research member will indicate that participation in the study is voluntary and will in no 
way affect their status as office professional at Centerstone. Office professional will have the opportunity to fill out the 
abbreviated battery of measures in person at this initial site visit or if the office professional would like to take additional 
time to review before filling out the abbreviated battery and therefore consenting, s/he will be given a pre-paid, return- 
addressed envelope to mail the abbreviated battery of measures when s/he is ready OR at the start of the next staff meet- 
ing a member of the research team will attend the first few minutes to answer questions and the abbreviated battery. For 
those office professionals who could not attend this staff meeting, 2) the site administrator for each site will place a wel- 
come packet in those office professionals’ mailboxes. State specific CRI RAs will then individually contact (via phone or 
email) these office professionals after a few days to see if s/he has any questions about the consent or participation pro- 
cess. The email recruitment/electronic implicit consent is described here: The other option for office professional recruit- 
ment and consent is via email recruitment for those office professionals who cannot attend the initial site visit with the 
research team. A recruitment email will (RCR.7) be sent out to these office professionals by a member of the research 
team with an e-version of the welcome packet should they prefer to complete the battery of measures through an elec- 
tronic survey option. These office professionals will then have the option to implicitly consent by filling out the abbrevi- 
ated baseline battery. These office professionals will be contacted by CRI RAs directly to see if they have any questions 
or concerns about participating. 
 
Consenting Staff 
Staff will be recruited and consented via two avenues: 1) in person, implicit consent during the initial site visit and 2) 
email recruitment with implicit consent. The in person, implicit consent is described here: During a staff meeting at each 
site, a member of the research team will provide staff members with a welcome packet and verbally give information 
about the study. Food will be provided to each staff member in attendance. The site administrator will send out an email 
(RCR.9) with information about this site visit to the staff. The welcome packet will include two staff informed consent 
forms, study recruitment letter (RCR.7), and the project coordinator’s contact information. The research member will 
indicate that participation in the study is voluntary and will in no way affect their status as a staff member at Centerstone. 
Staff will have the opportunity to fill out the abbreviated battery of measures in person at this initial site visit or if the 
staff member would like to take additional time to review before filling out the abbreviated battery and therefore con- 
senting, s/he will be given a pre-paid, return-addressed envelope to mail the abbreviated battery of measures when s/he is 
ready OR at the start of the next staff meeting a member of the research team will attend the first few minutes to answer 
questions and the abbreviated battery. For those staff members who could not attend this staff meeting, 2) the site 
35 

administrator for each site will place a welcome packet in those staff members’ mailboxes. State specific CRI RAs will 
then individually contact (via phone or email) these staff members after a few days to see if s/he has any questions about 
the consent or participation process. The email recruitment/electronic implicit consent is described here: The other option 
for staff recruitment and consent is via email recruitment for those staff who cannot attend the initial site visit with the 
research team. A recruitment email will containing the ICS be sent out to these staff members by a member of the re- 
search team with an e-version of the welcome packet should they prefer to complete the battery of measures through an 
electronic survey option. These staff members will then have the option to implicitly consent by filling out the abbrevi- 
ated baseline battery. These staff members will be contacted by CRI RAs directly to see if they have any questions or 
concerns about participating. 
 
2. Who will be responsible for obtaining initial and ongoing consent? (check all that apply) 
 
Principal Investigator 
Co-Investigator 
Other (specify): Approved investigators, project coordinator, research assistants 
 
a. 
Explain how these individuals will be adequately trained to conduct the consent interview and answer sub- 
ject’s questions (check all that apply): 
 
Passed the required Collaborative Institutional Training Initiative (CITI) modules 
Attended the Research Coordinator Education Program (RCEP) 
Attended the Research Coordinator Certification Program (RCCP) 
Received study-specific training from study personnel 
Other (specify): HIPAA training, National Institutes of Health Human Subjects Training 
 
b. Indicate in what language(s) the consent interview will be conducted. 
 
English 
Spanish 
Other (specify): 
 
c. 
If the consent interview will be conducted in a language other than English, state how the interview will be 
conducted (e.g. use of an interpreter): 
 
3. Explain how subjects’ privacy will be protected during the consent process. 
This refers to how access to subjects 
will be controlled (e.g. time, place, etc. of consent procedures). 
 
Clients 
A research assistant will call the potential client participant and ask if they are in a private setting and whether it would 
be convenient to speak about their interest in participating in the study. If it is not an optimal time for the client then the 
research assistant will schedule a more appropriate/convenient time in which to call the client back. 
 
Administrators 
A CRI RA will give the administrator participants an opportunity to set up a separate time to offer more information and 
answer any questions the administrator would have at their convenience over the phone, via email, or in person at Cen- 
terstone. 
 
Office Professionals 
A CRI RA will give the office professional participants an opportunity to set up a separate time to offer more infor- 
mation and answer any questions the office professional would have at their convenience over the phone, via email, or in 
person at Centerstone. 
 
Staff 
A CRI RA will give the staff participants an opportunity to set up a separate time to offer more information and answer 
any questions the office professional would have at their convenience over the phone, via email, or in person at Center- 
stone. 
 
 
1. Indicate any factors that might result in the possibility of coercion or undue influence. (check all that apply) 

the research will involve students of the investigator(s) 
the subjects will be recruited through institutions with which the PI has a close relationship 
Other (please specify): potential for clinicians to perceive coercion to participate in the study given that this proposal 
aligns with the agency’s overall strategic goals. 
 
Describe steps taken to mitigate the possible coercion: It is not possible to conceal clinician, administrator, or office pro- 
fessional study participation from supervisors. However, each supervisor will be instructed in human subjects research pro- 
tections and made aware of the seriousness of research participant coercion in any form. Further, the supervisors and their 
managers will be instructed not to use participation in research as criteria for adjustment of wages or job position. The clini- 
cian, administrator, and office professional participant will be made aware of this risk and given the contact numbers of the 
compliance office at Indiana University to report any unfair treatment based on their participation, or lack of, in research. 
Moreover, even though we plan to randomize at the site level, not all clinicians at each site will need to participate in order 
for recruitment goals to be met. Therefore, clinicians will be explicitly informed that even though their site is enrolled and 
they may be eligible, there is no obligation to participate in the research. 
 
   C. I am requesting a waiver of written documentation of informed consent (i.e. a consent process will occur, but no signature 
will be obtained from the subject). 
 
   Written statement regarding the research has been attached. Statement will be provided to subjects upon their request. 
Please explain: 
 
Consenting Clients 
Clients will consent for participation in the study after completing the eligibility screening over the phone with a CRI 
RA. Once the client has been determined eligible the CRI RA will go over the informed consent statement (study proce- 
dure, risks/benefits, confidentiality, etc.) and authorization for the release of protected health information and ask the 
client if s/he is willing to give verbal consent to participation in the study at that time or if they would like to first receive 
and review the informed consent statement. The CRI will stress that participation is voluntary and that the client may 
withdraw consent at any time. At this time, the client may verbally consent to participation or call back after reviewing 
the informed consent statement. Following the phone call, consented participants and those interested in consenting upon 
review of the informed consent statement will be mailed a Welcome Packet, which will contain the informed consent for 
the clients record. After clients give their verbal consent to participation, they will be enrolled in the study and data col- 
lection can begin. 
 
Consenting Clinicians  
All clinicians who meet the inclusion criteria presented below and who provide services at any of the 12 Centerstone 
sites involved in the proposed research will be eligible for participation in the study. There are three pathways through 
which the clinicians will be recruited and one way they may consent to participate in the research. The research team will 
recruit clinicians for participation via email and site visits. 
Recruitment Option 1: During a staff meeting at each site, a member of the research team will provide staff members 
with a welcome packet and verbally give information about the study. One hour of productivity credit coverage and food 
will be provided to each clinician in attendance. The site administrator will send out an email (RCR.1) with information 
about this site visit to the clinicians. The welcome packet will include two Clinician informed consent forms, a study 
recruitment letter (RCR.2), the baseline battery of questionnaires, and the project coordinator’s contact information. The 
research member will indicate that participation in the study is voluntary and will in no way affect their status as clini- 
cians at Centerstone. The recruitment letter as well as the research member(s) present at the meeting will inform clini- 
cians that by filling out and returning the baseline battery of questionnaires to the researchers, s/he is agreeing to partici- 
pate in the research project and all of its activities. However, if the clinician would like to take additional time to review 
the informed consent before implicitly consenting, s/he will be given a pre-paid, return-addressed envelope to mail the 
filled out baseline battery of questionnaires when s/he is ready OR at the start of the next staff meeting a member of the 
research team will attend the first few minutes to answer questions and collect the filled out battery of questionnaires. 
Recruitment Option 2: 
For those clinicians who could not attend this staff meeting, the site administrator for each site will place a welcome 
packet in the clinician mailboxes of those who could not attend. A research member will then individually contact (via 
phone or email) these clinicians after a few days to see if s/he has any questions about the consent or participation pro- 
cess. 
Recruitment Option 3: The other option for clinician recruitment and consenting is via email recruitment for those clini- 
cians who cannot attend the initial site visit with the research team. A recruitment email (RCR.2 [the bottom portion will 
be revised for email recruitment]) will be sent out to these clinicians by a member of the research team with an e-version 
of the welcome packet should they prefer to complete the battery of measures through an electronic survey option. This 
37 

recruitment email will have a link to the web-based survey of the baseline battery of questionnaires. Upon accessing the 
web-based survey, study participants will be presented with a web page containing the informed consent and will be in- 
formed that clicking the “proceed” button will be taken as implicit consent to the participate in the study and all its activ- 
ities. This page will include the contact information of the research team, so participants may contact the researchers 
with any questions they may have. In addition, these clinicians will be contacted by a research member directly after 
being sent the recruitment email to see if they have any questions or concerns about participating. 
 
Consenting Administrators 
Administrators will give their implicit consent for participation in the study by filling out the abbreviated baseline survey 
during the initial site visit. This implicit consent will be made clear verbally by a member of the research team during the 
visit and in the written recruitment materials and informed consent statement. See question A1. for full details of the 
informed consent process. 
 
Consenting Office Professionals 
Office processionals will give their implicit consent for participation in the study by filling out the abbreviated baseline 
survey during the initial site visit. This implicit consent will be made clear verbally by a member of the research team 
during the visit and in the written recruitment materials and informed consent statement. See question A1 for full details 
of the informed consent process. 
 
Staff 
Staff will give their implicit consent for participation in the study by filling out the abbreviated baseline survey during 
the initial site visit. This implicit consent will be made clear verbally by a member of the research team during the visit 
and in the written recruitment materials and informed consent statement. See question A1 for full details of the informed 
consent process. 
 
 
   2. 
The research presents no more than minimal risk of harm to subjects and involves no procedures for which written 
consent is normally required outside of the research context. Please explain: 
 
Because the research intends to remove identifiers in the data collection process, there is minimal risk to clinicians, 
clients, administrators, office professional and general staff. Per NIH requirements, we propose to make the quanti- 
tative data available for secondary data analysis, but careful removal of identifiers will occur prior to making these 
datasets available. Rather, participation in a study of this nature (implementation of measurement-based care) sug- 
gests a clinician’s client may benefit from improved treatment outcomes. Moreover, tracking client symptoms using 
a measurement-based care is a normal clinic procedure in many clinics and receiving training in measurement-based 
care and continuing education credits will likely benefit the clinician’s skills and expertise. Centerstone is trying to 
adopt this practice, so we will just be receiving de-identified data for research purposes to evaluate the effectiveness 
of the implementation process. Moreover, not all of the clinicians will be able to attend the staff meeting when the 
initial site visit will occur, making collecting face-to-face informed consent for 187 clinicians across 12 sites and 
two states (Indiana and Tennessee) for whom we will not be collecting identifiable information is not feasible. In 
addition, requiring clinicians to schedule a unique face-to-face individual meeting with a research assistant to pro- 
vide written consent is unnecessary particularly given the data will be coded so identification will not be possible. 
It is also unfeasible to collect written consent from clients because this study involves the recruitment of 625 clients 
across 12 Centerstone sites scattered (often great distances) across Tennessee and Indiana. It is impossible for our 
two Centerstone Research Institute research assistants (CRI RAs) to meet clients at the clinic to collect signed con- 
sent documents in person. Furthermore, it is not possible to properly train clinicians at each site to consent clients 
nor do we want to add this burden to clinicians’ already full workload. It is also difficult to collect signed consent 
forms from clients in the mail do to the nature of depression which all eligible clients have a part of their diagnosis. 
Specifically, depression is associated with a loss of interest in activities, difficulty concentrating and fatigue, all 
which interfere with daily functioning. Therefore, clients interested in participation in the study may be prevented 
from doing so due to the nature of their diagnosis and tedious, time consuming procedure of consenting to partici- 
pate which would involve reviewing and signing multiple forms and then returning them in the mail. Thus, waiving 
written documentation of informed consent improves feasibility and promotes recruitment rates and will likely bene- 
fit participating clinicians and clients.