CPIC is a Community Partnered Participatory Research (CPPR) Project of Community and Academic Partners Working Together to Learn the Best Way to Reduce Depression in Our Communities.
Phase: NA
Conditions: Depression, Information Dissemination, Social Determinants of Health
Sponsor: RAND

 
 
 
Study Protocol and Statistical Analysis Plan 
Community Partners in Care (CPIC) 
 
 
Principle Investigator: Kenneth B. Wells, MD, MPH 
 
Funded by: NIMH, RWJF, PCORI 
NCT01699789 
1-27-2021  
 
Note: Materials in this file were presented in previous publications (5/7/2013 for 6 months, 
11/18/2014 for 12 month, and 7/17/2017 for 3-years). This protocol integrates those materials 
and was edited for CT.gov in January 2021.

Page 2 of 72 
 
 
Table of Contents 
Page 
1 
Introduction 
3 
2 
Specific Aims 
5 
3 
Background and Significance 
6 
4 
Research Design and Methods 
15 
 
4.1 Sampling 
16 
 
4.2 Intervention Assignment 
19 
 
4.3 Intervention Conditions 
20 
 
4.4 Data Sources and Measures 
25 
5 
Statistical Analyses Plan and Power Analyses for Client Outcomes Analysis 
30 
6 
CPIC 3 years extension 
36 
7 
Limitations 
38 
8 
Protection of Human Subjects 
39 
9 
Data and Safety Monitoring Plan 
52 
10 
Inclusion of Women and Minorities   
56 
11 
12 
Summary of Initial Design, Community Input, Modifications, and Final Design 
Human Subjects Protocol 
56 
57 
13 
Literature Cited 
 
59 
 
Appendix A 
 
A1 
Sample 
A-2 
A2 
Weighting 
A-3 
A3 
Multiple Imputation  
A-4 
A4 
Analyses 
A-5 
A6 
References for Appendix 
A-9 
 

Page 3 of 72 
 
1. Introduction 
This document describes the study protocol and general analytic approach for the primary 
objectives of Community Partners in Care, including changes that were implemented over the 
course of the design phase post funding but prior to client enrollment, in response to planned 
community input, budget reductions, and additional grants obtained to support supplemental 
components. Specifically, there was an initial funded National Institute of Mental Health 
proposal and modifications due to budget reductions.  A key feature of the project was 
participatory collaboration in developing the design with community co-leadership and input.  
This led to further changes in the design, including an expansion of outcomes to include 
employment status and homelessness risk factors and prioritization among other outcomes; 
expansion of the scope of agencies, provider types, and client populations to include a wider 
range of community-based agencies and vulnerable client populations such as the homeless and 
prisoner re-entry clients, seniors, substance abusers, and neighborhoods with high concentrations 
of African Americans.  To respond to these community priorities, a competing supplement 
proposal was submitted and two of four requested years were funded, which resulted in an 
expansion in expected client sample size and restoration of some data collection elements 
dropped with budget cuts (e.g., 12-month client outcomes).  Subsequently, there was a second 
wave of community input with final changes in outcomes to include homelessness risk factors as 
outcomes resulting in a decision to increase client enrollment by a further 20% because with 
inclusion of the homeless and criminal justice populations there was an expected decrease in 
follow-up response from 80 to 65-70%.  In addition, at this stage community input led to some 
changes in intervention design, including: 1) inclusion for both intervention conditions of 
materials to support lower level staff in providing education, case management, screening and 
tracking outcomes; and resources for team management; as well as: 2) an increase in the 
intensity of the technical assistance intervention to be comparable to best-available technical 
assistance for evidence-based practice in public sector agencies.  The final protocol that was 
implemented remains broadly consistent with the original aims of the project but with outcomes 
more consistent with community priorities for functional status, wellness, and avoiding 
homelessness; a more intensive comparison intervention; a more diverse agency sample with 
more emphasis on community agencies; and greater planned client enrollment.  Because this 
document concerns client outcomes only, we present information in this Protocol only on aims, 

Page 4 of 72 
 
measures and analyses concerning client outcomes, rather than administrator or provider 
outcomes.  
To present the protocol and its evolution, we followed this strategy: 1) present the 
original funded proposal, modified to remove components that were eliminated with budget cuts 
but including components added or restored with the second revision proposal; 2) notation in 
italics in the text for each section changes due to community input.  We then summarize in Table 
5 (pages 39-40) major features of the protocol, highlighting changes in response to community 
input.  Please see the Appendix A for information on detailed technical handling of issues for the 
analysis of 6-month client data as well as 12-month and 3 years follow-ups.  By ignoring 
italicized information in the protocol the reader can understand the original protocol.  By 
including italicized information the reader can understand the evolution of protocol changes in 
response to planned community input and budget changes. 
 
Acronyms and glossary of terms: 
QI = Quality improvement or quality improvement program 
PIC=Partners in Care (a prior research study evaluating depression QI relative to UC care in 
primary care practices) 
WE Care (a prior research study evaluating treatments for depression in low-income, minority 
women) 
CPIC=Community Partners in Care 
CBPR=Community-based participatory research 
CPPR=Community-partnered participatory research (a form of CBPR emphasizing co-equal 
leadership in all phases of research through two-way knowledge exchange and a structure 
including a Council and workgroups following three development phases of Vision or planning, 
Valley or work, and Victory or products and celebration). 
CEP=Community Engagement and Planning Intervention for CPIC 
RS=Resources for Services (Technical Assistance) Intervention for CPIC 

Page 5 of 72 
 
2. Specific Aims  
Under-resourced communities of color in low income, largely ethnic-minority neighborhoods 
face an excessive burden of illness from depression due to higher prevalence of depression and 
lower access to quality care. 1   Evidence-based quality improvement (QI) programs for 
depression in primary care settings—where many low-income and minority patients receive their 
only mental health care—can enhance quality of depression care and improve health outcomes. 
These programs are under-utilized in community-based health care settings, and have not been 
adapted for use across diverse agencies (social service, faith based, primary and specialty care) 
that could partner to support disease management for depression.  Partners in Care (PIC)2, 3  and 
WE Care4 are interventions designed to improve access to evidence-based depression treatments 
(medication management or psychotherapy) for primary care patients and, in WE Care, social 
service clients. PIC evaluated a services delivery intervention while WE Care was an 
effectiveness trial with study-provided treatments. Both studies promoted use of the same 
evidence-based treatments. Both PIC and WE Care programs improved use of evidence-based 
treatments for depression and health outcomes for African Americans and Latinos.  The PIC 
interventions reduced health outcome disparities evident in usual care in the first follow-up year 
and at five-year follow-up.  While these findings offer hope to under-resourced communities, 
such communities have poor resources to support implementation of these programs, and may 
have historical distrust in research and health care settings. There is no evidence-based approach 
to support agency networks in under-resourced communities in implementing QI programs for 
depression.  To address this information gap, we proposed Community Partners in Care (CPIC), 
a group-level randomized, controlled trial, with randomization at the level of an agency site or 
“unit.” The trial is fielded in two underserved communities, Hollywood and South Los Angeles, 
and conducted through a community- participatory, partnered research (CPPR) approach.  The 
specific aims are: 
1. To engage two communities of color in improving safety-net care for depression.  
2. To examine the effects on three levels of stakeholders of an innovative, community-
engagement and network capacity building intervention that implements PIC and WE Care 
QI toolkits, compared to low-intensity dissemination (i.e., control).  The control condition 
that includes review of findings and toolkits in a conference followed by access to PIC 
toolkits on a website.  The stakeholders and outcomes of interest are:  

Page 6 of 72 
 
a. Agency adoption of QI toolkits and depression treatments. (Not discussed in this Protocol). 
b. Provider participation in QI trainings in each intervention condition; and intervention effects 
on provider attitudes and knowledge toward depression care and adoption of PIC/WE Care 
toolkits and depression treatments. (Not discussed in this Protocol). 
c. Client mental health-related quality of life and depressive symptoms, access to care and 
quality of care, employment status and work days lost, and risk factors for homelessness. 
(Discussed in this Protocol).  
This study builds on our extensive prior work in PIC/ WE Care and pilot work in the Witness 
for Wellness initiative to apply a model of community engagement in under-resourced 
communities of color to depression QI, and in the Community Partnership Initiative to develop 
agency networks in under-resourced communities in Los Angeles that are concerned with mental 
health and substance abuse issues. The proposal initiates a long-term research agenda, by 
developing evidence on the relative effectiveness of a community engagement approach and a 
more standard time-limited technical assistance approach to implementing community-wide 
quality improvement programs for depression in under-resourced communities. 
 
3. Background and Significance 
Overview: In the last decade, there has been increasing attention by policymakers, providers, the 
public, and scientists to the “quality chasm” or gap between the promise of clinical research and 
the realities of clinical practice, including for psychiatric conditions such as depression.5, 6  The 
Institute of Medicine defines quality as the effectiveness, efficiency, client-centeredness, 
timeliness, and equity of care. 6  McGlynn and her colleagues found that only 55% of persons 
with a chronic health condition received appropriate care; and quality of care for depression was 
close to this mediocre average.7  There has also been increasing attention to disparities in access, 
quality, and outcomes including for psychiatric disorders.1-3, 8  While appropriateness of care 
among treated patients is low across geographic regions and demographic groups,9, 10 
underserved minorities, including African Americans and Latinos, have lower access to 
appropriate care for depression overall and in primary care.4, 11-14  We propose to address this 
disparity-related gap in access to quality care, through evaluating the effectiveness of a 
community-engagement intervention, relative to a time-limited, individual program technical 
assistance intervention, to implement evidence-based QI programs for depression, originally 

Page 7 of 72 
 
developed for primary care settings but modified in this study to be applicable potentially to a 
broad range of community-based health and social services agencies.4, 13 We hypothesize that the 
community engagement approach will be more likely to increase agency and clinician use of the 
programs across the network, thereby increasing client access to appropriate care and health 
outcomes.  We also expect that the community engagement approach, through increasing access, 
will increase services use across health and social services sectors.  
 
While there has been increasing attention to workforce impacts of mental health 
conditions including depression,15 few data are available on the effects of improving care for 
depression on employment in underserved communities, particularly on entry into employment 
for the unemployed.  Employment and work productivity losses comprise 62% of the economic 
burden of depression, and are estimated to be 51.5 billion dollars in 2000.16, 17  QI programs for 
common mental disorders in primary care or through employer-managed programs can improve 
employment outcomes, either for employed subgroups, or across patients including employed 
and unemployed.18-21  Employment outcomes of such programs among minorities were reported 
in PIC.22  While intervention effect sizes were qualitatively similar or larger for minorities than 
for whites, they were statistically significant only among whites or whites and minorities pooled, 
owing to limited precision for analyses among minorities.  The present study also offers an 
opportunity to determine, in a largely low-income minority depressed group, whether a 
community engagement approach to implementing quality improvement for depression, 
compared to a more standard individual program technical assistance approach, leads to 
improved employment outcomes, addressing an important gap in the literature. 
 
The existence of a quality gap and outcome disparities constitute evidence of a “science 
dissemination” gap23-27  The NIH has proposed to address this gap through PAR-06-039, 
stimulating an improved science of dissemination and implementation for evidence-based 
practice.  Our trial proposed in response to this PAR features a community engagement 
intervention that promotes community commitment and leadership to form a network committed 
to evidence-based, quality improvement intervention approaches.  We suggest that a sound 
framework to improve depression care in underserved communities should incorporate effective 
interventions at three levels: evidence-based treatments; quality improvement strategies; and 
community-based implementation strategies that can be applied across the diverse agencies 
supporting underserved clients. Underserved communities of color often have poor resources to 

Page 8 of 72 
 
implement comprehensive programs, and may have historical issues of distrust in science and 
healthcare services.28, 29 Community engagement promotes organizational and community 
member participation and leadership in goal setting, program development, implementation and 
evaluation by shifting the authority for action to the community.30-33   
Community engagement is one strategy within community-based participatory research, 
the leading paradigm recommended for health intervention in underserved communities.34-39 
Recent recommendations for revitalizing clinical research call for increased public participation 
and community engagement,40-42 while management sciences have produced similar calls for 
“action research”43, 44 and “engaged scholarship.”45 We have proposed an approach to implement 
evidence-based QI programs in mental health, in which community agencies divide up the roles 
and functions required for evidence-based QI programs, among different network partners.37  
Following this framework, we piloted a community engagement approach 31, 38, 46 to building 
depression-related networks, and developed a manual for community members and a companion 
guide for academic leaders in the skills of community engagement.30, 32  We have not formally 
tested the effectiveness of this model relative to an alternative, however.  Previously, we 
provided evidence on the effectiveness of the QI interventions 4, 13, 47 including in African 
Americans and Latinos 2, 3  and on the effectiveness of treatments. 4, 48  This proposal takes the 
next step by testing the community engagement implementation model relative to a more 
standard, individual agency technical assistance model.  
Depression as a Tracer Condition: Depression, i.e., unipolar depressive disorder, is an excellent 
“tracer” condition for studying quality of care because of its high prevalence (at least 8-10% of 
the population in a year49 and associated morbidity.  Our prior work has shown that depression is 
associated with as much or more limitation in daily functioning and well-being as most major 
chronic conditions such as diabetes 50-52 and major depressive disorder is a top contributor to the 
total, worldwide disability-burden of disease.53, 54  Further, evidence-based treatments exist, 
including brief, structured psychotherapy such as Cognitive Behavioral Therapy and 
antidepressant medications.  Further, standards for appropriate care are formulated 55-61.  
Depressive symptoms below the threshold of depressive disorder are even more common than 
depressive disorders and substantially affect functional status. 62, 63   While the efficacy of 
standard treatments for depressive disorders among persons with minor depression is uncertain,64 
we found that QI programs for depression that support care decisions over time improved 

Page 9 of 72 
 
outcomes for persons with initial depressive disorder or subthreshold symptoms, and improved 
outcomes for those with subthreshold symptoms at 5-year follow-up.2, 3  We use a broad 
definition of depression for this proposal, “probable depressive disorder” that includes persons 
with depressive disorder and those with depressive symptoms without having a current 
depressive disorder, because that full range can benefit from disease management. 
Many persons with depression do not receive mental health care, and most receive such 
care through primary care or community agencies, such as faith-based settings.11, 65, 66   Primary 
care clinicians often do not detect depression62, 67 and many patients do not receive evidence-
based treatments.13, 66, 68 Depression was selected by the Institute of Medicine as one of ten 
conditions for tracking quality of care nationally. 69  Delivering evidence-based care for 
depression is challenging, owing to organizational and financing factors, such as limited 
psychotherapy coverage or diversity in third-party management of services; clinical features of 
depression such as social withdrawal; societal factors such as social stigma and limited public 
knowledge; and clinician factors such as limited knowledge or experience.5, 14, 29, 70-72  For 
example, while antidepressant medication use rose over the last decade, many in need of 
depression care did not receive counseling, medications, or referral.66 The poor and ethnic 
minorities are particularly unlikely to receive needed care.14, 29, 73   There are many barriers to 
provision of evidence-based psychotherapy such as lack of criteria for licensing providers in 
such treatments.5, 38  Outside of organized group practices, coordination between primary and 
mental health care is often difficult, given widespread use of “carve-out” behavioral health 
management companies that operate largely independently of health plans or medical practices.5, 
26  Public sector mental health agencies prioritize clients with severe and persistent illness, while 
public primary care agencies are not organized or financed to support ongoing mental health 
care.  The proposed community engagement intervention is designed to help compensate for 
these factors by developing networks that can share depression management and treatment 
responsibilities and roles. 
Quality Improvement for Depression: There have been several demonstrations of the 
effectiveness of QI programs for depression in primary care74 suggesting that: 1) collaborative 
care75 improves quality of care and health outcomes for depressed patients14, 76-83; more efficient 
methods, such as telephone case management improve outcomes84-86; 3) multimodal strategies 
that include case management tend to be more effective than improving clinician knowledge or 

Page 10 of 72 
 
skills alone87-89; 4) such programs can improve outcomes for adolescents90 and the elderly81, 91; 
and 4) active outreach strategies help engage low-income and minority groups in such care.4, 92 
The President’s New Freedom Commission on Mental Health recommended implementing QI 
programs for depression, citing the IMPACT study as a model.93 We lead or participated in a 
number of these trials. Other groups have been stimulating dissemination through financing 
innovations,94, 95 information technology,81, 96 clarifying gains for purchasers 
(caremangementfordepression.org),18 or developing private sector networks.97   
Partners in Care (PIC) was a group-level randomized trial in which managed, primary 
care practices were randomized to usual care or one of two QI programs that provided resources 
to encourage use of evidence-based treatments for depressed adults. Both interventions were 
implemented by local practice teams with study support and featured toolkits for educating 
clinicians, patients, and intervention staff.  One intervention facilitated use of evidence-based 
medication management through supporting nurse care managers for 6-12 months, while the 
other facilitated access to evidence-based therapy for depression through lowering the copay for 
sessions with therapists trained in Cognitive Behavioral Therapy (CBT) for Depression.60 The 
interventions improved quality of care, depression and quality-of-life outcomes and employment 
status over two years; and they generated cost-effectiveness ratios relative to usual care within 
the range expected for a widely-used medical therapy.13, 14, 47  African Americans and Latinos 
improved more in clinical outcomes than did comparable whites in the first follow-up year 4 and 
at five-years, particularly under the therapy-resource model.3  The PIC interventions facilitated 
use of evidence-based treatments following practice priorities and resources and patient and 
clinician preferences,14, 98, 99 and thus are suitable for use with a community-engagement 
approach because they permit the flexibility to fit implementation to network needs.  The PIC 
therapy-based QI is the proposed services intervention for this study because it was effective 
over time among Latinos and African Americans.3  WE Care is unique among depression 
interventions in encouraging and facilitating access to evidence-based treatments for depression 
among low-income, minority women who visited community social service agencies or health 
clinics. The study found that active outreach was necessary for increasing the engagement of 
low-income women in care, and that efforts to decrease structural barriers to care by facilitating 
transportation and childcare increased adherence to treatments.4  WE Care used an individual-
level randomized, controlled designed to compare enhanced community usual care to medication 

Page 11 of 72 
 
management or CBT; and the treatments improved outcomes relative to usual community 
services over the first year.4  WE Care has the social service setting and client activation 
approaches to complement the PIC services delivery approach.  The two studies used consistent 
treatment protocols. 
CPPR and Community Engagement Depression Initiatives:  Community Participatory Partnered 
Research promotes equal partnership of community and academic partners while building 
capacity for partnered planning and community implementation of research-informed 
programs.30  This is accomplished through a process of community engagement and sharing of 
resources and expertise.  A CPPR project involves identifying a health issue that fits community 
and academic priorities; developing a coalition of stakeholders who inform and support the 
initiative; engaging the community through activities that provide information, determine 
readiness to proceed, and obtain input; and initiating work groups that develop, implement, and 
evaluate action plans.31 CPPR is strength-based and celebrates community capacities, and can 
support use of evidence-based programs within a participatory approach.31, 38, 100 This project is 
conducted within a CPPR framework, not just because it evaluates a community engagement 
intervention, but because the study was designed and will be implemented with community input 
and co-leadership.  We use the term community to refer to persons who work, share recreation or 
live in a given area.  Community engagement refers to values, strategies and actions that support 
authentic partnerships, including mutual respect and active and inclusive participation; power 
sharing and equity; mutual benefit; and flexibility in pursuing goals, methods, and timeframes to 
fit priorities and capacities of communities.30, 36   
Because we found long-lasting outcome gains for depressed minorities in PIC and WE 
Care, we initiated pilot studies of CPPR and community engagement concerning depression in 
the communities for this proposal. One pilot, Witness for Wellness (W4W), led by Healthy 
African American Families (witness4wellness.org),31 developed a coalition of 40 community 
agencies and academic institutions to assess, in the community, the salience of depression and 
the PIC/WE Care interventions. The coalition sponsored a conference attended by over 500 
community members and providers, in which support for this focus emerged.31  Data from 
surveys and field notes of discussion groups framed the charge for 3 working groups that 
formulated action plans that were approved by community members.101  Supporting Wellness 
focused on policy and advocacy, and secured membership in Los Angeles County’s planning for 

Page 12 of 72 
 
the Mental Health Services Act.102  Talking Wellness sought to reduce stigma and promote 
community strength.103  Building Wellness developed a pilot randomized trial of community 
outreach and linkage to appropriate depression services.32 Community members and researchers 
have been adapting PIC toolkits to promote screening, education, and referral through social 
service agencies and have negotiated a partnership for collaborate care between a free clinic and 
a public mental health clinic.  We also used a community partnered approach to develop an 
evidence-based intervention for youth exposed to community violence through QueensCare,46 
and we initiated a pilot study of the use of parish nurses to screen community members for 
depression and refer to safety-net clinics. We adapted Cognitive Behavioral Therapy for non-
mental health practitioners and piloted the intervention with parish nurses, substance abuse 
counselors, and social workers. These pilots inform CPIC.  
Conceptual Framework: The field of implementation and dissemination research is emerging in 
mental health.  The relevant literature derives from diffusion of innovation theory,104-106 and 
communication and health behavior change,34, 107 healthcare organization, financing, and 
management,108, 109 provider education and behavior change,110, 111 chronic disease and health 
promotion literatures.34, 75, 112  Implementation involves various stakeholders, and studying it 
requires concepts from diverse frameworks.113  
Figure 1 presents a framework for this study.  It specifies factors that can influence 
implementation of evidence-based QI programs for depression through a community-
engagement approach.  Community capacity for mental health planning includes individual and 
organizational capacities that affect their willingness and ability to implement and maintain QI 
interventions, and provide information and resources for learning about, assessing, and 
reinforcing them.  Individual factors include knowledge and attitudes toward depression 
treatment and clinician training, values and skills.  Organizational factors include organizational 
culture and climate, such as receptivity to innovation and commitment to improve depression 
care, and structural features that affect capacity to deliver programs, such as care routines, 
funding, skilled personnel, information technologies, or agency linkages.  Figure 1 suggests that 
features of evidence-based programs such as training resources may affect depression services 
delivery. According to diffusion theory, ‘complexity’ and ‘trialability’ may affect adoption.  In 
addition, resources for dissemination may interact with community priorities and capacities to 
influence practice. The arrow between these constructs represents the community-engagement 

Page 13 of 72 
 
process that draws agencies and communities into dealing with depression and supports them in 
finding a feasible fit between the requirements of programs and the priorities and resources in the 
network, while building community leadership to support implementation.  The downward arrow 
represents the implementation plans, their adoption, fit to network, and sustainability. Care 
influences outcomes of stakeholders, which influence capacities for community planning, 
iteratively.  Our design mirrors this framework. 
In Figure 1, we indicate by a star areas potentially affected by the proposed interventions.  
We stimulate community engagement through a participatory, network development model.31, 32, 
38  We introduce into these partnerships the PIC/WE Care toolkits and training and their 
adaptations for the capacities of diverse safety-net agencies.  The proposed study design 
experimentally varies organizational exposure to the community engagement intervention versus 
low-intensity dissemination. During the study, the lead community agencies (QueensCare, 
HAAF, DMH), host a conference that facilitates feedback on the outcomes of implementation. 
The proposed planning process may affect organizational commitment, a component of culture, 
and receptivity to innovation, a component of climate.  By asking organizations to work as a 
network, we hope to increase partnership capacity.  By stimulating leadership for implementing 
evidence-based toolkits, we hope to improve knowledge, attitudes, skills, and commitment to 
sustained change.   
Figure 1 is a framework and not a specific theory, but is suited to applying concepts from 
theories to guide hypotheses. We rely on concepts from diffusion of innovation45, 104 
organizational learning114, 115 116 and network theories117, 118 social movement theory,16, 119, 120 
social learning theory,121 a community engagement framework31 and a community intervention 
model for fitting intervention needs to agency capacity.3  PIC applies the collaborative care 
model of chronic disease management,75 which is based on diverse theories including social 
influence theory110, 122, 123 and social learning theory,124 CBT, one of the treatments promoted by 
PIC/WE Care, is based on social cognitive learning theory.59   

Page 14 of 72 
 
Community 
Capacity for Mental
Health Planning
Evidence-based
Resources (PIC &
We Care)
Community Depression 
Practices & Services
Outcomes
Client, Provider, Agency, Community
*Community Engagement +
Network Development
*Stakeholder Feedback
Figure 1: Framework for Community Engagement Implementation of Evidence-Based 
Depression Quality Improvement Programs
Cultural values 
Knowledge & attitudes*
Social stigma
Clinician knowledge & attitudes*
Clinician skills & experience*
Organizational culture* & climate
Services structure & capacity*
Resources*
Agency linkages & partnerships*
Toolkits & training*
Dissemination resources & 
experience*
Academic partnership*
Implementation
*Intervention Influences
*Stakeholder Feedback
We suggest that stakeholder trust, ownership, and commitment generated by a community 
engagement approach is a necessary but not sufficient step to promoting implementation and 
sustainability of evidence-based programs in underserved communities. Resources to support use 
of programs and their availability and flexibility for adaptation to network capacities are 
required.  We rely on organizational learning and network theories to explicate how agencies 
learn to establish “communities of practice” by identifying common goals, sharing knowledge, 
and developing that promote social support and information flow among stakeholders.  We use 
diffusion of innovation theory to understand the process of implementation and the 
characteristics of interventions that promote adoption and sustainability.  In particular, we 
introduce systematic variation in organizations’ support for implementation. How organizations 
“fit” evidence-based programs to their structures, the structures they create, and the experience 
of participants in doing so, will be the subject of an implementation process evaluation.  We will 
use concepts from these diverse theories and frameworks and project data to specify a conceptual 
model of the process of implementation.       

Page 15 of 72 
 
For some services delivery interventions, such Multisystemic Therapy125 and Assertive 
Community Treatment,98, 126, 127 a high degree of fidelity is necessary to improve outcomes.  We 
think that variable adherence to PIC/WE Care will be effective, because positive outcomes have 
been observed across diverse QI programs65, 74 and less intensive models.84 A similar assumption 
underlies initiatives to promote use of depression QI programs in primary care through financing 
and other innovations.66, 95, 97  
 
4. Research Design and Methods  
We propose a group-level, randomized controlled trial to assess the effectiveness of a community 
engagement intervention, compared to an individual program technical assistance intervention, in 
increasing community agency and clinician use of quality improvement interventions for 
depression; and improving client quality of care and outcomes.  The project is longitudinal at 
three levels of stakeholders: agency/administrator, clinicians/case worker, and client.  The study 
is conducted in two, underserved, largely minority communities in Los Angeles: Hollywood and 
South Los Angeles.  We considered alternative designs, including an observational (pre-post) 
design and randomizing at the individual client or provider level.  An observational study has 
less internal validity; individual-level randomization risks contamination of conditions.   
We will recruit agencies (social service agencies, primary care agencies, and mental 
health agencies, and community partners such as faith-based organizations), using the 
Community Partnership Initiative sampling frame, and their “units,” or service locations or 
teams, which we also refer to as “programs.”  50-60 units or programs (expanded in the revision 
to 70-80; and then considering addition of homeless/prisoner re-entry to 80-100 programs), 
balanced across the two communities and including diverse agencies, will be selected and 
randomized to the community engagement or low-intensity control dissemination condition.  The 
first year is a planning phase for agency and provider recruitment and intervention and measures 
modification and documentation with community input. (With community input the planning 
year was expanded to 2 years to accommodate recruitment of the larger program and client 
sample; all benchmarks and timelines discussed below were shifted to one year later).  Between 
months 6-12, we will field the baseline web surveys for administrators and clinicians.  Agencies 
will be recruited in batches, South Los Angeles followed by Hollywood.  We will host Kick-Off 
Conferences to explain the study to agencies and their representatives and introduce the quality 

Page 16 of 72 
 
improvement toolkits that are common to both intervention conditions to all agencies/programs.  
Each conference is followed by planning and training for the intervention condition, followed six 
months later by client screening and recruitment of depressed patients into a panel study with 
baseline and 6- and 12-month measures.  Administrators in all agencies and clinicians in sites 
participating in the trial, as well as some sites not in the trial, will complete an 18 month follow-
up survey.  After several months of collaborative analysis involving community agency partners, 
the project will sponsor a community feedback conference to share the outcomes findings and 
provide assistance in use of intervention materials based on evidence for their effectiveness.  
Final analyses and feedback conference will occur in the last year of the study.  
4.1. Sampling  
Communities:  We selected by convenience two communities in Los Angeles (Hollywood and 
South Los Angeles) with large, low-income, mainly minority populations, where we have strong 
existing partnerships.   
The County of Los Angeles, with nearly 10 million people, is one of the most populous 
and diverse counties in the United States: of the population, 43.9% are Latino, 33.5% white, 
12.3% Asian, 10% Black and 0.3% American Indian.  Over 20% live below the federal poverty 
level128 with ethnic minorities over-represented among the poor.  South Los Angeles has a 
population of about 1.5 million, with 60% Hispanic, 35% African American, 3% non-Hispanic 
While and 2% Asian American. The communities of South Los Angeles suffer from some of the 
highest rate of morbidity and mortality and the lowest rate of educational attainment, family 
income and insurance coverage in Los Angeles County.128  Hollywood-Metro has a population 
of almost half a million: 60% Hispanic, 3% African American, 17% non-Hispanic White and 
17.3% Asian American.  Over half are foreign born and less than half have a high school 
diploma.  A little over a quarter live in poverty and the median household income is 25% less 
than the county as a whole.129  This area has the highest uninsured rates and highest percentages 
of adults and children with no regular source of care.130  
Agencies:  The agencies to be included in the study will be those who participated in the 
Community Partnership Initiative and their partners.131  The CPI frame of 94 organizations, 
which was developed through publicly available, comprehensive lists of services agencies in Los 
Angeles County coupled with recommendations from lead community partners, included mental 
health, substance abuse, primary care, social service, and homeless and other community 

Page 17 of 72 
 
agencies. Other agencies participated in the community feedback conference and expressed 
interest in further activities.  We will invite agencies to participate through letters and phone calls 
and ask them to complete a web survey, review a description of the study, and suggest additional 
partners.  Each consenting agency will provide a list of their sites or ‘units’ that provide services 
to adults and have at least one full-time equivalent clinician or case worker.  Agencies will be 
enrolled in a rolling fashion over six months, grouped into two main batches, one for each area.  
We will recruit clinicians from this pool of units and finalize unit selection at the conclusion of 
agency/clinician enrollment by month 13.  
(NOTE: With community input in the design period, the definition of agencies was expanded to 
include homeless-serving, prisoner re-entry and “community-trusted locations, including parks 
and recreation centers, senior citizen centers, faith-based programs, hair salons, and exercise 
clubs.  In addition, to incorporate community priorities for vulnerable populations into account, 
each community was invited to nominate two vulnerable groups for over-sampling.  South Los 
Angeles selected substance abuse agencies and neighborhoods with high concentration of 
African Americans.  Hollywood-Metro selected homeless clients and senior citizens.)  
Units or Programs:  Sampling will be carried out with sites or autonomous divisions within sites 
as the units to be sampled. We expect to identify at least 100-150, with some agencies having 
over 10-15 units while others have one. We may divide large primary care and mental health 
specialty clinics into teams as the units.  We will stratify units by agency type and randomly 
select within strata to obtain a sample of unit sample that is 30-40% primary care, 30-40% social 
service, and 20-30% mental health units that serve more “general community” populations, such 
as CalWorks contract agencies.  In this way, we anticipate having at least 10 units from each 
category to facilitate fitting statistical models.  We will attempt to enroll about 20-30% more 
units than needed, to allow for withdrawal/attrition.  We will include those units in the 
administrator/clinician surveys even if not selected for the main trial to track spill-over of 
intervention activities or parallel types of program exposure from the market. Social service 
agencies are included to participate in screening, education, referral, and case management.  
Primary care settings can play roles in screening, assessment, case management, and treatment.  
Mental health specialty clinics provide treatment, consultation and supervision, and some units 
may participate in screening.  We will replace units with others in the same stratum as needed.  
Units are the “unit of randomization” for this study.  We will enroll units in batches (groups of 

Page 18 of 72 
 
units) with “batch” as a blocking factor, randomizing units within batches to intervention or 
control.   
Administrators: Each participating agency will identify a lead administrator.  Some agencies will 
have a second administrator if they have units assigned to both study conditions.  Administrators 
in all agencies will be asked to complete web-surveys at baseline and 12-18 months.  They will 
be asked to facilitate agency participation in the intervention study and conferences.   
Case Workers and Clinicians:  Using lists of providers/case workers serving adults in 
participating units in the main study, we will randomly select up to 5 full-time case workers or 
clinicians per unit; but we will allow part-time (at least 50%) clinicians if necessary.  We will 
provide them with recruitment packets, followed by phone calls or site visits.  We will randomly 
select from the remainder in the same site/agency to replace refusers/drop-outs. (NOTE:  With 
community input, the definition of “provider” was expanded to all staff with direct client contact, 
including volunteers. We offered an invitation to all such staff to participate if they wished.  In 
addition, we permitted staff of participating programs to participate in intervention activities, 
whether or not they enrolled as providers in the study).  Consenting providers will be asked to 
complete baseline and follow-up web surveys at 18 months and to participate in implementation 
for PIC/WE Care, depending on their intervention assignment.  In PIC, 2 providers declined 
participation, and over 90% participated in 18-month surveys.  Providers will learn of their unit’s 
participation status by letter before the Kick-off Conference for their community.  We expect to 
have 175-225 case workers and providers in the main trial (Note: expanded with expansion of the 
overall study to 250-300).   
Clients:  In social service, community-based, and primary care units selected for the main trial, 
we will advertise the study with posters and fliers.  On designated screening days, clinic staff 
will invite a consecutive sample of adult clients (over 18) visiting participating clinicians to 
discuss study participation with RAND field staff.  (Note:  With community input, community 
members were hired and supervised by RAND).  Screening will occur over a one-month period 
for each unit. We expect 6,000 adult clients to be approached and 4,200 to complete screener 
surveys.  Those over 18 with a positive depression score will be asked to enroll and complete a 
baseline interview that day or by follow-up telephone call and to participate in a 6-month 
telephone survey.  Clients who do not speak English or Spanish, or who are too confused or 
agitated to complete the screener will be excluded.  

Page 19 of 72 
 
Sample size: We expect 20% of clients to be eligible, due to the low-income of the communities.  
Some of the community agencies at the CPI feedback conference reported that 75% of their 
population had mild to moderate depression.  Of 840 depressed adults, we expect 70% to enroll 
and complete baseline (588), 500 (85%) to complete 6-month, and 470 (80%) to complete 12-
month follow-up, based on PIC experience.  
(NOTE: Client screening protocols were changed in several ways.  First, because the type of 
programs was expanded to include homeless-serving programs and other programs for 
vulnerable populations that did not necessarily have traditional ‘waiting rooms’ we also planned 
to screen at events such as food lines, but for those locations used a random selection rather 
than consecutive selection strategy.  Second, all client targets were increased by more than 30% 
for the expanded sample for employment analyses and to compensate for lower follow-up 
retention rates due to inclusion of homeless clients.  However, because depression rates were 
much higher on average than expected (18-50% depending on the sector) and because 
acceptance of screener and study enrollment offer was higher than expected (greater than 90%), 
we were able to use the same expected number of completed screener surveys to achieve both the 
necessary increases in enrollment and analytic sample for the expanded study.)  
4.2. Intervention Assignment 
Within each community and stratified by type of agency, we will randomly assign, using a 
computerized random number generator, the intervention conditions (community engagement 
implementation or low-intensity control dissemination) to the 50-60 units (programs) for 
screening clients.  We will randomize additional units only providing implementation support 
(and not supporting client screening) to the two conditions.  Within large networks (e.g., 
QueensCare), we may assign common conditions to collaborating units to maintain network ties.  
We will consider randomizing units that are matched based on program size and client 
sociodemographics, to improve client comparability by intervention status, as in PIC.132  Agency 
administrators and participating clinicians will be informed of their intervention status by letter.  
(NOTE:  Because of the large number of programs to be recruited, and the very diverse type of 
sectors recruited, and not all sectors had available up-to-date county lists, we decided to 
determine program eligibility in stages, in order to randomize a set of probable final programs, 
after which we planned to conduct site visits to determine final eligibility and final agreement of 
programs to participate.)    

Page 20 of 72 
 
Programs will be informed of their intervention assignment status by a letter to their 
administrator after their enrollment is finalized.  Recruiters of programs, administrators and 
providers are blinded to program intervention status until program recruitment is finalized.  
During recruitment, the study Principal Investigator will have access to intervention assignment 
information on a “need-to-know” basis (such as to answer questions from programs with 
recruitment finalized and notified of intervention status).  Providers will be notified of program 
assignment through their program administrators.  Clients will not be directly notified of their 
program’s intervention status by the study, but could be notified by their program’s administrator 
or providers.  Survey staff for baseline and follow-up status will be blinded to program 
intervention assignment status.     
4.3. Intervention Conditions 
Resources for Services (Individual Program Technical Assistance): The lower-intensity 
condition is time-limited (3-4 months) technical assistance to individual programs to implement 
the toolkits of PIC/WE Care at a community conference, with post-conference access to PIC 
toolkits through a website (Table 1 below).  Control units will be invited to the final Community 
Feedback Conference and offered assistance post-trial in further use of the intervention 
materials, based on their preferences after reviewing the findings of CPIC.  
(NOTE:  In the community feedback during the design period, this intervention condition was 
significantly expanded in scope with increased intensity, to meet the goal of having technical 
assistance that reasonably approximates the highest standards of implementation of evidence-
based practices in the county based on current training practices.  The resulting model was 
offering programs in this condition up to 12 webinars covering all components of the training 
model as well as offering individual site visits to primary care clinics.  In addition, with 
community input, the RS Intervention had a leadership team of experts in medication 
management, case management, and Cognitive Behavioral Therapy for Depression; staff support 
as well as a community engagement and outreach specialist to help assure that individual 
agencies would participate.  
(Note:  The originally planned collaborative care toolkit from PIC/We-Care for both 
intervention conditions was expanded to accommodate the community’s preference to include all 
staff with direct client contact, including nonprofessional and volunteer staff.  We included a 

Page 21 of 72 
 
community health worker manual and team management tools from the Mental Health 
Infrastructure and Training Project using PIC/We-Care and IMPACT toolkits.) 
Table 1.  Quality Improvement Intervention Materials Available for CPIC 
Introductory Materials:  
Improving Depression Outcomes in Primary Care: A User's 
Guide to Implementing the Partners in Care Approach (PIC) 
To Be Developed:  A Community Network Guide to 
Implementing Partners in Care 
Training Materials: 
Training Agendas and Materials for Expert Leaders, 
Depression Nurse Specialists, and Psychotherapists 
Videotape of Nurse Specialist Assessment 
Resources for Team Management 
Materials for Primary Care Physicians & Care Managers: 
Clinician Guide to Depression Assessment & Management 
(PIC)  
Physician Pocket Reminder Cards 
Guidelines/Resources for Depression Nurse Specialist (PIC) 
Psychotherapy Materials: 
Guidelines for the Study Therapist   
Group and  Individual CBT Therapy Manuals for 
clinicians and clients (PIC, WE Care) 
Modified manuals for nurses, substance abuse 
counselors, and lay coaches  
Materials for Patients: 
Patient Education Brochure in English and Spanish); Patient and Family Education Videotape (English and Spanish); 
(PIC and WE Care);  
Audio taped testimonials, Fact Sheet (Witness for Wellness) 
Materials for Case Managers and Health Workers 
Case Manager Resources and Website (Building Wellness Pilot): Depression Screener and Scoring; Client Education 
and Referral Resources  
Community Health Worker Guide (Mental Health Infrastructure and Training Project) 
 
Community Engagement and Planning or CEP:  The CEP intervention condition follows the 
principles of community engagement reviewed above, and promotes the inclusion of relevant 
stakeholders in planning and recognizes participants as equal partners, promoting mutual respect, 
trust, ownership or products, and equal decision making authority. Inclusiveness is thought to be 
a key factor in the success of coalitions.133, 134  By promoting a negotiation of priorities and 
resources across academic and community partners, the intervention builds capacity for research-
informed action through knowledge dissemination and community leadership.31 To support 
community engagement, we developed a manual community members30 and guide for academic 
partners.32 The three main phases of this intervention will include a 4-month planning phase led 
by a partnered Council including community leaders from the assigned agencies and study QI 
and depression experts, guided by community engagement principles and activities, an 
implementation phase following the model of implementation developed in the planning phase, 
and monitoring and oversight to provide course corrections over that implementation phase.  
(NOTE: CEP was guided by a written manual developed during the design phase using 
community and academic partnered input. In addition, because the RS intervention had a more 
intensive intervention as described above, the CEP Councils were given a budget of $15K per 

Page 22 of 72 
 
community to support trainings, roughly comparable to the study’s costs of supporting the model 
of training in RS).   
CEP Planning and Training: Intervention activities will be initiated at the Kick-off Conference 
for each community.  Attendees will include administrative and clinicians from units in both 
intervention conditions, and grass-roots community members supported by HAAF, QueensCare 
and Behavioral Health Services, the lead community partners.  We will provide a study 
orientation and review PIC/WE Care findings and materials and access to the RAND PIC 
website and copies of therapy manuals used in PIC and WE Care (see Table 1). Grass roots 
members and civic leaders will join discussion groups for CE units only.  In this session, co-
chairs for the CE planning council will be elected.  The CE council will meet 1-2 times a month 
for four months and throughout implementation. Over the planning period, the council will 
participate in training and ongoing supervision in the CE model, following the manuals, led by 
HAAF, QueensCare, and Behavioral Health Services. The planning council will conduct an in-
depth review of the goals, structure, content, and training requirements for PIC/WE Care and our 
recently adapted materials, and summarize requirements in a spreadsheet. To develop network 
implementation capacity, the council will review the mission, services, staffing and resources of 
participating units, preparing summary tables.  Using CE negotiation principles (e.g., 
inclusion/fairness) the council will develop strategies to fit network capacities to PIC/WE Care 
requirements. A consensus on the implementation plan will be developed using a modified 
Delphi approach.135 The action plan will outline agency roles, provide a blueprint for services 
coordination, and identify key contacts.  For example, social service agencies may be assigned to 
education or case management roles; community members may be assigned as therapy coaches 
or educators; clinicians may determine which agency provides therapists or medication 
consultants.  During the planning period, study experts will train agency leaders in use of 
PIC/WE Care protocols.  Therapists will be trained in a 2-day conference followed by audio-
taped supervision of a first case when possible, as in PIC and WE Care.  Medication experts 
(primary care, specialty care) and case workers/care managers will be trained in one-day 
seminars with practice sessions.  Community members will receive 1-day training as client 
advocates/coaches. Manuals supporting activities by clinicians, therapists, caseworkers, and 
community coaches will be provided. (NOTE: CEP Councils developed plans for training that fit 

Page 23 of 72 
 
each community and were typically more intensive than outlined above, such as having more 
training sessions, opportunities for make-ups, and more intensive supervision). 
CEP Implementation: Agency intervention leaders, supported by study clinicians will train the 
unit clinicians/case workers in delivering PIC/WE Care protocols and provide consultation and 
supervision. The council will provide oversight, reviewing monthly reports from lead 
administrators and intervention experts, with feedback to unit leaders and intervention experts.  
These meetings will follow the CEP model for inclusiveness and fairness/respect and include 
grass-root community member perspectives supported by HAAF, QueensCare and Behavioral 
Health Services, the lead community-based agencies. The monthly reports from intervention 
leaders will describe progress, problems encountered.  As clients are enrolled, program 
administrative staff will be notified, with client permission.  Support for use of depression 
treatments will be encouraged for 6-12 months at the client level as in PIC.  (NOTE:  The 
training model for this condition was modified with community input to be joint training by study 
experts and community leaders.  We note that the design calls for notification of programs about 
client enrollment in CPIC but not RS programs).  
Community Feedback Conference: The lead CBOs will sponsor a Community Feedback 
Conference at 40 months (after all client outcomes data are collected), inviting all CPIC agencies 
as well as civic leaders and community members.  The overall study council co-chairs will 
review outcomes findings, implementation stories, and council recommendations.  Break out 
groups will discuss implications for implementing and sustaining depression services in the 
community.  All participating agencies will be supported in having access to and using the 
study’s intervention materials.  The toolkits and other intervention features for CEP and RS are 
summarized in Table 2. 

Page 24 of 72 
 
 
Table 2. Community Partners in Care Interventions and Training Features by Condition 
 
Resources for Services (RS) 
Community Engagement and Planning (CEP) 
Initial Model 
1) Depression care collaborative care toolkit 
(manuals, slides, medication pocket cards, 
patient education brochures and videos) via 
print, flash drives, and website. 
2) Trainings via 12 webinars / conference 
calls to all programs and site visits to 
primary care  
3) Expert trainers: nurse care manager, 
licensed psychologist cognitive behavioral 
therapy trainer, three board-certified 
psychiatrists for medication management, 
experienced community service 
administrator supporting cultural 
competence and participation 
4) Community engagement specialist for up 
to 5 outreach calls to encourage participation 
and fit toolkits to programs 
5) Study paid for trainings and materials at 
$16,333 per community.  
1) Depression care collaborative care toolkit 
(manuals, slides, medication pocket cards, 
patient education brochures and videos) via 
print, flash drives, and website. 
2) Expert trainers: nurse care manager, 
licensed psychologist cognitive behavioral 
therapy trainer, three board-certified 
psychiatrists for medication management, 
experienced community service 
administrator supporting cultural 
competence and participation 
3) 5 months of 2-hour, bi-weekly planning 
meetings for CEP councils to tailor materials 
and develop and implement a written 
training and depression service delivery plan 
for each community, guided by a manual 
and community engagement model. The 
goal of the plan was to support increased 
capacity for depression care through 
collaboration across a myriad of community 
programs.  
4) Co-leadership by study Council following 
community engagement and social justice 
principles to encourage collaboration and 
network building  
5) $15,000 per community for consultations 
and training modifications 
Implemented 
 
 
Overall 
21 Webinars and 1 primary care site visit 
Multiple one-day conferences with follow-
up trainings at sites; webinar and telephone-
based supervision  
Cognitive 
Behavioral 
Therapy 
(CBT) and 
clinical 
assessment 
Manuals (Individual and group) and 4 
webinars offered for licensed physicians, 
psychologists, social workers, nurses 
marriage and family therapists 
1) Manuals (Individual and group) 
2) Tiers of training: For licensed providers 
plus substance abuse counselors: a) intensive 
CBT support included feedback on 
audiotaped therapy session with one to two 
depression cases for 12-16 weeks, b) 10 
week webinar group consultation, and for 
any staff trainee, c) Orientation workshops 
for concepts and approaches.  

Page 25 of 72 
 
Table 2. Community Partners in Care Interventions and Training Features by Condition 
 
Resources for Services (RS) 
Community Engagement and Planning (CEP) 
Case 
management 
 
Manuals, 4 webinars and resources for 
depression screening, assessment of 
comorbid conditions, client education and 
referral, tracking visits to providers, 
medication adherence, and outcomes, and 
introduction to problem solving therapy and/ 
behavioral activation; for nurses, case 
workers, health educators, spiritual advisors, 
promotoras, lay counselors  
1)  Manuals 
2) In-person conferences, individual agency 
site visits, and telephone supervision for the 
same range of providers.  
3) Modifications included a focus on self-
care for providers, simplification of 
materials such as fact sheets and tracking 
with shorter outcome measures. Similar 
range of providers and staff as RS. 
3) Training in active listening in one 
community; training of volunteers to expand 
capacity in one community 
4) Development of an alternative “resiliency 
class” approach to support wellness for 
Village Clinic 
Medication 
and clinical 
assessment 
 
1) Manuals, medication pocket cards. 
2) For MD, Nurses, Nurse practitioners, 
physician’s assistants; training in medication 
management and diagnostic assessment; 
webinar and in-person site visit to primary 
care  
1) Manuals, medication pocket cards. 
2) Two-tiered approach with training for 
medication management and clinical 
assessment coupled with information on 
complementary / alternative therapies and 
prayer for depression, through training 
slides; and second tier of orientation to 
concepts for lay providers. 
Administrato
rs/Other 
Webinar on overview of intervention plan 
approaches to team building/management 
and team-building resources 
1) Conference break-outs for administrators 
on team management and building and team 
–building resources; support for grant-
writing for programs  
2) Administrative problem-solving to 
support “Village Clinic” including option of 
delegation of outreach to clients from 
RAND survey group, identification of 
programs to support case management, 
resiliency classes, and CBT for depression 
Training 
events 
 
21 webinars and 1 site visit (22 hours) 
(combined communities) 
CBT (8 hours) 
Care management (8 hours) 
Medication (1 hours) 
Implementation support for Administrators 
(5 hours) 
144 training events (220.5 total hours) 
(combined communities) 
CBT (135 hours) 
Care Management (60 hours) 
Medication (6 hours) 
Other Skills (19.5 hours) 
From Chung et al. Annals of Internal Medicine 161: S23-34, 2014. Copyright © 2014 American College of Physicians. Used 
with permission 
 
4.4. Data Sources and Measures 
Data on agencies include reports from administrators at baseline and follow-up web-surveys at 
12-18 months.  Data on clinicians/case workers will be obtained through self-report web-surveys 
at the same time points. Data on clients will be obtained through screening and baseline surveys 
among those positive for probable depression.  Enrolled depressed clients will participate in 6 
and 12-month follow-up surveys.  Data on implementation process include minutes, field notes, 

Page 26 of 72 
 
action plans, leader reports, and exit interviews of leaders in both conditions. We will develop 
each survey with community input into measures selection, based on the fuller measures given 
below.  To do this, the partner community organizations will recruit grass-roots community 
participants and some agency leaders that constitute a working group to review and discuss 
measures, pretest them, and select and suggest modifications or replacements.  In this process, 
academic investigators provide guidance on concepts of measurement construction and identify 
when measures need to be kept intact for validity and reliability. Table 2 summarizes key client 
concepts (prior to community input), with a star for priority measures (primary and secondary 
outcomes).  We also include important intermediate outcomes.  Client measures will be in 
English or Spanish.   
Intervention Status: The main independent variable is intervention status.   
Client Characteristics: Client demographics and insurance status will be reported at screening 
and baseline.  We will use data from the screener and baseline diagnostic instrument (MINI 136) 
to categorize patients as having depressive disorder (i.e., 12-month major depressive or 
dysthymic disorder plus having 30-day depressive symptoms) versus “subthreshold depression,” 
defined as not having recent disorder but screening positive on the CPIC screener. The 
depression screener will be the Patient Health Questionnaire, 8-item version (PHQ8) to permit 
comparisons to WE Care and other studies.  We will use items from HCC to assess self-report of 
lifetime diagnosis by a provider of schizophrenia or psychosis or lifetime hospitalization for 
psychosis.137  We will also include for outcome measurement the full PHQ9, including the 
suicide ideation item.   
Other patient covariates and potential intermediate outcomes include social support138 life 
stress events, patient treatment preferences,139 the presence of comorbid panic,140 and post-
traumatic stress disorder141 and alcohol problems142 use of avoidant and/or active coping 
strategies,14 patient beliefs about depression, and perceived stigma concerns (e.g., difficulty in 
getting a job, getting health insurance, or social relationships).143  The presence of 19 specific 
chronic conditions will be assessed as well as prior provider diagnosis of schizophrenia, prior 
psychiatric hospitalization (including for psychosis) and use of psychotropic medications.  We 
will include a subset of Katon’s self-efficacy for controlling depression items (e.g., how 
confident are you in your ability to recognize early on when you are starting to get depressed?).  
Suicidal ideation will be assessed as part of the MINI and PHQ9.  Current and past substance use 

Page 27 of 72 
 
will be assessed at baseline using a modification of the drug section of the MINI.  (NOTE:  
Given community input, the original planned use of the PIC depression screener was switched to 
the PHQ 8 because it was well-known in some of the practice sites.  In addition, one area of 
community concern was one item using the word “depression” and the thought was that some 
clients might not endorsed this item because of discomfort with the word.  We created an 
alternative version without the word depression, included both versions and allowed clients to be 
counted as depressed on the PHQ8 if they endorsed either version of this item; but Pearson 
correlation of the 2 versions was 0.99.)  
Process of Care:  Clients will also be asked about perceived need for treatment and how likely 
they would be to accept treatment or referral if it was recommended. Depressed clients will be 
asked at baseline and follow-up to report their use of primary care and mental health specialty 
services, including physician visits, prescription drugs, emergency room visits, and inpatient care 
over the prior six months.  Following AHRQ13, 56, 144 and prior work we will develop an indicator 
of two or more months of antidepressant medication or 4 or more counseling sessions.47  We will 
derive measures of use of services (e.g., primary care, mental health, emergency rooms, social 
service agencies, faith-based agencies).  Clients who improve may not need treatment over time, 
so we will develop a measure of unmet need for treatment, defined as having probable 
depression but not receiving appropriate care versus not improved or in care.  This measure 
combines outcomes and treatment data and thus has more power, while preserving intent to treat 
analyses.13  (NOTE: Based on community input, services use measures were expanded to include 
services in a wide range of services sectors  in the community relevant to depression, such as 
services in substance abuse agencies, telephone hotlines, self-help groups, community centers 
and other trusted locations).  
Outcomes of Care.   
Mental and Physical Health-Related Quality of Life: Clients will complete the SF-12 at baseline 
and follow-up.  Published results support the validity and responsiveness of the SF-12.145  We 
found high reliability for telephone and mail-survey administration for the global scale for 
mental health-related quality of life MCS12 (0.84).  The MCS12 is considered the primary 
outcome of interest as it integrates information on negative states such as psychological distress 
and on positive states such as wellness of particular interest to community members, and under 
the hypothesis that the community engagement intervention relative to technical assistance will 

Page 28 of 72 
 
increase both depression treatment rates, thus lowering distress as well as increase social 
supports and/or address social determinants of mental health improving mental wellness. 
Depression Status: We will also include the PHQ9.146 which has different cut-points but 
particularly examining whether there is mild/moderate depression (PHQ score greater than or 
equal to 10). The PHQ9 is a primary outcome measure under the hypothesis that the community 
engagement intervention would improve through networks access to depression treatments and 
thus lower symptoms.   
(NOTE: Based on community feedback, the highest priority outcome was mental health-
related quality of life and the primary measures for the study were designated as the MCS12 
(community and academic primary outcome) and the PHQ9 (academic primary outcome).  In 
addition, the community placed a high priority of “peace of mind” or mental wellness, so we 
created a separate scale using the positive well-being measures available from these measures 
counting as mental wellness having wellness or any of the 3 items. In addition, the community 
also placed an emphasis on having good physical activity, especially with the inclusion of senior 
citizen groups in the study.  We included a single-item measure of physical activity to assess this 
outcome.)   
Household and Work Productivity:  Current employment status will be assessed at baseline and 
follow-up as well as self-assessed work productivity (days of work missed; days late to work or 
left work early; rating of productivity) in the last month. employment and work productivity 
measures will be assessed in all client waves, and will use the World Health Organization Health 
and Productivity Questionnaire,  (http://www.hcp.med.harvard.edu/hpq/index.php) a well-
validated instrument49, 65, 147 that has been used to measure effects of depression treatment on 
employment outcomes (Wang 2007).(NOTE:  These measures were added with the Revision 
proposal, based on the community feedback that employment was a high priority for the 
community.  The study was fielded at the height of the economic recession in the United States).  
Homelessness Risk Factors:  After the community input prioritized inclusion of homeless 
populations, and after the prioritization of employment status and expansion of the sample to 
accommodate studying employment, the partnership determined that it would be important to 
track homelessness and risk factors for homelessness owing to the inclusion of this group and the 
possibility that agencies serving the homeless might be able to do so more effectively under the 
community engagement intervention model.  To this end, measures were included, similar to life 

Page 29 of 72 
 
stress event items, of the presence or absence in the prior six months of 4 factors:  Either being 
currently homeless or living in a shelter or having multiple risk factors for homelessness, 
including no place to stay for 2 or more nights; being concerned about not having enough to eat 
(food insecurity), severe financial crisis, and eviction from major place of residence.  These were 
combined into a simple count as a prioritized secondary outcome, from community interest.   
 
QI (PIC/WE Care) implementation.  We will assess a variety of implementation processes, 
including provider/staff exposure to training, specific implementation plans of intervention 
planning councils, depression QI toolkit adaptation, use of treatment protocols by clinicians, care 
managers, coaches, and therapists, and strategies to sustain or spread implementation, through 
reports by administrators and clinicians and from coding of project training logs, contact reports 
of care managers, session reports of therapists, council action plans and updates, and monthly 
reports (see Pearson et al 2005).148  In addition, we will conduct interviews of implementation 
leaders and administrators in both intervention conditions at the end of client data collection, to 
triangulate with the quantitative data and ensure validity of the quantitative survey measures.  
Contamination/exposure to external factors will be assessed through tracking contacts, sharing of 
information across conditions, and use of other QI programs from the web surveys, contact 
Table 3:  Client Measures* 
Client Characteristics 
Process of Depression Care 
Outcomes of Care 
Sociodemographics  (0) 
Insurance status (0,6,12) 
Family members (0,6, 12) 
*Physical comorbidities   
*Psychiatric comorbidities  
* Lifetime schizophrenia, 
*hospitalization for psychosis(0) 
* One year 
 
PTSD screener (0) 
 
Panic screener (0, 6, 12) 
   Alcohol screener (0, 6, 12) 
   Use of illicit substances (0) 
Stressful life events (0,6, 12) 
Social supports (0) 
Active/passive coping (0,6, 12) 
Ethnicity/acculturation (0) 
Depression knowledge (0,6) 
Stigma concerns (0) 
Treatment Preferences (0,6) 
Readiness for treatment (0) 
Treatment 
*Use of psychotropic medications 
(0,6,12) 
*Primary care counseling (0,6,12) 
*Specialty care referral and 
    counseling (0,6,12) 
 
Services Use 
Hospitalization for mental health or 
substance abuse 
Emergency Room Use 
Outpatient Services 
  Mental Health Specialty 
  Primary Care 
  Social Services/Substance Abuse 
  Other Community Services 
 
Satisfaction with treatment (6,12) 
*Unmet Need (6,12) 
Primary pre-planned: 
*Mental health related quality of 
life (0,6,12) 
*Probable depression (0,6,12) 
Secondary/community priorities: 
*Physical activity 
    (0,6,12) 
  Employment status (0,6,12) 
*Current work status 
*Missed days from work 
*Personal work productivity 
(0,6,12) 
Homelessness Risk Factors 
(0,6,12) 
Currently homeless or 2 or more 
nights homeless in 6 months 
Food insecurity (perceived) 
Severe financial crisis 
Eviction from major residence 
*0=baseline or screening, 6 or 12 = followup month; * Indicates priority measures  

Page 30 of 72 
 
reports, and meeting minutes. Survey data from non-trial administrators and clinicians will also 
be used.  
 
5. Statistical Analyses Plan and Power Analyses for Client Outcomes Analysis 
First, we review general analysis issues and then analysis plans for the outcomes Aim 2. 
Sampling and Nonresponse Weights:   We will create weights to account for the sampling design 
as well as for response. For example, weights at the provider level will be created as products of 
sample selection weights at the organizational unit and provider level, and response weights, 
which will be modeled based on logistic regressions of response indicators on provider 
characteristics149, 150  Weights for client data will be derived similarly.  We will use software that 
facilitates weighted analysis in multi-level models including HLM, Mplus, and SAS macro 
developed by Bell and McCaffrey using a bias reduction method for standard error estimation 
developed,150, 151 as a main analytical tool in PIC.  We will also explore alternative robust 
methods for variance estimation including generalized estimating equations or sandwich variance 
estimators)152-154  a main analytical tool in PIC. We will use software that facilitates analysis of 
data from complex sample designs, such as STATA, SUDAAN, PROC ROBUSTREG in SAS 
V9, and SAS macros developed by McCaffrey at RAND.150   
Missing Data:  Missing data can result in response bias. It is known that mixed-effects models 
handle cases with incomplete follow-up based on an implicit missing-at-random assumption.155   
If there are variables that are not included in the mixed-effects model but are relevant to the 
missing data mechanism, a model without those variables may lead to biased estimates.  In 
addition, many software programs for mixed-effect models drop participants from analyses when 
any explanatory variable is missing.  In CPIC, we will use multiple imputation techniques to 
account for missing data and the uncertainty in the imputed values.155, 156  The analytic team has 
extensive experience with multiple imputation.157-159 In line with our earlier findings about the 
sensitivity of inferences to underlying data distributions,158 we will consider hot-deck and model-
based imputation, relying on hot-deck strategies when data distributions are highly skewed but 
leaving open the possibility of handling several variables simultaneously using model-based 
strategies if data distributions appear suitably behaved.  
Analytic Strategies:  Our analysis uses an intent-to-treat (ITT) framework. To improve the 
precision of the estimated intervention effect, we will conduct a series of bivariate analyses to 

Page 31 of 72 
 
identify the potential covariates to be considered for a multiple regression model.  Confounding 
will be assessed by comparing the unadjusted coefficient for treatment condition with the 
adjusted coefficient. We will use model-building strategies to obtain final models.160  
Assumptions of normality will also be evaluated.  For example, we will explore various 
transformations (e.g., log and square root) for life event variables due to the often skewed 
distribution of event counts.  Smearing estimates will be used, if necessary, for retransformation, 
applying separate factors for each intervention group to ensure consistent estimates.161, 162  In 
general, we leave open the possibility of transforming variables with non-normal distributions.  
In certain cases, the outcome of interest will have a very skewed distribution, either because 
there will be many zeros (e.g. sick days) and/or because the distribution of observed values has a 
very long tail (e.g. services utilization).  In such cases, we will draw upon statistical models 
developed for handling these type of data, in particular a two-part model to separately handle 
zero values and a skewed distribution among non-zeros161 Tobit-type sample selection models163; 
split-sample techniques to distinguish between different functional forms and to avoid 
overfitting161; and dynamic models, such as episodes of care analytic models, duration analysis, 
and count models.164, 165  
Multiple Comparisons:  We will address multiple comparisons by comparing the proportion of 
results favoring one intervention to an assumed null-hypothesis value of 0.5.  We will consider 
but will not rely solely on Bonferroni adjustments and related methods that incorporate bounds 
on the probability of a single false finding of significance.  We will also consider the false 
discovery rate (FDR)166, a framework that offers more sensitive tests of significance when large 
numbers of tests are carried out by comparing observed significance findings with expected 
order statistics from a uniform distribution.   
Hypotheses and Analysis Plans for Aim 2:  To examine the effects on client access to care, 
quality of care, and depression-related outcomes.  
Hypotheses: We hypothesize that the community-engagement approach will lead to greater 
improvement over 6-12months in client’s access to care, quality of care, and health outcomes 
than the technical assistance condition.  In underserved communities, greater participation in 
planning, which community engagement instills, is anticipated to improve uptake of health 
interventions.34  Further, the community-engagement intervention utilizes many principles of 
social-science theory that support behavior change such as practice and skill development104, 124, 

Page 32 of 72 
 
167; development of opinion leaders,110 and stimulation of network development 118 and cross-
organizational “communities of practice” that promote identification with common goals and 
sharing of knowledge,119 improving capacity for implementation.  
 (NOTE:  Based on community input this hypothesis was modified to prioritize mental health-
related quality of life as the primary outcome because it represents both positive and negative 
mental health states, and then to separately examine both poor mental health and mental 
wellness as secondary outcomes; and to expand the include physical activity as both a social 
determinant of mental and physical health and especially relevant to elderly clients; and also to 
include employment status, missed work days, and risk factors for homelessness as important 
outcomes.  Further, the concept of quality of care was modified to reflect, in addition to 
traditional indicators of use of antidepressant medication and/or counseling for depression, use 
of depression-related case management and counseling services in alternative community 
services sectors. Services use was hypothesized to increase under the community engagement 
model relative to the technical assistance model)   
Overview:  The primary goal in this group-level randomized trial is to estimate the difference in 
effects on outcomes, between intervention arms (e.g., CE condition vs. low-intensity control). To 
test our main hypotheses (that CEP would be more effective than RS in improving client health 
outcomes and addressing community priorities for housing and employment, while likely 
increasing services utilization), we will conduct intent-to-treat, comparative effectiveness 
analyses using intervention status as the independent variable.  We will estimate logistic 
regression models for indicator variables and loglinear models for counts.  Consistent with 
recommendations for group-level trials, (8) main models will incorporate adjustment for baseline 
status of dependent variables and covariates: age, sex, >=3 chronic conditions, education, 
race/ethnicity, family poverty, 12-month alcohol abuse or illicit drug use, 12-month depressive 
disorder, and community.  We will weight data for clients with baseline or six-month data to 
characteristics of eligible clients.  For missing data, we will use item-level 155, 168 and wave-level 
169 imputation.  Estimates and p-values are calculated using SUDAAN 
(http://www.rti.org/sudaan/) taking into account clustering within programs and weighting.  We 
supplement findings from adjusted models with results from raw data to assess robustness.  We 
will use two-sided tests with p<.05 for statistical significance.   

Page 33 of 72 
 
We will also explore as an alternative approach multi-level models (also known as 
hierarchical models or random coefficient models) to account for the multilevel data structure 
with clients nested within units (agencies or relatively autonomous subdivisions thereof), and 
repeated measurements nested within clients.
170-173 For this approach, we will apply two-level 
models for the cross-sectional data with clients nested within units; and three-level models for 
the longitudinal data with repeated measurements nested within clients and clients nested within 
units.  To perform multilevel analyses, we will use special software such as the MIXED and 
GLIMMIX procedures in the SAS System V9, HLM6,170 MLWin159, Mplus
174 tailored for these 
analyses.   
Two-level model for cross-sectional data analysis   The statistical model for a “cross-sectional” 
analysis with one observation per client, for example Y=satisfaction at 6 months follow-up, is 
presented via two regression equations: 
Level 1 (individuals):    Yij =b0j + b1jXij + eij   e
ij ~ N(0, s2), 
Level 2 (organizational units):   b 0j=r00 + r01Vj + r02Wj +u0j   u0j ~ N(0, T), 
where Yij denotes the outcome for the i-th individual nested within the j-th organizational unit, 
i=1, …, n
j, j=1, …, J; X
ij denotes a vector of covariates at the individual level; V
j denotes 
treatment status for the jth organizational unit; and W
j denotes a vector of predictors at the 
organizational-unit level.  These two equations can be presented in the mixed model: 
Yij =r00 + b1jXij +r01Vj + r02Wj +u0j + eij eij ~ N(0, s2), u
0j ~ N(0, T). 
The random intercept model can be generalized to a random intercept, random slope mode that 
also specifies b1j to be random with a multivariate regression for (b
0j, b
1j)
T at Level 2. For binary 
outcomes, we will use generalized linear mixed models that use a binomial model and a logit 
link function at Level 1.  
Three-level model for longitudinal data analysis  The statistical models for a group-level 
randomized trial with repeated measures on individuals can be presented via a growth curve 
model for the trajectory of the outcome measure over time with three regression equations: 
Level 1 (Observation level, repeated measurements within individual):       
(1) Ymij =b0ij + b1ijTmij + emij  
Level 2 (individuals): 
(2.1) b0ij=C00j + C01jXij + r0ij 
(2.2) b1ij=C10j + C11jXij + r1ij 

Page 34 of 72 
 
Level 3 (organizational units): 
(3.1) 
C00j=d000 + d001Vj + d002Wj +u00j 
(3.2) 
C10j=d100 + d101Vj + d102Wj + u10j 
 
where Y mij denotes the outcome at the m-th occasion for the i-th client nested within the j-th 
unit, m=1, … t, i=1, …, nj, j=1, …, J; Tmij denotes time relative to baseline, Vj denotes the 
intervention indicator, Xij denotes vector of other covariates at individual level, and Wj denotes 
vector of predictors at the organizational-unit level.  Our analysis will be focused on the slope 
coefficient d101 that measures the main effect of treatment on the growth rates. Although 
clustering effects were nearly absent for most PIC analyses, specification of the covariance 
structure among the random effects will be modeled allowing for covariance between random 
intercepts and slopes.175  Given 3 or more repeated measures on individuals and reasonable 
smoothness assumptions, we will explore curvilinearity through non-linear terms such as a 
quadratic term in the Level-1 model, allowing insight into whether changes are greater in the first 
6 or subsequent 6 months.    
We will also apply multilevel models to clinician data, with clinicians taking on the role 
of “individuals.”   
The primary outcome comparisons will be based on client mental health-related quality 
of life and an indicator of probable depression (see Table 2).  Using the clients’ baseline and 6-
month follow-up data, we will evaluate the CE intervention effects on primary mental health 
outcomes, and community priorities/secondary outcomes of physical activity, homelessness risk 
factors and employment, indicators of quality of care (use of antidepressant medications and 
healthcare counseling for depression) and services use in all community services sectors.  We 
will take into account the clustered design for a group-randomized trial.  We will treat the 
intervention condition as the primary predictor in an intent-to-treat framework.  For a given 
outcome variable, we will include the baseline measure for that variable as a covariate along with 
demographic and baseline variables and stratification variables from the study design for 
additional adjustment.  We will also control for other baseline characteristics and stratification 
variables.  To examine intervention effects using longitudinal data (6 and 12 months), we will 
add individual-level random effects to account for repeated measurements within individuals and 
include interactions between time and the intervention indicator variable.    
Power for 6-Month Client Outcomes Analyses  

Page 35 of 72 
 
In group-level randomized trials design such as PIC and CPIC, the power analysis is complex 
because there are many factors that influence power176, 177.  For a simple group-level randomized 
trial, the power of a test such as a t-test in comparing two group means is a function of the cluster 
size (m), the number of clusters (J), and the intra-class correlation (ICC), the ratio of the 
between-cluster variability between clusters to the total variability. In our application where 
cluster is organization unit, we use Kish’s method176, 177 to adjust the standard errors by the 
variance inflation factor (VIF) defined as VIF=1+(m-1)×ICC, where ICC is the intra-class 
correlation at the organization unit level.  For analysis of repeated measures, we use an extended 
formula accounting for inter-temporal correlations within individuals and within units177  Based 
on PIC data, outcome analyses on depression care and clinic outcomes have shown that the ICC 
at the clinic level is either absent or around 0.01, and inter-temporal correlation coefficients 
within individuals and within units over a 12-month time period are greater than 0.60.  PIC 
provider data shows that ICCs are nearly absent for provider experience and around 0.00-0.05 for 
knowledge.  Taking a conservative approach, we assume that ICC at the unit level ranges from 
0.00 to 0.02 for client data and from 0.00 to 0.05 for provider data, and that the inter-temporal 
correlation is 0.50. In the calculation of power for testing intervention effects from a t-test, we 
use degrees of freedom equal to (number of clusters – 2) when ICC>0, which is conservative for 
assessing significance, although we also explore setting degrees of freedom equal to N-2 
(N=total sample size) for the scenario where ICC=0 (the optimistic scenario).  We also note that 
regression analyses adjusting for baseline covariates and stratification variables can improve 
statistical power if covariates explain a substantial portion of the uncertainty.177  All power 
analyses are conducted with power of 80% or above as adequate, assuming two-sided tests with 
significance level 0.05.  For client data, we provide detailed power analysis for two primary 
study variables (appropriate depression care and probable depression). 
Our initial power analyses were based on assumptions about effect sizes derived from 
prior studies of primary care patients, including ethnic minority patients, and that the main 
intervention effect would be related to increasing rates of treatments for depression.  During the 
partnered design year, as the community leaders requested inclusion of especially vulnerable 
populations and services sectors for inclusion expanded, for example to faith-based agencies, 
senior centers, homeless-serving providers, and prisoner-reentry programs, and outcomes were 
added such employment, we revisited our estimates of power.  First, to accommodate the interest 

Page 36 of 72 
 
in employment outcomes we submitted a supplemental grant application which was funded, 
allowing some expansion in sample.  However, even after this point, the focus on homelessness 
and homeless outcomes emerged in planning.  This raised several scientific and operations 
issues, including that we might have greater attrition than planned, necessitating a larger 
enrolled sample, and that effect sizes might not be as large as planned, because of the greater 
difficulty in improving outcomes in complex populations.  We therefore decided to assume that 
we needed approximately an increase in enrolled sample relative to our proposed sample for 
adequate power under more conventional conditions for fielding quality improvement 
interventions.  These adjustments are given in the Table 3 below, including power for originally 
proposed sample based on prior primary care studies; power for the analysis of the same sample 
to address employment; and then based on revised assumptions, prior to any recruitment, to 
capture smaller effect sizes expected for a more clinically and socially vulnerable sample with 
potentially higher attrition rates. After the participatory design phase, the final study was 
designed to have a power of .80 with an alpha of .05 (two-sided) and ICC=.00-.02 176, 177 to 
detect group difference of 10% to 11% or higher in comparing two proportions and standardized 
effects size of .20-.22  or higher in comparing two means. Enrolling 557-600 participants per 
condition (1114-1200 in total) allowed a retention rate of 65-70% at 6 month follow-up. 
Table 4. Minimum detectable effect sizes for main analyses (80% power (alpha=0.05), two-sided test)* 
Note 
Analytic sample size at 6 
months follow-up 
Effect Size† 
Point Change§ 
Original grant 
500 
0.25-0.27 
12.43-13.18% 
With 30% expansion  
650 
0.22-0.24 
10.92-11.78% 
With further 20% expansion, final 
780 
0.20-0.22 
9.98-10.92% 
†Detectable effect size (mean difference between two groups divided by the standard deviation)178.  §Detectable 
percentage point difference for two groups.  
 
6. CPIC 3-Year extension (PCORI award # 1845) 
Background:  Depression is a leading cause of morbidity with disparities in care.  Depression 
collaborative care, a team-based approach supporting care management, patient activation and 
evidence-based treatments, is effective relative to usual care but often unavailable in under-
resourced communities.  Community Partners in Care (CPIC) randomized 95 programs in 
healthcare and community-based (e.g., social services, faith-based) service sectors in two 
communities to Resources for Services (RS) for individual program technical assistance versus 

Page 37 of 72 
 
Community Engagement and Planning (CEP) for multi-sector coalition support for depression 
collaborative care.  For depressed clients, CEP relative to RS reduced having poor mental health-
related quality of life (MHRQL) and behavioral health hospitalization over 6-12 months. Longer-
term outcomes are unknown. 
Aims:  This extension study of CPIC aims to: 
1. Compare CEP and RS effects at 3-year follow-up, 2-years after intervention support, on 
primary (depression and MHRQL), community-prioritized outcomes of physical health-related 
quality of life (PHRQL) and behavioral health hospitalization, and outpatient services use 
(secondary); 
2. Compare effects of CEP and RS at 6, 12, and 36 months for CPIC participants enrolled in 
healthcare or community-based service sectors; 
3. Describe outcome priorities for African American and Latino depressed clients, and as 
perceived by providers. 
4. Describe views of stakeholders planning the Los Angeles County Health Neighborhood 
Initiative (HNI) informed by CPIC; and national stakeholders briefed on CPIC and HNI. 
We hypothesized that CEP relative to RS would improve long-term mental and physical health 
and reduce hospitalization nights, overall and within sector.  We expected stakeholders to 
prioritize social factors and mental health. 
Methods:  Extension Study (Aims 1-3): Between 1/2014-10/2014, we approached 1004 CPIC 
clients initially screened as depressed from 89 programs and eligible for 3-year surveys; 600 
participated (60%; RS 293, CEP 307). Aim 3: For client qualitative interviews, we approached 
163 consecutive 3-year survey participants stratified by depression, race/ethnicity, gender and 
intervention; 104 participated (64%).  For staff, we notified 289 completing 12 month surveys 
and used purposive follow-up stratified by community, service sector and intervention, to 
complete 51 interviews.  Aim 4:  We interviewed 49 county and community leaders planning 
HNI and 14 of 34 leaders attending a national briefing. 
Data Sources:  Self-report surveys (Aims 1 and 2) and semi-structured interviews (Aims 3 and 
4). 
Measures:  Client outcomes are poor MHRQL (MCS-12 ≤ 40) and depression (PHQ-8≥10) 
(primary); PHRQL (12-item physical composite score (PCS-12), mental wellness, and 
behavioral-health hospitalization nights (community-prioritized); and use of outpatient services 

Page 38 of 72 
 
(secondary).  Clients and providers were asked about priorities for care and national stakeholders 
for responses to CPIC findings and related policy initiatives. 
4-year follow-up of the 3-year sample (extension study).   
The CPIC 4-year follow-up was added based on community-stakeholder input that 
additional follow-up of the 3-year sample was desired to show longer-term effects on depression 
remission.  This follow-up built on the 3-year extension study as well as an opportunity to 
explore of using an existing funded cohort to recruit potential subjects into the Community and 
Patient Partnered Research Network (CPPRN).   
Analyses plan:  We will conduct 3-year end-status intent-to-treat analyses, and for 4-year 
follow-up of the 3-year sample, 4-year end status, with intervention status as the main 
independent variable, adjusted for baseline status and covariates, with response weights and 
multiple imputation (Aim 1); subanalyses of intervention effects within service sector at each 
follow-up (Aim 2); sensitivity analyses (raw data, longitudinal modeling and sector-by-
intervention interactions); and thematic analyses of qualitative data (Aims 3 and 4) 
 
7. Limitations 
The proposed study is based in two under-resourced communities in Los Angeles; findings may 
not necessarily apply to other communities. Other limitations include the group-level randomized 
design, which is quasi-experimental. We rely on self-report measures and for this analysis 6-
month client outcomes.  This remains an ambitious study in terms of design and scope in the 
context of a partnered research approach.   
 
8. Protection of Human Subjects 
8.1. RISKS TO THE SUBJECTS 
8.1a. Human Subjects Involvement and Characteristics  
The proposed research will involve five groups of human subjects, agencies/administrators; 
clinicians/case workers; clients of these clinicians, and community members and civic leaders, 
and the research team itself.  The study is a randomized controlled trial of a community-
engagement, network building implementation intervention versus a low-intensity dissemination 
approach using a conference and access to a website with Partners in Care (PIC) toolkits.  The 

Page 39 of 72 
 
study supports post-trial, wait-list intervention training for controls supported by intervention 
leaders.  
Communities:  We selected by convenience two of the largest communities in Los 
Angeles with substantial, low-income, and largely uninsured and predominantly minority 
populations, for which we had established partnerships.   
Agencies:  We will contact the community agencies participating in the Community 
Partnership Initiative interviews (66 to date, 2 more to be completed; plus 10 attending the 
Feedback Conference).  We will ask them to participate in a websurvey in which we will identify 
other potential partners.  We expect 15-20 new agencies to be identified and we will ask the new 
agencies to also complete a Community Partnership interview, and to complete the websurvey.  
The websurvey will describe the goals and activities of CPIC with contact information for study 
staff, and follow-up activities by study staff, including study presentations as desired and follow-
up calls.  We expect 80 agencies to agree.  These agencies will include social service and faith-
based community agencies, primary care clinics, and mental health specialty outpatient settings, 
serving low-income clients or community members.  Lead agency administrators will be asked to 
participate in a series of follow-up web surveys at 24, 36, and 48 months after their baseline 
survey, to participate in conferences, and if their agency is selected for the trial, in the main study 
activities. 
All 80 participating CPIC agencies will be asked to identify service locations in the 
community areas, for which they have at least one full-time equivalent clinicians/case workers 
that serve adults; and to provide the number of eligible clinicians or case workers in those 
locations.  These units will be the focus of clinician recruitment (see below).  We expect most 
participating agencies to have 1-4 eligible units, but some will have more than 10.  We will 
explore the feasibility of splitting some units into distinct teams.  We expect to identify 150-200 
eligible units across agencies. 
All participating agencies’ eligible units (150-200) will be considered for selection into 
the main trial.  We will select a stratified sample, by agency type, to achieve 50-60 participating 
units (25-30 per community) for the main study, but expect to have another 25-30 participate in 
the ongoing administrator and clinician surveys but not in the main study. 
Among the participating organizational units, we will randomly assign them (stratified by 
community and type of agency) to participation in the 2 study conditions; for very large, well-

Page 40 of 72 
 
established, multi-agency networks (such as QueensCare), we may randomize units within that 
network to preserve network ties.  Agencies and organizational units will be informed of their 
randomization status by letter prior to the study’s Phase 1 Kick-Off Conference.  Some agencies 
may have units in both Phase 1 intervention conditions in the trial, and for this purpose, will 
identify a lead administrator for that agency for each intervention condition, and ask the 
additional administrator to complete the baseline administrator survey.  This will increase the 
administrator pool across the 80 CPIC agencies to about 100. 
All agencies will be asked to commit to participate in the administrator/agency web 
surveys, clinician surveys, and Feedback Conference.  Agencies selected for the main trial will 
be asked to send 1-2 representatives to a Kick-Off Conference in their community, participate in 
the relevant intervention planning and implementation activities for each condition, and 
participate in a client outcomes study.  Intervention agencies will be asked to assist training for 
the controls in the wait-list training after the main trial. 
Agencies/units assigned to community engagement implementation will participate in 4 
months of planning and training for use of PIC/WE Care and to design an approach to fit the 
programs to their resources, priorities, and sites, with support of community members serving as 
co-intervention planners.  Then they will participate in 12-18 months of implementing PIC/WE 
Care, including training the clinicians within participating units and supporting access to 
evidence-based treatment treatments for depressed clients enrolled into a client study.  Those 
assigned to low-intensity, standard dissemination will attend a conference and have access to PIC 
toolkits on a website, but will not be asked to participate in other intervention planning or 
implementation at that time.  Agencies/units in both conditions will be asked to participate in 
client enrollment within each unit, designed to achieve screening of a defined client pool for 
depression and enrollment of several depressed clients for each unit into baseline surveys and 6-
month and 12-month follow-up surveys.  In community-engagement practices, those clients will 
be identified to the practice so that they can initiate PIC/WE Care intervention activities to 
facilitate access to evidence-based depression treatments, following the approach developed 
within their implementation planning period.  Following the 6-month outcomes data collection 
for the client study, the agencies in the community engagement condition will be asked to 
identify representatives to participate in collaborative analyses and preparation of findings for 
presentation at the Community Feedback Conference, attended by all participating Agencies, to 

Page 41 of 72 
 
formulate recommendations for the field and for future community implementation, and initiate 
wait-list training for controls. 
Administrators:  The administrator sample will be recruited from all participating CPIC 
agencies (see above), including those from the original respondents to the Community 
Partnership Initiative, those for agencies joining CPIC after being nominated for inclusion (i.e., 
primary care and specialty care partners of social service agencies in CPIC), and the additional 
lead administrators for those agencies with units in both Phase 1 conditions.  We expect to have 
100 administrators in the study.   Administrators will be asked to complete baseline, 12-month, 
24-month, and 36-month follow-up websurveys on behalf of their agency, and to participate in 
the other relevant study activities for their agency, see above. 
Clinicians:  We will ask administrators to complete a form for each selected unit for the 
study within their agency (see above), listing the number of full-time providers of services to 
adults within that unit.  Using lists of providers we will select 5 at random, using other providers 
(including part time or 50% working clinicians) for replacements as needed, using random 
selection from the remainder.  We will send recruitment packets with follow-up telephone 
contact/fax, and as needed, and for larger agencies make presentations to units to explain the 
study.  Clinicians/case workers will be asked to complete a baseline websurvey on the 
background and knowledge of and treatment practices for depression.  All clinicians will be 
asked to participate in follow-up websurveys at 12, 24, and 36 months.  Clinicians will be 
informed that there will be different types of participation depending on whether they are in an 
organization in the main study, and whether in the main study, in the community engagement 
intervention or low intensity dissemination condition.  Clinicians will be informed of their 
participation status by letter at the time that their agencies are notified.  We expect about 200-
250 clinicians/case workers to enroll in the units participating in the main trail and another 100-
150 in nonparticipating units. 
Clinicians in units assigned to the main study will be asked to participate in the client 
outcomes study. One or two clinicians for each participating unit will be identified by their 
agency as “lead” clinicians for purpose of participating in planning and assuming roles of 
trainers of other clinicians in interventions.  These lead clinicians will be nominated by their 
agencies, not selected by the study. The lead clinicians for their unit, may be asked by their 
agency to attend the Kick-off Community Conference and Feedback Community Conference.  

Page 42 of 72 
 
Lead clinicians (and key administrators) may be asked to participate in semi-structured 
interviews about implementation at the end of the trial. 
When participating in the community engagement intervention (initial trial or wait-list 
period for controls after main trial), clinicians/case workers will be asked to participate in 
PIC/WE Care training activities relevant to the scope of their practice (care managers, therapists, 
medication specialists, outreach workers, educations, etc.), using training materials from the 
study, with help from their agency’s lead clinician(s) and study investigators. 
Clinicians will be free to participate or not in CPIC, and in the training activities, 
conferences, expert leader roles, and other activities of the project.  They will be able to select 
treatments with their clients or not use treatments.  There is no control by the study over their 
activities.  Clinicians will not be paid for their participation but will receive token gift certificates 
or comparably small thank-you tokens. 
Clients:  The clients will be recruited from a consecutive sample of adults visiting each 
participating clinician/case worker, over a one to two month period for that site.  Agencies will 
be given fliers and other materials to advertise the study and screening days if they wish.  
Agency staff at the unit will ask clients if they would be interested in participating in a research 
study in which the agency/clinic is participating.  If interested, the client will be referred to the 
study field interviewer.  The interviewer will ask the client to complete a screening survey.  
Clinicians can also refer to the study. We expect 6000 clients to be referred for screening across 
the 50-60 units, and expect 4200 to complete the screener survey.  This phase will not require 
written consent, because we will not obtain client identifiers.  Those with a positive depression 
score (840), which indicates probable (but not certain) depression, will be asked to consent to 
participate in a baseline assessment and two follow-up surveys at 6 and 12 months.  The client 
baseline survey will be completed on the same day at the agency/unit or, if this is not feasible for 
the client, in a follow-up telephone call from RAND survey staff.  An estimated 540 subjects 
identified as depressed on the screener will sign the study consent and complete the baseline 
survey.  The clinician visited that day will be asked to complete a brief post-visit encounter form 
describing the process of care for the same visit, collected by the survey staff.  All depressed 
patients completing the screener, at the time of enrollment, will receive a list of local mental 
health agencies in case they perceive a need for care. 

Page 43 of 72 
 
Clients from agency units/clinicians in the community engagement condition will be 
identified to their practice for initiation of the PIC/WE Care program, and clients will be given 
contact information for a care manager in the intervention group.  Depending on the scope of 
practice for the agency unit in which they were screened, clients may be asked to participate in 
developing a strategy for education and referral, clinical assessment for depression, or counseling 
or medication management, and to work with a case manager for follow-up; for some of these 
activities, clients may be referred to other agencies within the intervention network, for example, 
to receive medication management or Cognitive Behavioral Therapy.  Clients and clinicians will 
be able to select their choice of treatments and fit them to client needs.  Clients will be able to 
select other resources available to them, outside of the study or network, or other services within 
the network, and remain in the study. 
Clients from agencies in the low-intensity dissemination condition (control) will be given 
the standard referral information and encouraged if they are concerned about their mental health 
or welfare to talk to their agency clinician or other provider.  Further, their participating CPIC 
clinician or case worker will have previously had exposure to PIC/WE Care materials, through 
an initial conference presentation and discussion and access to toolkits on the website. Clients 
from the low-intensity control will be able to receive medication or counseling supported 
through usual community services. 
Client participants in both intervention conditions will be asked to complete a follow-up 
telephone survey at 6 months and again at 12 months from baseline.  All participants at 12 month 
follow-up will be given the same mental health referral information from baseline.  We expect 
85% response at 6 months (N=500) and 80% at 12 months (N=470). 
Collaborating Institutions and Sites: The lead collaborating institutions for the research will be 
RAND, UCLA, Healthy African American Families, QueensCare, and Los Angeles County 
Department of Mental Health.  In addition, research activities conducted by study staff, and 
intervention activities by community providers, will occur in the participating agencies.  Because 
those agencies are not yet recruited, we cannot identify those agencies at this time.  Agencies 
recruited may have their own Internal Review Board, and we will support submissions of the 
study as needed to those IRBs, and otherwise these agencies will be asked to assign their IRB 
needs to RAND.  We will collect information on whether the organization has its own IRB for 

Page 44 of 72 
 
participating in research, during the baseline agency administrator survey and the recruitment 
activities for that survey. 
UCLA and RAND executed a Memo of Understanding and filed necessary documents 
permitting deferred review, such that studies involving both institutions have only one institution 
as the sole IRB for that pair of institutions.  For research not involving human tissues, requiring 
FDA approval, or involving UCLA HIPPA issues (i.e., involving UCLA patient records), the 
reviewing institution is RAND.  This policy went into effect September 1, 2006. Thus for CPIC, 
RAND will be the lead IRB; separate UCLA IRB review will not be required. 
Community Leaders and Advisors:  For intervention groups, key agency leaders 
participate in the 4-month implementation planning, serve as leaders in the community 
conferences, help provide oversight during implementation, and participate in the collaborative 
analyses of data from each stage.  Along with agency leaders, we will include community 
members who will serve as key community advisors, representing the community voice outside 
of formal agencies.  In addition, there will be representation from key civic agencies, such as the 
Los Angeles County Board of Supervisors and the Los Angeles Mayor’s Office (Dr. Patel, and 
representatives from local neighborhood planning councils), as part of the advisory group for 
planning.  This addition of grass-roots community members and civic leaders, helps make the 
overall project a community-wide project, supports the community-engagement model, and 
builds community support for implementation.  These are public roles, but the participants are 
also research subjects because they will be asked to complete conference surveys; notes of 
planning meetings are project data; and training and planning meetings and conferences, may be 
attended by study research staff making observations and taking field notes as qualitative 
research data for the project.  For these activities, the community leaders and advisors are also 
research subjects and will be asked to sign a consent form for participation in the research 
activities of the project.  Community members will be recruited for their advisory roles by the 
lead CBOs for the project, through advertising among the community contacts for these roles.  
The Community members participating in these roles will receive thank-you stipends from the 
CBOs.  The civic leaders will be recruited from the mayor’s and Board of Supervisor offices (see 
support letters), with additional nominations from those offices and the lead CBOs, as they have 
broad knowledge of community leadership in their areas.  Civic leaders will not receive a stipend 
for participating, as it is assumed to be within the scope of their usual duties.      

Page 45 of 72 
 
 
Research Investigators and Staff: Project research investigators and staff participating in training, 
implementation planning meetings, and conference activities including the collaborative research 
analyses of data with agency administrators, will also be research subjects as they will be 
involved in minutes from meetings and subject to observation and other qualitative data 
collection activities, just like the community members and civic leaders.  Those project staff 
serving in those collaborator/consulting roles with community agencies, will be asked to sign a 
consent form as a research participant, as in Witness for Wellness.  
8.1b. Sources of Materials  
Data Sources:  Data on agencies and units derives from reports from lead administrators from the 
pre-study Community Partnership Initiative and four longitudinal web-surveys at baseline, 12, 
24, and 36 follow-up.  Data on clinicians/case workers is obtained through four self-report web-
surveys at the same time points.  Data on clients is obtained through screening surveys with a 
follow-up baseline face-to-face or telephone-administered interview for those screening positive 
for depression; and, for enrolled depressed clients only, at 6-and12-month follow-up telephone 
interviews.  Other sources of data are minutes and action plans from planning meetings and 
conferences, field notes of conferences, meetings, and training sessions, before-after surveys for 
conference participants (for Kick-Off, Feedback, and Quality Assurance Conferences), and other 
qualitative interviews for the process evaluation, conducted periodically (such as at trial exit for 
administrators) as part of the implementation process evaluation.   
8.1c. 
Potential Risks  
Administrators and Clinicians:  There is some minor risk of social harm (embarrassment) to 
administrators or clinicians due to the evaluation process (for example, through feeling compared 
with other administrators or clinicians).  Also, there will be an increase in the caseloads of the 
providers in the intervention agencies, but a given provider will only have a few enrolled 
depressed patients in the study, that load is not great.  Thus, we think it unlikely that agencies 
will assume an undue burden in selecting the scope of implementation.  Over time, this may or 
may not translate into economic risk for the agencies or providers, since improved patient 
outcomes may well reduce intensity of services over time for some clients, and increase it for 
others.   

Page 46 of 72 
 
Providers will help define the implementation plans for use of service delivery 
interventions that support use of evidence-based treatments.  None of the treatments supported 
by the study, and none of the service delivery interventions (PIC/WE Care), are new 
interventions; they are not “experimental” treatments or interventions, as their evidence-basis is 
already well-established.  Providers and clients are not mandated to use particular treatments, 
i.e., they are assigned to resources to support care decisions, not mandated to particular 
treatments; and providers can refuse to use study materials and remain in the study, and clients 
can refuse treatments or select others, and remain in the study.  Providers face potential 
malpractice suits for adverse treatment outcomes under usual practice, and while there is always 
a risk for an adverse treatment outcome (such as a side effect, or no treatment response), these 
are less likely with evidence-based care, so if anything the overall chances of malpractice suits 
for treating depression should be less for providers supported by the study to consider evidence-
based care strategies.  In the unlikely event that a malpractice claim is brought (for example, 
because of a complication from medication), treating providers would be covered by their own 
malpractice insurance or that of their agencies. 
Clients:  Potential risks to clients participating in this study, from the study itself, are 
minimal, and chiefly consist of the embarrassment of revealing personal information to a 
stranger or neutral party. There is also some risk that information on a mental health condition, if 
entered into the medical record or reported on an insurance claim form, could establish a history 
of a pre-existing condition making it more difficult to be covered by insurance for that problem, 
but only in some companies, should the client change jobs or try to change insurance.  Consents 
will suggest that clients discuss such concerns with their providers, and providing training will 
alert providers to this standard concern in clinical practice.  The interventions will be in addition 
to the patient’s usual course of treatment, not a replacement for it, and clients can choose to use 
any study materials, such as the PIC client brochure, or use a treatment (such as Cognitive 
Behavioral Therapy that was supported by study-training of a community provider), or to not use 
these resources, and stay in the study.  Clients are not assigned to treatments, and all treatment 
decisions will be up to clients and the providers they choose to visit.  The client’s usual provider 
will continue to be responsible for treatment and will be responsible for application of the 
protocols, or providers to which the client is referred. 

Page 47 of 72 
 
Because the treatments the study interventions support are not experimental, side effects 
from them (for example, normal side effects of antidepressant medications) experienced by 
clients, would not be considered “adverse events” of the study itself, but expected side effects of 
standard practice treatments.   
We will collect data from the clients about lifetime and current depression and other 
psychiatric disorders (alcoholism, panic disorder, psychotic symptoms).  If a life-threatening 
situation is encountered during screening, telephone interviews or other contact with project 
staff, emergency protocols similar to those developed for PIC will be followed.  Depending on 
whether the situation comes up in the agency/unit or through a study telephone interview, clients 
with an emergency will either be walked to a clinician in their agency and the study staff will 
assist the client in explaining the situation; or the telephone survey staff will contact the PI (or if 
an urgent life-threatening situation, either contact 911 or if feasible have the respondent contact 
911, which greatly speeds up the response).  In over 10 years of conducting depression studies, 
emergency protocols are activated for 1-3 out of 500 depressed clients per wave; in the last (8-
year) follow-up of PIC participants, there were no emergency events; but in tests of the PI’s 
response time through randomly selected times for calls, the PI responded to the telephone 
survey center on average, with 2 minutes of the call from survey staff.  Thus, we are very 
experienced with facilitating activation of and effective use of an emergency response system for 
studies of depressed clients.   
Clients will be paid $20 for the baseline survey and $30 for the 6-month follow-up 
survey. Clients will receive a small gift certificate for completing the screener. 
Community Members, Civic Leaders, and Researchers:  For other participants in implementation 
planning and collaborative research activities, we expect individuals to agree to serve these roles 
who are comfortable with public visibility.  Community members will be supported in their roles 
by the lead CBOs.  Community members are in a more vulnerable position and will receive 
stipends for their roles by the CBOs, but civic leaders and researchers will not receive stipends.  
The main risks to these participants is embarrassment or insecurity in serving in a public role.  
 
8.2. ADEQUACY OF PROTECTION AGAINST RISKS 
8.2a.  Recruitment and Informed Consent   
 

Page 48 of 72 
 
Lead administrators from the participating agencies will be asked to review a description of 
CPIC on the web survey, and will review that description with appropriate leaders within their 
agency.  Agencies will submit a signed letter to initiate their participation.  The composition and 
recruitment procedures are provided in detail above, but briefly include agencies participating in 
the Community Partnership Initiative; and additional primary care and specialty care agencies 
identified by social service agencies as their main partners of that type.  There will be 80 
organizations that participate.  The organizations will be invited by mail to enroll as a 
participating agency and the subsequent study activities.  Organizations will be free not to 
participate.  
Administrators and Clinicians: We will ask each participating agency to identify 1 
administrator, supplemented in agencies with organizational units in both intervention 
conditions, by administrators that support the additional intervention, for an approximate sample 
of 100 administrators. Administrators identify organizational units with full time clinicians/case 
workers for adults, and these units will be arrayed by the study to select 50-60 for random, 
stratified assignment to the intervention conditions in the main study.  To have alternative sites 
and due to the timing of obtaining the information to select units and recruiting clinicians, we 
expect to have some clinicians participating in  units that are not selected for the study. 
Clinicians/Case Workers:  Using lists of all full-time clinicians or case workers for adults 
within selected units, we will use recruitment packets and follow-up calls to recruit 5 clinicians 
per unit, replacing as necessary from remaining clinicians; and if necessary replacing units with 
others from the remaining nonselected pool.  In PIC, all but 2 clinicians agreed to participate in 
selected clinics, so we expect a similarly high rate here given high agency interest and 
cooperation.  We expect 200-250 clinicians in the 60 main study units and another 100-150 from 
other units.  Clinicians will be free to choose not to participate in the study or in particular 
activities of the study. 
Clients: We will recruit a consecutive sample of adult clients visiting participating 
clinicians, over a two week period for that site.  Following posting of notices for the study or 
sending fliers, the agency’s staff will ask their clients if they are interested in participating in a 
research project in which the agency is participating.  If the client is interested, he/she will be 
directed to see the RAND field staff person assigned to that agency.  We hope to talk to 6000 
clients across the 48 units, and expect to complete 4200 screeners.  Those with a positive 

Page 49 of 72 
 
depression score, an estimated 840, will be asked to complete a baseline survey at the agency or 
if this is not feasible, in a follow-up telephone call.  An estimated 500 will agree to the survey 
and sign consent forms to participate in the study.  The field staff who do the screening give the 
client a written study summary and a question and answer booklet, review these materials with 
the client, and get written consent from the client.  In addition, clients will be asked to give 
permission for their clinician to complete a post-visit report.  Enrolled, depressed clients will also 
be asked to consent to a 6-month follow-up telephone survey and a 12-month survey.  We expect 
500 enrolled participants to complete the 6-month interview and 470 to complete the 12-month 
survey.  Clients are free not to participate, receive referrals to mental health providers whether or 
not they are in the higher intensity condition, and all clients will be free to accept treatments or 
referrals or not.     
8.2b.  Protection against Risk   
Subjects will know that they are free to refuse to participate in each stage of the study, and to 
leave questions or issues unanswered.  Subjects will know that they will have the option of 
completing a survey at the agency or by telephone.  If a life-threatening situation is encountered 
during the telephone interview or other contact with participants, emergency protocols similar to 
those developed for PIC will be followed.  For example, for life-threatening physical 
emergencies, subjects are instructed to call 911 and assisted if necessary.  The main other 
emergencies are a patient becoming disoriented or severely confused, or disclosing that they are 
currently suicidal.  Project interviewers will be trained in protocols for contacting the project on-
call physician who can make contact with the participant and/or provide referrals to a specific 
doctor or clinic identified for this purpose within each geographical area.  During PIC telephone 
interviews there were no instances where interviewers encountered life-threatening physical 
health situations.  However, there were about 2 instances per survey wave where the interviewer 
was concerned because a subject suggested they might have been recently suicidal (out of about 
1000 interviews per wave).  Following the approved protocol, the study PI contacted these 
participants (usually within 24 hours or as soon as they could be reached by telephone) and 
discussed the situation.  In all instances, the patients were well aware of their options for 
treatment, all were in active treatment, and all were discussing their symptoms with their 
providers and felt they did not need additional assistance contacting their doctor.  Most subjects 

Page 50 of 72 
 
in this category had already seen their provider that day or were scheduled to see their provider 
the following day.  All appreciated the contact by the PI.   
Responding to mental health emergencies by telephone is challenging, particularly at 
some distance where there is no current rapport with a treating provider, or where there is no 
active treatment provider.  Because of this, we will increase the chances the subjects will have 
information needed to avoid emergencies.  In our recently completed 8-year follow-up of PIC 
participants, we had no telephone emergencies.  To assure responsiveness in the case of an 
emergency, we developed a protocol to have test the PI’s responsiveness by having calls at 
random times, and the response to the telephone center was within 2 minutes over 10 calls. 
All patient-identifying information will be kept confidential.  RAND’s Survey Research 
Group (SRG) will use one set of identifiers (Field IDs) in their contacts with participants.  Our 
senior programmer will receive field data from SRG and transform the Field IDs to another set of 
IDs (Analytic IDs) before the data are available to the investigators.  The file that links the 2 IDs 
will be secured and accessible only to the senior programmer and RAND’s privacy office.  
Identifiable hard copy data will be maintained separately by SRG with limited access and kept 
only as long as needed to maintain consent files or to contact patients for follow-up surveys.  A 
detailed data safeguarding plan will be developed for the study and submitted to the IRBs at 
RAND and UCLA for approval.  See section on Data and Safety Monitoring Plan below. 
All field staff who interact with clients will receive at least 4 days of paid training at 
RAND and are required to complete the UCLA online course for social behavioral training in the 
protection of human subjects.  General training includes general interviewing skills such as 
maintaining neutrality and use of non-leading probes, use of the interview software, SRG 
confidentiality, data safeguarding and quality control policy and procedures, and how to handle 
emergency situations such as intent to harm self or others.  During PIC clinician training, 
clinicians are encouraged to review potential side effects of treatments with patients, and to elicit 
their treatment preferences and negotiate with clients to receive their treatments of choice.   
 
8.3.  POTENTIAL BENEFITS OF THE PROPOSED RESEARCH TO THE SUBJECTS 
AND OTHERS 
Many national studies have shown that underserved minority populations have low rates of 
access to mental health services and lower access in particular to appropriate care for depression.  

Page 51 of 72 
 
Depression is a leading cause of disability worldwide, and improving access to appropriate 
services can improve health outcomes and functional status, and improve employment.  Our 
prior work in PIC has shown that evidence-based service delivery programs for depression can 
improve health outcomes in depressed patients, and especially in minorities, largely overcoming 
disparities in outcomes from care between whites and minorities.  Yet in safety-net systems that 
are the main sources of healthcare for low-income, communities of color, such evidence-based 
programs have not been available, because of poor resources, competing priorities, and poor 
infrastructure to support evidence-based quality improvement programs.  Determining how to 
best support safety-net agency networks in implementing these service delivery programs, is a 
critical issue for improving the health-related quality of life of depressed persons in underserved 
communities. 
The proposed study offers participants an opportunity to receive their care in systems, 
whether social service agencies or health clinics, that can support a network approach to 
evidence-based care.  This study focuses on determining whether a community-engagement, 
network-building approach to implementation is more effective than a standard, low-intensity 
dissemination approach. 
Overall, we think that the benefits of the study, in achieving a better knowledge base and 
network structure for service delivery interventions that support provision of evidence-based 
care, are potentially very substantial, and will benefit clients and their families.  The risks are 
embarrassment, some having side effects from treatment whereas they might not have had 
treatments (and not gotten better), and for clinicians, administrators, and other participants, 
embarrassment, extra efforts, or for agencies, the possibility of increased costs; but all 
participants are free to participate or not in the study and activities within the study, and the 
study is designed to adjust implementation of the services delivery interventions to the priorities 
and resources of the agencies, and the treatment preferences of providers and clients. The more 
serious risks of clients having difficulty with insurance in future jobs from a history of treatment 
are rare compared to the common risks of untreated depression.  Further the implementation 
strategy being studied, community engagement, is specifically designed to provide the 
empowerment and flexibility for participate meaningfully in social planning, such as improving 
healthcare delivery. 

Page 52 of 72 
 
This study will provide several types of new knowledge: 1) the effectiveness of a 
community-engagement based implementation intervention relative to standard, low-intensity 
dissemination; 2) strategies that community agencies and members use to learn about 
implementing evidence-based interventions; and 3) the process of implementation of evidence-
based depression quality improvement programs in underserved communities of color.    
These knowledge gains for the field, will likely help develop a stronger focus on 
dissemination and implementation for a stigmatized condition such as depression.We believe that 
the gains of the study, in terms of knowledge and substantive gains to the community, exceed the 
risks for participants.  
 
9. Data and Safety Monitoring Plan  
The administrator and clinician website will be maintained by UCLA.  Most of the data collected 
from administrators and clinicians will be entered into a secure web data base that is password 
protected.  Access to this data base will be limited to a senior programmer and Ruth Klap, the 
data manager at UCLA.  Field IDs will be replaced with Analytic IDs before data are released for 
analysis. Contact information and contact histories for administrators and clinicians will be 
shared with the RAND survey group, to enable their contact of all participants for further 
surveys, conferences, etc. A program, such as PGP, that provides RSA level security will be used 
to encrypt files before sending them by email. The passwords for encrypted files will be 
transmitted directly to the intended recipient over the telephone (no messages will be left) or by 
US mail.  Hard copy administrator and clinician agreements and interview forms will be kept at 
RAND.  All client data will be maintained at RAND.  RAND will initiate all written contacts, 
telephone or other personal contact with participants; UCLA initiates contact with administrators 
and providers through the web survey or emails to cue them to complete a web survey.  Hard 
copy administrator and clinician agreements and interview forms will be kept in locked file 
cabinets with limited access. 
The use of Web-based technology presents additional concerns in relation to loss of 
confidentiality. To ensure the security and integrity of our data, we use technology similar to that 
used to protect credit card information in e-commerce applications: all exchanges between a 
user's browser and the server are two-way encrypted with the secure sockets layer (SSL) protocol 
to prevent eavesdropping, tampering, or data forgery. SSL protocols use 128-bit cryptography 

Page 53 of 72 
 
algorithms, such as RC4, which have emerged as standard approaches to high-grade online 
privacy protection. As a further safeguard, access to all data on the server is controlled by 
password protection. Different user profiles allow access to relevant sections of the database. For 
example, study clinicians can view and edit data on their own patients only. Investigators can see 
recruitment and intervention data for their study site and de-identified summary data for all study 
sites for comparison.  For additional security of the data, we use only unique study identification 
numbers to identify study patients on data forms, and we do not transmit patients' names, 
medical record numbers, or other identifying information over the Internet. 
Hard copy patient participation agreements will be sent to the Survey Research Group at 
RAND on a weekly basis from each of the screening sites.  They will be logged in and put on-
line by designated person(s) and then stored in locked file cabinets.  The screening forms will be 
sent to data entry after which the hard copy will be stored in locked file cabinets. Judy Perlman, 
SRG, will be responsible for the safeguarding of the hard copy of these forms while they are in 
the field and once the forms are at RAND. 
Baseline surveys will be collected on laptop computers sing Computer Assisted Personal 
Interviewing (CAPI).  Access to the notebook computer will require a password.  An additional 
password will be required for access to the survey data.  Data will be transferred from the laptops 
to network drives with restricted access regularly.   The follow-up surveys will be conducted 
using Computer Assisted Telephone Interviewing (CAPI) by the RAND survey Research group.  
This data will be safeguarded at RAND.  Only de-identified data will be shared with UCLA. 
Worksheets, logs and call records that contain the patient names and RAND IDs used by 
operations staff are needed for monitoring the flow of the field work during the data collection 
period.  After the field period, the worksheets will be kept in locked file cabinets.  As much as 
possible, Field IDs will be used as identifiers on worksheets rather than patient identifiers. 
We will have an on-line file of patient names and telephone numbers during the data 
collection period.  This system will track the status of each patient and provider until the data 
collection is completed and will help in the scheduling of cases.  These files have access 
restricted by passwords to interviewers and supervisors who have a need to use them to operate 
the study and who have signed the oath of confidentiality.  Judy Perlman will be responsible for 
safeguarding these files during the data collection period.  When data collection is completed, 

Page 54 of 72 
 
these data files will be handled as described in below before being turned over to the analysis 
team. 
Bernadette Benjamin will create analytic IDs to replace the IDs assigned during the data 
collection.  A file that links the analytic IDs and data collection IDs will be kept by Bernadette.  
This linking file will have read and write access restricted to Bernadette.  The analytic ID for 
participants and/or other RAND-assigned identifiers will replace the identifying information in 
all fields that would identify individuals or clinics before primary data are made available to the 
analysis team.  Following analysis, reports will be prepared.  These reports will not contain any 
data that could identify individual clients or providers. 
The raw data and the computer data files will be kept for at least one year following the 
publication of the final reports on the study.  Upon the completion of the study, the link files, and 
the survey data files will be turned over to the RAND Privacy Resource Team for safeguarding. 
Employees who will be handling any participant-identifying computer files, worksheets, 
or medical record information as well as the survey interviewers and their supervisor will sign an 
oath of confidentiality.  As part of their training, they will be instructed in the rules of 
confidentiality and data safeguarding for the study. 
Process for Handling and Reporting Adverse Events (AEs):  If an AE in the nature of 
threat to harm self or others occurs in the agency during the screening, consenting, baseline 
survey interaction, the client’s provider or a provider designated by the clinic will be notified 
with the client’s consent.  Events occurring during telephone interviews with clients will be 
reported to the SRG supervisor who will immediately contact the Principal Investigator, Dr. 
Wells or a project clinician designated by Dr. Wells (Dr. Miranda, Dr. Ong).  Dr. Wells (or a 
designated backup clinician) will be always on call during the field period.  Dr. Wells decides 
how the event is to be handled (e.g., phones a suicidal patient, reports elderly abuse).  When the 
event is resolved, Dr. Wells will write a summary of the occurrence and submit it to the IRB.  
The IRB determines whether further notification is warranted. 
Data Safeguarding and Management Board (DSMB): The DSMB is charged as an external, 
independent oversight board, to monitor the conduct of the study for ongoing feasibility, data 
integrity and safety. NIMH has its own DSMP for clinical trials.  If determined that the study 
requires a DSMP, we will use the NIMH DSMP.  However, we think this study may not require 
a DSMP.  The reason is that it is a randomized, controlled trial of an intervention to implement 

Page 55 of 72 
 
evidence-based services delivery programs that support use of evidence-based treatments, 
relative to a low-intensity, standard dissemination approach.  Neither approach increases patient 
risks appreciably in that they both support improved treatment approaches if implemented.  The 
study does not promote use of new treatments or unproven treatments, except for the value of the 
implementation approach itself, which is directed at a network and organizational unit level. 
If recommended for DSMP, CPIC will work with the selected DSMB members identified 
by NIMH; they will be selected to have appropriate expertise and no conflicts of interests. These 
members will monitor the study and ensure that monitoring is timely and effective. The PI will 
provide necessary data to the DSMB, respond to any DSMB queries or suggestions and submit 
the DSMB reports to all IRBs after DSMB meetings. 
Data to be Reviewed: Prior to the start of the study, the board would review the protocol and 
overall plans for data and safety monitoring. The board then will review patient flow and subject 
entry to ensure adequate recruitment and retention of subjects and will monitor the occurrence of 
adverse events related to participation in the protocol. However, we point out that the main 
adverse events relevant to a study of this nature, are likely to be either expected complications of 
depression or of treatments.  Overall in clinical trials of depression, adverse events may include: 
deaths, suicide attempts, study dropout, psychiatric hospitalizations, and clinical deterioration 
defined as emergent suicidal or plan, development of serious substance abuse, or emergence of a 
new psychiatric or medical diagnosis or behavior posing significant risk to the subjects or others.   
Relative to clinical treatment trials, this study has infrequent monitoring of patients and is 
mainly able to identify clinical problems at the time of the subject being in a screening visit 
when a social service worker or clinician will be present; and at telephone follow-up, when 
emergency procedures are handled by the PI on notification by survey staff.  During main 
community-wide implementation, community organizations are implementing the protocols 
under their own priorities and resources, and the study does not monitor their patient care, except 
through a sample of clients in the second cross-sectional survey, i.e., it is real community 
implementation, and that phase is not randomized.  This study is more like a “fifth” or “sixth” 
level trial, and is not of treatment per se, or services delivery per se, but of implementation of 
evidence-based interventions at these levels. 

Page 56 of 72 
 
Schedule of Meetings: If recommended for a DSMB, we suggest teleconferences for the 
meetings, once in the first year; at 6-month intervals years 2 and 3, and once in year 4, when 
clients are participating in the study.  
 
10. Inclusion of Women and Minorities   
This study will focus on communities of color and will have predominantly minority participants 
at the client level; 2/3 of depressed clients are women and we should accordingly address 
important issues for women in communities of color. The project will use bilingual interviewers 
and all instruments will be available in English and in Spanish. 
 
11. Summary of Initial Design, Community Input, Modifications, and Final Design   
Table 5:  Summary of Initial Design, Community Input, Modifications, and Final Design 
Design 
Feature 
Initial 
Community Input 
Modified 
Final 
Communities 
South Los 
Angeles/Hollywood-
Metropolitan Los 
Angeles 
Include Downtown Los 
Angeles with Hollywood 
Affirmed 
Same as initial 
Services 
Sectors 
Mental health specialty, 
primary care, substance 
abuse, social services 
Include public health, 
homeless-serving, 
prisoner re-entry, family 
preservation, faith-based, 
senior centers of parks 
and recreation, exercise 
clubs, hair salons 
Expansion to meet 
community goals, 
required modification of 
expected retention for 
client surveys and of 
client recruitment plans 
Mental health 
specialty, primary 
care, public health, 
substance abuse, 
social services, faith-
based, senior centers, 
exercise clubs, and 
hair salons  
Providers 
Professional staff and 
case workers 
Include all staff, paid and 
volunteer with direct 
patient contact 
Invitation to all staff 
Staff with patient 
contact 
Clients 
Persons on designated 
days showing to sites 
for services (adults), 
consecutive sample 
Include parents of child 
clients that accompanied 
child if they could also 
be supported; expand to 
events/food lines 
Clients and parents of 
child clinics; 
events/food lines 
included 
Adult clients and 
parents of child 
clients in waiting 
rooms (consecutive 
selection) and events 
(random selection) 
Depression 
Quality 
Improvement 
Toolkits 
 
Toolkits for clinician 
assessment, medication 
management, Cognitive 
Behavioral Therapy, 
case management, 
patient education from 
Partners in Care and 
We-Care 
Modify for lay staff, 
simplify language and 
forms, available on-line, 
expand  health worker 
tools 
Modified clinician 
assessment and case 
manager tools; added 
health worker guide and 
team  leader tools from 
Mental Health 
Infrastructure and 
Training 
Modified PIC, We-
Care and MHIT 
materials, simplified 
and offered on web 
site, hardcopy, and 
flashdrives 
Resources for 
Services 
One-day conference 
and written/posted 
Expand to meet Los 
Angeles County 
Offer of up to 12 
webinars on all toolkit 
“Standard” technical 
assistant to individual 

Page 57 of 72 
 
Table 5:  Summary of Initial Design, Community Input, Modifications, and Final Design 
Design 
Feature 
Initial 
Community Input 
Modified 
Final 
Intervention 
toolkit materials 
standards for training 
support for evidence-
based practices 
components plus site 
visit to primary care; 
plus toolkits online, 
flashdrive/hardcopy 
agencies as described 
in modifications 
Community 
Engagement 
and Planning 
Intervention 
4 months of planning 
followed by 2 day 
training and follow-up 
supervision led by 
research/QI experts 
supported by written 
toolkits 
Trainings coled by 
community leaders; 
follow community 
specifications; resources 
to match RS  
($15K/Council); network 
innovation 
Modified 4 months of 
planning with written 
plan, coled training, 
monthly oversight of 
implementation, 
network innovations  
4 months of capacity 
building and network 
development to re-
engineer toolkit 
training as partnered 
training and 
innovation 
Randomization 
Group-randomized at 
program level (agency 
site/team) 
Develop trust in 
randomization/explore 
alternatives 
Community forums on 
Tuskegee legacy and 
scientific advantages of 
randomization; Council 
supplies seed numbers 
to initiate randomization 
Group-randomization 
implemented with 
trust-building 
activities  
Outcomes 
Risk for depressive 
disorder, mental health-
related quality of life, 
physical functioning, 
access and use of 
services, quality of care 
Priority for quality of 
life, wellness and 
resilience; good physical 
functioning and physical 
activity; think of access 
broadly to include 
services in alternative 
locations; add 
employment and risk 
factors for homelessness 
because of economy  
Measures re-prioritized, 
employment and 
homelessness, food 
insecurity, financial 
problems, eviction 
added as homelessness 
risk factors; physical 
activity added to 
physical functioning for 
overall good physical 
health; resiliency items 
explored;  
Mental health quality 
of life main outcome;  
mental wellness, life 
perceived as 
organized, poor 
mental health; 
employment and work 
loss days; 
homelessness risk 
factors primary for 
community; services 
use in diverse sectors 
Sampling 
Initial: 50 programs and 
screening of 6000 
clients to identify and 
enroll and complete 
outcomes on 500 
To add employment 
outcomes, 30% increase 
in sample size required; 
homeless sample would 
reduce expected retention 
from 80% to 65-70% 
Expand expected sample 
size to achieve power 
for employment 
analyses; expand 
enrollment 20% more 
due to expected attrition 
80-100 programs; 
with screening to 
enroll 1200 depressed 
clients yielding 780 at 
last follow-up 
Analysis 
Approach 
Intent to treat; control 
for baseline value of 
outcome and 
covariates; nonresponse 
weights; adjust for 
clustering; impute 
missing data 
Include raw data 
Conclusions based on 
modeled data and raw 
presented to confirm 
modeling 
Modeling as planned 
plus raw data 
presented 
 
 
12. Human Subjects Protocol 
 
Human Subjects Study Protocols were approved by RAND’s Institutional Review Board (HSPC) 
prior to study initiation in 3/18/2008. HSPC id: w2812-06-01; Federalwide Assurance number 
(FWA): FWA00003425; IRB number: IRB00000051.   

Page 58 of 72 
 
 
Client components including informed consent forms for screener, enrollment, baseline, 6 
months, and 12 months follow-up were approved in 10/22/2009 through 10/21/2010 (w2812-06-
01-CR02), with extensions through 10/3/2014 (w2812-0601-AM09, w2812-06-01-CR03, 
w2812-06-01-CR04, w2812-06-01-CR05, w2812-06-01-CR06).   
 
Partner agencies approved the referenced research by signing an authorization agreement letter, 
or if requiring their own IRB, through approval by their IRB, prior to client recruitment at their 
agency.  For agencies with their own IRB, the IRB focused on initial client recruitment into the 
study.  The initial consents covered screening for eligibility and enrollment in the trial, including 
baseline through 12-month follow-up, in the enrollment consent. All longer-term follow-up 
human subjects approval was through RAND IRB with the recruited clients .   
   
Three-year extension was approved by HSPC in 10/3/2014 through 10/3/2016 (w2812-06-01-
CR07, w2812-06-01-CR08). An amendment was approved in 2/29/2016 (w2812-06-01-AM25) 
for contacting the 600 CPIC clients who completed the 36-month follow-up interviews by phone, 
for the 4-year follow-up survey and enrollment into a new program (CPPRN).  
 

Page 59 of 72 
 
12. Literature Cited 
1. U.S. Department of Health and Human Services. Mental health: culture, race, and ethnicity - a 
supplement to mental health: a report of the Surgeon General. Rockville: USDHHS; 2001. 
2. Wells K, Sherbourne C, Duan N, et al. Quality improvement for depression in primary care: 
do patients with subthreshold depression benefit in the long run? Am J Psych 2005;162(6):1149-
57. 
3. Wells K, Sherbourne C, Schoenbaum M, et al. Five-year impact of quality improvement for 
depression: results of a group-level randomized controlled trial. Arch Gen Psychiatry 
2004;61(4):378-86. 
4. Wells K, Sherbourne C, Duan N, et al. Quality improvement for depression in primary care: 
do patients with subthreshold depression benefit in the long run? Am J Psychiatry 
2005;162(6):1149-57. 
5. Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality 
Chasm Series. 2006. (Accessed at http://www.nap.edu/catalog/11470.html.) 
6. Institute of Medicine. Crossing the Quality Chasm : A New Health System for the 21st 
Century. Washington: National Academy Press; 2001. 
7. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the 
United States. New England Journal of Medicine 2003;348(26):2635-45. 
8. Smedley BD, Stith AY. Unequal treatment: Confronting racial and ethnic disparities in health 
care: National Academy Press; 2003. 
9. McGlynn EA. Six challenges in measuring the quality of health care. Health Affairs 
1997;16(3):7-21. 
10. Cordasco KM, Asch SM, Bell DS, et al. A low-literacy medication education tool for safety-
net hospital patients. Am J Prev Med 2009;37(6):S209-S16. 
11. Young AS, Klap R, Sherbourne CD, Wells KB. The quality of care for depressive and 
anxiety disorders in the United States. Archives of General Psychiatry 2001;58(1):55-61. 
12. Wang PS, Bergland P, Kessler RC. Recent Care for common mental disorders in the United 
States, Prevalence and conformance with evidence-based recommendations. Journal of General 
Internal Medicine 2000;15(5):284-92. 

Page 60 of 72 
 
13. Meredith LS, Jackson-Triche M, Duan N, Rubenstein LV, Camp P, Wells KB. Quality 
improvement for depression enhances long-term treatment knowledge for primary care 
clinicians. J Gen Intern Med 2000;15(12):868-77. 
14. Rost KM, Duan N, Rubenstein LV, et al. The Quality Improvement for Depression 
Collaboration: general analytic strategies for a coordinated study of quality improvement in 
depression care. General Hospital Psychiatry 2001;23(5):239-53. 
15. Dewa CS, McDaid D, Ettner SL. An International Perspective on Worker Mental Health 
Problems: Who Bears the Burden and How Are Costs Addressed? Can J Psychiatry 
2007;52(6):346. 
16. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised 
controlled trials of complex interventions. BMJ 2006;332(7538):413-6. 
17. Greenberg PE, Kessler RC, Birnbaum HG, et al. The economic burden of depression in the 
United States: how did it change between 1990 and 2000? J Clin Psychiatry 2003;64(12):1465-
75. 
18. Rost K, Smith J, Dickinson M. The effect of improving primary care depression management 
on employee absenteeism and productivity:  A randomized trial. Med Care 2004;42(12):1202-10. 
19. Wells KB, Sherbourne CD, Schoenbaum M, et al. Impact of disseminating quality 
improvement programs for depression in managed primary care: A randomized controlled trial. 
Journal of American Medical Association 2000;283(2):212-20. 
20. Schoenbaum M, Unutzer J, Sherbourne C, et al. Cost-effectiveness of practice-initiated 
quality improvement for depression: results of a randomized controlled trial. JAMA 
2001;286(11):1325-30. 
21. Rollman BL, Belnap BH, Mazumdar S, et al. A Randomized Trial to Improve the Quality of 
Treatment for Panic and Generalized Anxiety Disorders in Primary Care. Archives of General 
Psychiatry 2005;62(12):1332. 
22. Miranda J, Duan, N., Sherbourne, C., Schoenbaum, M., Lagomasino, I., Jackson-Triche, M., 
Wells, K. B. Improving Care for Minorities: Can Quality Improvement Interventions Improve 
Care and Outcomes For Depressed Minorities? Results of a Randomized, Controlled Trial. 
Health Services Research 2003;38(2):613-30. 

Page 61 of 72 
 
23. NIMH. Bridging Science and Service:  A report by the National Advisory Mental Health 
Council's Treatment and Services Workgroup. Washington, DC: National Institutes of Mental 
Health, Clinical Treatment and Services Research Workgroup; 1998. 
24. Lenfant C. Clinical research to clinical practice-lost in translation? New England Journal of 
Medicine 2003;349:868-74. 
25. Bloom F. Science as a way of life:  Perplexities of a physician-scientist. Science 
2003;300(5626):1680-5. 
26. Druss BG, Goldman HH. New Freedom Commission Report: introduction to the special 
section on the president's New Freedom Commission Report. Psychiatr Serv 2003;54(11):1465-6. 
27. Stein MB, Roy-Byrne PP, Craske MG, et al. Functional impact and health utility of anxiety 
disorders in primary care outpatients. Med Care 2005;43(12):1164. 
28. Gamble VN. Under the shadow of Tuskegee: African Americans and health care. Am J 
Public Health 1997;87(11):1773-8. 
29. DHHS. Mental Health: Culture, Race and Ethnicity: A Supplement to Mental Health: A 
Report of the Surgeon General. Rockville, MD: Department of Health and Human Services, 
Public Health Service, Office of the Surgeon General; 2001. 
30. Moini M, Fackler-Lowrie N, Jones L. Community Engagement: Moving from Community 
Involvement to Community Engagement—a Paradigm Shift. PHP Consulting 2005. 
31. Bluthenthal R, Jones L, Fackler-Lowrie N, et al. Witness for Wellness: preliminary findings 
from a community-academic participatory research mental health initiative. Ethn Dis 
2006;16(Suppl):S18-34. 
32. Jones L, Wells KB. Strategies for Academic and Clinician Engagement in Community 
Participatory Partnered Research JAMA Submitted. 
33. Shortell SM, Jones RH, Rademaker AW, et al. Assessing the impact of total quality 
management and organizational culture on multiple outcomes of care for coronary artery bypass 
graft surgery patients. Med Care 2000;38(2):207. 
34. Institute of Medicine. Promoting Health: Intervention Strategies From Social and Behavioral 
Research. Washington, DC.: Institute of Medicine, National Academy Press; 2000. 
35. Israel BA, Eng E, Schulz AJ, Parker EA, eds. Methods in Community-Based Participatory 
Research for Health. San Francisco, CA: Jossey-Bass; 2005. 

Page 62 of 72 
 
36. Minkler M. Community-based participatory research for health. San Francisco, CA: Jossey-
Bass; 2003. 
37. Wells KB, Miranda J, Bruce ML, Alegria M, Wallerstein N. Bridging community 
intervention and mental health services research. American Journal of Psychiatry 
2004a;161(6):955-63. 
38. Watkins KE, Paddock SM, Zhang L, Wells KB. Improving Care for Depression in Patients 
with Comorbid Substance Misuse. American Journal of Psychiatry 2006;163(1):125-32. 
39. Smedley BD, Syme SL. Promoting health: intervention strategies from social and behavioral 
research: National Academies Press; 2000. 
40. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials:  Increasing the value of clinical 
research for decision making in clinical and health policy. Journal of the American Medical 
Association 2003;290(12):1624-32. 
41. Zerhouni EA. Translational and Clinical Science -- Time for a New Vision. N Engl J Med 
2005;353(15):1621-3. 
42. Zerhouni EA. US Biomedical Research: Basic, Translational, and Clinical Sciences. JAMA 
2005;294(11):1352-8. 
43. Argyris C, Putname R, Smith DM. Action Science. San Francisco: Jossey-Bass.; 1985. 
44. Beer M. Why Management Research Findings Are Unimplementable: An Action Science 
Perspective Reflections: The SoL Journal 2001;2(3):58-65. 
45. VandeVen A, Johnson P. Knowledge for Theory and Practice. Academy of Management 
Review Forthcoming. 
46. Kataoka SH, Fuentes S, O Donoghue VP, et al. A community participatory research 
partnership: the development of a faith-based intervention for children exposed to violence. 
Ethnicity and Disease 2006;16(1):1. 
47. Unutzer J, Katon W, Williams Jr JW, et al. Improving primary care for depression in late life: 
the design of a multicenter randomized trial. Med Care 2001;39(8):785-99. 
48. Schoenbaum M, Unützer J, McCaffrey D, Duan N, Sherbourne C, Wells KB. The effects of 
primary care depression treatment on patients' clinical status and employment. Health Serv Res 
2002;37(5):1145-58. 

Page 63 of 72 
 
49. Kessler RC, Berglund P, Demler O, et al. The epidemiology of major depressive disorder: 
results from the National Comorbidity Survey Replication (NCS-R). Jama 2003;289(23):3095-
105. 
50. Sherbourne CD, Hays RD, Ordway L, DiMatteo MR, Kravitz RL. Antecedents of adherence 
to medical recommendations: results from the Medical Outcomes Study. Journal of Behavioral 
Medicine 1992;15(5):447-68. 
51. Wells KB, Burnam MA, Rogers W, Hays RD, Camp P. The course of depression in adult 
outpatients.  Results from the Medical Outcomes Study. Archives of General Psychiatry 
1992;49(10):788-94. 
52. Wells KB, Sherbourne CD. Functioning and Utility for Current Health of Patients With 
Depression or Chronic Medical Conditions in Managed, Primary Care Practices. Archives of 
General Psychiatry 1999b;56:897-904. 
53. Murray CJ, Lopez AD. The Global Burden of Disease: A comprehensive assessment of 
mortality and disability from disease, injuries, and risk factors in 1990 and projected to 2020. 
Boston, MA: The Harvard Schoolof Public Health on behalf of the World Health Organization 
and the World Bank; 1996. 
54. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and 
comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. 
Arch Gen Psychiatry 2005;62(6):617-27. 
55. Depression Guidelines Panel. Depression in Primary Care: Volume 1: Detection and 
Diagnosis. Rockville, Maryland: USDHHS, PHS, AHRQ (formerly Agency for Health Care 
Policy Research); 1993a. Report No.: 93-0550. 
56. Depression Guidelines Panel. Depression in Primary Care: Volume 2: Treatment of Major 
Depresson. Rockville, MD: USDHHS, PHS, AHRQ (formerly Agency for Health Care Policy 
and Research); 1993b. Report No.: 93-0551. 
57. Association AP. Practice Guideline for the Treatment of Patients With Major Depressive 
Disorder. 2nd ed. Washington, DC: APA Publishing Inc.; 2000. 
58. Karasu TB, Gelenberg A, Merriam A, Wang P. Practice Guideline for the Treatment of 
Patients With Major Depressive Disorder, Second Edition. Arlington, VA: American Psychiatric 
Association; 2000. 

Page 64 of 72 
 
59. Beck AT, Rush AJ, Shaw BF, Emery G. Cognitive Therapy of Depression. New York, NY: 
The Guilford Press; 1979. 
60. Muñoz RF, Miranda J, Aguilar-Gaxiola S. Individual therapy manual for cognitive-
behavioral treatment of depression: Rand Corp; 2000. 
61. Practice guideline for major depressive disorder in adults. American Psychiatric Association. 
Am J Psychiatry 1993;150(4):1-26. 
62. Wells KB, Golding JM, Hough RL, Burnam MA, Karno M. Acculturation and the 
probability of use of health services by Mexican Americans. Health Serv Res 1989;24(2):237. 
63. Judd CM, Jessor R, Donovan JE. Structural equation models and personality research. J Pers 
1986;54(1):149-98. 
64. Williams JWJ, Barrett J, Oxman TE, et al. Treatment of dysthymia and minor depression in 
primary care: A randomized trial comparing placebo, paroxetine and problem-solving therapy. 
Journal of American Medical Association 2000;284:1570-2. 
65. Wang PS, Berglund PA, Kessler RC. Patterns and correlates of contacting clergy for mental 
disorders in the United States. Health Serv Res 2003;38(2):647-73. 
66. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of 
mental health services in the United States: results from the National Comorbidity Survey 
Replication. Archives of general psychiatry 2005;62(6):629. 
67. Borowsky SJ, Rubenstein LV, Meredith LS, Camp P, Jackson-Triche M, Wells KB. Who is 
at risk of nondetection of mental health problems in primary care? J Gen Intern Med 
2000;15(6):381-8. 
68. Katz SJ, Kessler RC, Lin E, Wells KB. Medication management of depression in the United 
States and Ontario. Journal of General Internal Medicine 1998;13:77-86. 
69. Committee on Bridges to Independence: Identifying Opportunities for and Challenges to 
Fostering the Independence of Young Investigators in the Life Sciences NRCotNA. Bridges to 
Independence: Fostering the Independence of New Investigators in Biomedical Research (2005). 
Washington, D.C.: National Academies; 2005. 
70. Gitterman DP, Sturm R, Scheffler RM. Toward full mental health parity and beyond. Health 
Aff (Millwood) 2001;20(4):68-76. 
71. DHHS. Mental Health: A Report of the Surgeon General. Rockville, MD:: Department of 
Health and Human Services, Public Health Service, Office of the Surgeon General; 1999. 

Page 65 of 72 
 
72. Markowitz FE. The effects of stigma on the psychological well being and life satisfaction of 
persons with mental illness. Journal of Health and Social Behavior 1998;35:335-47. 
73. Kataoka SH, Zhang L, Wells KB. Unmet need for mental health care among US children: 
variation by ethnicity and insurance status. Am J Psych 2002;159(9):1548-55. 
74. Gilbody S, Whitty P, Grimshaw J, Thomas R. Educational and organizational interventions 
to improve the management of depression in primary care. JAMA 2003;289(23):3145-51. 
75. Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. The 
Milbank Quarterly 1996:511-44. 
76. Katon W, Robinson P, Von Korff M, et al. A multifaceted intervention to improve treatment 
of depression in primary care. Archives of General Psychiatry 1996;53(10):924. 
77. Katon W, Von Korff M, Lin E, et al. Collaborative management to achieve treatment 
guidelines. JAMA 1995;273(13):1026-31. 
78. Simon GE, Manning WG, Katzelnick DJ, Pearson SD, Henk HJ, Helstad CP. Cost-
effectiveness of systematic depression treatment for high utilizers of general medical care. 
Archives of general psychiatry 2001;58(2):181. 
79. Rost KM, Nutting P, Smith J, Werner J, Duan N. Improving depression outcomes in 
commmunity primary care practice:  a randomized trial of the quEST intervention.  Quality 
enhancement by stragegic teaming. Journal of General Internal Medicine 2001;16:143-9. 
80. Von Korff M, Katon W, Bush T, et al. Treatment costs, cost offset, and cost-effectiveness of 
collaborative management of depression. Psychosomatic medicine 1998;60(2):143-9. 
81. Unutzer J, Katon W, Callahan CM, et al. Collaborative Care Management of Late-Life 
Depression in the Primary Care SettingA Randomized Controlled Trial. In: American Medical 
Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government 
Use.; 2002. 
82. Rost K, Nutting P, Smith JL, Elliott CE, Dickinson M. Managing depression as a chronic 
disease: a randomised trial of ongoing treatment in primary care. BMJ 2002;325(7370):934. 
83. Lin EHB, Simon GE, Katon WJ, et al. Can enhanced acute-phase treatment of depression 
improve long-term outcomes? A report of randomized trials in primary care. Am J Psych 
1999;156(4):643-5. 

Page 66 of 72 
 
84. Hunkeler EM, Meresman JF, Hargreaves WA, et al. Efficacy of nurse telehealth care and 
peer support in augmenting treatment of depression in primary care. Arch Fam Med 
2000;9(8):700-8. 
85. Simon GE, Ludman EJ, Tutty S, Operskalski B, Von Korff M. Telephone psychotherapy and 
telephone care management for primary care patients starting antidepressant treatment: a 
randomized controlled trial. Jama 2004;292(8):935-42. 
86. Simon GE, Ludman EJ, Unutzer J, Bauer MS, Operskalski B, Rutter C. Randomized trial of 
a population-based care program for people with bipolar disorder. Psychol Med 2005;35(1):13-
24. 
87. Gilbody S, House AO, Sheldon TA. Screening and case finding instruments for depression. 
Cochrane Database Syst Rev 2005;4. 
88. Thompson C, Kinmonth AL, Stevens L, et al. Effects of a clinical-practice guideline and 
practice-based education on detection and outcome of depression in primary care: Hampshire 
Depression Project randomised controlled trial. The Lancet 2000;355(9199):185-91. 
89. King M, Davidson O, Taylor F, Haines A, Sharp D, Turner R. Effectiveness of teaching 
general practitioners skills in brief cognitive behaviour therapy to treat patients with depression: 
randomised controlled trial. BMJ 2002;324(7343):947. 
90. Asarnow JR, Jaycox LH, Duan N, et al. Effectiveness of a quality improvement intervention 
for adolescent depression in primary care clinics. JAMA 2005;293(3):311-9. 
91. Bruce MI, Ten Have TR, Reynolds CF, et al. Reducing suicidal ideation and depressive 
symptoms in depressed older primary care patients: A randomized controlled trial. Journal of the 
American Medical Association 2004;291:1081-91. 
92. Areán PA, Ayalon L, Hunkeler E, et al. Improving depression care for older, minority 
patients in primary care. Med Care 2005;43(4):381. 
93. USDHHS. The President's New Freedom Commission on Mental Health.  Achieving the 
Promise:  Transforming mental health care in America. Rockville, MD: U.S. Department of 
Health and Human Services; 2003. Report No.: SMA-03-3832. 
94. Frank RG, Glied S. Changes in mental health financing since 1971: implications for 
policymakers and patients. Health Aff (Millwood) 2006;25(3):601-13. 

Page 67 of 72 
 
95. Horvitz-Lennon M, Kilbourne AM, Pincus HA. From silos to bridges: meeting the general 
health care needs of adults with severe mental illnesses. Health Aff (Millwood) 2006;25(3):659-
69. 
96. Rubenstein LV, Mittman BS, Yano EM, Mulrow CD. From understanding health care 
provider behavior to improving health care: the QUERI framework for quality improvement. 
Quality Enhancement Research Initiative. Med Care 2000;38(6 Suppl 1):I129-41. 
97. Dietrich AJ, Oxman TE, Williams JW, et al. Re-engineering systems for the treatment of 
depression in primary care: cluster randomised controlled trial. BMJ 2004;329(7466):602. 
98. Rubenstein LV, Jackson-Triche M, Unutzer J, et al. Evidence-based care for depression in 
managed primary care practices. Health Aff 1999;18(5):89-105. 
99. Rubenstein LV, Mittman BS, Yano EM, Mulrow CD. From understanding health care 
provider behavior to improving health care: the QUERI framework for quality improvement. 
Quality Enhancement Research Initiative. Medical Care 2000;38(6 Suppl 1):I129-41. 
100. 
Wells K, Miranda J, Bruce M, Alegria M, Wallerstein N. Bridging community 
intervention and mental health services research. Am J Psych 2004;161(6):955-63. 
101. 
Patel K, Koegel P, Booker T, Jones L, Wells KB. Innovative approaches to participatory 
evaluation in the Witness for Wellness experience. Ethnicity and Disease, special issue 2006, In 
Press. 
102. 
Stockdale S, Patel K, Gray R, Hill DA, Franklin C, Madyun N. Supporting wellness 
through policy and advocacy:  A case history of a working group in a community partnership 
initiative to address depression. Ethnicity and Disease, special issue 2006, In Press. 
103. 
Chung B, Corbett CE, Boulet B, et al. Talking Wellness: A Description of a Community-
Academic Partnered Project to Engage an African American Community around Depression 
through the Use of Poetry, Film, and Photography. Ethnicity and Disease 2006;16 (suppl 1):S1-
67 - S1-78. 
104. 
Jackson-Triche ME, Greer Sullivan J, Wells KB, Rogers W, Camp P, Mazel R. 
Depression and health-related quality of life in ethnic minorities seeking care in general medical 
settings. Journal of Affective Disorders 2000;58(2):89-97. 
105. 
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of Innovations 
in Service Organizations:  Systematic Review and Recommendations. Milbank Q 
2004;82(4):581-629. 

Page 68 of 72 
 
106. 
Wejnert B. Integrating Models of Diffusion of Innovations: A Conceptual Framework. 
Annual Review of Sociology 2002;28:297-326. 
107. 
Institute of Medicine. Speaking of Health: Assessing Health Communication Strategies 
for Diverse Populations. Washington, DC: National Academy Press; 2002. 
108. 
Blumenthal DS, Smith SA, Majett CD, Alema-Mensah E. A trial of 3 interventions to 
promote colorectal cancer screening in African Americans. Cancer 2010;116(4):922-9. 
109. 
Manning WG, Wells KB, Buchanan JL, Keeler EB, Valdez RB, Newhouse JP. Effects of 
Mental Health Insurance: Evidence from the Health Insurance Experiment.: The RAND 
Corporation; 1989. Report No.: R-3815-NIMH/HCFA. 
110. 
Rubenstein LV, Mittman BS, Yano EM, Mulrow CD. From understanding health care 
provider behavior to improving health care: The QUERI framework for quality improvement. 
Med Care 2000;38(6):I. 
111. 
Valente TW. Evaluating Health Promotion Programs. New York: Oxford University 
Press; 2002. 
112. 
Von Korff M, Gruman J, Schaefer J, Curry SJ, Wagner EH. Collaborative management 
of chronic illness. Annals of Internal Medicine 1997;127(12):1097-102. 
113. 
Shortell SM, Marsteller JA, Lin M, et al. The role of perceived team effectiveness in 
improving chronic illness care. Med Care 2004;42(11):1040. 
114. 
Argyris C, 1996 DDSDD. Organizational Learning II: Theory, Method, and Practice. 
Reading, MA: Addision-Wesley; 1996. 
115. 
Rasbash J, Steele F, Browne W, Prosser B. A user's guide to MLwiN version 2.0. 
London: Institute of Education 2004. 
116. 
Wenger E, Snyder WM. Communities of Practice: The Organizational Frontier. Harvard 
Business Review 2000;78(1):139. 
117. 
Bennett KJ, Torrance GW, Boyle MH, Guscott R, Moran LA. Development and testing 
of a utility measure for major, unipolar depression (McSad). Qual Life Res 2000;9(1):109-20. 
118. 
Valente TW. Network Models of the Diffusion of Innovations. Cresskill, NJ: Hampton; 
1995. 
119. 
Bate SP, Robert G, Bevan H. Changing and improving health care systems: social 
movements and other radical approaches. Aldershot: Ashgate Publishing; In press. 

Page 69 of 72 
 
120. 
Davis GF, McAdam D, Scott WR, Zald MN. Social Movements and Organization 
Theory. New York: Cambridge University Press; 2005. 
121. 
Bandura A. Comments on the crusade against the causal efficacy of human thought. J 
Behav Ther Exp Psychiatry 1995;26(3):179-90. 
122. 
Lomas J, Enkin M, Anderson GM, Hannah WJ, Vayda E, Singer J. Opinion leaders vs 
audit and feedback to implement practice guidelines. Delivery after previous cesarean section. 
Jama 1991;265(17):2202-7. 
123. 
Soumerai SB, McLaughlin TJ, Gurwitz JH, et al. Effect of local medical opinion leaders 
on quality of care for acute myocardial infarction: a randomized controlled trial. Jama 
1998;279(17):1358-63. 
124. 
Bandura A. Self-Efficacy:The Exercise of Control. New York, NY: W.H. Freeman; 1997. 
125. 
Schoenwald SK, Hoagwood K. Effectiveness, transportability and dissemination of 
interventions:  What matters when? Psychiatric Services 2001;52:1190-7. 
126. 
Essock SM, Frisman LK, Kontos NJ. Cost-Effectiveness of Assertive Community 
Treatment Teams. American Journal of Orthopsychiatry;68(2):179-90. 
127. 
Rosenheck R, Stroup S, Keefe RSE, et al. Measuring outcome priorities and preferences 
in people with schizophrenia. Brit J Psychiat 2005;187(6):529-36. 
128. 
Unutzer J, Simon G, Belin TR, Datt M, Katon W, Patrick D. Care for depression in HMO 
patients aged 65 and older. J Am Geriatr Soc 2000;48(8):1012-3. 
129. 
U.S. Census Bureau. United States Census 2000. Washington, D.C.; 2000. 
130. 
Los Angeles County Department of Health Services. LA County Department of Health 
Services Key Health Indicators. Los Angeles; 2003. 
131. 
Stockdale SE, Mendel P, Jones L, Arroyo W, Gilmore J. Assessing organizational 
readiness and change in community intervention research:  Framework for participatory 
evaluation. Ethnicity and Disease 2006a;16 (suppl 1):S1-136 - S1-45. 
132. 
Wells KB. The Design of Partners in Care: Evaluating the Cost-Effectiveness of 
Improving Care for Depression in Primary Care. Soc Psychiatry Psychiatr Epidemiol 
1999a;34:20-9. 
133. 
Rohrer JE. Planning for community-oriented health systems. Washington, DC: American 
Public Health Association 1999. 

Page 70 of 72 
 
134. 
Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for 
improving community health. Annu Rev Public Health 2000;21:369-402. 
135. 
Patel K, Koegel P, Booker T, Jones L, Wells KB. Innovative approaches to participatory 
evaluation in the Witness for Wellness experience. Ethnicity and Disease 2006;16 (suppl 1):S35-
42. 
136. 
Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric 
Interview (MINI): the development and validation of a structured diagnostic psychiatric 
interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59:22-33. 
137. 
Sherbourne CD, Sturm R, Wells KB. What outcomes matter to patients? J Gen Intern 
Med 1999;14(6):357-63. 
138. 
Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med 
1991;32(6):705-14. 
139. 
Dwight-Johnson M, Sherbourne CD, Liao D, Wells KB. Treatment preferences among 
depressed primary care patients. J Gen Intern Med 2000;15(8):527-34. 
140. 
Stein MB, Roy-Byrne PP, McQuaid JR, et al. Development of a brief diagnostic screen 
for panic disorders in primary care. Psychosomatic Medicine 1999;61(3):359-64. 
141. 
Prins A, Ouimette P, Kimerling R, et al. The primary care PTSD screen (PC-PTSD): 
development and operating characteristics. International Journal of Psychiatry in Clinical 
Practice 2004;9(1):9-14. 
142. 
Rost K, Burnam MA, Smith GR. Development of screeners for depressive disorders and 
substance disorder history. Med Care 1993;31(3):189-200. 
143. 
Roeloffs C, Sherbourne C, Unützer J, Fink A, Tang L, Wells KB. Stigma and depression 
among primary care patients. Gen Hosp Psychiatry 2003;25(5):311-5. 
144. 
Katon W, von Korff M, Lin E, Bush T, Ormel J. Adequacy and duration of antidepressant 
treatment in primary care. Med Care 1992;30(1):67-76. 
145. 
Ware Jr JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. 
Conceptual framework and item selection. Med Care 1992:473-83. 
146. 
Kroenke K, Strine TW, Spitzer RL, Williams JBW, Berry JT, Mokdad AH. The PHQ-8 
as a measure of current depression in the general population. Journal of Affective Disorders 
2009;114(1-3):163-73. 

Page 71 of 72 
 
147. 
Kessler RC, Ustun TB. The World Mental Health (WMH) Survey Initiative Version of 
the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI). Int 
J Methods Psychiatr Res 2004;13(2):93-121. 
148. 
Pearson ML, Wu S, Schaefer J, et al. Assessing the implementation of the chronic care 
model in quality improvement collaboratives. Health Serv Res 2005;40(4):978-96. 
149. 
Huber P. The behavior of maximum likelihood estimates under nonstandard conditions. 
Fifth Berkeley Symposium on Mathematical Statistics and Probability 1967;1:221-33. 
150. 
Little R, Yau L. Intent-to-treat analysis in longitudinal studies with drop-outs. Biometrics 
1996; 52(4):1324-33. 
151. 
McCaffrey DF, Bell RM. Improved hypothesis testing for coefficients in generalized 
estimating equations with small samples of clusters. Stat Med 2006;25(23):4081-98. 
152. 
Bell RM, McCaffrey DF. Bias reduction in standard errors for linear regression with 
multi-stage samples. Surv Methodol 2002;28(2):169-82. 
153. 
Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. 
Biometrika 1986;73(1):13-22. 
154. 
White H. A heteroskedasticity-consistent covariance matrix estimator and a direct test for 
heteroskedasticity. Econometrica: Journal of the Econometric Society 1980:817-38. 
155. 
Rubin D. Multiple imputation for nonresponse in surveys. Hoboken, NJ: John Wiley & 
Sons; 1987. 
156. 
Schafer JL. Analysis of incomplete multivariate data: Chapman & Hall/CRC; 1997. 
157. 
Song J, Belin TR. Imputation for incomplete high-dimensional multivariate normal data 
using a common factor model. Stat Med 2004;23(18):2827-43. 
158. 
Tang L, Song J, Belin TR, Unützer J. A comparison of imputation methods in a 
longitudinal randomized clinical trial. Stat Med 2005;24(14):2111-28. 
159. 
Yang M, Rasbash J, Goldstein H, Barbosa M. MLwiN macros for advanced multilevel 
modelling. London: Institute of Education 1999. 
160. 
Hosmer DW, Lemeshow S. Applied logistic regression: Wiley-Interscience; 2000. 
161. 
Duan N, Manning Jr WG, Morris CN, Newhouse JP. A comparison of alternative models 
for the demand for medical care. J Bus Econ Stat 1983:115-26. 
162. 
Manning WG. The logged dependent variable, heteroscedasticity, and the 
retransformation problem. J Health Econ 1998;17(3):283-95. 

Page 72 of 72 
 
163. 
Maddala GS. Limited-dependent and qualitative variables in econometrics: Cambridge 
Univ Pr; 1983. 
164. 
Meredith LS, Mendel P, Pearson M, et al. Implementation and maintenance of quality 
improvement for treating depression in primary care. Psychiatr Serv 2006;57(1):48-55. 
165. 
Lancaster T. The econometric analysis of transition data New York: Cambridge 
University Press; 1990. 
166. 
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful 
approach to multiple testing. Journal of the Royal Statistical Society Series B (Methodological) 
1995:289-300. 
167. 
Bandura A. Social foundations of thought and action: A social cognitive theory. 
Englewood Cliffs, NJ: Prentice-Hall; 1986. 
168. 
Little RJA. Missing-data adjustments in large surveys. J Bus Econ Stat 1988:287-96. 
169. 
Lavori PW, Dawson R, Shera D. A multiple imputation strategy for clinical trials with 
truncation of patient data. Stat Med 1995;14(17):1913-25. 
170. 
Raudenbush SW, Bryk AS, Cheong YF, Congdon R. HLM6. 01: Hierarchical Linear and 
Nonlinear Modeling [manual]. Lincolnwood, Ill: Scientific Software International 2005. 
171. 
Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for 
missing data in longitudinal studies. Psychological Methods 1997;2(1):64. 
172. 
Reise SP, Duan N. Multilevel modeling and its application in counseling psychology 
research. The Counseling Psychologist 1999;27(4):528-51. 
173. 
Rosenbaum P, Rubin DB. Difficulties with regression analyses of age-adjusted rates. 
Biometrics 1984 40(2):437-43. 
174. 
Muthén LK, Muthén BO. Mplus: statistical analysis with latent variables. Los Angeles: 
Muthén & Muthén; 2005. 
175. 
Littell RC, Milliken G, Stroup W, Wolfinger R. SAS system for mixed models. Cary, 
NC: SAS Institute Inc.; 1996. 
176. 
Kish L. Survey sampling: New York: Wiley; 1965. 
177. 
Murray DM. Design and analysis of group-randomized trials: Oxford University Press, 
USA; 1998. 
178. 
Cohen J. Statistical power analysis for the behavioral sciences: Lawrence Erlbaum; 1988. 
 

A-1 
 
Appendix A 
This report provides information on detailed technical methods for handling of issues for the 
analysis of 6-month client data as well as 12-month and 3 years follow-ups. 
 
Table of Contents 
Page 
A1 
Analytic Sample 
A-2 
A2 
Weighting 
A-3 
A3 
Multiple Imputation 
A-4 
A4 
Analyses 
A-6 
A5 
References for Appendix 
A-10 
 
 

A-2 
 
A1. Analytic Sample 
Enrollment  
The study settings were South Los Angeles and Hollywood-Metro. Participant sampling 
(program and client recruitment) and randomization are described in greater detail in prior 
publications. Within enrolled programs, clients were screened for eligibility in waiting rooms 
(consecutive selection) or events (random selection) between March 2010 to November 2010 by 
community members blinded to intervention condition and supervised by RAND. Staff 
approached 4,645 adults (age≥18) over 2-3 days per program; 4440 (96%) agreed to screening. 
Study eligibility was limited to clients providing contact information and depressed based on a 
score of 10 or greater on a modified 8-item Patient Health Questionnaire (PHQ-8) (3). Of 4,440 
screened, 1,322 (30%) were eligible and 1246 (94%) consented. In our previous publications,(1-
3) we created enrollment weights based on propensity weighting adjustment, by fitting logistic 
regression models to predict the enrollment among those eligible. The reciprocal of the predicted 
response probability was used as the enrollment weight for each participant. See Supplementary 
Materials in publications. 1-5 
Telephone survey 
Baseline survey. Between April 27, 2010-January 2, 2011, we approached 1,246 
consented clients for baseline telephone survey conducted by survey staff blinded to intervention 
condition and 981 clients (79% of 1246 consented, RS: 492, CEP: 489) completed the baseline, 
36 refused (RS: 17, CEP: 19), 227 were unable to contact (RS: 96, CEP: 131), and 2 deceased 
(RS: 1, CEP: 1). 
6 month follow-up survey. Between November 2, 2010 and August 11, 2011, we 
approached 1093 participants (RS: 540, CEP: 553) for 6-month follow-up telephone survey; 153 
enrolled participants were excluded from the contact list because their baseline survey response 
status was in one of following categories: final refusal, ill or incarceration, unable to contact, or 
deceased. Of 1093 attempted for 6-month follow-up, N=759 (69% of attempted, RS: 380, CEP: 
359) completed the survey, 12 refused (RS: 3, CEP: 9), 321 were unable to contact (RS: 157, 
CEP: 164), and 3 deceased (RS: 1, CEP: 2).  
12 month follow-up survey. Between May 10, 2011 and March 12, 2012, we approached 
974 participants for 12-month follow-up telephone survey; 272 enrolled participants were not 
attempted for 12-month follow-up based on their baseline or 6 month survey status. Of 974 

A-3 
 
attempted for 12-month follow-up, N=733 (75% of attempted, RS: 364, CEP: 369) completed 
the survey, 7 refused (RS: 4, CEP: 3), 229 were unable to contact (RS: 109, CEP: 120), and 5 
deceased (RS: 3, CEP: 2).  
3 year follow-up survey (extension study). Between 1/14/2014 and 10/14/2014, we 
attempted to contact 1004 participants for 36-month follow-up with exclusion who deceased 
(RS: 4, CEP: 4), who made final refusal in previous wave (RS: 22; CEP: 27), or who had no data 
on baseline, 6, and 12 months (RS: 84; CEP: 101).  Of 1014 attempted for 36-month follow-up, 
600 (600/1014=59%) participated (RS 293, CEP:307), 24 were deceased (RS:14, CEP:11), 10 
refused (RS:7, CEP:3), 3 were too ill/incapable (RS:2, CEP:1), and 367 (RE:180, CEP:186) were 
not able to be reached. 
4-year follow-up of the 3-year sample (extension study).  The CPIC 4-year study was 
added to explore longer-term outcomes for the sample completing the 3-year follow-up, based on 
community stakeholder interest.  In addition, this follow-up offered CPIC participants the 
opportunity to be participants in an additional project, the Community and Patient Partnered 
Research Network (CPPRN).  Between 3/31/2016 and 5/26/2016, we attempted to contact 598 
clients (RS: 292, CEP: 306) who completed CPIC 3 years interview participants excluding 2 that 
declined follow-up (RS 1, CEP 1). The telephone interview took about 15-20 minutes. At the end 
of the interview, a respondent was asked “Would you like to hear about the CPPRN program?” 
and “do you agree to be part of the CPPRN?” if answered ‘yes’.  Of those, 283 (47.3%) 
participated (RS 143, CEP 140), with 308 not reached (RS 143, CEP 165), 1 deceased (RS 1, 
CEP 0), 5 refused (RS 5, CEP 0), and 1 too ill to participate (RS 0, CEP 1). 5 
 
A2. Weighting 
Enrollment The enrollment weight was intended to make the enrolled sample (n=1,246) 
representative of a specific target population. 6, 7  That target population was defined as people 
who were: age 18 or older, visited a participating CPIC sties during our screening window, 
screened positive for PHQ-8, and provided contact information.  We created enrollment weights 
based on propensity weighting adjustment, by fitting logistic regression models to predict the 
enrollment among those eligible.  The reciprocal of the predicted response probability was used 
as the enrollment weight for each participant.  Five versions of the enrollment weight were 
created corresponding to five imputed screener data, because imputed predictors from the 

A-4 
 
screener data were used in fitting logistic regressions.  Common predictors of age, community, 
and type of programs were used in all models. 
Attrition The analytic sample for 6 months is comprised of 1,018 participants who completed 
baseline, or 6-month telephone follow-up surveys.  We used nonresponse weighting to address 
missing data for subjects who did not who did neither complete baseline nor 6 months follow-up 
telephone surveys.  We started with a large set of independent categorical variables to be 
considered for a logistic regression on the outcome of response among enrolled participants.  The 
final model included predictors that were significant (p<0.05) for either intervention arm (age, 
gender, ethnicity, living situation, income, US born), as well as for two design variables 
(community and sector of the screening program).  The final weights defined on the analytic 
sample were the product of the two adjustment factors for enrollment and nonresponse. See 
Supplementary Materials in publications. We applied the weights to the 36-month outcome 
analysis excluding 24 deceased cases (1,004 attempted - 24 deceased = 980). The weights for 3-
year main analysis defined on the analytic sample were the product of the two adjustment factors 
for enrollment and nonresponse. See Supplementary Materials in previous publications. 1-3  
For 4-year follow-up, we tested bivariate associations between an indicator for 
responding at the 4-year survey and demographic and clinical characteristics at baseline and 3 
years. 5 The weighting model to predict follow-up status from 3-year sample included screener 
sector, age, education, working for pay, PCS-12, poor mental health-related quality of life at 
baseline, and risk for homelessness at 3 years. 
 
A3. Multiple Imputation 
Item-level Missing Data We used an extended hot deck multiple imputation technique to impute 
missing values for item-level nonresponse. 8, 9  The procedure was based on cycling through each 
missing-data pattern on each variable with incomplete items.  This method involved two steps: 1) 
forming imputation classes based on the predicted mean of the variable being imputed from a 
multiple regression model, and 2) drawing imputations at random from observed data within 
each class based on an approximate Bayesian bootstrap.  To reflect the uncertainty of donor cells 
we created bootstrap weights and then used the product of the bootstrap weights in the multiple 
imputation model.  Five imputed datasets were created.  Each of the imputed data sets differs by 
the bootstrap weight and the seed used to obtain the random number employed in the hot deck 

A-5 
 
imputation.  Data on several hundred, multi-item scales were collected at screener, baseline and 
follow-up time points.  Most variables had item-level missingness rates of less than 5% except 
for baseline income and MINI variables.  With imputations stratified by intervention arms, 5 
alternative imputed datasets were produced for screener, baseline, 6 month, 12 month, and 3 year 
follow-ups, and multiple imputation inferences were used in all analysis. 10, 11   
The approach for selecting variables for multiple regression models was intended to 
preserve the associations and relationships among variables.  In general, we identified common 
predictors for all imputation models including design variables (community, type of programs), 
social demographic variables (age, gender, ethnicity, marital status, education, living situation, 
income, and working status), and PHQ-8 score.  For baseline, 6 months, 12 months, and three 
year data, baseline health variables (count of chronic medical conditions, PCS12 and MCS12) 
were included.  In addition to these common predictors, each imputation model also included 
other predictors to be used in later analyses of interest. The order in which variables were 
imputed was determined based on a judgment of the analytic importance of the variables and the 
degree of missing data. Earlier imputed values were used during subsequent imputation steps, 
implying some dependence on the order in which variables were imputed. 
Unit-Level Multiple Imputation We used a hot deck multiple imputation procedure based on an 
approximate Bayesian bootstrap method for unit-level missing data. 12, 13  This model assumes 
that both missingness and dropout arise from mechanisms that are missing at random (MAR) in 
the sense defined by Rubin.  Our imputation techniques attempted to include information related 
to the missing values whenever possible.  We first modeled the propensity of response at a given 
time point (coded 1 if response and 0 if nonresponse).  In Step 2, we stratified cases based on the 
quintiles of the propensity scores and used the approximate Bayesian bootstrap to select donors. 
In practice these procedures were applied in sequence for the baseline, 6-month, 12-month, and 
3-year data, with imputations stratified by two intervention arms.  We started with imputing 
baseline.  For each of the 5 item-level imputed screener datasets, we imputed a unit-level 
imputation baseline dataset.  Limited to the analytic sample of 1,018, we then used baseline 
variables as predictors for modeling 6 and 12-month follow-up data and produced unit-level 
imputation datasets.  In modeling the logistic regression of predicting response propensities, we 
started with a large set of independent variables.  The final baseline model included the 
predictors: age, gender, ethnicity, income, living situation, US born, community, and type of 

A-6 
 
screening program.  The 6-month models included participants characteristics assessed at 
screener (age, gender, ethnicity, health insurance, and type of screening program.), and baseline 
clinic and service variables (multiple chronic conditions, alcohol abuse or use of illicit drugs, any 
depression care).  The 12-month models included additional variables: community, PHQ-8 
assesses at screener, mental wellness, homeless status at baseline.  The 3 years models included 
age, gender, ethnicity, employment status, ≥3 chronic conditions, homeless, 12-month alcohol 
abuse or use of illicit drugs, no place to stay for at least two nights in the past 6 months, type of 
screening program, and community with additional stratum variable sector (social-community 
screening sector vs healthcare Screening Sector) in Step 2.  Values for participants who were 
deceased were not imputed. 
 
A4. Analyses 
For all study components (including supplement components discussed below), we convened 
working groups co-chaired by academic and community members to review analysis plans, data 
output and discuss findings. 
Analyses for 6-month outcome: We conducted intent-to-treat, comparative-effectiveness 
analyses with intervention status as the independent variable, using logistic regression for 
dichotomous measures and Poisson regression models for count variables.  Covariates were 
selected to account for known associations of sociodemographic and clinical status indicators 
with primary outcome measures.  We adjusted for baseline status of dependent variables and 
covariates (age, sex, ≥3 chronic general medical conditions from a list of 18, education, 
race/ethnicity, family income < federal poverty level, 12-month alcohol abuse or use of illicit 
drugs at baseline, 12-month depressive disorder at baseline, and community).  All analyses were 
conducted using SUDAAN Version 10.0 (http://www.rti.org/sudaan/) and accounted for 
clustering (clients within programs), weighting, and multiple imputations.  Significance of 
comparisons by intervention status was based on regression coefficients.  Results of logistic 
regression models are presented as odds ratios (OR) and Poisson models as rate ratios (RR) with 
95 % confidence intervals.  We illustrate results for intervention groups adjusted for covariates 
using standardized predictions generated from fitted regression models. 14  We present Cohen’s 
effect size index h, defined for dichotomous variables,15 where h=0.20 is small, h=0.50 is 
medium and h=0.80 is large.  We supplement adjusted models with unadjusted raw data to assess 

A-7 
 
robustness.  We applied a Bonferroni adjustment considering two primary outcomes (Poor 
mental health qualify of life MCS-12<=40, and at least mild depression PHQ-9>=10) and 
compared the proportion of results across outcomes favoring one intervention to an assumed 
null-hypothesis of 0.5. 
Analyses for 12-month outcome: We conducted intent-to-treat analyses of repeated measures 
including all participants with available data at baseline, 6-, or 12-months by using SAS software 
version 9.2 (SAS Institute).  We used a model-based approach with unweighted data.  Initial 
explorations of 3-level, random-effects logistic models using SAS PROC GLIMMIX for binary 
outcomes yielded unstable estimates for program-speciﬁc random effects. 16, 17 
We analyzed dichotomous and count outcomes by using a generalized estimating 
equation (GEE) framework. 18 Speciﬁcally, we ﬁtted logistic regression models for binary out-
comes and Poisson models for count data using SAS PROC GENMOD, specifying exchangeable 
correlation at the program level, with regression adjustment for baseline covariates (age, sex, 
>=3 chronic conditions, education, race/ethnicity, family poverty, alcohol abuse or use of illicit 
drugs in the past 12 months, depressive disorder in the past 12 months, and community).  We 
then developed a contrast involving a linear combination of coefﬁcients to test intervention 
effects at each end point (baseline, 6 months, and 12 months) and tested differences between 
intervention groups in change from baseline to 6 and 12 months. 
The results of analyses of binary outcomes are presented as odds ratios, and the results of 
Poisson regression analyses of count data are presented as rate ratios.  We treated time as a 
continuous variable and examined the ﬁxed effects for time and intervention, and their 
interactions.  We included quadratic terms (squared effect of time and its interaction with the 
intervention) which allowed insight into whether changes are greater from baseline to 6 months 
or subsequent months. 
In analyzing continuously scaled MCS-12 as the dependent variable, we used a 3-level, 
mixed-effect regression model by using SAS PROC MIXED.  We accounted for the multilevel 
data structure with clients nested within pro-grams and repeated measurements nested within 
clients.  To account for the intraclass correlation expected in the data, we speciﬁed random 
effects at the program level and an autoregressive covariance structure within clients to account 
for within-client correlation over time. 

A-8 
 
We conducted sensitivity analyses for alternative representations of time as a continuous 
or class variable and for alternative weighting approaches.  To investigate possible nonignorable 
effects, we used 2 methods.  For continuous measures (such as MCS-12 and number of service 
visits), we multiplied ignorable-model imputations alternatively by 1.1 and 0.9 to reveal 
sensitivity to 10% departures from ignorable-model predictions with dichotomized versions of 
continuous measures (MCS-12 <= 40) based on the imputed continuous value.  For categorical 
imputations where reference cells were based on an underlying continuous measure (that is, 
predicted response propensity) including an indicator for any utilization and adjusted Bayesian 
bootstrap imputations reﬂecting unit nonresponse at a particular time point, nonignorable 
imputations for cases in nonboundary reference cells were generated by borrowing values from 
the reference cell with either the next higher or next lower value of the underlying continuous 
measure. 19  We also conducted 12-month endpoint analysis controlling baseline with weighting 
adjustment and presented standardized predictions. 2, 14 
Analyses for 3-year outcome: 
We conducted 3 years endpoint analysis controlling baseline with weighting adjustment (see 
analysis strategy for 6 months).  Given multiple secondary outcomes, 19 we built on the false-
discovery-rate (FDR) framework 20 as extended by Yekutieli and Benjamini 21 and used both 
standard and FDR-adjusted p values (pFDR) in interpreting results across a large number of 
regression analyses.  Results with pFDR <.05 are viewed as convincing evidence of a difference, 
and higher pFDR thresholds are considered as suggestive evidence.  We separately calculated 
pFDR for the two primary outcomes, the community-prioritized outcomes, service use in the 
health care sector, service use in the social-community sector, medication use, and summary 
utilization indicators. 
 
We conducted a longitudinal sensitivity analysis using all waves of data (baseline, 6-
months, 12-months, 3-years) without response weights, adjusting for baseline covariates as in the 
primary analysis.  We specified a spline model, with a linear segment between baseline and the 
first follow-up for initial improvement and another linear segment for the subsequent follow-ups; 
the two linear segments were speciﬁed to join at the ﬁrst follow-up.  In analyzing continuously 
scaled PCS-12 as the dependent variable, we used a 3-level, mixed-effects regression model by 
using SAS PROC MIXED.  To account for the intraclass correlation due to the multilevel 
structure, we specified random effects at the program level and a spatial power covariance 

A-9 
 
structure at the client level to account for unequal spacing of waves.  Initial explorations of 3-
level, random-effects logistic models using SAS PROC GLIMMIX for binary outcomes yielded 
unstable estimates for program-specific random effects.  We utilized a generalized estimating 
equation (GEE) framework with logistic regression models for binary outcomes and Poisson 
models for counts using SAS PROC GENMOD, specifying exchangeable correlation at the 
program level.  From the estimated spline models, we developed a contrast involving a linear 
combination of coefficients to test intervention effects at each end point (baseline, 6-months, 12-
months, and 3-years) and tested differences between intervention groups in change from baseline 
to 6 months, 12 months, and 3 years. 
 
Exploratory analyses. The CPIC study had 2 primary outcomes: poor MHRQL (12-item 
mental health composite score, MCS-12<=40), and depressive symptoms (PHQ-8 >=10 or PHQ-
2 < 3 for 4-year follow-up); and a community-prioritized outcome of mental wellness (a response 
of at least “a good bit of the time” in the previous 4 weeks to feeling calm or peaceful, having 
energy, or being happy), based on community input matched to the 36-item short-form health 
survey quality-of-life items. We developed 2 indicators: “clinical remission” (PHQ-8 < 10 or 
PHQ-2 < 3) and, with stakeholder input, “community-deﬁned remission” (PHQ-8 < 10 or PHQ-2 
< 3 or MCS-12 > 40 or any mental wellness). We also used a count of survey periods in 
remission from baseline up to 3-year follow-up. Intervention status was the client’s program 
assignment at enrollment to CEP or RS. 
For time (survey period) to ﬁrst remission, we used Cox proportional hazard models for 
the full sample with any survey data (n = 1018), limited to those at risk for subsequent clinical (n 
= 995) and community-deﬁned (n = 408) remission at baseline. We used linear regression for 
number of periods in remission during the course of 3 years for the full sample (n = 1018). We 
developed models including intervention status, adding social determinants, then depression 
treatment. 5 
Analyses for 4-year outcome: 
We conducted intent-to-treat analyses with intervention status as the main independent 
variable, using logistic regression for probability of remission (clinical, community-deﬁned) for 
the sample with 4-year data (n = 298). We conducted analyses parallel to the 3-year analyses 
noted previously, using the same covariates plus base-line measure of the outcome and the 1 
baseline item differing by intervention status among 4-year completers. 5 

A-10 
 
A5. References 
 
1. Wells KB, Jones L, Chung B, Dixon EL, Tang L, Gilmore J, Sherbourne C, Ngo VK, Ong MK, 
Stockdale S, Ramos E, Belin TR, Miranda J. Community-partnered cluster-randomized 
comparative effectiveness trial of community engagement and planning or resources for services 
to address depression disparities. J Gen Intern Med. 2013 Oct;28(10):1268-78. doi: 
10.1007/s11606-013-2484-3. Epub 2013 May 7. Erratum in: J Gen Intern Med. 2013 
Nov;28(11):1534.  PubMed ID: 23649787 
2. Chung B, Ong M, Ettner SL, Jones F, Gilmore J, McCreary M, Sherbourne C, Ngo V, Koegel P, 
Tang L, Dixon E, Miranda J, Belin TR, Wells KB. 12-month outcomes of community 
engagement versus technical assistance to implement depression collaborative care: a partnered, 
cluster, randomized, comparative effectiveness trial. Ann Intern Med. 2014 Nov 18;161(10 
Suppl):S23-34. doi: 10.7326/M13-3011.  PubMed ID: 25402400 
3. Ong MK, Jones L, Aoki W, Belin TR, Bromley E, Chung B, Dixon E, Johnson MD, Jones F, 
Koegel P, Khodyakov D, Landry CM, Lizaola E, Mtume N, Ngo VK, Perlman J, Pulido E, Sauer 
V, Sherbourne CD, Tang L, Vidaurri E, Whittington Y, Williams P, Lucas-Wright A, Zhang L, 
Southard M, Miranda J, Wells K. A Community-Partnered, Participatory, Cluster-Randomized 
Study of Depression Care Quality Improvement: Three-Year Outcomes. Psychiatr Serv. 2017 
Dec 1;68(12):1262-1270. doi: 10.1176/appi.ps.201600488. Epub 2017 Jul 17.  PubMed ID: 
28712349 
4. Belin TR, Jones A, Tang L, Chung B, Stockdale SE, Jones F, Wright A, Sherbourne CD, Perlman 
J, Pulido E, Ong MK, Gilmore J, Miranda J, Dixon E, Jones L, Wells KB. Maintaining Internal 
Validity in Community Partnered Participatory Research: Experience from the Community 
Partners in Care Study. Ethn Dis. 2018 Sep 6;28(Suppl 2):357-364. PMID: 30202188 
5. Arevian AC, Jones F, Tang L, Sherbourne CD, Jones L, Miranda J; Community Partners in Care 
Writing Group. Depression Remission From Community Coalitions Versus Individual Program 
Support for Services: Findings From Community Partners in Care, Los Angeles, California, 
2010-2016. Am J Public Health. 2019 Jun;109(S3):S205-S213. doi: 10.2105/AJPH.2019.305082.   
PubMed ID: 31242001 
6. Groves RM: Survey nonresponse: Wiley-Interscience, 2002 
7. Korn EL, Graubard BI: Analysis of health surveys. Hoboken, New Jersey: Wiley-Interscience, 
1999 
8. Bell RM, McCaffrey DF. Bias reduction in standard errors for linear regression with multi-stage 
samples. Survey Methodology 2002;28(2):169-82. 

A-11 
 
9. Little RJA. Missing-data adjustments in large surveys. Journal of Business & Economic Statistic 
1988;6(3):287-96. 
10. Rubin D. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ, Wiley, 1987 
11. Schafer JL: Analysis of incomplete multivariate data: CRC press, 1997 
12. Lavori PW, Dawson R, Shera D: A multiple imputation strategy for clinical trials with truncation 
of patient data. Statistics in medicine 14:1913-25, 1995 
13. Tang L, Song J, Belin TR, et al.: A comparison of imputation methods in a longitudinal 
randomized clinical trial. Statistics in medicine 24:2111-28, 2005 
14. Graubard BI, Korn EL. Predictive margins with survey data. Biometrics. 2004;55(2):652-9. 
15. Cohen J. Statistical power analysis for the behavioral sciences: Lawrence Erlbaum; 1988 
16. Murray DM: Design and analysis of group-randomized trials: Oxford University Press, USA, 
1998 
17. Littel RM, G; Stroup, W; et al.: SAS System for Mixed Models. Cary, NC: SAS Institute, Inc, 
1996 
18. Liang K-Y, Zeger SL: Longitudinal data analysis using generalized linear models. Biometrika 
73:13-22, 1986 
19. Belin TR, Hu MY, Young AS, Grusky O. Performance of a general location model with an 
ignorable missing-data assumption in a multivariate mental health services study. Stat Med. 
1999;18:3123-35. [PMID: 10544311] 
20. Benjamini Y, Yekutieli D: The control of the false discovery rate in multiple testing under 
dependency. Annals of Statistics 29:1165–1188, 2001 
21. Benjamini Y, Hochberg Y: Controlling the false discovery rate: a practical and powerful 
approach to multiple testing. Journal of the Royal Statistical Society. Series B. Methodological 
57:289–300, 1995 
22. Yekutieli D, Benjamini Y: Resampling-based false discovery rate controlling multiple test 
procedures for correlated test statistics. Journal of Statistical Planning and Inference 82:171–196, 
1999 

 
Study Name: 
Community Partners in Care (CPIC) 
NCT Number: 
NCT01699789 
HSPC (RAND’s Institutional Review Board)  
HSPC Project ID: 
w2812-06-01 
Federalwide Assurance number (FWA): 
FWA00003425 
IRB number: 
IRB00000051 
 
Content: 
Informed consent forms 
Dates of Document 
 
Screener: 
2/16/2010 
Enrollment: 
2/16/2010 
Three years: 
10/2/2013 
Four years:  
2/29/2016 
 

COMMUNITY PARTNERS IN CARE (CPIC) 
 
 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 2 
Rev. 2-16-10 
SCREENER CONSENT 
 
We are asking you and over 7,000 other adults in Los Angeles County to participate in a 
10 minute interview. The purpose of this interview is to see who may be asked to 
participate in a new research project. 
 
WHO IS LEADING THIS RESEARCH PROJECT? 
The name of the project is Community Partners in Care (CPIC). The project is carried 
out by community-based organizations led by Healthy African American Families, 
QueensCare Health and Faith Partnership, and the Los Angeles County Department of 
Mental Health Services, as well as the RAND Corporation and UCLA. 
 
WHAT IS THIS INTERVIEW ABOUT? 
• This short interview is a screening to see who may be asked to participate in a 
research project.  
• During the interview, I will ask you about how you have been feeling and services 
you may be getting. I will also ask you questions to see if you have symptoms of 
depression or too much stress. By depression, we mean feeling sad, blue, feeling 
down, feeling less than, or feeling stressed almost all of the time and having 
other symptoms like poor concentration or poor appetite. 
 
WHAT WILL WE ASK YOU TO DO? 
• If you agree, we can complete the interview right here in a private place and it will 
take 10 minutes. You can either read it yourself or I can assist you if you would 
like. As a thank you, I will give you $5 in cash for completing this interview.  
• There are no right or wrong answers. Based on your answers, we will know right 
away if I will talk with you about participating in the next part of the research 
project. I will explain that project and you can agree or refuse to participate in it at 
that time.  
 
POTENTIAL BENEFITS 
If you agree to be interviewed today: 
• After the interview, we give everyone a CPIC Resource Guide which provides 
information on mental health and other services available in the community.  
• The results of the research may help improve understanding of how to help 
people in the community who have depression.  
• I will give you $5 as a thank you. 
 
RISKS OF PARTICIPATION 
• Answering some questions might upset or embarrass you. However, you can 
skip any questions you do not want to answer. 
• If the results suggest you may be depressed, you might become concerned or 
worried; but this is only a screening. To determine if you might have depression 
or another problem that needs treatment, you would need to visit a provider for a 
clinical assessment.  
 
CLIENT ID 

COMMUNITY PARTNERS IN CARE (CPIC) 
 
 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 2 of 2 
Rev. 2-16-10 
CONFIDENTIALITY 
• Any information you give us will only be used to a) find out if we might ask you to 
continue in the project and b) to help us describe the different types of people 
who agreed to be interviewed.  
• Information you provide will be kept in locked file cabinets and in secure 
computer files. Only research staff will have access to these files.  
• I will NOT ask for your name for this interview except for certain emergencies 
described below. 
• If you are asked to participate in the main project and agree, I will ask for your 
name at that time.  
 
While we don’t ask about this subject, if you tell me that a child or elderly person is 
being abused, I will ask for your name and must report this to my supervisor who may 
then report it to the appropriate authorities. If you tell me you might harm yourself or 
others, I will ask for your name and I may give information about you to people who 
could help protect you from that harm, which could include needed medical attention. A 
project clinician or provider might also call you if you tell me you might harm yourself or 
others. 
 
COSTS 
There is no cost to you for participating in this interview.  
 
PARTICIPATION IS VOLUNTARY 
You do not have to participate if you do not want to. You can skip any question or stop 
the interview at any time. You may drop out of the project at anytime. Your choice about 
being in this project will not affect the care or services you are receiving in any way. 
Participation will have no effect on decisions about probation, parole, detention or 
sentencing, if any apply to you.  
 
WHO CAN YOU CONTACT? 
• Judy Perlman is the CPIC Survey Director at RAND and can be reached at (310) 
393-0411 x7811. Our toll free number for CPIC is (877) 499-7411.  
• Dr. Kenneth Wells is the Principal Investigator at RAND and can be reached 
through the CPIC project coordinator, Esmeralda Ramos and can be reached at 
(310) 794-3719. 
• If you have questions regarding your rights as a research subject, contact Marilyn 
Yokota at the RAND Institutional Review Board at (310) 393-0411 x6369. 
• You may contact the LA County Department of Mental Health Research 
Committee at (213) 738-4600. 
• You can have a copy of this information sheet. 
 
 
CPIC STAFF USE ONLY 
My signature indicates that I have explained the research to the person and answered all of his/her questions and 
that I believe he/she understands the information described in this document. 
 
NAME:______________________________________   SIGNATURE:_____________________________________ 
 
DATE:______________________________________ 

COMMUNITY PARTNERS IN CARE (CPIC) 
 
 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-16-10 
 
ENROLLMENT CONSENT 
CONSENT PROVIDED BY SIGNATURE ON-SITE OR ORALLY BY PHONE 
 
 
WHY HAVE YOU BEEN SELECTED? 
You were selected for the main project because you reported some symptoms of depression or 
stress in the short screening interview. Many problems and stresses can cause similar symptoms. 
Only a provider who has done a clinical assessment can tell whether you have depression or 
another health issue that might benefit from treatment.  
 
WHAT WILL WE ASK YOU TO DO? 
For this research project we will ask you to: 
1.  Complete a 1 hour telephone interview in the next two weeks for which we will give you 
$15 to thank you for your time.  
2. Complete another interview in 6 months and one more in 12 months. You will receive $20 
for the 6-month interview and $25 for the 12-month interview to thank you for your time. 
We may ask you to participate in additional interviews past 12 months but you can choose 
whether or not you wish to at that later time. 
3.  Give us your telephone number and other ways to contact you so we can do the telephone 
interviews. 
4. We may ask you to participate in related research regarding services you received, but you 
can choose whether or not you wish to at that later time.  
 
WILL CPIC GIVE YOU TREATMENT AS PART OF THIS PROJECT? 
No, the project does not provide treatment. The project can only give you a list of places where 
you can go to receive treatment. It is up to you to decide if you want treatment. We cannot 
guarantee that any agency on this list can take you for treatment, or take you quickly.  
 
If you get assessed or treated for depression, it may be important for you to discuss what goes into 
your written record with your provider. This is important because some private insurance 
companies sometimes deny coverage or set prices based on an individual’s treatment history.  
 
BENEFITS OF PARTICIPATING IN THIS PROJECT 
• Some people find the interviews interesting and helpful in their understanding some of their 
feelings.  
• You might receive information about depression that might help you or someone you know. 
 
RISKS OF PARTICIPATING IN THIS PROJECT 
• If you participate in this project, you might seek services that you did not consider before.  
• You might be embarrassed by some of the questions we will ask you during the interview.  
 
CONFIDENTIALITY 
• All the information you give us for this project will be kept confidential. Information you 
provide will be kept in locked file cabinets and in secure computer files. Only research staff 
will have access to these files.  
 
 
 
CLIENT ID 

COMMUNITY PARTNERS IN CARE (CPIC) 
 
 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 2 of 4 
Rev. 2-16-10 
 
• The only exceptions are: while we don’t ask about this subject, if you tell me that a child or 
elderly person is being abused, I must report this to my supervisor who may then report it to 
the appropriate authorities. If you tell me you might harm yourself or others, I will give 
information about you to people who could help protect you from that harm, which could 
include needed medical attention. A project clinician or provider might also call you if you 
tell me you might harm yourself or others.  
• The project has received a Certificate of Confidentiality from the National Institutes of 
Health which gives participants the added protection that we do not have to release 
information about individuals from this project even under court order.  
• In some cases, we may give your contact information to a staff person at a service agency. 
This person may contact you about a visit with them or refer you to another agency. 
 
PARTICIPATION IS VOLUNTARY 
You do not have to participate if you do not want to. You can skip any question or stop the 
interview at any time. You may also drop out of the project at anytime. Your choice about being 
in this project will not affect the care or services you are receiving in any way. Participation will 
have no effect on decisions about probation, parole, detention or sentencing, if any apply to you.  
 
WHO CAN YOU CONTACT? 
• Judy Perlman is the CPIC Survey Director at RAND (310) 393-0411 x 7811. Our toll free 
number for CPIC is (877) 499-7411. 
• Dr. Kenneth Wells is the Principal Investigator at RAND and can be reached through the 
CPIC project coordinator, Esmeralda Ramos at (310) 794-3719. 
• If you have questions regarding your rights as a research subject, contact Marilyn Yokota at 
the RAND Institutional Review Board at (310) 393-0411 x 6369. 
• You may contact the LA County Department of Mental Health Research Committee at 
(213) 738-4600. 
• You can have a copy of this information sheet. 
 
SIGNATURE OF RESEARCH PARTICIPANT 
My signature indicates that I have read (or someone has read to me) this consent form and that I 
understand the information. Any questions that I have asked have been answered, and I agree to 
take part in the telephone interviews. 
 
YOUR NAME: _____________________________________________ 
 
YOUR SIGNATURE: ________________________________________ 
 
DATE: ______________________________________________ 
 

COMMUNITY PARTNERS IN CARE (CPIC) 
 
 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 3 of 4 
Rev. 2-16-10 
 
 Please check off each statement after you read it. 
 YOU CAN KEEP THIS FORM 
Community Partners in Care 
 
 
 
 
 
  
Things you should know about the project: 
 
1.   We are asking you to complete a one hour telephone interview in 
the next two weeks, another in 6 months, and another in 12 
months. Each interview will last about an hour. We will pay you 
$15 for the first interview, $20 for the second, and $25 for the 
third.  
 
2. 
We are asking you to participate in CPIC because you reported 
symptoms of feeling sad, depressed, or stressed.  
 
3.   You don’t have to be in this project. You can leave the project at 
any time.  
 
4.   If you do not want to be in the project, the care or services you 
are receiving will not change.  
 
5.   Participation in CPIC will have no effect on decisions about 
probation, parole, detention or sentencing, if any apply to you.  
 
6.   We might tell a provider or staff member at the agency where you 
completed the short interview that you are enrolled in CPIC.  
 
7.   You could be contacted by someone at that agency or a partner 
agency about further services available to you. 
 
8. 
You will be given a CPIC Resource Guide for places to go for 
services. It is up to you to decide whether to go and where to go.   
 
9. 
CPIC cannot guarantee that you will get an appointment for an 
assessment or what services you will receive. 
 
10. Do you have any questions? 
 
11. Do you agree to be in this project?

COMMUNITY PARTNERS IN CARE (CPIC) 
 
 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 4 of 4 
Rev. 2-16-10 
 
 
Want to Know More? 
We are only asking you to participate in telephone interviews, but you may be 
wondering why we are doing this project or what is going to happen during 
the project. This sheet will give you more information about CPIC.  
 
WHAT IS THE MAIN PROJECT ABOUT? 
This project compares two different ways of supporting community-based agencies in 
providing services for depression. All participating agencies are being provided with 
some help. Some agencies are supported independently by the project to provide care 
for depression. Other agencies are supported to work together as a group to improve 
services. For example, they might share resources or staff.  
 
Agencies are placed in the individual support or group planning by chance, like a coin 
toss, also called randomization. Regardless of which support program they are in, all 
agencies are free to use or not to use the resources provided by the project.  
 
The resources provided by the project support education, assessment, and treatments 
that are proven to help people with depression. There are no new or experimental 
treatments being supported in the project.  
 
The agency that you are in now has been assigned to one of the two ways we 
mentioned above for improving services.  
 
• You are free to tell a provider or staff member at this agency about your 
participation in this project if you wish.  
• The provider or staff member at this agency is free to work with you using 
resources from the project or to follow their usual procedures.  
 
If this agency has been assigned to the group planning model for improving services, 
you may receive a follow-up call or be asked to return for a visit at either this agency or 
another agency from the group of agencies working together. That is the main way that 
you will learn that your agency is in the group-support program.  
 
Regardless, you will be able to visit or use agencies in your community and receive 
services as you normally would. You and your providers will also be able to choose 
whether or not to use any resources from this project that are offered to you. That is, 
these resources may be new options that you can choose to use or not.  
 
Either way, all agencies have support from the information and resources provided by 
the project to help people with depression in some way. We hope to learn whether it is 
better for agencies to do their best by themselves or to spend the time working 
together, and that is why the project is being conducted by community and academic 
partners. We will use the results of the project to share what we learned together about 
how to improve depression services in the community.  
 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 2 
Rev. 10--2-2013 
Community Partners in Care 
40-Month Interview Consent Script 
 
Hello, my name is [INTERVIEWER NAME] and I'm calling from RAND about Community 
Partners in Care to speak with [NAME]. 
 
You may remember that you completed a telephone interview with us some time ago. Thank 
you for doing that. I am calling now to conduct another interview with you. The interview will take 
about 30-35 minutes depending on your experiences and we will send you a $40 gift card as a 
thank you. 
 
We sent you a letter to let you know we would be calling. Did you receive that letter? 
 
IF YES: Great. First I need to confirm that I'm speaking to the right person. What is your date of 
birth? 
IF MATCHES  GO TO intro 
IF DOB DOES NOT MATCH  GO TO dobprob 
 
IF NO: No problem. The letter explained that in the interview, we will be asking you some 
questions about your health and how you have been feeling lately, as well as your experiences 
using community services. 
 
First I need to confirm that I'm speaking to the right person. What is your date of birth? 
 
IF MATCHES  GO TO intro 
IF DOB DOES NOT MATCH  GO TO dobprob 
 
>dobprob< 
That does not match the date listed in our records. Is this (RESPONDENT NAME)? 
  
<1> YES, DOB MATCHES  GO TO intro 
<2> YES, BUT DOB DOES NOT MATCH  GO TO dobprob2 
<3> NO, WRONG PERSON  END CALL. 
 
>dobprob2< 
I cannot continue the interview if the dates do not match. I need to speak to my supervisor. 
Someone will call you back when we have more information.  END CALL. 
 
>intro< 
Thank you. Next, I need to go over a few important points with you.  
 
You were selected for this interview because you reported some symptoms of depression or 
stress in a past CPIC interview. Many problems and stresses can cause similar symptoms. Only 
a provider who has done a clinical assessment can tell whether you have depression or another 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 2 of 2 
Rev. 10--2-2013 
health issue that might benefit from treatment. Because you participated in a past interview, we 
want to follow up with you and see how you are doing now. We want you to know that the 
information you can give us is important, but you do not have to take part in this study.  
Everything you tell us is confidential. The only exceptions are: although I will not be asking 
questions about this, if you tell me that a child or elderly person is being abused, we may report 
it to the appropriate authorities. If you tell me that you might harm yourself, I will give information 
about you to people who could help protect you from that harm, which could include needed 
medical attention. A project clinician or provider will likely call you if you tell me you might harm 
yourself or others.  
 
You can skip any question you do not want to answer or stop the interview at any time. 
Information you provide will be kept in locked file cabinets and in secure computer files. Only 
research staff will have access to these files. That means we will not tell anyone outside of the 
study your answers. 
 
* 
The project has received a Certificate of Confidentiality from the National Institutes of Health 
which gives participants the added protection that we do not have to release information 
about individuals from this project even under court order.  
* 
In some cases, we may give your contact information to a staff person at a service agency. 
This person may contact you about a visit with them or refer you to another agency.  
 
There are phone numbers to call if you have questions about this study or your rights. Would 
you like me to give you those numbers now? 
   
IF YES: Judy Perlman is the CPIC Survey Director at RAND. You can reach her at 310-393-
0411 x 7811. Dr. Kenneth Wells is the Principal Investigator at RAND and can be reached 
through the CPIC project coordinator, Esmeralda Ramos, at (310) 794-3719. 
 
IF NEEDED: If you have questions regarding your rights as a research subject, contact the 
RAND Human Subjects Protection Committee at (310) 393-0411 x6369. You may also contact 
the LA County Department of Mental Health Research Committee at (213) 738-4611. 
 
Do you agree to take part in the telephone interview? 
 
IF YES  START INTERVIEW. 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
CPPRN Enrollment Consent Script 
 
Hello, my name is [INTERVIEWER NAME] and I'm calling from RAND about Community 
Partners in Care to speak with [NAME]. 
 
You may remember that you completed a telephone interview with us some time ago. Thank 
you for doing that. I am calling now to conduct another interview with you. The interview will take 
about 15-20 minutes depending on your experiences and we will send you a $20 gift card as a 
thank you. We also would like to discuss another project with you, and will send you an 
additional $5 gift card as a thank you for taking the time to consider that project, regardless of 
whether you agree to participate. 
 
We sent you a letter to let you know we would be calling. Did you receive that letter? 
 
IF YES: Great. First I need to confirm that I'm speaking to the right person. What is your date of 
birth? 
IF MATCHES  GO TO intro 
IF DOB DOES NOT MATCH  GO TO dobprob 
 
IF NO: No problem. The letter explained that in the interview, we will be asking you some 
questions about your health and how you have been feeling lately, as well as your experiences 
using community services. 
 
First I need to confirm that I'm speaking to the right person. What is your date of birth? 
 
IF MATCHES  GO TO intro 
IF DOB DOES NOT MATCH  GO TO dobprob 
 
>dobprob< 
That does not match the date listed in our records. Is this (RESPONDENT NAME)? 
  
<1> YES, DOB MATCHES  GO TO intro 
<2> YES, BUT DOB DOES NOT MATCH  GO TO dobprob2 
<3> NO, WRONG PERSON  END CALL. 
 
>dobprob2< 
I cannot continue the interview if the dates do not match. I need to speak to my supervisor. 
Someone will call you back when we have more information.  END CALL. 
 
Thank you for participating in CPIC!  Because you participated in a past interview, we want to 
follow up with you and see how you are doing now.  This interview will take about 15-20 
minutes. 
 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
We want you to know that the information you can give us is important, but you do not have to 
take part in this interview.  Everything you tell us is confidential.  The only exceptions are: 
although I will not be asking questions about this, if you tell me that a child or elderly person is 
being abused, we may report it to the appropriate authorities. If you tell me that you might harm 
yourself, I will give information about you to people who could help protect you from that harm, 
which could include needed medical attention. A project clinician or provider will likely call you if 
you tell me you might harm yourself or others.  
 
You can skip any question you do not want to answer or stop the interview at any time. 
Information you provide will be kept in locked file cabinets and in secure computer files. Only 
research staff will have access to these files. That means we will not tell anyone outside of the 
study your answers. 
 
The project has received a Certificate of Confidentiality from the National Institutes of Health 
which gives participants the added protection that we do not have to release information about 
individuals from this project even under court order.  
 
START INTERVIEW. 
 
 
END INTERVIEW 
 
Now, we would like to take a few minutes to invite you to participate in a new program of 
research called Community and Patient Partnered Research Network, or CPPRN for short. 
CPPRN is part of a national network called PCORnet, working to improve through research the 
information available to patients, providers and communities, about health and healthcare 
decisions that affect them. CPPRN is focused on under-resourced communities and behavioral 
health and social issues such as stress, sadness, resiliency, housing, and jobs.  CPPRN is a 
joint effort across Los Angeles and New Orleans.  If you can take a few minutes to hear our 
invitation to participate in CPPRN, we will send you a $5 gift card as a thank you. This is in 
addition to the $20 gift card you will receive for participating in the interview.  Would you like to 
hear about the CPPRN program?   
 
IF NO: [Proceed to questions about contact information and sending the $20 gift card, if they 
completed an interview.] 
 
IF YES: CPPRN is setting up a program of research to understand people’s priorities for health 
and healthcare about behavioral health and social factors like housing.  People can participate 
in CPPRN by providing information on their priorities as well as some information about their 
background and health.   
 
If you agree to participate in CPPRN, we will ask you to let us use the contact information you 
gave to CPIC, your information from today’s CPIC survey and some data from prior CPIC 
surveys for the new CPPRN.  We will use these data to determine if you are eligible for other 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
studies of ours or of other PCORnet programs, as they come up, and then contact you either by 
our CPPRN staff or lead investigators of the other PCORnet programs about those studies. We 
also may contact you in the future to ask permission to obtain information on services use from 
your providers for any health, mental health, or social services.  For any of these requests, you 
can choose whether or not to give us permission at that time.  While this will take some of your 
time to participate, we hope it may be meaningful to be part of a new national set of programs 
working to provide better information for patients, providers and communities.  Any information 
we obtain will only be used for research purposes and will be presented anonymously, that is 
without naming you.   
 
Do you agree to be part of the CPPRN? 
 
IF YES: Great.  
 
Do you have any questions about CPPRN?    
Just to confirm, you have agreed to be part of CPPRN, are willing to be contacted for future 
studies, and agree that we can use your information from CPIC in combination with other’s 
information, without naming you.  Correct?  
 
IF NO: OK, thanks.  Would you still like to be on a contact list for other activities in the CPPRN, 
such as having access to our website or giving opinions on health issues?  IF NO THANK 
RESPONDENT AND GO TO SECTION ON SENDING THE $20 AND $5 GIFT CARDS.   
 
IF YES: Great. (ASK FOR UPDATED CONTACT INFORMATION) 
 
 
GET CONTACT INFORMATION 
 
 I really appreciate your patience and honesty.  I need to get some information to help us 
contact you in the future.  
  
  
NAME 
We have your name as [FILL R NAME].  Any other names you go by? 
  
        
            @a___________________OTHER NAMES/ALIAS 
  
MAILADD 
What is your mailing address?  This is the address we will send your gift card to.  Please allow 
3-4 weeks for processing. 
_________________________   STREET ADDRESS  
 
 
 
 
 (C/O, Apt.) 
_________________________  CITY            

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
_________ STATE  
_________________________  COUNTRY  
__________________________  ZIPCODE  
  
 
 
SAMEADD 
Is your mailing address the same as your home address?  
  
1           YES  à GO TO HPNUM 
0          NO 
  
HOMEADD 
What is your home address?  
_________________________   STREET ADDRESS  
 
 
 
 
 (C/O, Apt.) 
_________________________  CITY            
_________ STATE  
_________________________  COUNTRY  
__________________________  ZIPCODE  
  
  
HPNUM 
What is your correct home telephone number, including area code?  
  
            |__|__|__|  (Area Code)  |__|__|__| - |__|__|__|__|  
  
ENTER 000-000-0000 IF NO PHONE 
ENTER r if REFUSED 
  
HPLIST 
Is this telephone number listed in your name?  
INTERVIEWER: IF NO HOME PHONE CODE YES  
1          YES  à GO TO CPNUM 
0          NO    
 
 
HPNAME 
In whose name is the telephone number listed?  
[INTERVIEWER:  WRITE FULL NAME VERBATIM.  CHECK WITH R TO MAKE SURE YOU 
HAVE FULL NAME AND CORRECT SPELLING] 
            __________________________________ (VERBATIM-LIMITED)  
  
CPHONE 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
Do you also have a separate cell phone number? 
1          YES   
0          NO   à GO TO EMAIL 
 
 
CPNUM 
What is your correct cell number, including area code?  
  
            |__|__|__|  (Area Code)  |__|__|__| - |__|__|__|__|  
ENTER r if REFUSED 
  
HEMAIL 
Do you have an e-mail address you use?  
1          YES   
0          NO   à GO TO CONT1a 
 
 
HEMAILAD 
What is this e-mail address?  
            ___________@_________________  (END SHOULD BE .com, .edu, .gov, .net, .org)  
  
  
 
  
CONT1a 
In case we have difficulty getting in touch with you in the future, could you give me the name, 
address, and phone number of one person not currently living with you who will always know 
your whereabouts? PROBE:  This might be a family member or a close friend or someone else 
who knows where you are. IF NECESSARY:  We would only get in touch with this person if we 
could not reach you. 
  
  
What is this person’s  name? 
  
__________________________NAME 
  
ENTER 0 IF REFUSED AND GO TO INPUT 
  
CONT1b 
How is this person related to you?  
DO NOT READ ANSWERS.  CODE R'S RESPONSE.  
INTERVIEWER NOTE:  THIS SHOULD BE A PERSON NOT CURRENTLY LIVING WITH R  
1.   MOTHER 
2.   FATHER 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
3.   STEPMOTHER 
4.   STEPFATHER 
5.   GRANDMOTHER 
6.   GRANDFATHER 
7.   DAUGHTER 
8.   SON 
9.   AUNT 
10. UNCLE 
11. SISTER (INCLUDING HALF SISTER, STEP SISTER) 
12. BROTHER (INCLUDING HALF BROTHER, STEP BROTHER) 
13. OTHER RELATIVES 
14. FRIEND 
15. COWORKER 
16. OTHER, SPECIFY-LIMITED __________________________ 
  
CONT1c 
What is his/her address? 
______________________________  (ADDRESS)  
 
 
 
 
 (C/O, Apt.) 
__________________________ (CITY)  
________  (STATE)  
______________________________  (COUNTRY) 
_____________(ZIP) 
  
CONT1d 
What is his/her home phone number? 
(__ __ __) __ __ __ - __ __ __ __  EXT. __ __ __ __  (PHONE NUMBER) 
  
CONT1e 
How about a cell phone number? 
(__ __ __) __ __ __ - __ __ __ __  (CELL PHONE NUMBER) 
ENTER 000-000-0000 IF NO CELL NUMBER 
ENTER r IF REFUSED 
  
 
INPUT 
Those are all the questions I have for you.  Thank you for participating.   
 
GET INFORMATION ON BEST WAY TO SEND GIFT CARDS 
  
I also wanted to let you know that there are phone numbers to call if you have questions about 
the CPPRNs study or your rights. Would you like me to give you those numbers now? 
 

COMMUNITY PARTNERS IN CARE (CPIC) 
RAND/UCLA INFORMATION FOR PARTICIPATION IN RESEARCH 
Page 1 of 4 
Rev. 2-29-2016 
IF NEEDED: Dr. Kenneth Wells is the Principal Investigator of CPPRN at RAND and UCLA and 
can be reached through the CPPRN project coordinator, Krystal Griffith, at (310) 794-2967. 
 
IF NEEDED: If you have questions regarding your rights as a research subject, contact the 
RAND Human Subjects Protection Committee at (310) 393-0411 x6369.