INTEGRATE-D: A Pilot Test of Implementation Strategies to Support Integration of Medical and Psychosocial Care for People With Type II Diabetes
Phase: NA
Conditions: Type II Diabetes, Primary Health Care, Behavioral Health
Sponsor: Oregon Health and Science University

Page 1 of 26 
 
 
Study title: INTEGRATE-D: A Pilot-test of Implementation Strategies to Support Integration of Medical and Psychosocial 
Care for People with Type II Diabetes 
 
Document date: 10/26/2022 
 
NCT: 04461405 
 
 

Page 2 of 26 
 
Minimal Risk Protocol 
1. Protocol Title 
INTEGRATE-D: A Pilot-test of Implementation Strategies to Support Integration of Medical and 
Psychosocial Care for People with Type II Diabetes 
2. Objectives 
The proposed study consists of two aims: 1) To refine INTEGRATE-D, an intervention that offers primary 
care practices implementation support to align type II diabetes mellitus care with America Diabetes 
Association recommendations for integrating medical and psychosocial care. 2) The study will pilot test 
INTEGRATE-D, recruiting and randomly selecting four primary care practices to an experimental condition 
(i.e., treatment and control group). The study team will use mixed methods to assess the acceptability, 
feasibility, and cost of implementing INTEGRATE-D and to understand the preliminary impact on DMII and 
behavioral health process and outcome measures. The study hypothesis are: 
1. Practices receiving INTEGRATE-D will find it acceptable and feasible and worth the effort and 
cost. 
2. Practices receiving INTEGRATE-D will meet a higher number of process-of-care metrics for 
DMII and depression (pre-post) as compared to control practices (primary hypothesis). 
3. Intervention practices will report using more quality-improving strategies post-intervention than at 
baseline, compared to control practices. 
4. Practices receiving INTEGRATE-D will demonstrate a positive trend (improvement in PHQ-9 and 
HbA1c scores (pre-post) as compared to control practices (secondary hypothesis). 
The anticipated methodological approaches include, chart audits, observations, surveys, and semi- structured 
interviews. Practice member staff and patients will be recruited from the participating sites. 
3. Background 
The American Diabetes Association1 (ADA) recommends integrating medical and psychosocial care 
(integration) to improve health outcomes for patients with comorbid diabetes mellitus (DM) and common 
mental and behavioral health (BH) problems. This is because when patients’ depression, anxiety and diabetes 
distress are addressed, patients are better able to engage in self-management activities (e.g., medication 
adherence, physical activity, diet) and manage their DM.2-19,78 
In the US, 30 million people live with type II diabetes (DMII),20 and the majority of care for this chronic 
disease is delivered in primary care setting.21,22 By implementing the ADA recommendations, primary care 
practices can help patients manage their behavioral health needs, meet recommended goals for DMII 
management, and reduce the risk of adverse outcomes.1,2,4,79-87 In response to changes in federal and local 
policies and funding, some primary care practices have added staff, such as behavioral health clinicians 
(BHCs) and / or developed relationships with externally located BHCs, that can support integrated care for 
individuals with DMII.28,88-95 
While some practices may strive to provide integrated care per ADA recommendations, few have been able 
to do so comprehensively.96,97 Comprehensive change to align with ADA guidance would include 
systematic screening for depression and anxiety, diabetes distress, cognitive impairment 

Page 3 of 26 
 
and self-management behaviors, delivering consistent treatments for DMII and for psychosocial need, and 
having treatment pathways for engaging BHCs and others when need is detected. 
Implementing practice changes to align care with ADA recommendations could address persistent quality of 
care gaps and national disparities in DMII care quality and outcomes.2,23-29 Yet, few primary care 
practices have made or sustained changes to routinize integration of medical and psychosocial care for 
patients with DMII.98-101 
Implementing integrated care for patients with DMII involves complex practice change.25-29,38-40 This 
includes adopting systematic case finding and diagnosis, patient engagement/education, treatment approaches 
based on emerging evidence, engaging clinicians with different backgrounds (e.g., psychologists) and 
sometimes in different locations in treatment and following-up (including adjusting care plans) to ensure that 
improvement occurs and is sustained.23,25 To accomplish this, changes are needed to practice operational 
processes and workflows, documentation and information sharing, and communication.28,29,39,40 Multi-
faceted implementation support approaches that engage leadership and multidisciplinary teams are needed to 
implement ADA recommendations.98-100,102 Effective combinations of ‘implementation strategies’ enable 
clinical teams to put evidence-based care in place, and involve quality reporting/feedback46-48 in 
combination with technical assistance,57-62 health information technology (HIT) support.28,63-66 and 
facilitation50-54,57 which helps practices make changes that are tailored to practices needs and 
resources.54,62,67-75 
4. Study Design 
The study team will conduct a practice-randomized pilot study to estimate effect of the intervention 
on practice-level outcomes. The study team will also evaluate feasibility of recruitment, 
randomization, retention, assessment procedures, implementation of the INTEGRATE-D intervention 
and cost of doing so using mixed methods. Four practices will be recruited and randomized to either 
an intervention (N=2) or control arm (N=2). The study team will collect mixed methods data prior to 
implementation of the intervention (baseline) to assess practice capacity as well as detailed 
information about integrated behavioral health and DMII care processes. During implementation, a 
qualitative researcher will conduct monthly telephone calls with a key informant at the intervention 
practices. Post- intervention, the study team will collect follow-up surveys and conduct observation 
and interviews with intervention practices to assess what was or was not implemented and why, as 
well as acceptability, feasibility and perceived cost / benefits. The study team will conduct chart 
audits with intervention and control practices to measure key behavioral health and DMII process and 
outcome measures, as is commonly performed when assessing an array of behavior health integration 
process measures.189,240 Although this design is not optimal to conclusively demonstrate 
effectiveness, it is appropriate to gather preliminary evidence as to the promise of INTEGRATE-D in 
achieving desired outcomes. 
5. Study Population 
i. 
Number of Subjects 
i. 
Primary Care Practice Sites. The study team will recruit four practices, two rural and two non-
rural practices that vary on whether they employ a behavioral health consultant. Oregon Rural 
Practice-based Research Network (ORPRN) practice sites that meet the eligibility criteria will 
participate in data collection activities that will involve surveys, observation, interviews (staff 
and patients), and chart audits. 

Page 4 of 26 
 
ii. 
Practice Members. Staff at all four participating practices will be invited to complete the Practice 
Member Questionnaire, which asks about staff demographics (e.g., gender, race/ethnicity, role, 
years in practice) and elements of the practice’s culture that align with adaptive capacity (e.g., 
leadership, team-work, communication). We will use a random selection process, where practice 
members are stratified and randomly selected by role, to select a few (1-3) practice members from 
each role in the practice (e.g., clinician, office manager, medical assistant) to participate in the 
survey. We anticipate that 10–15 staff members per practice, depending on practice size, will be 
invited to complete the questionnaire. Our anticipated sample size for the proposed study will be 
40-60 practice members. 
i. 
Patients. Patients with DMII diagnosis seen at least once at the practice in the 15 months prior to 
the intervention start date and at least once after the intervention start date will be eligible for 
chart audits. Because it is costly and time prohibitive to audit medical records for “all” eligible 
patients, the study team will randomly select 80 patients per practice from those eligible; random 
sampling will ensure that the results from our analyses will generalize to the sample of patients 
who are eligible to receive the intervention. For patient interviews, with assistance from 
practices, the study team will approach the 30 patients selected for chart audits about 
participation in semi- structured interviews. Our total sample size for the pilot study will be 320 
patients across four practices. 
ii. 
Inclusion and Exclusion Criteria 
Inclusion Criteria 
i. 
Primary care practices must be affiliated with ORPRN. 
ii. 
Primary care practices must have more than 100 adult patients with DMII. 
iii. 
Patients, from the selected primary care practices, must be at least 18 years old. 
iv. 
Patients, from the selected primary care practices, must have a diagnosis of DMII at the 
time of study enrollment. 
v. 
Patients must be seen at least once at a participating practice in the 15 months prior to the 
intervention start date and at least once after the intervention start date. 
iii. 
Vulnerable Populations 
Vulnerable populations will not be specifically targeted or recruited for study participation as defined 
by the OHSU IRB. This study will not include any vulnerable populations. We will not collect any 
information about subjects’ status as prisoners, pregnant women, or adults lacking capacity. 
iv. 
Setting 
ORPRN is a statewide practice-based research network of primary care providers, community partners, 
and academicians dedicated to studying health care delivery and how to improve the health of 
Oregonians and to reduce health disparities. ORPRN works with 279 primary care practices, including 
51% that are rural and 42% provider-owned. Practices have participated in studies related to care 
delivery redesign, heart health, cancer prevention, and social 

Page 5 of 26 
 
determinants of health. ORPRN is organized into regions, each served by a facilitator that supports 
improvement initiatives and collects research data. 
v. 
Recruitment Methods 
ORPRN’s experienced recruiters will personally contact practices through a faxback process to assess 
interest, capacity and eligibility. ORPRN will distribute a recruitment flyer to its network of primary care 
practices, detailing the purpose of the study, eligibility requirements, as well as responsibilities of 
participating practices. Interested sites will submit a non-binding agreement to ORPRN to participate in 
the pilot study, either by fax or email. ORPRN and the study team will select four practices. 
INTEGRATE-D aligns with programs relevant for practices in the Northwestern region (e.g., CPC+; 
Accountable Care Organization changes) and recruiting should not pose a challenge. Practices that agree 
to participate must sign a Memorandum of Understanding outlining expectations and responsibilities for 
participation. To encourage practice member participation, full explanation of the study and the purpose 
of the data collection activities will be given to practice members by the study team. For patient 
participation in interviews, the study team will work with a practice liaison, who will make the first 
contact with patients to ascertain permission to be contacted by a study member. The research team will 
contact them to explain the study, review the consent information sheet, and provide details of the 
interview. Patients will receive compensation in the form of a $25.00 gift for their participation. Patients 
will be informed that the interviews are entirely voluntary. A patient sample will be drawn from the chart 
audits. Medical records are a secondary analysis of existing data; therefore, the study team will not recruit 
or seek consent from patients for the chart audits. 
vi. 
Consent Process 
Modifications to the Consent Process 
i. 
Primary Care Practice Sites. The study team will distribute informational material to 
practice leads of participating sites to explain the study to practice members. Additionally, 
the research team will be available to explain the study. 
ii. 
Practice Members. For the survey and interviews, members of the team will explain the study 
using an information sheet and answer any questions before data collection begins. The 
observation, surveys and interviews present no more than minimal risk, do not involve 
procedures for which consent is required outside a research context, and avoids any 
documentation with interviewees’ names. 
iii. 
Patients. For interviews, the study team will explain the study, review the information sheet and 
answer any questions before conducting an interview. As with practice members, the information 
sheet presents no more than minimal risk and do not involve procedures for which consent is 
required outside a research context. The IRB submission includes a waiver of consent for patient-
level data, which includes the chart audits and patient interviews, as consent is not feasible given 
our process for obtaining chart audits. The study team anticipates a sample of 80 chart audits per 
practice (320 chart audits in total). The number of charts to be reviewed renders the contact of 
each individual impracticable. Additionally, current contact information for potential subjects 
may not be available. Therefore, the time and resources needed to conduct an informed consent 
process are not reasonable in relation to the time to conduct the study. 

Page 6 of 26 
 
6. Procedures Involved 
As mentioned, the pilot study consists of two aims. Aim 1 is focused on the development of the 
intervention (i.e., INTEGRATE-D), beginning in Year 1, quarters 1 and 2. Data collection will not during 
this time as the study team will compile and refine the intervention. When the refinement period for 
INTEGRATE-D is complete, the study team will move to an active phase of the intervention (i.e., Aim 2), 
set to begin Year 1, quarter 2. Aim 2 will demonstrate the feasibility, acceptability, and cost of 
implementing INTEGRATE-D. See Table 1. The study team will employ mixed methods. 
Table 1. Data Collection Timeline 
 
Data Collection Activities 
Year 1 
Year 2 
Quarter 
1 
2 
3 
4 
1 
2 
3 
4 
Semi-Structured Interviews 
Direct Observations (i.e., Site Visit Observation Guide) 
 
X
B 
 
 
 
 
X
F 
• 
Pre-Site Visit Interview 
 
X
B 
 
 
 
 
 
• 
Practice Member Staff Interviews 
 
X
B 
 
 
 
 
X
F 
• 
Interim Interviews 
 
X 
X 
X 
X 
X 
X 
• 
Patient Interviews 
 
 
 
 
 
X
F 
Survey Administration 
• 
Practice Survey 
 
X
B 
 
 
 
X
F 
• 
Practice Member Questionnaire 
 
X
B 
 
 
 
X
F 
Chart Audits 
• 
Test Chart Audits 
 
 
 
X  
 
• 
Conduct Chart Audits 
 
 
 
 
 
X 
 
Qualitative Methodology. The procedures for qualitative and quantitative approaches are 
documented as follows. 
i. 
Primary Care Practices. In Year 1, quarter 1, the study team will randomly assign the four primary 
care practices to an experimental condition (i.e., treatment and control group). Following the 
assignments, in Year 1, quarter 3 the study team will conduct semi-structured interviews with staff at 
all four practices. Originally, the study team proposed site visit observations; however, due to 
COVID-19, semi-structured data collection will occur by phone or via web-conferencing software 
(i.e., Cisco Webex). Follow-up interviews will occur 15 months, post-intervention. 
During the semi-structured interviews, study team members will take notes. Neither protected health 
information (PHI) nor identifiers will be recorded. Field notes will be prepared within 24 hours of the 
site visit’s end. All notes will be de-identified and destroyed after the full field notes are prepared. 
After the baseline interviews, the study team will also conduct interim interviews with one point of 
contact from each practice implementing INTEGRATE-D. The interviews will be 30- minute 
monthly check-in phone calls to minimize recall bias and gather real-time experiences with 
INTEGRATE-D. The audio recordings will be professionally transcribed, checked for accuracy and 
de-identified. 

Page 7 of 26 
 
ii. 
Practice Member Staff. Five to ten individuals per practice will be recruited to participate in formal 
and informal interviews. The study team will ask open-ended questions to explore organizational and 
contextual factors and discuss their experiences with delivering integrated DMII care. The interview 
will take about 60 minutes, and with permission, the interviews will audio-recorded. The audio 
recordings will be professionally transcribed, checked for accuracy and de-identified. As with the 
observations, the study team will return for follow-up interviews. 
iii. 
Patients. Post-intervention (Year 2, quarter 3), the study team will conduct interviews with five 
patients, at each practice, that were exposed to the intervention to understand patients’ experiences 
with the practice changes that were implemented and the acceptability of those changes. Patient 
interviews will also consist of open-ended questions. All interviews will be audio-recorded and last 
approximately 60 minutes. The audio recordings will be professionally transcribed, checked for 
accuracy and de-identified. 
Quantitative Methodology 
i. 
Chart Audits. The study team will conduct virtual chart audits all four primary care practices (80 
individuals per practice). The data will measure key behavioral health (i.e., depression), DMII 
process (i.e., self-management activities). The study team will conduct the audit, pre- intervention for 
a baseline, and 15 months post-intervention for follow-up. As noted earlier, chart audits are a 
secondary analysis of existing data; therefore, the study team will not recruit or seek consent from 
patients for the chart audits. 
ii. 
Surveys. The study team will administer two surveys during the study. 
The first of which is a Practice Survey. One practice staff person (e.g., office manager, clinical lead) 
will complete this survey for each practice. The Practice Survey collects demographic data (e.g., 
ownership) and include items from the Change Process Capability Questionnaire to measure quality 
improvement strategies the practice uses. The survey will be administered online though RedCap. 
The Practice Member Questionnaire will be administered online via Qualtrics to a select number of 
staff. We will ask the office manager for each practice to provide a list with employees’ names, role 
and email addresses. The study team will use a stratified random process to select staff who will 
receive a survey. Staff who are selected will receive an email invitation from us to complete an 
anonymous online survey. We anticipate that there will be 10-15 practice members asked to complete 
this survey at each practice, but this will vary based on practice size and diversity of team roles. Due 
to COVID-19, the study team altered its initial approach, which would have taken place at an in-
person all-staff meeting to collected baseline and follow-up data. For now, we are altering baseline 
data collection only. 
For the initial submission, the study team is submitting for approval of the following qualitative and 
quantitative measures at baseline: the Pre-Site Visit Interview Guide, Site Visit Observation Guide, Practice 
Member Staff Interview, Patient Interview, Practice Survey, and Practice Member Questionnaire. As the 
study team refines the intervention, modifications to the IRB will be submitted. 

Page 8 of 26 
 
For Study Modification #3, the study team re-submitted the Practice Survey. The revised version removes a 
number of descriptive items already available to the team (e.g., practice size, practice ownership), amended 
questions for the Oregon context (i.e., Patient Centered Primary Care Home (PCPCH) program), updated 
question phrasing, expanded answer choices (including “Don’t Know” and “Behavioral health team”) and 
incorporated three items about the use of an electronic health record system in primary care practices. 
7. Data and Specimens 
a. Handling of Data and Specimens 
i. Primary Care Practices. As noted earlier, team members will produce field notes from their 
observations. Neither protected health information nor identifiers will be recorded. Field 
notes will be prepared within 24 hours of the site visit’s end. Field notes will not include 
names, but refer to persons by role. Practices will be referred to using an identifying number 
(practice 1, 2). When reporting data in publications, reports, or grants data will remain de-
identified. 
ii. Practice Member Staff. Interview data will be organized by participant code #. Interview 
recordings will be stored on the encrypted OHSU network. The interview will take about 60 
minutes, and with permission, the interviews will audio-recorded. The audio recordings will 
be professionally transcribed, checked for accuracy and de- identified. As with the 
observations, the study will return for follow-up interviews. 
For the Practice Survey, data will be collected and stored on the Oregon Clinical and 
Translation Research Institute (OCTRI) installation of REDCap, a highly secure and robust 
web-based research data collection and management system. Qualtrics, an online survey tool 
will be used for the Practice Member Questionnaire. Qualtrics is approved for use with 
OHSU-restricted information, including PHI. The study team included Qualtrics to 
anonymize survey data. Both surveys will be launched through the respective platforms, at 
baseline and follow-up. 
iii. Patients. Post-intervention (Year 2, quarter 3), the study team will conduct interviews with 
five patients, at each practice, that were exposed to the intervention to understand patients’ 
experiences with the practice changes that were implemented and the acceptability of those 
changes. Patient interviews will also consist of open-ended questions. All interviews will be 
audio-recorded and last approximately 60 minutes. The audio recordings will be 
professionally transcribed, checked for accuracy and de-identified. 
b. Sharing of Results with Subjects 
Findings will be shared with participants if requested, at completion of the study. Results will be 
aggregated and will not include identifying information of the clinic site, staff/providers, or patients. 
c. Data and Specimen Banking 
Data will not be used for future research planned at this time nor will specimens be collected. 

Page 9 of 26 
 
8. Data Analysis 
Rapid Qualitative Assessment. The qualitative team (Dr. Cohen, Ms. Bonsu, Ms. Baron) will create baseline 
summaries for each intervention practice that include summarized practice characteristics, capacity data (e.g., 
Adaptive Reserve, CPCQ, HIT capacity) and preliminary insights from observation and interviews related to 
the delivery of psychosocial and DMII care. The aim is to rapidly summarize strengths, weakness and 
opportunities for change to inform the facilitator’s work with practices. 
Hypothesis 1 Analyses—Assessing Feasibility and Acceptability and estimating cost of INTEGRATE-D. 
Analyses for Aim 1 will be mixed methods. To evaluate the hypothesis that INTEGRATE-D is acceptable 
and feasible, the study team will use standard descriptive statistics to summarize each intervention practice’s 
scores on the acceptability of intervention, intervention appropriateness and feasibility measures. Each 
measure includes four items, each on a 5-point Likert scale. The study team will create and summarize scales 
for each measure by averaging responses (where higher scores denote higher feasibility and acceptability) as 
is recommended.250 
The qualitative team will examine observation and interview data following the five-step process outlined by 
Miller and Crabtree276 (see Table 5, next page). The first step is to understand each practice as a separate 
case in its own context. To do so, the study team will analyze data as a group, reading field notes and listening 
to interviews from a practice to identify aspects of culture, team structure and operations, to identify how 
DMII care is delivered, how psychosocial care is integrated into DMII care process, and the factors 
influencing integration. The study team will compare pre- and post-intervention data to identify changes the 
practice made during INTEGRATE- D and why. Preliminary findings will be triangulated with what we learn 
in monthly phone calls with the key informants and survey data. The study team will also examine practice 
member experiences during the intervention (both good and bad), identify recommended improvements to the 
intervention, explore what tools and implementation strategies practices found most and least useful, and 
examine acceptability and feasibility. During this process, the qualitative team tags text with analytically 
relevant names (e.g., behavioral health screening) called “codes.” Once within- practice analyses are 
complete, we analyze data a second time to make comparisons across the practices; the study team will create 
matrices to display data across practices for comparative analyses as suggested by Miles and Huberman.276 
The study team will seek additional data sources to confirm/disconfirm and refute findings. 
Interview data will be use to assess the perceived costs / benefits of the INTEGRATE-D. These data will be 
analyzed and summarized using the procedures outlined above. Facilitator tracking data will be used to count 
the number of visits to each practice, the time spent during each visit (and between visits, if applicable) and 
the practice members who participated in QI meetings. The study team will use the average facilitator salary 
and estimates for practice member salaries (using national averages) to estimate the cost of the intervention. 
This will lay a foundation for a more thorough and systematic cost-effectiveness analysis planned for a 
subsequent R18. 
Hypotheses 2 and 3 Analyses—Assessing Change in Process of Care Measures. The primary goal of 
hypothesis 2 is to evaluate whether the intervention had a higher number of process care metrics for DMII 
and depression as compared to control practices. For this hypothesis, our unit of analysis is at the patient 
level. To compare the effect of INTEGRATE-D on process outcome measures in the context of a cluster 
randomized design, the study team will implement Generalized Linear Mixed 

Page 10 of 26 
 
Models (GLMMs).277 For our primary outcomes (which we treat as continuous variables), the study team will 
use a model that addresses the following design features of the study: (1) the study team will include a random 
intercept for practice to account for clustering of patients within practices; 
(2) the study team will perform an intent-to-treat analysis; (3) due to the pragmatic nature of the study, the 
patient sample will be an open cohort, with some patients remaining in the study for the entire duration and 
others joining the study when they come in during the study period. Because outcomes will be assessed 
multiple times, it may be possible that a patient shows up in both the pre and post period; thus, the study team 
will consider including random effects for the patient, if needed. Depending on the outcome type, the study 
team will take advantage of the flexibility of GLMM to model various forms of dependent variables, 
including the mixed effects versions of logistic regression (binary data), beta regression (percent data), 
Poisson regression (count data), and Gaussian regression (normally distributed data). For example, to assess 
changes in screening for PHQ-2 and PHQ-9, the study team will consider a logistic mixed effects model 
where we model the outcome of screened (yes versus no) as a function of an indicator for period (post vs. 
pre), indicator for intervention arm (intervention vs. control), and the interaction between period and 
intervention. If sample size allows and depending on imbalance between arms, we may consider adding a 
small set of unbalanced patient and practice-level confounders. 
For the larger pragmatic hybrid implementation-effectiveness trial, the study team will extend this model to 
consider moderation analyses. Because we hypothesize that implementation support directly helps improve 
practice capacity for QI, the study team will evaluate whether practice capacity measured using Adaptive 
Reserve and CPCQ moderates the relationship between implementation support and distal and process of care 
outcomes. To do this, the study team will consider moderation analyses (e.g., heterogeneity of implementation 
support over time by practice capacity) by including interaction terms between study arm, time, and practice 
capacity in the models above.278 
For hypothesis 3, which compares the change in QI strategies over time between intervention and control 
practices, our unit of analysis is at the practice level, as only one member of the practice answers these 
questions. Given the low sample size (four practices each providing two surveys, resulting in eight data 
points), the study team will describe the change from pre to post in both groups using descriptive means, 
standard deviations and ranges for the overall CPCQ score and for each individual item. For the larger 
pragmatic hybrid implementation-effectiveness trial, we would consider a linear mixed effects model where 
the dependent variable would be the score (range = - 28 to +28) and fixed effects of period (post vs. pre), 
intervention arm (intervention vs. control) and the interaction between period and intervention plus 
covariates. 
Hypothesis 4 Analysis—Assessing Trends in Outcome Measures. For hypothesis 4, the study team will utilize 
the GLMM modeling similar to what is proposed above. The study team will consider an analysis at both the 
patient and practice level. For patients that showed up in both pre and post periods, we are able to estimate 
their change in PHQ-9 and HbA1c values, and the study team will use linear mixed effect modeling to 
compare change across study groups. For practice-level analyses, the study team will take the average PHQ-9 
and HbA1c values at both pre- and post- intervention periods for both arms and compare them descriptively. 
For the large pragmatic trial, the study team will extend the linear mixed model to account for a longer 
follow-up time and the multiple observations per patient over that follow-up time. The study team will 
consider different functional forms of time (e.g., linear splines) to model pre-post changes.279 Multiple 
observations in 

Page 11 of 26 
 
such a longitudinal structure provides more rigor than just a pre-post analysis. The study team will also 
examine goodness of fit statistics and model fitting diagnostics to identify influential points and to evaluate 
alternative model specifications (e.g., Poisson mixed-effects models for PHQ-9). 
9. Privacy, Confidentiality, and Data Security 
Privacy. Among the study team are four members tasked with data management, LeAnn Michaels, ORPRN 
program manager, will oversee the REDCap project (i.e., database). Pamela Bonsu, project director, will build 
and manage the Practice Member Questionnaire on Qualtrics for data collection. Ms. Michaels and Ms. Bonsu 
will share the data with Rachel Springer, MS, a biostatistician, who will manage data from the chart audits 
and surveys for analysis. Andrea Baron, MPH, a qualitative analyst, who will manage the data from 
qualitative interviews. REDCap and Qualtrics offer controlled user access, which allows Ms. Michaels the 
ability to provide completely anonymized or deidentified data. OCTRI's REDCap software is housed on 
servers located in ITG's Advanced Computing Center providing locked physical security. Qualtrics’ servers 
are protected by high-end firewall systems audited using an industry standard (i.e., SSAE-18 method). 
Ms. Springer will remove PHI from chart audits and survey data. Additionally, she will perform data quality 
control checks and implement cleaning procedures. Data will be stored on a secure, password-protected 
network at OHSU. Ms. Baron will also remove PHI field notes from observations and interview data. All 
study team members are highly trained and appropriately certified in confidentiality, data security and the 
current protection measures represent the cutting edge of electronic protection. Ms. Baron will also store data 
on a secure, password-protected network at OHSU. 
Confidentiality. Many safeguards are in place to protect the ORPRN data. All data from the practice sites will 
be stored on secure servers at ORPRN. ORPRN staff will remove all personal identifiers from the charts and 
will assign each patient a unique identification number. ORPRN staff work with patient data from their 
member clinics on a regular basis and are appropriately trained in data security. Limited data will only be 
shared by ORPRN practice sites with OHSU once memorandums of understanding are fully executed. 
Analysts will maintain the analytical data file for a period of two years following the end of the study period 
to permit appropriate ongoing analysis. After two years from the study conclusion date, the files will either be 
destroyed, or will be used for ongoing research. In the latter instance, Dr. Cohen will file a new application 
for IRB approval. The protocol and procedures for using the data ensures no patients are individually 
identifiable during data transfers and analyses. We have previous experience in conducting these types of 
data transfers and data sharing between OHSU and ORPRN. All research staff are highly trained and 
appropriately certified in data security and confidentiality, and the current protection measures represent the 
cutting edge of electronic protection. 
Data Security. The proposed study includes two teams, quantitative and qualitative, managing outcome 
data. All of the qualitative data collected (i.e., site observations and interviews) will be securely managed by 
Ms. Andrea Baron under the direct supervision of Dr. Cohen. Ms. Baron will manage and coordinate the 
sending and receiving of qualitative study data using strict HIPAA compliant procedures. Under the 
direction of Dr. Miguel Marino, co-investigator, a biostatistician, Ms. Rachel Springer, will securely 
manage all chart audit and survey data from ORPRN practice sites. Chart audit and survey data will be 
securely collected and transferred to OHSU, and quality control checks will be performed. For chart audits, 
OHSU will work in collaboration with Ms. Julia 

Page 12 of 26 
 
Heinlein, ORPRN practice facilitator. Ms. Heinlein will participate in weekly meetings with the quantitative 
team to review data quality and to make decisions about included data. In addition, OHSU and ORPRN will 
follow a four-part data quality monitoring procedure consisting of using a standardized abstraction protocol, 
extensive training, monitoring of Kappa values between abstractors, and continuous quality improvement 
through feedback. For surveys, Ms. Springer will work with Ms. Michaels, ORPRN program manager, to 
securely access survey data from REDCap. The REDCap servers are housed behind both the OHSU firewall 
and a second ACC firewall. All transmissions of data from the application are encrypted over HTTPS with 
the industry standard TLS 1.1 protocol (AES 256-bit encryption). To maintain data integrity, REDCap is 
jointly managed in accordance with OHSU Information Security Directives by ACC staff and members of 
OCTRI's Biomedical Informatics Program, ensuring fidelity of database configuration and back-ups. User 
activities are logged to enable auditing of all data changes. Qualtrics services are hosted by trusted data 
centers, which are audited using the SSAE-18 method. Additionally, Qualtrics is ISO 27001 certified and is 
FedRAMP authorized, which is considered a gold standard of security certification used by U.S. federal 
government. Qualtrics also possesses a HITRUST certification for compliance with HIPAA. 
OHSU and ORPRN have both standard and Internet 2 connectivity through a link to Seattle's Pacific 
Northwest Gigapop. Its IP network is based on meshed switched gigabit ethernet with distributed routing, 
with 100 Mbps to the desktop. The network can be accessed via standard TCP/IP or Novell Netware. 
OHSU’s Information Technology Group (ITG) supports both Windows and MAC workstations for OHSU 
personnel. It also maintains web and application server clusters for application hosting and MS SQL Server 
and Oracle database servers for departmental use, including support for SSL communications and databases 
optimized for security of protected health information. Secure communications by electronic mail are also 
provided using Tumbleweed Secure messaging software. In addition, all data will be computerized and 
managed on OHSU and ORPRN HIPAA-complaint computers. All data will be stored and backed up 
password-protected secure servers. 
10. Risks and Benefits 
a. Risks to Subjects 
Primary Care Practices. The Practice Survey and the Practice Member Questionnaire descriptively 
illustrate the participating sites in the study. The surveys are not designed to capture PHI on 
practices’ patient population. For practices, risks associated with participation are minimal as the 
study seeks to measure quality improvement strategies employed by practice sites through surveys. 
The survey items are non-intrusive and poses minimal emotional, social, and legal harm to 
practices. All participating practices will sign a Memorandum of Understanding outlining 
expectations, responsibilities for participation, and the steps OHSU will take to mitigate risk during 
the pilot study. 
Patients. Patient outcomes are integral to interpretation of findings. Chart audits will be conducted 
such that protected health information (PHI) from patients is not collected and removed from the 
practice. This complies with federal HIPAA regulations regarding the transferring, storage, and 
reporting of PHI and ensures patient privacy and confidentiality. 
Five patients at each intervention practice (N=20) will be identified and asked to participate in an 
interview. Using the list of randomly selected patients who meet study criteria and 

Page 13 of 26 
 
screen positive for depression, a practice staff member will reach out to patients, using a script that 
we prepare and is approved by OHSU IRB. This script will guide the staff member to briefly explain 
the study and ask the patient if they are willing to be contacted (and have their contact information 
shared) with research staff for further details. Research staff will then contact patients, describe the 
study and provide more details. If the patient agrees to participate, a member of the research team 
will schedule an interview. Interviews can be conducted in person at the practice or at a convenient 
location (e.g., coffee shop, library study room) and / or can be conducted by telephone, if an in-
person interview is not possible. Patients will be mailed or emailed an Information Sheet approved by 
OHSU IRB prior to the interview and the interviewer will bring an Information Sheet to the 
interview. The interviewer will answer any of the patient’s questions about the study, the interview 
and the Information Sheet, and ask for permission to audio-record the interview. If granted, the 
interviewer will conduct and record the interview. If not, the interviewer will conduct the interview 
and take notes, or not conduct the interview at all, if the patient has changed his or her mind about 
participation. Participants will be instructed that the interview can stop and the recording can end at 
any time they indicate. The interviewer will ask about the patient’s experiences with the 
INTEGRATE-D intervention at his or her practice and these questions will pose very minimal risk to 
patients. Interview recordings are professionally transcribed and reviewed for accuracy by the study 
team. During that process, all information that could be used to identify the patient is removed from 
the transcript. 
Findings from these data, will only be communicated in aggregate and as high-level findings with 
practices. 
b. Potential Benefits to Subjects 
The minimal risk to subjects is offset by the anticipated benefits of this study. The participants in 
this study are practices and their members and there are several direct benefits to these participants. 
Behaviors health clinicians will receive cutting-edge training in the ADA recommendations from 
the experts who created that curriculum free of cost. Practices will then receive practice support to 
help make practice improvements. Practice sites will also receive tailored reports and education in 
how to use them. Practice will also receive expert consultation from leaders in integration. Practices 
receive this consultation for free. Participation in this study could improve practice capacity and 
care quality. In addition, the findings from INTEGRATE-D may be used to inform policies and 
practices regarding how to best implement the ADA recommendations for integrating medical and 
psychosocial are for patients with DMII, and will be used to inform a scale-up effort among primary 
care sites, other practices nationwide and the quality improvement organizations that support 
practice improvement in the US. 

Page 13 of 26 
 
REFEREN
CES 
 
1. 
Young-Hyman D, de Groot M, Hill-Briggs F, Gonzalez JS, Hood K, Peyrot M. Psychosocial Care for People 
With Diabetes: A Position Statement of the American Diabetes Association. Diabetes Care. 2016;39(12):2126-
2140. 
2. 
Kahn R, Anderson JE. Improving diabetes care: the model for health care reform. Diabetes Care. 
2009;32(6):1115-1118. 
3. 
Ducat L, Philipson LH, Anderson BJ. The Mental Health Comorbidities of Diabetes. JAMA. 2014;312(7):691-692. 
4. 
Katon W, Felz-Cornelis Cvd. Treatment of Depression in Patients with Diabetes: Efficacy, Effectiveness and 
Maintenance Trials, and New Service Models. In: Depression and Diabetes. John Wiley & Sons, Ltd; 2010:81-
107. 
5. 
Jones A, Vallis M, Pouwer F. If it does not significantly change HbA1c levels why should we waste time on it? A 
plea for the prioritization of psychological well-being in people with diabetes. Diabetic medicine : a journal of 
the British Diabetic Association. 2015;32(2):155-163. 
6. 
Gonzalez JS, Peyrot M, McCarl LA, et al. Depression and diabetes treatment nonadherence: a meta-analysis. 
Diabetes Care. 2008;31(12):2398-2403. 
7. 
Kilbourne AM, Reynolds CF, 3rd, Good CB, Sereika SM, Justice AC, Fine MJ. How does depression influence 
diabetes medication adherence in older patients? The American journal of geriatric psychiatry : official journal of 
the American Association for Geriatric Psychiatry. 2005;13(3):202-210. 
8. 
Egede LE, Ellis C, Grubaugh AL. The effect of depression on self-care behaviors and quality of care in a 
national sample of adults with diabetes. General hospital psychiatry. 2009;31(5):422-427. 
9. 
Gonzalez JS. Is the glass half full or half empty? Treatment of depression and HIV/AIDS medication adherence: 
a comment on Sin and DiMatteo. Annals of behavioral medicine : a publication of the Society of Behavioral 
Medicine. 2014;47(3):253-254. 
10. 
Lin EH, Katon W, Von Korff M, et al. Relationship of depression and diabetes self-care, medication 
adherence, and preventive care. Diabetes Care. 2004;27(9):2154-2160. 
11. 
Egede LE, Ellis C. The effects of depression on diabetes knowledge, diabetes self-management, and perceived 
control in indigent patients with type 2 diabetes. Diabetes technology & therapeutics. 2008;10(3):213-219. 
12. 
Katon W, Von Korff M, Ciechanowski P, et al. Behavioral and Clinical Factors Associated With Depression 
Among Individuals With Diabetes. Diabetes Care. 2004;27(4):914. 
13. 
Lustman PJ, Anderson RJ, Freedland KE, de Groot M, Carney RM, Clouse RE. Depression and poor 
glycemic control: a meta-analytic review of the literature. Diabetes Care. 2000;23(7):934. 
14. 
Ciechanowski PS, Katon WJ, Russo JE. Depression and diabetes: Impact of depressive symptoms on adherence, 
function, and costs. Archives of Internal Medicine. 2000;160(21):3278-3285. 
15. 
Park H, Hong Y, Lee H, Ha E, Sung Y. Individuals with type 2 diabetes and depressive symptoms exhibited 
lower adherence with self-care. Journal of clinical epidemiology. 2004;57(9):978-984. 
16. 
Holt RI, Katon WJ. Dialogue on Diabetes and Depression: Dealing with the double burden of co-morbidity. 
Journal of affective disorders. 2012;142 Suppl:S1-3. 
17. 
Gonzalez JS, Safren SA, Cagliero E, et al. Depression, self-care, and medication adherence in type 2 diabetes: 
relationships across the full range of symptom severity. Diabetes Care. 2007;30(9):2222-2227. 
18. 
Ali S, Stone M, Skinner TC, Robertson N, Davies M, Khunti K. The association between depression and 
health- related quality of life in people with type 2 diabetes: a systematic literature review. 
Diabetes/metabolism research and reviews. 2010;26(2):75-89. 
19. 
Ali S, Stone MA, Peters JL, Davies MJ, Khunti K. The prevalence of co-morbid depression in adults with Type 2 
diabetes: a systematic review and meta-analysis. Diabetic medicine : a journal of the British Diabetic Association. 
2006;23(11):1165-1173. 
20. 
Centers for Disease Control and Prevention. National Diabetes Statistics Report. Atlanta, GA: U.S. Dept of 
Health and Human Services;2017. 
21. 
Kaissi AA, Parchman M. Assessing Chronic Illness Care for Diabetes in Primary Care Clinics. The Joint 
Commission Journal on Quality and Patient Safety. 2006;32(6):318-323. 

Page 14 of 26 
 
22. 
Janes GR. Ambulatory medical care for diabetes. In: NDD G, ed. Diabetes in America. Vol 95. Bethesda: 
National Institutes of Health; 1995:541-552. 
23. 
Unutzer J, Park M. Strategies to improve the management of depression in primary care. Primary care. 
2012;39(2):415-431. 
24. 
Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, 
efficiency, and costs of medical care. Annals of Internal Medicine. 2006;144(10):742-752. 
25. 
Cohen DJ, Davis M.M., Hall J.D, Gilchrist E, B.F M. A Guidebook of Professional Practices for Behavioral 
Health and Primary Care Integration: Observations from Exemplary Sites. Rockville, MD: Agency for 
Healthcare Research and Quality;2015. 
26. 
Davis M, Balasubramanian BA, Waller E, Miller BF, Green LA, Cohen DJ. Re: integrating behavioral and 
physical health care in the real world: early lessons from advancing care together. Journal of the American 
Board of Family Medicine : JABFM. 2014;27(1):161. 
27. 
Davis M, Balasubramanian BA, Waller E, Miller BF, Green LA, Cohen DJ. Integrating behavioral and 
physical health care in the real world: early lessons from advancing care together. Journal of the American 
Board of Family Medicine : JABFM. 2013;26(5):588-602. 
28. 
Cifuentes M, Davis M, Fernald D, Gunn R, Dickinson P, Cohen DJ. Electronic Health Record Challenges, 
Workarounds, and Solutions Observed in Practices Integrating Behavioral Health and Primary Care. Journal 
of the American Board of Family Medicine : JABFM. 2015;28 Suppl 1:S63-72. 
29. 
Cohen DJ, Balasubramanian BA, Davis M, et al. Understanding Care Integration from the Ground Up: 
Five Organizing Constructs that Shape Integrated Practices. Journal of the American Board of Family 
Medicine : JABFM. 2015;28 Suppl 1:S7-20. 
30. 
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and 
innovations: a review of the empirical literature and recommendations for future research. Implementation 
science : IS. 2012;7(1):17. 
31. 
Mittman B. Factors That Influence the Scale Up and Spread of Innovations. 2016. 
32. 
McCannon CJ, Schall MW, Perla RJ. Planning for Scale: A Guide for Designing Large-Scale Improvement 
Initiatives. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 
2008. In. 
33. 
Simmons R and Shiffman J. Scaling up health service innovations: a framework for action. In: Simmons R, 
Fajans P, and Ghiron L, editors. Scaling up health service delivery: from pilot innovations to policies and 
programmes. Geneva: World Health Organization; 2007. p. 1-30. Available at 
http://whqlibdoc.who.int/publications/2007/9789241563512_eng.pdf?ua=1. In. 
34. 
Peek ME, Cargill A, Huang ES. Diabetes health disparities: a systematic review of health care interventions. 
Medical Care Research and Review. 2007;64(5 Suppl):101s-156s. 
35. 
Wilkes AE, Bordenave K, Vinci L, Peek ME. Addressing diabetes racial and ethnic disparities: lessons learned 
from quality improvement collaboratives. Diabetes management (London, England). 2011;1(6):653-660. 
36. 
Clark KD, Miller BF, Green LA, de Gruy FV, Davis M, Cohen DJ. Implementation of behavioral health 
interventions in real world scenarios: Managing complex change. Families, systems & health : the journal of 
collaborative family healthcare. 2017;35(1):36-45. 
37. 
Cohen DJ, Davis MM, Gunn R, et al. Integrating behavioral health and primary care: Consulting, coordinating and 
collaborating among Professionals. Journal of the American Board of Family Medicine : JABFM. 2015;September- 
October(Supplement 1):S21-S31. 
38. 
Gunn R, Davis MM, Hall J, et al. Designing Clinical Space for the Delivery of Integrated Behavioral Health 
and Primary Care. Journal of the American Board of Family Medicine : JABFM. 2015;28 Suppl 1:S52-62. 
39. 
Hall J, Cohen DJ, Davis M, et al. Preparing the Workforce for Behavioral Health and Primary Care Integration. 
Journal of the American Board of Family Medicine : JABFM. 2015;28(S1):S41-S51. 

Page 15 of 26 
 
40. 
Davis MM, Balasubramanian BA, Cifuentes M, et al. Clinician Staffing, Scheduling, and Engagement 
Strategies Among Primary Care Practices Delivering Integrated Care. Journal of the American Board of Family 
Medicine : JABFM. 2015;28 Suppl 1:S32-40. 
41. 
Zgibor JC, Solano FX, Siminerio L. A Primary Health-Care System Approach to Improving Quality of Care 
and Outcomes in People with Diabetes: The University of Pittsburgh Medical Center Experience. In: Simmons 
D, Wenzel H, Zgibor JC, eds. Integrated Diabetes Care: A Multidisciplinary Approach. Cham: Springer 
International Publishing; 2017:51-64. 
42. 
Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a 
review of systematic reviews. International journal of technology assessment in health care. 2005;21(3):380-385. 
43. 
Byrne JL, Davies MJ, Willaing I, et al. Deficiencies in postgraduate training for healthcare professionals who 
provide diabetes education and support: results from the Diabetes Attitudes, Wishes and Needs (DAWN2) study. 
Diabetic medicine : a journal of the British Diabetic Association. 2017;34(8):1074-1083. 
44. 
Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects 
on professional practice and health care outcomes. The Cochrane database of systematic reviews. 
2009(2):Cd003030. 
45. 
Beidas RS, Edmunds JM, Marcus SC, Kendall PC. Training and consultation to promote implementation of an 
empirically supported treatment: a randomized trial. Psychiatric services (Washington, DC). 2012;63(7):660-665. 
46. 
Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making 
feedback actionable. Implementation science : IS. 2006;1. 
47. 
Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare 
outcomes. The Cochrane database of systematic reviews. 2012;6:CD000259. 
48. 
Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on 
professional practice and health care outcomes. The Cochrane database of systematic reviews. 2006;2(2). 
49. 
Harvey G, Lynch E. Enabling Continuous Quality Improvement in Practice: The Role and Contribution 
of Facilitation. Frontiers in Public Health. 2017;5(27). 
50. 
Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC. Outcomes of a partnered facilitation 
strategy to implement primary care-mental health. Journal of general internal medicine. 2014;29 Suppl 4:904- 
912. 
51. 
Ritchie MJ. Implementation Facilitation Training Manual: Using Implementation Facilitation to Improve Care 
in the Veterans Health Administration. 2017. 
52. 
Ritchie MJ, Parker LE, Edlund CN, Kirchner JE. Using implementation facilitation to foster clinical practice 
quality and adherence to evidence in challenged settings: a qualitative study. BMC Health Services Research. 
2017;17(1):294. 
53. 
Roderick SS, Burdette N, Hurwitz D, Yeracaris P. Integrated behavioral health practice facilitation in patient 
centered medical homes: A promising application. Families, systems & health : the journal of collaborative 
family healthcare. 2017;35(2):227-237. 
54. 
Mader EM, Fox CH, Epling JW, et al. A Practice Facilitation and Academic Detailing Intervention Can Improve 
Cancer Screening Rates in Primary Care Safety Net Clinics. The Journal of the American Board of Family 
Medicine. 2016;29(5):533-542. 
55. 
Knox L, Taylor E, Geonnotti K, et al. Developing and Running a Primary Care Practice Facilitation 
Program: A How-to Guide (Prepared by Mathematica Policy Research under Contract No. 
HHSA290200900019I TO 5.). Rockville, MD: Agency for Healthcare Research and Quality;2011. 
56. 
Liddy C, Hogg W, Singh J, et al. A real-world stepped wedge cluster randomized trial of practice facilitation 
to improve cardiovascular care. Implement Sci. 2015;10:150. 
57. 
Avorn J. Academic detailing: "marketing" the best evidence to clinicians. JAMA. 2017;317(4). 
58. 
Fischer MA, Avorn J. Academic detailing can play a key role in assessing and implementing 
comparative effectiveness research findings. Health Affairs. 2012;31(10):2206-2212. 

Page 16 of 26 
 
59. 
Van Hoof TJ, Harrison LG, Miller NE, Pappas MS, Fischer MA. Characteristics of academic detailing: results 
of a literature review. American Health & Drug Benefits. 2015;8(8):414. 
60. 
Leone FT, Evers-Casey S, Graden S, Schnoll R, Mallya G. Academic Detailing Interventions Improve Tobacco 
Use Treatment among Physicians Working in Underserved Communities. Ann Am Thorac Soc. 2015;12(6):854-
858. 
61. 
Curry WJ, Lengerich EJ, Kluhsman BC. Academic detailing to increase colorectal cancer screening by primary 
care practices in Appalachian Pennsylvania. BMC Health Services Research. 2011;11. 
62. 
Powell BJ, Beidas RS, Lewis CC, et al. Methods to Improve the Selection and Tailoring of 
Implementation Strategies. The journal of behavioral health services & research. 2017;44(2):177-194. 
63. 
Blijleven V, Koelemeijer K, Wetzels M, Jaspers M. Workarounds Emerging From Electronic Health Record 
System Usage: Consequences for Patient Safety, Effectiveness of Care, and Efficiency of Care. JMIR Human 
Factors. 2017;4(4):e27. 
64. 
Aultman JM, Dean E. Beyond privacy: Benefits and burdens of e-health technologies in primary care. Journal 
of Clinical Ethics. 2014;25(1):50-64. 
65. 
George S, Garth B, Fish A, Baker R. Factors shaping effective utilization of health information technology in 
urban safety-net clinics. Health Informatics Journal. 2013;19(3):183-197. 
66. 
Baron RJ. Quality improvement with an electronic health record: achievable, but not automatic. Annals of 
Internal Medicine. 2007;147(8):549-552. 
67. 
Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported 
mental health interventions. Research on Social Work Practice. 2014;24(2):192-212. 
68. 
Waltz TJ, Powell BJ, Matthieu MM, et al. Use of concept mapping to characterize relationships 
among implementation strategies and assess their feasibility and importance: results from the Expert 
Recommendations for Implementing Change (ERIC) study. Implementation science : IS. 2015;10:109. 
69. 
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from 
the Expert Recommendations for Implementing Change (ERIC) project. Implementation science : IS. 
2015;10:21. 
70. 
Powell BJ, McMillen JC, Proctor EK, et al. A compilation of strategies for implementing clinical innovations 
in health and mental health. Medical Care Research and Review. 2012;69(2):123-157. 
71. 
Waltz TJ, Powell BJ, Chinman MJ, et al. Expert Recommendations for Implementing Change (ERIC): protocol 
for a mixed methods study. Implementation science : IS. 2014;9:39. 
72. 
Irwin R, Stokes T, Marshall T. Practice-level quality improvement interventions in primary care: a review 
of systematic reviews. Primary health care research & development. 2015;16(6):556-577. 
73. 
Lemelin J, Hogg W, Baskerville N. Evidence to action: a tailored multifaceted approach to changing 
family physician practice patterns and improving preventive care. Canadian Medical Association Journal. 
2001;164(6):757-763. 
74. 
Tennison J, Rajeev D, Woolsey S, Black J, Oostema SJ, North C. The utah beacon experience: integrating 
quality improvement, health information technology, and practice facilitation to improve diabetes outcomes in 
small health care facilities. EGEMS (Washington, DC). 2014;2(3):1100. 
75. 
Wensing M, Grol R. Single and combined strategies for implementing changes in primary care: a literature 
review. International Journal of Quality in Health Care. 1994;6(2):115-132. 
76. 
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Journal of 
general internal medicine. 2001;16(9):606-613. 
77. 
Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatric Annals. 
2002;32(9):509-515. 
78. 
Naranjo DM, Fisher L, Arean PA, Hessler D, Mullan J. Patients with type 2 diabetes at risk for major 
depressive disorder over time. Annals of family medicine. 2011;9(2):115-120. 
79. 
Lustman PJ, Griffith LS, Freedland KE, Kissel SS, Clouse RE. Cognitive behavior therapy for depression in 
type 2 diabetes mellitusA randomized, controlled trial. Annals of Internal Medicine. 1998;129(8):613-621. 
80. 
Williams JW, Jr, Katon W, Lin EB, et al. The effectiveness of depression care management on diabetes-
related outcomes in older patients. Annals of Internal Medicine. 2004;140(12):1015-1024. 

Page 17 of 26 
 
81. 
de Groot M, Kushnick M, Doyle T, et al. Depression Among Adults With Diabetes: Prevalence, Impact, and 
Treatment Options. Diabetes spectrum : a publication of the American Diabetes Association. 2010;23(1):15-18. 
82. 
Miller BF, Gilchrist, E., Ross, K., Wong, S., Green, L. Creating a Culture of Whole Health: Recommendations 
for Integrating Behavioral Health and Primary Care. Eugene S. Farley, Jr. Health Policy Center University of 
Colorado School of Medicine;2016. 
83. 
Holt RI. Undoing Descartes: integrating diabetes care for those with mental illness. Practical Diabetes. 
2011;28(6):270-275. 
84. 
Fisher L, Skaff M, Mullan J, Arean P, Glasgow R, Masharani U. A longitudinal study of affective and anxiety 
disorders, depressive affect and diabetes distress in adults with Type 2 diabetes. Diabetic medicine : a journal of 
the British Diabetic Association. 2008;25(9):1096-1101. 
85. 
Young-Hyman D, Peyrot M. Psychosocial care for people with diabetes. Arlington, VA: American 
Diabetes Association; 2012. 
86. 
Ismail K, Winkley K, Rabe-Hesketh S. Systematic review and meta-analysis of randomised controlled trials 
of psychological interventions to improve glycaemic control in patients with type 2 diabetes. Lancet 
(London, England). 2004;363(9421):1589-1597. 
87. 
Fisher L, Dickinson WP. Psychology and primary care: New collaborations for providing effective care for 
adults with chronic health conditions. The American psychologist. 2014;69(4):355-363. 
88. 
Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ. Collaborative Care for Depression: A Cumulative 
Meta- analysis and Review of Longer-term Outcomes. Archives of Internal Medicine. 2006;166(21):2314-
2321. 
89. 
Williams JW, Jr., Gerrity M, Holsinger T, Dobscha S, Gaynes B, Dietrich A. Systematic review of 
multifaceted interventions to improve depression care. General hospital psychiatry. 2007;29(2):91-116. 
90. 
Ell K, Katon W, Xie B, et al. Collaborative care management of major depression among low-income, 
predominantly Hispanic subjects with diabetes: a randomized controlled trial. Diabetes Care. 2010;33(4):706- 
713. 
91. 
Katon WJ, Russo JE, Von Korff M, Lin EH, Ludman E, Ciechanowski PS. Long-term effects on medical costs of 
improving depression outcomes in patients with depression and diabetes. Diabetes Care. 2008;31(6):1155-1159. 
92. 
deGruy FV. Treatment of depression in primary care. Annals of family medicine. 2015;13(1):3-5. 
93. 
Levey SM, Miller BF, Degruy FV, 3rd. Behavioral health integration: an essential element of population-
based healthcare redesign. Translational behavioral medicine. 2012;2(3):364-371. 
94. 
Peek ME, Ferguson M, Bergeron N, Maltby D, Chin MH. Integrated community-healthcare diabetes interventions 
to reduce disparities. Current diabetes reports. 2014;14(3):467. 
95. 
Chin MH, Goddu AP, Ferguson MJ, Peek ME. Expanding and sustaining integrated health care-community 
efforts to reduce diabetes disparities. Health promotion practice. 2014;15(2 Suppl):29s-39s. 
96. 
McGough PM, Bauer AM, Collins L, Dugdale DC. Integrating Behavioral Health into Primary Care. 
Population Health Management. 2016;19(2):81-87. 
97. 
Price-Haywood EG, Dunn-Lombard D, Harden-Barrios J, Lefante JJ. Collaborative Depression Care in a Safety 
Net Medical Home: Facilitators and Barriers to Quality Improvement. Population Health Management. 
2016;19(1):46-55. 
98. 
Phillips RL, Kaufman A, Mold JW, et al. The Primary Care Extension Program: A Catalyst for Change. The 
Annals of Family Medicine. 2013;11(2):173-178. 
99. 
Genevro J, Meyers D, Peikes D, Taylor EF. Creating Capacity for Improvement in Primary Care: The Case 
for Developing a Quality Improvement Infrastructure. In: Rockville, MD: Agency for Healthcare Research 
and Quality; 2013: http://www.ahrq.gov/professionals/prevention-chronic-care/improve/capacity- 
building/pcmhqi1.html. 
100. 
Peyrot M, Rubin RR, Lauritzen T, Snoek FJ, Matthews DR, Skovlund SE. Psychosocial problems and barriers 
to improved diabetes management: results of the Cross-National Diabetes Attitudes, Wishes and Needs 
(DAWN) Study. Diabetic medicine : a journal of the British Diabetic Association. 2005;22(10):1379-1385. 

Page 18 of 26 
 
101. 
Mold JW, Aspy CA, Nagykaldi Z, Oklahoma Physicians Resource/Research Network. Implementation of 
evidence- based preventive services delivery processes in primary care: an Oklahoma Physicians 
Resource/Research Network (OKPRN) study. Journal of the American Board of Family Medicine : JABFM. 
2008;21(4):334-344. 
102. 
Powell BJ, Beidas RS. Advancing Implementation Research and Practice in Behavioral Health Systems. 
Administration and Policy in Mental Health and Mental Health Services Research. 2016;43(6):825-833. 
103. 
Advisors PH. History. https://pha.purdue.edu/about/history/. Accessed. 
104. 
Centers NCAHE. What We Do: Practice Support. https://www.ncahec.net/practice-support/what-we-do/. 
Accessed. 
105. 
Organizations QI. Improving Behavioral Health for Older Adults. Healthy People, Healthy Communities Web 
site. https://healthinsight.org/quality-improvement. Accessed. 
106. 
Organizations QI. Practice Innovation Network. http://medicare.qualishealth.org/projects/practice-innovation- 
network. Accessed. 
107. 
Organizations QI. Quality Improvement Initiatives and Other Technical Assistance 
Projects. https://healthcarefornewengland.org/initiatives/qii-ta/. Accessed. 
108. 
Organizations QI. Celebrating Five Years of QIO Program Success. 
https://qioprogram.org/qionews/articles/celebrating-five-years-qio-program-success. Published 2019. Accessed. 
109. 
Woodson TT, Gunn R, Clark KD, et al. Designing health information technology tools for behavioural 
health clinicians integrated within US-based primary care teams. Journal of Innovation in Health 
Informatics. 2018;25(3):158-168. 
110. 
Jetelina KK, Woodson TT, Gunn R, et al. Evaluation of an Electronic Health Record (EHR) Tool for 
Integrated Behavioral Health in Primary Care. J Am Board Fam Med. 2018;31(5):712-723. 
111. 
Jaen CR, Crabtree BF, Palmer RF, et al. Methods for evaluating practice change toward a patient-centered 
medical home. Annals of family medicine. 2010;8 Suppl 1:S9-20; S92. 
112. 
Miller WL, Crabtree BF, Nutting PA, Stange KC, Jaen CR. Primary care practice development: a 
relationship- centered approach. Annals of family medicine. 2010;8 Suppl 1:S68-79; s92. 
113. 
Balasubramanian BA, Cohen DJ, Davis MM. Learning evaluation: blending quality improvement and 
implementation research methods to study healthcare innovations. Implementation science : IS. 2015;10. 
114. 
Balasubramanian BA, Cohen DJ, Jetelina KK, et al. Outcomes of Integrated Behavioral Health with Primary Care. 
Journal of the American Board of Family Medicine : JABFM. 2017;30(2):130-139. 
115. 
Davis MM, Gunn R, Gowen LK, Miller BF, Green LA, Cohen DJ. A Qualitative Study of Patient Experience 
of Integrated Care in Behavioral Health and Primary Care Settings: More Similar than Different. 
Translational behavioral medicine. 2018. 
116. 
Gunn R, Davis MM, Hall J, et al. Designing Clinical Space for the Delivery of Integrated Behavioral Health 
and Primary Care. J Am Board Fam Med. 2015;28 Suppl 1:S52-62. 
117. 
Wallace NT, Cohen DJ, Gunn R, et al. Start-Up and Ongoing Practice Expenses of Behavioral Health and Primary 
Care Integration Interventions in the Advancing Care Together (ACT) Program. Journal of the American Board 
of Family Medicine : JABFM. 2015;28 Suppl 1:S86-97. 
118. 
Davis S, Collins KS, Hall A. Community health centers in a changing US health care system. Policy 
brief (Commonwealth Fund). 1999(300):1. 
119. 
Druss BG, Zhao L, Cummings JR, Shim RS, Rust GS, Marcus SC. Mental comorbidity and quality of diabetes 
care under Medicaid: a 50-state analysis. Medical care. 2012;50(5):428-433. 
120. 
Katon W, Schulberg H. Epidemiology of depression in primary care. General hospital psychiatry. 
1992;14(4):237- 247. 
121. 
Regier DA, Narrow WE, Rae DS, Manderscheid RW, Locke BZ, Goodwin FK. The de facto US mental and 
addictive disorders service system. Epidemiologic catchment area prospective 1-year prevalence rates of 
disorders and services. Archives of general psychiatry. 1993;50(2):85-94. 
122. 
Petterson SM, Phillips RL, Jr., Bazemore AW, Dodoo MS, Zhang X, Green LA. Why there must be room for 
mental health in the medical home. American family physician. 2008;77(6):757. 

Page 19 of 26 
 
123. 
Loeppke R, Taitel M, Haufle V, Parry T, Kessler RC, Jinnett K. Health and productivity as a business 
strategy: a multiemployer study. Journal of occupational and environmental medicine. 2009;51(4):411-428. 
124. 
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: 
combining elements of clinical effectiveness and implementation research to enhance public health impact. 
Medical care. 2012;50(3):217-226. 
125. 
Gold R, Bunce AE, Cohen DJ, et al. Reporting on the Strategies Needed to Implement Proven Interventions: An 
Example From a "Real-World" Cross-Setting Implementation Study. Mayo Clinic proceedings. 
2016;91(8):1074- 1083. 
126. 
Friedman A, Howard J, Shaw EK, Cohen DJ, Shahidi L, Ferrante JM. Facilitators and Barriers to Care 
Coordination in Patient-centered Medical Homes (PCMHs) from Coordinators' Perspectives. Journal of the 
American Board of Family Medicine : JABFM. 2016;29(1):90-101. 
127. 
DeVoe JE, Huguet N, Likumahuwa-Ackman S, et al. Testing health information technology tools to 
facilitate health insurance support: a protocol for an effectiveness-implementation hybrid randomized trial. 
Implementation science : IS. 2015;10:123. 
128. 
Balasubramanian BA, Fernald D, Dickinson LM, et al. REACH of Interventions Integrating Primary Care 
and Behavioral Health. Journal of the American Board of Family Medicine : JABFM. 2015;28 Suppl 
1:S73-85. 
129. 
Chase DA, Ash JS, Cohen DJ, Hall J, Olson GM, Dorr DA. The EHR's roles in collaboration between 
providers: A qualitative study. AMIA Annual Symposium proceedings AMIA Symposium. 2014;2014:1718-
1727. 
130. 
Peek CJ, Cohen DJ, deGruy FV, 3rd. Research and evaluation in the transformation of primary care. The 
American psychologist. 2014;69(4):430-442. 
131. 
Crosson JC, Ohman-Strickland PA, Cohen DJ, Clark EC, Crabtree BF. Typical electronic health record 
use in primary care practices and the quality of diabetes care. Annals of family medicine. 2012;10(3):221-
227. 
132. 
Cohen DJ, Balasubramanian BA, Isaacson NF, Clark EC, Etz RS, Crabtree BF. Coordination of health 
behavior counseling in primary care. Annals of family medicine. 2011;9(5):406-415. 
133. 
Balasubramanian BA, Chase SM, Nutting PA, et al. Using Learning Teams for Reflective Adaptation 
(ULTRA): insights from a team-based change management strategy in primary care. Ann Fam Med. 
2010;8(5):425-432. 
134. 
Cohen DJ, Crabtree BF, Etz RS, et al. Fidelity versus flexibility: translating evidence-based research into practice. 
American journal of preventive medicine. 2008;35(5 Suppl):S381-389. 
135. 
Balasubramanian BA, Cohen DJ, Dodoo MS, Bazemore AW, Green LA. Behavioral change counseling in 
the medical home. American family physician. 2007;76(10):1472. 
136. 
Cohen DJ, Tallia AF, Crabtree BF, Young DM. Implementing health behavior change in primary care: lessons 
from prescription for health. Annals of family medicine. 2005;3 Suppl 2:S12-19. 
137. 
Cohen D, McDaniel RR, Jr., Crabtree BF, et al. A practice change model for quality improvement in primary care 
practice. Journal of healthcare management / American College of Healthcare Executives. 2004;49(3):155-168; 
discussion 169-170. 
138. 
Gold R, Cottrell E, Bunce A, et al. Developing Electronic Health Record (EHR) Strategies Related to Health 
Center Patients' Social Determinants of Health. Journal of the American Board of Family Medicine : JABFM. 
2017;30(4):428-447. 
139. 
Cohen DJ, Balasubramanian BA, Gordon L, et al. A national evaluation of a dissemination and implementation 
initiative to enhance primary care practice capacity and improve cardiovascular disease care: the 
ESCALATES study protocol. Implementation science : IS. 2016;11(1):86. 
140. 
Balasubramanian B, Ezekiel D, Sweeney S, et al. Measuring the Dose of External Practice Facilitation. 11th 
Annual Conference on the Science of Dissemination and Implementation in Health; December, 2018, 2018; 
Washington D.C. 
141. 
Balasubramanian B, Valenzuela S, Marino M, et al. (Dis)agreement between practice members in practices’ 
capability for implementing change. North American Primary Care Research Group meeting; November, 2018; 
Chicago, IL. 

Page 20 of 26 
 
142. 
Cohen D, Sweeney S, Miller W, et al. Implementing improved cardiovascular preventive care in primary care 
practices. 11th Annual Conference on the Science of Dissemination and Implementation in Health; December, 
2018, 2018; Washington D.C. 
143. 
Cohen DJ, Dorr DA, Knierim K, et al. Primary Care Practices’ Abilities And Challenges In Using Electronic 
Health Record Data For Quality Improvement. Health Affairs. 2018;37(4):635-643. 
144. 
Ono S, Tuzzio L, Sweeney S, et al. Contextual Factors Influence the Scale and Spread of EvidenceNOW. 
Academy Health Annual Research Meeting; June, 2018; Seattle, WA. 
145. 
Holman GT, Waldren SE, Beasley JW, et al. Meaningful use’s benefits and burdens for US family physicians. 
Journal of the American Medical Informatics Association : JAMIA. 2018:ocx158-ocx158. 
146. 
Hemler JR, Hall JD, Cholan RA, et al. Practice Facilitator Strategies for Addressing Electronic Health Record 
Data Challenges for Quality Improvement: EvidenceNOW. J Am Board Fam Med. 2018;31(3):398-409. 
147. 
Dorr DA, Cohen DJ, Adler-Milstein J. Data-Driven Diffusion Of Innovations: Successes And Challenges In 3 
Large- Scale Innovative Delivery Models. Health Aff (Millwood). 2018;37(2):257-265. 
148. 
Muench J, Levy S, Rdesinski R, Schiefer R, Gilbert K, Fleishman J. Personal characteristics associated with the 
effect of childhood trauma on health. The International Journal of Psychiatry in Medicine. 2018;53(5-6):384-394. 
149. 
Gold R, Nelson C, Cowburn S, et al. Feasibility and impact of implementing a private care system’s diabetes 
quality improvement intervention in the safety net: a cluster-randomized trial. Implementation science : IS. 
2015;10(1):83. 
150. 
Gold R, Muench J, Hill C, et al. Collaborative development of a randomized study to adapt a diabetes quality 
improvement initiative for federally qualified health centers. Journal of health care for the poor and 
underserved. 2012;23(3 Suppl):236-246. 
151. 
Muench J, Blenning C, Judkins DZ, Vincent C. Clinical inquiries. Angiotensin blockade for diabetes: 
monitor microalbuminuria? J Fam Pract. 2007;56(2):145-146. 
152. 
Elder NC, Muench J. Diabetes care as public health. Journal of Family Practice. 2000;49(6):513-513. 
153. 
Muench J, Carey M. Diabetes mellitus associated with atypical antipsychotic medications: new case report and 
review of the literature. The Journal of the American Board of Family Practice. 2001;14(4):278-282. 
154. 
Romain AM, Muench J, Phillips JP. Preparing family physicians for the care of patients with severe and 
persistent mental illness: Examples from two U.S. residency programs. International journal of psychiatry in 
medicine. 2015;50(1):25-35. 
155. 
Muench J, Hamer AM. Adverse effects of antipsychotic medications. Am Fam Physician. 2010;81(5):617-622. 
156. 
Cantone RE, Fleishman J, Garvey B, Gideonse N. Interdisciplinary Management of Opioid Use Disorder in 
Primary Care. The Annals of Family Medicine. 2018;16(1):83. 
157. 
Boone J, Elder N, Short A, et al. (266) Minimal versus intensive interprofessional quality improvement for 
chronic pain management in primary care. The Journal of Pain. 2016;17(4):S42. 
158. 
Elder N, Penm M, Pallerla H, et al. Provision of Recommended Chronic Pain Assessment and Management in 
Primary Care: Does Patient-Centered Medical Home (PCMH) Recognition Make a Difference? J Am Board 
Fam Med. 2016;29(4):474-481. 
159. 
Winders WT, Ariizumi R, Hart K, et al. Hypertensive ED patients: Missed opportunities for 
addressing hypertension and facilitating outpatient follow-up. The American journal of emergency 
medicine. 2018;36(12):2268-2275. 
160. 
Angier H, Gold R, Gallia C, et al. Variation in outcomes of quality measurement by data source. Pediatrics. 
2014;133(6):e1676-1682. 
161. 
Heintzman J, Marino M, Hoopes M, et al. Supporting health insurance expansion: do electronic health records 
have valid insurance verification and enrollment data? Journal of the American Medical Informatics Association 
: JAMIA. 2015;22(4):909-913. 
162. 
Huguet N, Angier H, Marino M, et al. Protocol for the analysis of a natural experiment on the impact of the 
Affordable Care Act on diabetes care in community health centers. Implementation science : IS. 2017;12(1):14. 

Page 21 of 26 
 
163. 
Marino M, Killerby M, Lee S, et al. The Effects of a Cluster Randomized Controlled Workplace Intervention 
on Sleep and Work-Family Conflict Outcomes in an Extended Care Setting. Sleep health. 2016;2(4):297-308. 
164. 
DeVoe JE, Hoopes M, Nelson CA, et al. Electronic health record tools to assist with children's insurance 
coverage: a mixed methods study. BMC Health Serv Res. 2018;18(1):354. 
165. 
Cohen DJ, Dorr DA, Knierim K, et al. Primary Care Practices' Abilities And Challenges In Using Electronic 
Health Record Data For Quality Improvement. Health Aff (Millwood). 2018;37(4):635-643. 
166. 
Olson R, Thompson SV, Elliot DL, et al. Safety and Health Support for Home Care Workers: The 
COMPASS Randomized Controlled Trial. American journal of public health. 2016;106(10):1823-1832. 
167. 
Melnick AL, Rdesinski RE, Marino M, et al. Randomized Controlled Trial of Home-Based Hormonal 
Contraceptive Dispensing for Women At Risk of Unintended Pregnancy. Perspectives on sexual and 
reproductive health. 2016;48(2):93-99. 
168. 
Balasubramanian BA, Marino M, Cohen DJ, et al. Use of quality improvement strategies among small to 
medium- size us primary care practices. Annals of family medicine. 2018;16:S35-S43. 
169. 
Bailey SR, Heintzman JD, Marino M, et al. Measuring Preventive Care Delivery: Comparing Rates Across 
Three Data Sources. Am J Prev Med. 2016;51(5):752-761. 
170. 
Heintzman J, Bailey SR, Hoopes MJ, et al. Agreement of Medicaid claims and electronic health records for 
assessing preventive care quality among adults. Journal of the American Medical Informatics Association : 
JAMIA. 2014;21(4):720-724. 
171. 
Fisher L, Polonsky WH, Hessler D, Potter MB. A practical framework for encouraging and supporting 
positive behaviour change in diabetes. Diabetic medicine : a journal of the British Diabetic Association. 
2017. 
172. 
Sandberg J, Yorgason JB, Fisher L, et al. Does Length of Relationship or Gender Predict Response to 
Behavioral Diabetes Intervention? The Diabetes educator. 2017;43(2):216-222. 
173. 
Fisher L, Hessler D, Polonsky W, Strycker L, Masharani U, Peters A. Diabetes distress in adults with 
type 1 diabetes: Prevalence, incidence and change over time. Journal of Diabetes and its Complications. 
2016;30(6):1123-1128. 
174. 
Fisher L, Polonsky WH, Hessler DM, et al. Understanding the sources of diabetes distress in adults with type 
1 diabetes. Journal of Diabetes and Its Complications. 2015;29(4):572-577. 
175. 
Fisher L, Gonzalez JS, Polonsky WH. The confusing tale of depression and distress in patients with diabetes: a 
call for greater clarity and precision. Deiabetic Medicine. 2014;31(7):764-772. 
176. 
Fisher L, Hessler D, Masharani U, Strycker L. Impact of baseline patient characteristics on interventions to 
reduce diabetes distress: the role of personal conscientiousness and diabetes self-efficacy. Diabetic medicine : a 
journal of the British Diabetic Association. 2014;31(6):739-746. 
177. 
Fisher L, Hessler DM, Polonsky WH, Mullan J. When is diabetes distress clinically meaningful?: establishing 
cut points for the Diabetes Distress Scale. Diabetes Care. 2012;35(2):259-264. 
178. 
Hessler DM, Fisher L, Mullan JT, Glasgow RE, Masharani U. Patient age: a neglected factor when 
considering disease management in adults with type 2 diabetes. Patient education and counseling. 
2011;85(2):154-159. 
179. 
Fisher L, Mullan JT, Skaff MM, Glasgow RE, Arean P, Hessler D. Predicting diabetes distress in patients with 
Type 2 diabetes: a longitudinal study. Diabetic medicine : a journal of the British Diabetic Association. 
2009;26(6):622- 627. 
180. 
Fisher L, Hessler D, Glasgow RE, et al. REDEEM: a pragmatic trial to reduce diabetes distress. Diabetes Care. 
2013;36(9):2551-2558. 
181. 
Fisher L, Hessler D, Polonsky W, et al. Emotion regulation contributes to the development of diabetes distress 
among adults with type 1 diabetes. Patient education and counseling. 2017. 
182. 
Fisher L, Hessler DM, Polonsky WH, et al. Prevalence of depression in Type 1 diabetes and the problem of 
over- diagnosis. Diabetic medicine : a journal of the British Diabetic Association. 2016;33(11):1590-1597. 
183. 
Hessler D, Fisher L, Strycker LA, Arean PA, Bowyer V. Causal and bidirectional linkages over time between 
depression and diabetes regimen distress in adults with type 2 diabetes. Diabetes research and clinical practice. 
2015;108(2):360-366. 

Page 22 of 26 
 
184. 
Hessler DM, Fisher L, Naranjo D, Masharani U. Young adult African American patients with type 2 diabetes: 
a high risk patient sub-group with few supports for good diabetes management. Journal of health psychology. 
2012;17(4):535-544. 
185. 
Hilliard ME, Yi-Frazier JP, Hessler D, Butler AM, Anderson BJ, Jaser S. Stress and A1c Among People with 
Diabetes Across the Lifespan. Current Diabetes Report. 2016;16(8):67. 
186. 
Huang B, Willard-Grace R, De Vore D, et al. Health coaching to improve self-management and quality of life 
for low income patients with chronic obstructive pulmonary disease (COPD): protocol for a randomized 
controlled trial. BMC Pulmonary Medicine. 2017;17(1):90. 
187. 
Fisher L, Hessler D, Polonsky W, et al. Emotion regulation contributes to the development of diabetes distress 
among adults with type 1 diabetes. Patient Educ Couns. 2018;101(1):124-131. 
188. 
Fisher L, Hessler D, Polonsky WH, et al. T1-REDEEM: A Randomized Controlled Trial to Reduce Diabetes 
Distress Among Adults With Type 1 Diabetes. Diabetes Care. 2018;41(9):1862-1869. 
189. 
Dickinson WP, Dickinson LM, Jortberg BT, Hessler DM, Fernald DH, Fisher L. A protocol for a cluster 
randomized trial comparing strategies for translating self-management support into primary care practices. BMC 
Fam Pract. 2018;19(1):126. 
190. 
Dickinson WP, Dickinson LM, Jortberg BT, et al. A Cluster Randomized Trial Comparing Strategies for 
Translating Self-Management Support into Primary Care Practices. Journal of the American Board of Family 
Medicine : JABFM. 2019;32(3):341-352. 
191. 
Naranjo D, Tanenbaum ML, Iturralde E, Hood KK. Diabetes Technology: Uptake, Outcomes, Barriers, and 
the Intersection With Distress. Journal of diabetes science and technology. 2016;10(4):852-858. 
192. 
Naranjo D, Schwartz DD, Delamater AM. Diabetes in ethnically diverse youth: disparate burden and intervention 
approaches. Current diabetes reviews. 2015;11(4):251-260. 
193. 
Naranjo D, Mulvaney S, McGrath M, Garnero T, Hood K. Predictors of self-management in pediatric type 1 
diabetes: individual, family, systemic, and technologic influences. Current diabetes reports. 2014;14(11):544. 
194. 
Tanenbaum ML, Iturralde E, Hanes SJ, et al. Trust in hybrid closed loop among people with diabetes: 
Perspectives of experienced system users. Journal of health psychology. 2017:1359105317718615. 
195. 
Butler AM, Weller BE, Yi-Frazier JP, et al. Diabetes-Specific and General Life Stress and Glycemic 
Outcomes in Emerging Adults With Type 1 Diabetes: Is Race/Ethnicity a Moderator? Journal of pediatric 
psychology. 2017. 
196. 
Hilliard ME, Powell PW, Anderson BJ. Evidence-based behavioral interventions to promote diabetes 
management in children, adolescents, and families. The American psychologist. 2016;71(7):590-601. 
197. 
Hilliard ME, De Wit M, Wasserman RM, et al. Screening and support for emotional burdens of youth with type 
1 diabetes: Strategies for diabetes care providers. Pediatric diabetes. 2017. 
198. 
Hilliard ME, Herzer M, Dolan LM, Hood KK. Psychological screening in adolescents with type 1 diabetes 
predicts outcomes one year later. Diabetes research and clinical practice. 2011;94(1):39-44. 
199. 
Stone MA, Gill PS. Screening for Depression in People with Diabetes in Primary Care. In: Lloyd CE, Pouwer 
F, Hermanns N, eds. Screening for Depression and Other Psychological Problems in Diabetes: A Practical 
Guide. London: Springer London; 2013:161-180. 
200. 
Wasserman RM, Hilliard ME, Schwartz DD, Anderson BJ. Practical strategies to enhance executive functioning 
and strengthen diabetes management across the lifespan. Current Diabetes Report. 2015;15(8):52. 
201. 
Balasubramanian BA, Heurtin-Roberts S, Krasny S, et al. Factors Related to Implementation and Reach of a 
Pragmatic Multisite Trial: The My Own Health Report (MOHR) Study. Journal of the American Board of 
Family Medicine : JABFM. 2017;30(3):337-349. 
202. 
Lee SJ, Clark MA, Cox JV, Needles BM, Seigel C, Balasubramanian BA. Achieving Coordinated Care for 
Patients With Complex Cases of Cancer: A Multiteam System Approach. Journal of oncology practice. 
2016;12(11):1029- 1038. 
203. 
Krist AH, Glasgow RE, Heurtin-Roberts S, et al. The impact of behavioral and mental health risk assessments 
on goal setting in primary care. Translational behavioral medicine. 2016;6(2):212-219. 

Page 23 of 26 
 
204. 
Krist AH, Phillips SM, Sabo RT, et al. Adoption, reach, implementation, and maintenance of a behavioral 
and mental health assessment in primary care. Annals of family medicine. 2014;12(6):525-533. 
205. 
Prezio EA, Balasubramanian BA, Shuval K, Cheng D, Kendzor DE, Culica D. Evaluation of quality improvement 
performance in the Community Diabetes Education (CoDE) program for uninsured Mexican Americans: results of 
a randomized controlled trial. American journal of medical quality : the official journal of the American College 
of Medical Quality. 2014;29(2):124-134. 
206. 
Krist AH, Glenn BA, Glasgow RE, et al. Designing a valid randomized pragmatic primary care 
implementation trial: the my own health report (MOHR) project. Implementation science : IS. 2013;8:73. 
207. 
Prezio EA, Cheng D, Balasubramanian BA, Shuval K, Kendzor DE, Culica D. Community Diabetes Education 
(CoDE) for uninsured Mexican Americans: a randomized controlled trial of a culturally tailored diabetes 
education and management program led by a community health worker. Diabetes research and clinical practice. 
2013;100(1):19-28. 
208. 
Fernald DH, Dickinson LM, Froshaug DB, et al. Improving multiple health risk behaviors in primary care: lessons 
from the Prescription for Health Common Measures, Better Outcomes (COMBO) study. Journal of the American 
Board of Family Medicine : JABFM. 2012;25(5):701-711. 
209. 
Ohman-Strickland PA, Orzano AJ, Hudson SV, et al. Quality of diabetes care in family medicine 
practices: influence of nurse-practitioners and physician's assistants. Annals of family medicine. 
2008;6(1):14-22. 
210. 
Orzano AJ, Strickland PO, Tallia AF, et al. Improving outcomes for high-risk diabetics using information systems. 
Journal of the American Board of Family Medicine : JABFM. 2007;20(3):245-251. 
211. 
Jetelina KK, Reingle Gonzalez JM, Vaeth PA, Mills BA, Caetano R. An Investigation of the Relationship 
Between Alcohol Use and Major Depressive Disorder Across Hispanic National Groups. Alcoholism, clinical 
and experimental research. 2016;40(3):536-542. 
212. 
Woodson TT, Gunn R, Clark KD, et al. Designing health information technology tools for behavioral 
health clinicians integrated within a primary care team. J Innov Health Inform. 2018;25(3):158-168. 
213. 
Crabtree BF, Miller WL, Stange KC. Understanding practice from the ground up. The Journal of Family Practice. 
2001;50(10):881-887. 
214. 
Peek C, National Integration Academy Council. Lexicon for behavioral health and primary care integration: 
Concepts and definitions developed by expert consensus. In. Vol 2015. Rockville, MD: Agency for 
Healthcare Research and Quality; 2013. 
215. 
Ono SS, Crabtree BF, Hemler JR, et al. Taking Innovation To Scale In Primary Care Practices: The 
Functions Of Health Care Extension. Health Aff (Millwood). 2018;37(2):222-230. 
216. 
Perry CK, Damschroder LJ, Hemler JR, Woodson TT, Ono SS, Cohen DJ. Specifying and comparing 
implementation strategies across seven large implementation interventions: a practical application of theory. 
Implement Sci. 2019;14(1):32. 
217. 
Hung DY, Rundall TG, Crabtree BF, Tallia AF, Cohen DJ, Halpin HA. Influence of primary care practice and 
provider attributes on preventive service delivery. American journal of preventive medicine. 2006;30(5):413-422. 
218. 
Solberg LI, Asche SE, Margolis KL, Whitebird RR. Measuring an organization's ability to manage change: the 
change process capability questionnaire and its use for improving depression care. American journal of medical 
quality : the official journal of the American College of Medical Quality. 2008;23(3):193-200. 
219. 
Cohen DJ, Dorr DA, Knierim K, et al. Primary care practices' abilities and challenges in using electronic 
health record data for quality improvement. Health Affairs. 2018;37(4):635-643. 
220. 
Goeschel C, Pronovost P. Harnessing the potential of health care collaboratives: lessons from the Keystone ICU 
Project. In: Henriksen K, Battles J, Keyes M, Grady M, eds. Advances in patient safety: new directions and 
alternative approaches. Vol. 2, culture and redesign. Rockville: Agency for Healthcare Research and Quality; 
2008. 
221. 
Mold JW. A cooperative extension service for primary care in Oklahoma. J Okla State Med Assoc. 
2011;104(11- 12):414-418. 

Page 24 of 26 
 
222. 
Dorr DA, Anastas T, Ramsey K, et al. Effect of a Pragmatic, Cluster-randomized Controlled Trial on Patient 
Experience With Care: The Transforming Outcomes for Patients Through Medical Home Evaluation and 
reDesign (TOPMED) Study. Med Care. 2016;54(8):745-751. 
223. 
Nease D, Daly J, Dickinson LM, et al. Impact of a Boot Camp Translation Intervention on Self-
Management Support in Primary Care. Vol 52018. 
224. 
McConnell KJ, Renfro S, Chan BK, et al. Early Performance in Medicaid Accountable Care 
Organizations: A Comparison of Oregon and Colorado. JAMA internal medicine. 2017;177(4):538-545. 
225. 
Chang AM, Cohen DJ, McCarty D, Rieckmann T, McConnell KJ. Oregon's medicaid transformation -- 
observations on organizational structure and strategy. Journal of health politics, policy and law. 2015;40(1):257-
264. 
226. 
Cohen DJ, Hall JD, Reece DA, Schwarz E. Chapter 7 - The Path to Integrating Medical, Behavioral, and Oral 
Health Care: Oregon’s Experience With Change. In: Stock R, Goldberg B, eds. Health Reform Policy to Practice. 
Academic Press; 2017:101-119. 
227. 
Kroening-Roche J, Hall JD, Cameron DC, Rowland R, Cohen DJ. Integrating behavioral health under an ACO 
global budget: barriers and progress in Oregon. The American journal of managed care. 2017;23(9):e303-e309. 
228. 
McConnell KJ, Renfro S, Lindrooth RC, Cohen DJ, Wallace NT, Chernew ME. Oregon's Medicaid Reform 
And Transition To Global Budgets Were Associated With Reductions In Expenditures. Health Aff 
(Millwood). 2017;36(3):451-459. 
229. 
Stock R, Hall J, Chang AM, Cohen D. Physicians' early perspectives on Oregon's Coordinated Care Organizations. 
Healthcare (Amsterdam, Netherlands). 2016;4(2):92-97. 
230. 
Cohen D. Addressing behavioral health integration with payment reform. In. Health Affairs Blog2015. 
231. 
New York City Department of Health and Mental Hygiene. ABCS Toolkit for the practice facilitator. In:2016. 
232. 
Agency for Healthcare Research and Quality. EvidenceNOW tools for change for practice 
facilitators. https://www.ahrq.gov/evidencenow/tools/facilitation/index.html. Published 2018. 
Accessed. 
233. 
Bodenheimer T, Ghorob A, Willard-Grace R, Grumbach K. The 10 Building Blocks of High-Performing 
Primary Care. The Annals of Family Medicine. 2014;12(2):166-171. 
234. 
Margolis PA, DeWalt DA, Simon JE, et al. Designing a large-scale multilevel improvement initiative: the 
improving performance in practice program. The Journal of continuing education in the health professions. 
2010;30(3):187- 196. 
235. 
Halladay JR, DeWalt DA, Wise A, et al. More extensive implementation of the chronic care model is associated 
with better lipid control in diabetes. Journal of the American Board of Family Medicine : JABFM. 
2014;27(1):34- 41. 
236. 
Quality AfHRa. The EvidenceNOW Key Driver Diagram. 
http://www.ahrq.gov/evidencenow/tools/keydrivers/index.html. Published 2019. Accessed. 
237. 
Crabtree BF, Miller WL, Aita VA, Flocke SA, Stange KC. Primary care practice organization and preventive 
services delivery: a qualitative analysis. Journal of Family Practice. 1998;46(5):403-409. 
238. 
Miller WL, Crabtree BF, McDaniel R, Stange KC. Understanding change in primary care practice using 
complexity theory. Journal of Family Practice. 1998;46(5):369-376. 
239. 
Sweeney SM, Hall JD, Ono SS, et al. Recruiting Practices for Change Initiatives Is Hard: Findings From 
EvidenceNOW. American journal of medical quality : the official journal of the American College of 
Medical Quality. 2017:1062860617728791. 
240. 
Dickinson LM, Beaty B, Fox C, et al. Pragmatic Cluster Randomized Trials Using Covariate Constrained 
Randomization: A Method for Practice-based Research Networks (PBRNs). Journal of the American Board 
of Family Medicine : JABFM. 2015;28(5):663-672. 
241. 
Murphy K, Dickinson C, Fagnan LJ, Parchman ML. Recruitment methodology for EvidenceNOW: describing 
Oregon's healthy hearts northwest recruitment process (Poster presentation). Practice-based Research Network 
Conference; 2017; Bethesda, MD. 
242. 
Raab GM, Butcher I. Balance in cluster randomized trials. Statistics in medicine. 2001;20(3):351-365. 

Page 25 of 26 
 
243. 
Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clinical trials 
(London, England). 2004;1(3):297-305. 
244. 
Greene EJ. A SAS Macro for Covariate-Constrained Randomization of General Cluster-Randomized 
and Unstratified Designs. Journal of statistical software. 2017;77(Cs1). 
245. 
Li F, Lokhnygina Y, Murray DM, Heagerty PJ, DeLong ER. An evaluation of constrained randomization for 
the design and analysis of group-randomized trials. Statistics in medicine. 2016;35(10):1565-1579. 
246. 
Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. 
Journal of evaluation in clinical practice. 2004;10(2):307-312. 
247. 
Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical 
and translational research. Clinical and translational science. 2011;4(5):332-337. 
248. 
Paxton AE, Strycker LA, Toobert DJ, Ammerman AS, Glasgow RE. Starting the conversation performance of a 
brief dietary assessment and intervention tool for health professionals. American journal of preventive medicine. 
2011;40(1):67-71. 
249. 
Dickinson WP, Dickinson LM, Jortberg BT, Hessler DM, Fernald DH, Fisher L. A protocol for a cluster 
randomized trial comparing strategies for translating self-management support into primary care practices. BMC 
family practice. 2018;19(1):126. 
250. 
Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation 
outcome measures. Implementation science : IS. 2017;12(1):108. 
251. 
Mays N, Pope C. Rigour and qualitative research. BMJ (Clinical research ed). 1995;311(6997):109-112. 
252. 
Marshall MN. The key informant technique. Family practice. 1996;13(1):92-97. 
253. 
Patterson RE, Kristal AR, Tinker LF, Carter RA, Bolton MP, Agurs-Collins T. Measurement characteristics of 
the Women's Health Initiative food frequency questionnaire. Annals of epidemiology. 1999;9(3):178-187. 
254. 
Yore MM, Ham SA, Ainsworth BE, et al. Reliability and validity of the instrument used in BRFSS to assess 
physical activity. Medicine and science in sports and exercise. 2007;39(8):1267-1274. 
255. 
U.S.Department of Health and Human Services. Behavioral Risk Factor Surveillance System. 2004. 
256. 
Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions 
(AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement 
Project (ACQUIP). Alcohol Use Disorders Identification Test. Archives of Internal Medicine. 
1998;158(16):1789-1795. 
257. 
Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of 
medication adherence. Medical care. 1986;24(1):67-74. 
258. 
Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure (PAM): 
Conceptualizing and Measuring Activation in Patients and Consumers. Health services research. 2004;39(4 
Pt 1):1005-1026. 
259. 
Polonsky WH, Fisher L, Earles J, et al. Assessing psychosocial distress in diabetes: development of the 
diabetes distress scale. Diabetes Care. 2005;28(3):626-631. 
260. 
Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a two-item 
depression screener. Medical care. 2003;41(11):1284-1292. 
261. 
Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the 
GAD- 
7. Archives of Internal Medicine. 2006;166(10):1092-1097. 
262. 
Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening 
tool for mild cognitive impairment. Journal of the American Geriatrics Society. 2005;53(4):695-699. 
263. 
Bickel G, Nord M, Price C, Hamilton W, Cook J. Guide to Measuring Household Food Security. Alexandria 
VA: U.S. Department of Agriculture, Food and Nutrition Service,;2000. 
264. 
Hall MH, Matthews KA, Kravitz HM, et al. Race and financial strain are independent correlates of sleep in 
midlife women: the SWAN sleep study. Sleep. 2009;32(1):73-82. 
265. 
Cohen S, Doyle WJ, Skoner DP, Rabin BS, Gwaltney JM, Jr. Social ties and susceptibility to the common cold. 
Jama. 1997;277(24):1940-1944. 

Page 26 of 26 
 
266. 
Health Measures. Available PROMIS® Measures for Adults. http://www.healthmeasures.net/explore- 
measurement-systems/promis/intro-to-promis/list-of-adult-measures. Published 2017. Accessed. 
267. 
Davis M, Balasubramanian BA, Cifuentes M, et al. Clinician Staffing, Scheduling, and Engagement Strategies 
Among Primary Care Practices Delivering Integrated Care. Journal of the American Board of Family 
Medicine : JABFM. 2015 EXPORT;28(S1):S32-S40. 
268. 
Cohen DJ. Online Diaries for Qualitative Evaluation: Gaining Real-Time Insights. American Journal of Evaluation. 
2006;27(2):163-184. 
269. 
Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and 
recommendations. Annals of family medicine. 2008;6(4):331-339. 
270. 
Ash JS, Sittig DF, McMullen CK, et al. Multiple perspectives on clinical decision support: a qualitative study 
of fifteen clinical and vendor organizations. BMC medical informatics and decision making. 2015;15:35. 
271. 
Miller WL, Crabtree BF. Primary care research: A multimethod typology and qualitative road map. Vol 31992. 
272. 
H3: Healthy Hearts in the Heartland. Chart review protocol for blood pressure measure (CMS 165V4; NQF 
0018) Version 1.7.16. https://escalates.org/wp-content/uploads/2019/03/H3-Chart-Review-Protocol.pdf. 
Published 2016. Accessed. 
273. 
Liddy C, Wiens M, Hogg W. Methods to achieve high interrater reliability in data collection from primary 
care medical records. Ann Fam Med. 2011;9(1):57-62. 
274. 
ESCALATES Study Team. Method to sample charts. https://escalates.org/wp- 
content/uploads/2019/03/ESCALATES-Chart-Sampling.pdf. Published 2015. 
Accessed. 
275. 
Beebe J. Rapid assessment process : an introduction. Walnut Creek, CA: AltaMira Press; 2001. 
276. 
Miles MB, Huberman AM. Qualitative data analysis : an expanded sourcebook. Thousand Oaks: 
Sage Publications; 1994. 
277. 
McCulloch CE, Neuhaus JM. Generalized linear mixed models. Wiley Online Library; 2001. 
278. 
Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine--reporting of subgroup analyses 
in clinical trials. The New England journal of medicine. 2007;357(21):2189-2194. 
279. 
Cnaan A, Laird NM, Slasor P. Using the general linear mixed model to analyse unbalanced repeated measures 
and longitudinal data. Statistics in medicine. 1997;16(20):2349-2380. 
280. 
Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster 
randomised trials. Bmj. 2012;345:e5661.